Editorial - How do you see music?


Until the dawning of the era of recorded sound, most music was a very visual medium - if you heard music, the chances were very strong that you'd also be seeing it being played right before your eyes. With the perhaps mystical exception of church organs and hidden choirs (suggesting something not of this world?) music was by definition live and if you could hear it you could see it taking place in front of you.

The change to recorded sound must have been profound - to actually be able to stop time and hear something that had already taken place, after millions of years during which this was impossible, could only have been seen as truly mind-blowing by the generation not born into the era of recorded sound. How, they might have asked, could this cylinder, or this grooved disc, possibly capture and replay someone's voice, or an instrument?

An early ad for Edison's cylinders advised the listener to shut his eyes and visualise the musicians performing before him to enhance the realism of the recording - bear in mind that this was in the middle of the acoustic (pre-microphone) recording era when I tell you that if you didn't hear the total realism, according to the advertisement you probably weren't trying hard enough and should try again, and this time do it properly!

Today we've gone far beyond the era where anyone alive remembers a time before sound recordings. We take it for granted that they exist, and that in the case of older ones, there are people who spend time and effort trying to make them sound better. It's a shame we can't go back in time to hear the first performances of works by Beethoven, Mozart or Bach - or perhaps the orators of Ancient Rome and Greece - but in a relatively short time we have amassed a huge amount of recorded sound. As busy as people like I am restoring it, new recordings are being generated on a massive scale. It's a never-ending document of human existence, and much of it now comes with moving pictures.

So when you listen to, for example, Lotte Lenya singing the part of the prostitute Jenny in Kurt Weill's The Rise and Fall of the City of Mahagonny, what do you see? Do you visualise her as depicted on the CD sleeve? Or perhaps as she appeared in the 1962 Bond movie From Russia With Love? Perhaps you see notes on a page, or are too busy concentrating on the words to visualise anything. Some people see music as colours - literally streaming apparently before their eyes.

For the sound restoration engineer, visualising music (and sound) has become incredibly important over the last decade. Until very recently the visual aspect of a sound recording was limited to patterns of grooves on a record, or the sound you get when playing through a chewed up tape, or perhaps something unintelligible on an oscilloscope screen.

But today, the advent of powerful computers and digital recording allows us to literally visualise music and sound, see how it's made up and how it fits together. Have a look at this example:

Clarinet solo - Low strings - Lotte Lenya speaks (dur. 5", from 5'24" in)

I've tried to caption the picture above to coincide with what's happening in the music. From left to right we begin with a single solo clarinet note. The very bright line at the bottom is the note itself, those above it the quieter, and here visually dimmer, harmonics. A low strings (double basses and cellos, by the sound of it) counter-melody quickly joins in, creating something of a visual mess (at this resolution) - all those low frequencies are at the bottom of the picture.

The entry of Lenya's voice is dramatically different, as her half-sung, half-spoken dialogue slides up and down in pitch, together with its multiple harmonics. Yet still we can see the clear, straight lines of the clarinet underneath.

Once sound "exists" like this, and you're given a set of tools with which to manipulate its image, all sorts of things become possible:

Perhaps you hear a distant car moving off, its low but rising note captured in the concert hall by the microphone. You zoom right in on that frequency range, looking for the rising lines of the car as you listen, then select it and, with the click of a mouse, 'zap' it to oblivion. You're using a highly intelligent "zapper", set up carefully to recognise what is music (that you want to keep) and what is noise (that you want to zap). It's good too for coughs, sneezes, chair scrapes, dropped pages and any number of other unwanted noises.

Or maybe there's a dropout in the tape - suddenly the "picture" dims above 3500Hz and the treble does likewise. So you select the dim area and brighten it with a volume control - suddenly restoring the missing sound and restoring that dull section to its original brightness.

You can get creative - copying and pasting notes, harmonics and so on to cover bad edits, dropouts or even fix mistakes (the latter is not something I would indulge in due to the nature of what I'm producing). You can see radio interference and follow its pattern with a mouse pointer, then interpolate what's above and below it to remove it without trace. You can spot electrical hum and see how many harmonics are there, before isolating them and filtering them out.

And on it goes. This is how I see music and sound every day when I'm working. My job as a sound engineer, which used to be an entirely auditory one (I was taught early on at the BBC to ignore the desk meters and train my ears to be better than the electronics), is now often as much about visuals as it is about sound, especially when one has a bronchial audience to contend with! It's hard for me to imagine a world without this - or to think of pure sound in any other way, and it's hard to conceive of a world and a way of working without it.

Just like sound recording itself - so quickly you take it for granted, and wonder how you ever got along without it.

Andrew Rose, December 3rd, 2010