XR remastering's equalisation mystery, part 3

And why we're not going to get $1,000,000,000 for it, probably    

Last week's newsletter got a bit too scientific for some - but not enough for others, judging by the feedback I've received. This week I want to carry on trying to bridge that gap, and I'll start by going a lot less (I hope) science-y.


The Billion-Dollar Question

OK, so maybe this is pushing the limits of credibility just a tad, but is Pristine Audio about to be bought up by Facebook for a billion dollars? The short answer is no. The longer answer, well it's very similar, I've reluctantly concluded.

What Facebook has recently splashed out on (and I love the tale that this takeover was cooked up between the two young-ish company bosses alone on a sofa, with no reference to lawyers, company boards, banks or anyone else) is a company called Instagram. Yes, Facebook blew a billion on a company that makes an iPhone app, has 13 employees, has never made a penny, and didn't exist a year or two ago.

What does Instagram's app do? The opposite of XR - it makes new things seem older. Specifically, you take a picture on your phone, run it through the "retro-cool" filters in Instagram, then upload it so all your friends can see just how clever you are. Hmm. Still, what do I know. Anyway, here's a nice collage I found online demonstrating those super "retro" effects:



Thrilling isn't it?

Each picture takes the original and messes with its colour balance, brightness, contrast, gamma, saturation and a whole host of other things I don't know enough about to explain here, in order to "create" something that looks a bit like a faded colour photo you might have had stuffed in a drawer since 1974. Even the company logo (bottom right) has a 1970s design vibe about it.

Yes, a certain take on "authentic" retro is apparently cool. Hip young people are supposedly discovering mysterious "old" things, like film photography, vinyl LPs, typewriters and the like. Some seem to prefer the vintage aesthetic now that it's just another optional extra.

All that money got me thinking, as it would. It's relatively easy to doctor a photo to make it look a bit older, or even quite a lot older - though anyone who's an expert ought to be able to spot the fakes, especially if there's someone else holding something obviously modern in the shot. But doing the opposite is quite a lot harder - even more so when you're in the audio rather than our primary, visual domain as humans, and that's what XR is trying to do.

Imagine an old colour photograph - a real one this time - such as a single shot (taken from a reel of film you still have) which contains a scene that still exists identically today, one with a multitude of colours which haven't changed a bit over the last forty years. Perhaps it's a summer's day with a clear blue sky and the sun shining directly overhead. The grass is a rich green, as are the leaves on the trees. A stone building remains a multitude of fine grained detail, and a bright red British post box sits to one side, its precise shade clearly defined by law and unchanged for many, many decades.

If you could take that old photo and a brand new photo of the same scene and analyse all the colour data of the two, you'd probably be able to come up with a kind of reverse-Instagram filter which could "modernise" that old shot. And if there was enough colour information in that single shot to cover the entire range of the visual spectrum adequately, you might even be able to use the same filter to restore the rest of the film roll to its full-colour glory. But if you were to take a second roll of film from another camera and another year, it probably wouldn't work as well - the results might look a little off-colour at best, and could actually be considerably worse.

This isn't too bad an analogy for what XR remastering does. Our 1930s recording this week of Weingartner conducting Brahms shares sufficient characteristics with a modern recording of the same - violins, cellos, flutes, trumpets and so on, not to mention them playing the same notes in the same order and following, roughly at least, some kind of instruction regarding volume - that we can compare these recordings as we compared the two photos described above, noting the differences in the audio frequencies rather than the visual frequencies, and then go about correcting the older version using the newer version as a reference.

Neither is likely to yield 100% perfect results. Just as a convincing ageing filter for photographs will have to discard information as it flattens colours and burns out brightness, so some of the fine-detail information was lost or distorted in the real thing, or at best poorly captured, back in 1938. In the case of Weingartner's Brahms this degree of loss has proved unpredictable - the detail seems more clear in the Third Symphony where in the Fourth the sound is a little more flattened out, if you like, despite the recordings taking place in the same studio just a few short months apart in 1938.

There's also the question of noise. I don't know whether Instagram adds simulated photographic grain to an image, but there's real audio "grain" in all analogue recordings: in the tape hiss, the disc noise, the low level background into which the quietest detail is lost. Sometimes this noise is random, which can make it easier to deal with, but sometimes it's not as random as I would like. The Weingartner Fourth Symphony in particular suffered from a kind of swirly "something" that's almost as loud as the real music between 5kHz and 10kHz, and was a real nuisance to deal with. And our Backhaus piano sonatas series has suffered from noise which gets louder as the playing gets louder - and is especially annoying with single, isolated high notes, which sometimes emerge as smudges rather than cleanly hit strings.

But all is not lost - we do at last have tools, developed largely over the last decade and especially the last 3-5 years, which offer unprecedented control in the audio domain, including the ability to go in and "airbrush out" those smudges manually, whilst leaving the notes intact. Alas it's not as easy to display the results of a before-and-after audio restoration session in an e-mail as it is to show what Instagram's photo filters do, but I did make my own little sound collage for you to try.

If you click here you can download a short, high quality MP3 sound file. It's the beginning of Backhaus's Hammerklavier, and I've set it up to contrast the "old" original with the XR "new".

We start with a short phrase in its original state, followed by the XR equalised version of the same. Then we move on to the next phrase - again the "old" first then the "new".

There then follows a longer section. Partway through we cross-fade from original to XR. You'll also hear the change from mono to Ambient Stereo, with just the tiniest bit of convolution reverb to fill out the picture. The fade is complete at about 50 seconds into the clip and we stay with XR until the end.

Now you'll have to take my word for this, but I've not messed around with the dynamics here to address the criticism made about the recording in the 1950s - that it was dynamically flat. I've done nothing to make the loud sections louder or the quiet ones quieter. But correcting the tonal balance has achieved precisely that - the first "before" (a loud phrase) is clearly quieter than its "after". Yet the second "before" and "after" are much more closely balanced during this quieter phrase.

Meanwhile the longer, cross-faded section contains more of a mix of dynamics. In its original state it does sound flat, dull and lifeless. But listen to how it comes to life once all the frequencies are harmonically balanced and the full vibrancy of the performance is unleashed. This is one reason I've written most weeks that this sonata cycle desperately needs a re-evaluation - it's never had a chance to really convey Backhaus's playing or vision before and had been quietly consigned to the waste bin of history by many.

Finally, and perhaps to stretch the old picture/old recording analogy to near breaking point but it's a question that comes up regularly, I still think the chances of recreating true, positional stereo or surround sound from mono recordings of the historic era are about as remote as recreating truly believable 3D colour images from black and white photographs and movies of the same era. I can conceive of a time not too far away where we can fake that separation, just as we can colourise old films, but even if the actual positional information is buried in the mono signal waiting for someone to figure out how to extract it and use it (and it's highly doubtful that it is), we're a long, long way away from being able to do so.


A question of zoom and resolution - the vaguely scientific bit

You'll remember all those lovely graphs from last week, like this one:



And this one:



I talked a lot about "zooming in" and "resolution", yet the axes on the graphs remained the same. How could this be so? If the two graphs above represent the same thing how can one be a "zoomed in" version of the other?

This comes down to the way we analyse audio data using something called Fast Fourier Transforms (FFT), and it's the resolution of these which gives us the differences in our graphs above. Now I don't want to go into the maths with regard to FFT's, not least because it's probably beyond me now, but what I do know is that in practise when using the technique to split a waveform up into component frequencies for display purposes one can vary the number of sample windows one analyses and thus the resolution or "zoom".

We can increase the resolution by upping the number of FFT bands. This means that we can split the audio frequency range into a set number of "chunks" and work out how loud they are at any particular point in time. There is rarely an optimum number of chunks for this - it depends on what you're trying to achieve. The more chunks you split into the more processing power and time required and the longer you have to wait for the results. You so get finer frequency differentiation, but you may also lose out, because this is offset by coarser time resolution - in other words, using FFT, the more you find out what frequency is being played, the less you know about when it started and stopped playing.

In a visual sense, or in the mind's eye, you can however still think of it as "zooming in on sound" - start the FFT resolution low and you first might see blocks of octaves on screen. Increase it a little and you'll get blocks of fifths, then thirds, individual tones, semitones and finer. And it turns out that at different resolutions, different degrees of "zoom", characterise different aspects of a recording's characteristics - the zoom level required for XR remastering turns out to be reasonably high, whereas a lower zoom level gives you a better idea of the brightness or boominess of a recording.

We then turn in XR to digital EQ filters, which offer a degree of ultra-high selectivity not available in the analogue domain. We can create and use filters with a very high "Q factor" (read all about that here if you wish) that can target specific frequencies very precisely without affecting neighbouring frequencies in order to boost or reduce their volume. This is something we can do very well mathematically in the digital domain - it's a lot harder with capacitors and resistors in the "real" analogue world - and this, combined with fine-resolution FFT analysis under computer control is what has allowed the development of XR remastering.

Some reading this will have a vague idea of what I'm going on about. Others will know a lot more about it than me (and may even have spotted a mistake or two in my explanation - go easy on me if you have!). Many won't have a clue what all this is about, technically speaking. But in my view it doesn't really matter just as long as the results sound right to your ears.

Right now I'm listening through to a recording I've been working on for a week or two, of Elgar's Second Symphony by Barbirolli and the Hallé Orchestra. It was made in 1954, and what's just jumped out at me is Elgar's delicious use of the bass drum. It was somewhat lost in the original recording, likewise in the reissue I've just managed to track down, but there it is in the XR remaster of the last movement, a quiet underpinning you feel as much as hear.

Good job I had plenty of FFT and Q for that then, because it sounds lovely at last - an essential Elgarian detail finally revealed in this wonderful performance, and coming to a website near you soon...

 

Andrew Rose
20 April 2012