Synthesised Surround Sound Department of Electronics and Computer Science University of Southampton, Southampton, SO17 2GQ

Transcription

1 Synthesised Surround Sound Department of Electronics and Computer Science University of Southampton, Southampton, SO17 2GQ Author Abstract This paper discusses the concept of producing surround sound with just two speakers. Traditionally, a single pair of speakers could only produce phantom images that appear to originate from a location at or between the physical speakers. Synthesised surround sound, however, is attempts to simulate a convincing three dimensional environment where sounds are perceived to originate from any given direction. Two ways of synthesising surround sound are discussed: binaural sound, and synthesised HRTFs (Head Related Transfer Functions). The relative advantages and disadvantages of each system are analysed and discussed, accounting for such subjects as crosstalk, the 'sweet spot' phenomenon, and the relative sociability of the systems. Suggestions as to the future possibilities and market for such technology are also ventured. Keywords Psychoacoustics, binaural sound, music, synthesised surround sound, stereo, headphones, localisation, HRTF Head Related Transfer Functions, crosstalk, multi-channel. Introduction This paper will aim to review the development and drive behind the belief that it is possible to produce surround sound through a simple stereo speaker system. Synthesised surround sound works on a similar principle to that of stereo, where sounds can be panned to locations between the speakers, thereby creating phantom images where there are no speakers. Stereo cannot, however, create phantom images to the sides, above, below or behind the listener and this is where synthesised surround sound seeks to succeed. The main competing technologies at present will be compared and contrasted with a view to explaining some of the technical issues this topic encompasses and the theory behind them. Suggestions will then be made as to the kind of future such a system may have in our society, and whether there is a possibility it could become as prevalent as stereo or Dolby- Pro Logic is today. Background Early monophonic 1 music reproduction managed to convey the most important features, such as melody, tempo, pitch and amplitude. However, it lacked the vital spatial element. That is to say, when listening to a live sound source, our auditory system will automatically process where it is originating (see HRTFs). Playing a monophonic recording, then, will be simply relayed to the listener as originating from the speaker. By the 1930s the theory behind recording the directional and spatial elements were understood, but both cost and technical limitations restrained realisation of these concepts. 1 Monophonic refers to recording through a single channel. This is in contrast to humans who listen binaurally (we have two ears) even if two or more channels are used. - Page 1 of 8 -

2 The 1950s welcomed the arrival of twochannel stereo recordings even though it suffered greatly from over emphasised leftright effects or a hole in the middle. The idea of stereo music was to place the featured artist as a phantom image between the two speakers. Even today, products attempting to further the feeling of listener interaction by utilising products such as spatialisers 2 are common place. It can only be concluded, then, that two-channel stereo still is not adequate. Here, the market divides into two main standpoints that which requires additional channels and speakers, and that which looks to evolve stereo by differing and/or better encoding/decoding techniques. This paper will pay most attention to the second standpoint. It should also be noted that the paper is not dealing with stereo enhancements whereby sounds that are normally limited to the left and right speakers are widened to extend this. This is not considered three-dimensional or surround sound, although this is often claimed! A Sociable System? Firstly, it needs to be noted that the two approaches mentioned previously are largely based around methods to tackle the sweet spot 3 phenomenon. A system that produces a very localised sweet spot is often termed an anti-social system as it only caters for a single listener, whilst others will experience a degraded performance. The majority of stereo systems will create a sweet spot that is very long, tall and narrow the listener may move towards and away from the speakers or vertically up and down. Any movement from left to right will progressively impede the sound quality. Headphones could be viewed as the most 2 A stereo enhancement where sound appears to extend laterally beyond the two speakers. 3 A sweet spot refers to the area in which the listener receives the most aurally rewarding performance Figure 1: Multi-channel, multi-speaker cinema system [1] extreme in an anti-social system. This type of system offers itself to research in new ways of recording and playback. At the other end of the scale, is a multichannel, multi-speaker system yielding a large sweet spot that can provide an audience in a room (such as a cinema, fig1) with a sound that will hardly alter from seat to seat. Clearly, there may be worse and better house seats but on the whole, this would be regarded a very sociable system. In focusing on synthesised surround sound, the remainder of this paper will concentrate on the stereo, two speaker systems that are generally anti-social, although there are some viable exceptions available now. Head Related Transfer Functions As sound waves encounter the head, they diffract around it and the outer ears. The brain receives this information; along with the small time delay ( 0.4ms) that occurs as the sound travels from one ear to the other; and accurately processes it for the location, distance and even the type of acoustic surroundings at the sound source [4]. More precisely, as sound waves come in contact with the intricate folds of the outer ear (the pinna) they are amplified or suppressed depending on their direction. From this, the head, pinna and auditory canal the first three stages of the ear a sound wave encounters - are - Page 2 of 8 -

3 all considered spatial encoders (fig2). This is then decoded once transmitted to the brain [5]. It was soon deduced that the characteristics of a sound wave being spatially encoded were quantifiable and became known as the Head Related Transfer Functions (HRTFs). This incorporated three main areas a near ear response, a far ear response and the interaural time delay. Finally, and most importantly, it was found possible to electronically synthesise the HRTFs, which could then be relayed audibly through headphones or two speakers. When done well, an extremely convincing reproduction of the original sound is conveyed to the listener, whose brain is tricked into believing sounds are originating from all around. Clearly then, it is possible to generate a surround sound environment from just two speakers what we perceive relies on the encoding or decoding methods when reproducing sound waves that reach our ears. Reproducing Surround Sound This can be achieved using two methods. The first relies on capturing the HRTF at the original live sound location, and requires just two channels and no additional hardware or software for playback. This is called binaural sound and is largely unheard of. The second synthesises the HRTFs on reproduction of the sounds. It uses multiple channels (the Dolby Digital 5.1 standard, for example) and needs extra processing before output to a pair of speakers. Binaural Sound Binaural sound was conceived and proven at the beginning of the 1900s and yet still remains unknown for many people. Anyone who perceives sounds binaurally does so with Figure 2: Acoustic transmission into the ear[2] two ears. The term refers to the paths from the ears to the brain which are separate, that is, each ear converts the sound reaching it into electrical impulses, so the sound from the two ears mix in the brain not as physical vibrations, but as mixed electrical signals [3,6]. Recording sounds binaurally, works by capturing the sound waves as they enter the ear canal of the listener - literally by placing small microphones on or in the outer ear, depending on the accuracy sought. Microphones located inside the ear canal near the tympanic eardrum give the greatest realism because the sound waves have received the maximum amount of natural processing by the human hearing system. This is best achieved by the use of an accurately modelled mannequin head complete with implanted microphones. Alternatively, microphones situated on the side of the head are really only capturing the interaural time delay of the sound wave; along with slight processing from the pinna, although this is the simplest and most cost effective method. Whatever method is settled on, the result can be recorded to any stereo compliant device with no further processing and in turn, listened to using any stereo system. - Page 3 of 8 -

4 The end result is a fairly realistic representation of the original sound especially with the use of headphones. However, a common complaint with headphones is the sounds that should be perceived far out in front are often localised very close to or inside the listeners head [1]. This problem is compounded by the lack of visual cue for a sound. Naturally, a listener may turn their head to help localise a distant sound, but this is ineffective for a synthesised sound using headphones its location moves with the listener. To enable synthesised surround sound with this technique, more channels are required when recording. As used in the film industry, Dolby 5.1 Surround does exactly this (fig 3), with one speaker situated in each corner, a central speaker and an non-directional lowfrequency speaker (subwoofer) the.1. It is Dolby Surround that will continue to be used as an example. To combat this, loudspeakers need to be used to deliver an impressive sound externally and in front of the listener. This in turn creates a new problem crosstalk (see later). Synthesising HRTFs This is where the most research and development is currently being deployed. Here, the head related transfer functions are applied to a given signal that was recorded at the source. The simplest system is a stereo system where there are two channels left and right. The HRTFs are applied to each channel that creates a signal, one for each ear. A twochannel system still will not allow for synthesised surround sound just a more convincing stereo experience. Figure 4: Dolby Surround speaker positioning[2] Each channel is encoded onto a stereo device using a matrix such as Dolby ProLogic. To reproduce the sound, normally the data will be passed through a Dolby ProLogic decoding amplifier which then passes each channel straight on to one of the five speakers it is intended. To synthesise this same effect for two speakers, the same process must occur, but an additional step is inserted between the decoder and the speakers. This step takes each separate channel, and according to their intended location, a specific HRTF is applied to each to synthesise a phantom speaker. Figure 3: Dolby 5.1 Surround[1] For example, the rear right channel is expected to be situated +120 from the listener (fig 4) and so a HRTF is calculated to simulate this. The function is then applied to the rear right - Page 4 of 8 -

5 channel to simulate its location on a stereo system. Crosstalk The signal produced from the right and left speaker must be kept isolated. More definitively, the sound waves travelling from the right hand speaker should only reach the listeners right ear and likewise for the left speaker. In reality, however, small amounts of sound will always cross over to the wrong ear and thereby create crosstalk (fig5). Both binaural sound and synthesised surround sound using HRTFs suffer from crosstalk. However, the opposite speaker can electronically generate crosstalk counteraction so that a sound wave equal in magnitude and with inverted phase will cancel each other out on arrival at the listener s ears. though, the sounds on a speaker system are correctly perceived to be external and out in front of the listener. In summary, there are two ways to create synthesised surround sound - microphones located on a mannequin head and electronic processing with HRTFs and two ways to reproduce them headphones and crosstalk cancelled loudspeakers Conclusion It would seem there is a definite market place for synthesised surround sound technology, whether it is crosstalk-cancelled speakers or headphones both have a valid niche. At present, it would be safe to say that neither is as well known as say the fully-fledged Dolby Digital home cinema Surround Sound entertainment systems. However, there are several clear advantages to choosing a synthesised version. Figure 5: Crosstalk This relies on the assumption that the listener is in a pre-defined location that will not change, and so, even with the best system, the sweet spot is very localised no better than conventional stereo. The advantage lies in the auditory reward stereo where a phantom image can be created between two speakers; and alternatively being taken to another three-dimensional world. The end result of crosstalk-cancelled speakers may be likened to the use of headphones, which automatically ensures that only the right ear hears the right channel and the left ear, the left channel. An important difference remains, Playback: Crosstalk-cancelled speakers vs. Headphones Before the advantages can be realised in the market place, peoples preconceptions regarding what is needed to achieve surround sound must be challenged. These preconceptions are largely based on the familiar multi-channel, multi-speaker set-ups that are found in cinemas everywhere. In fact, the idea of attempting to recreate the cinema experience in one s home could be considered naive. Very often the room is really too small to situate five physical speakers effectively and so the intended effect of space and distance is lost. When considering synthesised surround sound this isn t a problem at all - the room s physical walls do not restrict the sound environment. Instead, it is impressive to have a room with two cross-talk cancelled speakers that create phantom speakers that can be heard but not seen. These phantom images are not - Page 5 of 8 -

6 constricted to the room either they may even appear to originate outside the walls! Not only this, but the cost five speakers can be spent on a pair of higher quality ones that will all sound perfectly matched - the two physical speakers create the phantom images, which means the characteristics (or timbre[7]) of all five speakers will be the same, differing only by the HRTFs used. Crosstalk-cancelled speakers are ideally suited to the use of personal computers. A problem noted earlier was the distance of the listener from the speakers was assumed, and auditory performance degraded if this assumption did not hold true. With a computer, this assumption can be safely calculated and relied on, which thereby offers itself to films and computer games (many of which now incorporate surround sound). Another advantage of loudspeakers is the ability to confirm the location of a sound. Naturally we rotate our head to help locate a sound, so by using the same technique with loudspeakers we conclude the sound is external and in front. With headphones, this is quite ineffective as the sound source moves with the head. More trivial factors to assess are details such as cost and type of use (social or antisocial). For example, if a system is wanted to watch films with friends then speakers are a necessity. If music or films want to be played without disturbing others, headphones would be most valid. Cost is less of an issue, but headphones are generally found to be cheaper than speakers. Recording: Binaural vs. Multi-channel with HRTFs The main drawback with binaural recording is it suffers from the type of model the sound was recorded with. The human head and ears vary greatly; so by recording music with just one model greatly restricts its effect. Humans grow accustomed to their own hearing system to localise sounds, and when using someone else s our ability to localise the same sound is reduced. Binaural sounds are also plagued with the confusion between front and back sounds. This is due to the front/back (and up/down) symmetry of our ears and is found in natural, live localisation problems[6]. Normally, we are provided with visual cues to help and so games or film assist in the reproduced world. Binaural is also a very rare medium to be able to purchase. There are few pieces of music produced in binaural available, although some sources can be found on Internet web sites[9]. Multi-channel recordings are abundant however. They are mainly found in film, but with the advent of DVD many musical albums also feature multi-channel. As F.E Toole[1] notes,..ironically, it has been the film industry, not the audio industry or audiophiles, that has driven the introduction of multi-channel sound reproduction in homes. Both methods of recording produce a relatively small sweet spot. They require the listener to be in a certain location for the optimal result. There is always a trade-off available if a larger sweet spot is preferred it becomes both fuzzier and the synthesised sound locations become less well defined. The Future With regard to preference of a recording technique, I would have to put forward multichannel. The sheer fact that binaural technology has been known about for over a century, but has a negligible share of the market suggests to me it would be very unlikely to see a sudden interest. Alternatively, multi-channel recording has already obtained phenomenal amounts of public acceptance in the form of Dolby Digital. - Page 6 of 8 -

7 The public market is very slow to adopt or change a product they have grown accustomed to, and considering Dolby s heavy involvement with the film industry it would be great surprise to see it disappear or even reduce in favour of any alternatives available. of multi-channel headphone listening can only grow in popularity. Additionally, HRTFs are so realistic now, it may be easier for the public to be persuaded to a synthesised surround system. Moving on to methods of playback, both headphones and crosstalk-cancelled speakers seem to have their different advantages. Loudspeakers for small, home cinemas rooms and headphones for private listening. Again, as the complete Dolby Digital decoding amplifiers and speakers systems are readily available, and have been for years, I feel it unlikely that crosstalk-cancelled speakers will become a system on par. However, I would expect it to be a more commonplace in home entertainment and especially in the use of personal computing and gaming. The main area of growth, I foresee is the use of headphones. It can only be a good thing to be able to watch a film privately through headphones - whether commuting on a train or plane, or late night viewing at home but still with the full surround sound experience. Already, in the last year, Dolby have released a product named Dolby Headphone. This manages to perform a simulation using HRTFs of all 5.1 channels in normal stereo headphones and there is no longer the crosstalk issue [8]. Dolby Headphone has already been incorporated into several software DVD players (namely Cyberlink PowerDVD) as well as several airlines that have adopted the idea for their in-flight entertainment. After experiencing the capabilities of Dolby Headphone along with the increase of laptop computers and portable DVD players such use - Page 7 of 8 -

8 References [1] Floyd E. Toole, Direction and Space, The Final Frontiers How many channels do we need to be able to believe that we are there, Harman International Industries Inc., March 2000, (accessed: 5/11/2001) [2] A Sibbald, Virtual Audio for Headphones, Sensaura White Paper, 1999, (accessed: 02/11/2001) [3] Britannica Macropædia, Knowledge in Depth (vol. 27, page 569), 15 th Edition ( ), Binaural Perception. [4] Britannica Macropædia, Knowledge in Depth (vol. 27, page 569), 15 th Edition ( ), Locating Sound by Sound Phase. [5] Britannica Macropædia, Knowledge in Depth (vol. 27, page 205), 15 th Edition ( ), Sensory Reception: Structure of the Inner Ear. [6] D. Howard and J. Angus, Acoustics and psychoacoustics, 2 nd Edition, Focal Press (2001); page 97, 102, 359 [7] P.Cook, Music Cognition and Computerised Sound And Introduction to Psychoacoustics, Introduction to Timbre, First MIT paperback edition (1999), Chapter: Introduction to Timbre page 79. [8] The Dolby Headphone Story (accessed 12/11/2001) [9] The Binaural Source (accessed 13/12/2001) Further Reading R. Gilkey, Binaural and Spatial Hearing in Real and Virtual Environments, Lawrence Erlbaum Associates, (1997) E. Schubert, Psychological acoustics benchmark paper in acoustics, v.13, Dowden, Hutchinson & Ross, (1979) B. Gardener HRTF Measurements of a KEMAR Dummy-Head Microphone, (1996: accessed 1211/2001) Surround Sound Past, Present and Future (accessed 12/11/2001) B. Moore, An Introduction to the Psychology of Hearing, 2 nd Edition, Academic Press (1982), page Page 8 of 8 -