A Perceptual Model for Sinusoidal Audio Coding Based on Spectral Integration

Transcription

1 EURASIP Journal on Appled Sgnal Processng 25:9, c 25 Steven van de Par et al. A Perceptual Model for Snusodal Audo Codng Based on Spectral Integraton Steven van de Par Dgtal Sgnal Processng Group, Phlps Research Laboratores, 5656 AA Endhoven, The Netherlands Emal: steven.van.de.par@phlps.com Armn Kohlrausch Dgtal Sgnal Processng Group, Phlps Research Laboratores, 5656 AA Endhoven, The Netherlands Department of Technology Management, Endhoven Unversty of Technology, 56 MB Endhoven, The Netherlands Emal: armn.kohlrausch@phlps.com Rchard Heusdens Department of Medamatcs, Delft Unversty of Technology, 26 GA Delft, The Netherlands Emal: r.heusdens@ew.tudelft.nl Jesper Jensen Department of Medamatcs, Delft Unversty of Technology, 26 GA Delft, The Netherlands Emal: j.jensen@ew.tudelft.nl Søren Holdt Jensen Department of Communcaton Technology, Insttute of Electronc Systems, Aalborg Unversty, DK-922 Aalborg, Denmark Emal: shj@kom.aau.dk Receved 31 October 23; Revsed 22 July 24 Psychoacoustcal models have been used extensvely wthn audo codng applcatons over the past decades. Recently, parametrc codng technques have been appled to general audo and ths has created the need for a psychoacoustcal model that s specfcally suted for snusodal modellng of audo sgnals. In ths paper, we present a new perceptual model that predcts masked thresholds for snusodal dstortons. The model reles on sgnal detecton theory and ncorporates more recent nsghts about spectral and temporal ntegraton n audtory maskng. As a consequence, the model s able to predct the dstorton detectablty. In fact, the dstorton detectablty defnes a (perceptually relevant) norm on the underlyng sgnal space whch s benefcal for optmsaton algorthms such as rate-dstorton optmsaton or lnear predctve codng. We evaluate the merts of the model by combnng t wth a snusodal extracton method and compare the results wth those obtaned wth the ISO MPEG-1 Layer I-II recommended model. Lstenng tests show a clear preference for the new model. More specfcally, the model presented here leads to a reducton of more than 2% n terms of number of snusods needed to represent sgnals at a gven qualty level. Keywords and phrases: audo codng, psychoacoustcal modellng, audtory maskng, spectral maskng, snusodal modellng, psychoacoustcal matchng pursut. 1. INTRODUCTION The ever-ncreasng growth of applcaton areas such as consumer electroncs, broadcastng (dgtal rado and televson), and multmeda/internet has created a demand for Ths s an open-access artcle dstrbuted under the Creatve Commons Attrbuton Lcense, whch permts unrestrcted use, dstrbuton, and reproducton n any medum, provded the orgnal work s properly cted. hgh-qualty dgtal audo at low bt rates. Over the last decade, ths has led to the development of new codng technques based on models of human audtory percepton (psychoacoustcal maskng models). Examples nclude the codng technques used n the ISO/IEC MPEG famly, for example, [1], the MnDsc from Sony [2], and the dgtal compact cassette (DCC) from Phlps [3]. For an overvew of recently proposed perceptual audo codng schemes and standards, we refer to the tutoral paper by Panter and Spanas [4].

2 Perceptual Model for Snusodal Audo Codng 1293 A promsng approach to acheve low bt rate codng of dgtal audo sgnals wth mnmum perceved loss of qualty s to use percepton-based hybrd codng schemes, where audo sgnals are decomposed and coded as a snusodal part and a resdual. In these codng schemes, dfferent sgnal components occurrng smultaneously are encoded wth dfferent encoders. Usually, tonal components are encoded wth a specfc encoder amed at sgnals composed of snusods and the remanng sgnal components are coded wth a waveform or nose encoder [5, 6, 7, 8, 9]. To enable the selecton of the perceptually most sutable snusodal descrpton of an audo sgnal, dedcated psychoacoustcal models are needed and ths wll be the topc of ths paper. One mportant prncple by whch audtory percepton can be exploted n general audo codng s that the modellng error generated by the audo codng algorthm s masked by the orgnal sgnal. When the error sgnal s masked, the modfed audo sgnal generated by the audo codng algorthm s ndstngushable from the orgnal sgnal. To determne what level of dstorton sgnal s allowable, an audtory maskng model can be used. We, for example, consder the case where the maskng model s used n a transform coder. Here the model wll specfy, for each spectrotemporal nterval wthn the orgnal audo sgnal, what dstorton level can be allowed wthn that nterval such that t s perceptually just not detectable. Wth an approprate sgnal transformaton, for example, an MDCT flter bank [1, 11], t s possble to selectvely adapt the accuracy wth whch each dfferent spectro-temporal nterval s descrbed, that s, the number of bts used for quantsaton. In ths way, the spectro-temporal characterstcs of the error sgnal, can be adapted such that audtory maskng s exploted effectvely, leadng to the lowest possble bt rate wthout perceptble dstortons. Most exstng audtory maskng models are based on the psychoacoustcal lterature that predomnantly studed the maskng of tones by nose sgnals (e.g., [12]). Interestngly, for subband coders and transform coders the nature of the sgnals s just the reverse; the dstorton s nose-lke, whle the masker, or orgnal sgnal, s often tonal n character. Nevertheless, based on ths psychoacoustcal lterature dedcated psychoacoustcal models have been developed for audo codng for the stuaton where the dstorton sgnal s nose-lke such as the ISO MPEG model [1]. Maskng models are also used for snusodal codng, where the sgnal s modelled by a sum of snusodal components. Most exstng snusodal audo coders, for example, [5, 6, 13] rely on maskng curves derved from spectralspreadng-based perceptual models n order to decde whch components are masked by the orgnal sgnal, and whch are not. As a consequence of ths decson process, a number of masked components are rejected by the coder, resultng n a dstorton sgnal that s snusodal n nature. In ths paper a model s ntroduced that s specfcally desgned for predctng the maskng of snusodal components. In addton, the proposed model takes nto account some new fndngs n the psychoacoustcal lterature about spectral and temporal ntegraton n audtory maskng. Thspaper sorgansed asfollows. InSecton 2 we dscuss the psychoacoustcal background of the proposed model. Next, n Secton 3, the new psychoacoustcal model wll be ntroduced, followed by Secton 4, whch descrbes the calbraton of the model. Secton 5 compares predctons of the model wth some basc psychoacoustcal fndngs. In Secton 6, we apply the proposed model n a snusodal audo modellng method and n Secton 7 we compare, n a lstenng test, the resultng audo qualty to that obtaned wth the ISO MPEG model [1]. Fnally, n Secton 8, wewllpresent some conclusons. 2. PSYCHOACOUSTICAL BACKGROUND Audtory maskng models that are used n audo codng are predomnantly based on a phenomenon known as smultaneous maskng (see, e.g., [14]). One of the earler relevant studes goes back to Fletcher [15] who performed lstenng experments wth tones that were masked by nose. In hs experments the lsteners had to detect a tone that was presented smultaneously wth a bandpass nose masker that was spectrally centred around the tone. The threshold level for detectng the tones was measured as a functon of the masker bandwdth whle the power spectral densty (spectrum level) was kept constant. Results showed that an ncrease of bandwdth, thus ncreasng the total masker power, led to an ncrease of the detecton thresholds. However, ths ncrease was only observed when the bandwdth was below a certan crtcal bandwdth; beyond ths crtcal bandwdth, thresholds were ndependent of bandwdth. These observatons led to the crtcal band concept whch s the spectral nterval across whch masker power s ntegrated to contrbute to the maskng of a tone centred wthn the nterval. An explanaton for these observatons s that the sgnal processng n the perpheral audtory system, specfcally by the baslar membrane n the cochlea, can be represented as a seres of bandpass flters whch are excted by the nput sgnal, and whch produce parallel bandpass-fltered outputs (see, e.g., [16]). The detecton of the tone s thought to be governed by the bandpass flter (or audtory flter) that s centred around the tone. When the power rato between the tone and the masker at the output of ths flter exceeds a certan crteron value, the tone s assumed to be detectable. Wth these assumptons the observatons of Fletcher can be explaned; as long as the masker has a bandwdth smaller than that of the audtory flter, an ncrease n bandwdth wll also lead to an ncrease n the masker power seen at the output of the audtory flter, whch, n turn, leads to an ncrease n detecton threshold. Beyond the audtory flter bandwdth the added masker components wll not contrbute to the masker power at the output of the audtory flter because they are rejected by the bandpass characterstc of the audtory flter. Whereas n Fletchers experments the tone was centred wthn the nose masker, later on experments were conducted where the masker dd not spectrally overlap wth the tone to be detected (see, e.g., [17]). Such experments reveal more nformaton on the audtory flter characterstc, specfcally about the tals of the flters.

3 1294 EURASIP Journal on Appled Sgnal Processng The mplcaton of such experments should be treated wth care. When dfferent maskers and sgnals are chosen, the resultng conclusons about the audtory flter shape are qute dfferent.forexample,atonalmaskerprovestobeamuch poorer masker than a nose sgnal [17]. In addton, the flter shapes seem to depend on the masker type as well as on the masker level. These observatons suggest that the basc assumptons of lnear, that s, level ndependent, audtory flters and an energy crteron that defnes audblty of dstorton components, are only a frst-order approxmaton and that other factors play a role n maskng. For nstance, t s known that the baslar membrane behaves nonlnearly [18], whch may explan, for nstance, the level dependence of the audtory flter shape. For a more elaborate dscusson of audtory maskng and audtory flters, the reader s referred to [19, 2, 21]. Despte the fact that the assumpton of a lnear audtory flter and an energy detector can only be regarded as a frstorder approxmaton of the actual processng n the audtory system, we wll proceed wth ths assumpton because t proves to gve very satsfactory results n the context of audo codng wth relatvely smple means n terms of computatonal complexty. Along smlar lnes as outlned above, the ISO MPEG model [1] assumes that the dstorton or nose level that s allowed wthn a specfc crtcal band s determned by the weghted power addton of all masker components spread on and around the crtcal band contanng the dstorton. The shape of the weghtng functon that s appled s based on audtory maskng data and essentally reflects the underlyng audtory flter propertes. These spectral-spreadng - based perceptual models have been used n varous parametrc codng schemes for snusodal component selecton [5, 6, 13]. It should be noted that n these models, t s assumed that only the audtory flter centred around the dstorton determnes the detectablty of the dstorton. When the dstorton-to-masker rato s below a predefned threshold value n each audtory flter, the dstorton s assumed to be naudble. On the other hand, when one sngle flter exceeds ths threshold value, the dstorton s assumed to be audble. Ths assumpton s not n lne wth more recent nsghts n the psychoacoustcal lterature on maskng and wll later n the paper be shown to have a consderable mpact on the predcted maskng curves. Moreover, n the ISO MPEG model [1], a dstncton s made between maskng by nosy and tonal spectral components to be able to account for the dfference n maskng power of these sgnal types. For ths purpose a tonalty detector s requred whch, n the Layer I model, s based on a spectral peak detector. Threshold measurements n psychoacoustcal lterature consstently show that a detecton threshold s not a rgd threshold. A rgd threshold would mply that f the sgnal to be detected would be just above the detecton threshold, the sgnal would always be detected whle t would never be detected when t would be just below the threshold. Contrary to ths pattern, t s observed n detecton threshold measurements that the percentages of correct detecton as a functon of sgnal level follow a sgmod psychometrc functon [22]. The detecton threshold s defned as the level for whch the sgnal s detected correctly wth a certan probablty of, typcally, 7% 75%. In varous theoretcal consderatons, the shape of the psychometrc functon s explaned by assumng that wthn the audtory system some varable, for example, the stmulus power at the output of an audtory flter, s observed. In addton, t s assumed that nose s present n ths observaton due to, for example, nternal nose n the audtory system. When the nternal nose s assumed to be Gaussan and addtve, the shape of the sgmod functon can be predcted. For the case that a tone has to be detected wthn broadband nose, the assumpton of a stmulus power measurement wth addtve Gaussan nose leads to good predctons of the psychometrc functon. When the ncrease n the stmulus power caused by the presence of the tonal sgnal s large compared to the standard devaton of the nternal nose, hgh percentages of correct detecton are expected whle the reverse s true for small ncreases n stmulus power. The rato between the ncrease n stmulus power and the standard devaton of the nternal nose s defned as the senstvty ndex d and can be calculated from the percentage of correct responses of the subjects. Ths theoretcal framework s based on sgnal detecton theory and s descrbed more extensvely n, for example, [23]. In several more recent studes t s shown that the audblty of dstorton components s not determned solely by the crtcal band wth the largest audble dstorton [24, 25]. Buus et al. [24] performed lstenng tests where tone complexes had to be detected when presented n a nose masker. They frst measured the threshold levels of several tones separately each of whch were presented smultaneously wth wdeband nose. Due to the specfc spectral shape of the maskng nose, thresholds for ndvdual tones were found to be constant across frequency. In addton to the threshold measurements for a sngle tone, thresholds were also measured for a complex of 18 equal-level tones. The frequency spacng of the tones was such that each audtory crtcal band contaned only a sngle tone. If the detectablty of the tones was only determned by the flter wth the best detectable tone, the complex of tones would be just audble when one ndvdual component of the complex had the same level as the measured threshold level of the ndvdual tones. However, the experments showed that thresholds for the tone complex were consderably lower than expected based on the bestflter assumpton, ndcatng that nformaton s ntegrated across audtory flters. In the paper by Buus et al. [24],anumberoftheoretcal explanatons are presented. We wll dscuss only the multband detector model [23]. Ths model assumes that the changes n sgnal power at the output of each audtory flter are degraded by addtve nternal nose that s ndependent n each audtory flter. It s then assumed that an optmally weghted sum of the sgnal powers at the outputs of the varous audtory flters s computed whch serves as a new decson varable. Based on these assumptons, t can be shown that the senstvty ndex of a tone complex,

4 Perceptual Model for Snusodal Audo Codng 1295 d total, can be derved from the ndvdual senstvty ndces d n as follows: d total = K d n 2, (1) n=1 x ĥ om γ C a + Wthn D channel dstorton detectablty C s + x D where K denotes the number of tones and where each ndvdual senstvty ndex s proportonal to the tone-to-masker power rato [22]. Accordng to such a framework, each doublng of the number of audtory flters that can contrbute to the detecton process wll lead to a reducton of 1.5dBn threshold. The measured thresholds by Buus et al. are well n lne wth ths predcton. In ther experments, the complex of 18 tones leads to a reducton of 6 db n detecton threshold as compared to the detecton threshold of a sngle tone. Based on (1)achangeof6.3dBwasexpected.More recently, Langhans and Kohlrausch [25] performed smlar experments wth complex tones havng a constant spacng of1hzpresentednabroadbandnosemasker,confrmng that nformaton s ntegrated across audtory flters. In addton, results obtaned by van de Par et al. [26] ndcate that also for bandpass nose sgnals that had to be detected aganst the background of wdeband nose maskers, the same ntegraton across audtory flters s observed. As ndcated, ntegraton of nformaton across a wde range of frequences s found n audtory maskng. Smlarly, ntegraton across tme has been shown to occur n the audtory system. Van den Brnk [27] nvestgated the detecton of tones of varable duraton that were presented smultaneously wth a nose masker wth a fxed duraton that was always longer than that of the tone. Increasng the duraton of the tone reduced the detecton thresholds up to a duraton of about 3 mllseconds. Whle ths result s an ndcaton of ntegraton across tme, t also shows that there s a lmtaton n the nterval for whch temporal ntegraton occurs. The above fndngs wth respect to spectral and temporal ntegraton of nformaton n audtory maskng have mplcatons for audo codng whch have not been consdered n prevous studes. On the one hand t nfluences the maskng propertes of complex sgnals as wll be dscussed n Secton 5, on the other hand t has mplcatons for rate dstorton optmsaton algorthms. To understand ths, consder the case where for one partcular frequency regon a threshold level s determned for dstortons that can be ntroduced by an audo coder. For another frequency regon a threshold can be determned smlarly. When both dstortons are presented at the same tme, the total dstorton s expected to become audble due to the spectral ntegraton gven by (1). Ths s n contrast to the more conventonal models, such as the ISO MPEG model [1], whch would predct ths smultaneous dstorton to be naudble. The effect of spectral ntegraton, of course, can easly be compensated for by reducng the level of the maskng thresholds such that the total dstorton wll be naudble. But, based on (1), assumng that t holds for maskng by complex audo sgnals, there are many dfferent solutons to ths equaton whch lead to the same d total.inotherwords,many Fgure 1: Block dagram of the maskng model. dfferent dstrbutons of dstorton levels per spectral regon wll lead to the same total senstvty ndex. However, not every dstrbuton of dstorton levels wll lead to the same amount of bts spent by the audo coder. Thus, the concept of a maskng curve whch determnes the maxmum level of dstorton allowed wthn each frequency regon s too restrctve and can be expected to lead to suboptmal audo coders. In fact, spectral dstorton can be shaped such that the assocated bt rate s mnmsed. For more nformaton the reader s referred to a study where these deas were confrmed by lstenng tests [28]. 3. DESCRIPTION OF THE MODEL In lne wth varous state-of-the-art audtory models that have been presented n the psychoacoustcal lterature, for example, [29], the structure of the proposed model follows the varous stages of audtory sgnal processng. In vew of the computatonal complexty, the model s based on frequency doman processng and consequently neglects some parts of perpheral processng, such as the har cell transformaton whch performs nherent nonlnear tme-doman processng. A block dagram of the model s gven n Fgure 1. The model nput x s the frequency doman representaton of a short wndowed segment of audo. The wndow should lead to suffcent rejecton of spectral sde lobes n order to facltate adequate spectral resoluton of the audtory flters. The frst stage of the model resembles the outer- and mddleear transfer functon ĥom, whch s related to the flterng of the ear canal and the osscles n the mddle ear. The transfer functon s chosen to be the nverse of the threshold-n-quet functon ĥtq. Ths partcular shape s chosen to obtan an accurate predcton of the threshold-n-quet functon when no masker sgnal s present. The outer- and mddle-ear transfer functon s followed by a gammatone flter bank (see, e.g., [3]) whch resembles the flterng property of the baslar membrane n the nner ear. The transfer functon of an nth-order gammatone flter hasamagntudespectrumthatsapproxmatedwellby ( ( ) f 2 ) n/2 f γ( f ) = 1+ k ERB ( ), (2) f where f s the centre frequency of the flter, ERB( f ) s the equvalent rectangular bandwdth of the audtory flter centred at f as suggested by Glasberg and Moore [31], n s

5 1296 EURASIP Journal on Appled Sgnal Processng the flter order whch s commonly assumed to be 4, and k = 2 (n 1) (n 1)!/π(2n 3)!!, a factor needed to ensure that the flter ndeed has the specfed ERB. The centre frequences of the flters are unformly spaced on an ERB-rate scale and follow the bandwdths as specfed by the ERB scale [31]. The power at the output of each audtory flter s measured and a constant C a s added to ths output as a means to lmt the detectablty of very weak sgnals at or below the threshold n quet. In the next stage, wthn-channel dstorton detectabltes are computed and are defned as the ratos between the dstorton and the masker-plus-nternal nose seen at the output of each audtory flter. In fact, the wthn-channel dstorton detectablty D s proportonal to the senstvty ndex d as descrbed earler. Ths s an mportant step; the dstorton detectablty (or d )wllbeusedasameasureofperceptual dstorton. Ths perceptual dstorton measure can be nterpreted as a measure of the probablty that subjects can detect a dstorton sgnal n the presence of a maskng sgnal. The masker power wthn the th flter due to an orgnal (maskng) sgnal x s gven by M = 1 N ĥ om ( f ) 2 γ ( f ) 2 x( f ) 2, (3) f where N s the segment sze n number of samples. Equvalently, the dstorton power wthn the th flter due to a dstorton sgnal ε s gven by S = 1 N ĥ om ( f ) 2 γ ( f ) 2 ε( f ) 2. (4) f Note that (1/N) x( f ) 2 denotes the power spectral densty of the orgnal, maskng sgnal n sound pressure level (SPL) per frequency bn, and smlarly (1/N) ε( f ) 2 s the power spectral densty of the dstortng sgnal. The wthn-channel dstorton detectablty D s gven by D = S M +(1/N)C a. (5) From ths equaton two propertes of the wthn-channel dstorton detectablty D can be seen. When the dstorton-tomasker rato S /M s kept constant whle the masker power s much larger than (1/N)C a, dstorton detectablty s also constant. In other words, at medum and hgh masker levels the detectablty D s manly determned by the dstortonto-masker rato. Secondly, when the masker power s small compared to (1/N)C a, the dstorton detectablty s ndependent of the masker power, whch resembles the percepton of sgnals near the threshold n quet. In lne wth the multband energy detector model [23], we assume that wthn-channel dstorton detectabltes D are combned nto a total dstorton detectablty by an addtve operaton. However, we do not add the squared senstvty ndces as n (1), but we smply add the ndces drectly. Although ths may ntroduce naccuraces, these wll later turn out to be small. A beneft of ths choce s that the dstorton measure that wll be derved from ths assumpton wll have propertes that allow a computatonally smple formulaton of the model (see (11)). In addton, recent results [26] show that at least for the detecton of closely spaced tones (2 Hz spacng) masked by nose, the reducton n thresholds when ncreasng the sgnal bandwdth s more n lne wth a drect addton of dstorton detectabltes than wth (1). Therefore, we state that D(x, ε) = C s L eff D (6) = C s L eff f ĥ om ( f ) 2 γ ( f ) 2 ε( f ) 2, (7) NM + C a where D(x, ε) s the total dstorton detectablty as t s predcted for a human observer gven an orgnal sgnal x and a dstorton sgnal ε. The calbraton constant C s s chosen such that D = 1 at the threshold of detectablty. To account for the dependency of dstorton detectablty on the duraton of the dstorton sgnal (n lne wth [27]), a scalng factor L eff s ntroduced defned as ( ) L L eff = mn 3 ms,1, (8) where L s the segment duraton n mllseconds. Equaton (8) resembles the temporal ntegraton tme of the human audtory system whch has an upper bound of 3 mllseconds [27]. 1 Equaton (7) gves a complete descrpton of the model. However, t defnes only a perceptual dstorton measure and not a maskng curve such as s wdely used n audo codng nor a masked threshold such as s often used n psychoacoustcal experments. In order to derve a masked threshold, we assume that the dstorton sgnal ε( f ) = A ɛ. Here,A s the ampltude of the dstorton sgnal and ɛ the normalsed spectrum of the dstorton sgnal such that ɛ 2 = 1 whch s assumed to correspond to a sound pressure level of db. Wthout yet makng an assumpton about the spectral shape of ɛ, wecanderve that, assumng that D = 1 at the threshold of detectablty, the masked threshold A 2 for the dstorton sgnal ɛ s gven by 1 A 2 = C sl eff f ĥ om ( f ) 2 γ ( f ) 2 ɛ( f ) 2. (9) NM + C a When dervng a maskng curve t s mportant to consder exactly what type of sgnal s masked. When a maskng model s used n the context of a waveform coder, the 1 An alternatve defnton would be to state that L eff = N,thetotalduraton of the segment n number of samples. Accordng to ths defnton t s assumed that dstortons are ntegrated over the complete excerpt at hand, whch s not n lne wth perceptual maskng data, but whch n our experence stll leads to very satsfactory results [32].

6 Perceptual Model for Snusodal Audo Codng 1297 dstorton sgnal ntroduced by the coder s typcally assumed to consst of bands of nose. For a snusodal coder, however, the dstorton sgnal contans the snusods that are rejected by the perceptual model. Thus, the components of the dstorton sgnal are n fact more snusodal n nature. Assumng now that a dstorton component s present n only one bn of the spectrum, we can derve the masked thresholds for snusodal dstortons. We assume that ɛ( f ) = v( f m )δ( f f m ) wth v( f m ) beng the snusodal ampltude and f m the snusodal frequency. Together wth the assumpton that D = 1at the threshold of detectablty, v can be derved such that the dstorton s just not detectable. In ths way, by varyng f m over the entre frequency range, v 2 consttutes the maskng curve for snusodal dstortons n the presence of a masker x. By substtutng the above assumptons n (7) weobtan 1 v 2( ) = C s L ( ) ĥ om fm 2 ( ) γ fm 2 eff. (1) f m NM + C a Substtutng (1)n(7), we get D(x, ε) = f ε( f ) 2 v 2 ( f ). (11) Ths expresson shows that the computatonal load for calculatng the perceptual dstorton D(x, ε) can be very low once the maskng curve v 2 has been calculated. Ths smple form of the perceptual dstorton, such as gven n (11), arses due to the specfc choce of the addton as defned n (6). 4. CALIBRATION OF THE MODEL For the purpose of calbraton of the model, the constants C a for absolute thresholds and C s for the general senstvty of the model n (7) need to be determned. Ths wll be done usng two basc fndngs from the psychoacoustcal lterature, namely the threshold n quet and the just notceable dfference (JND) n level of about.5-1 db for snusodal sgnals [33]. When consderng the threshold n quet, we assume that the maskng sgnal s equal to zero, that s, x = and that the just detectable snusodal dstorton sgnal s gven by ε( f ) = ĥtq( f m )δ( f f m )forsome f m,whereĥtq s the threshold-n-quet curve. By substtutng these assumptons n (7) (assumng that D = 1 corresponds to a just detectable dstorton sgnal), we obtan ( ) C a = C s L eff γ fm 2. (12) Note that (12) only holds f γ ( f m ) 2 s constant for all f m, whch s approxmately true for gammatone flters. We assume a 1 db JND whch corresponds to a maskng condton where a snusodal dstorton s just detectable n the presence of a snusodal masker at the same frequency, say f m. For ths to be the case, the dstorton level has to be 18 db lower than the masker level, assumng that the masker and dstorton are added nphase. Ths specfc phase assumpton s made because t leads to smlar thresholds as when the masker and sgnal are slghtly off-frequency wth respect to one another, the case whch s most lkely to occur n audo codng contexts. We therefore assume that the masker sgnal s x( f ) = A 7 δ( f f m ) and the dstorton sgnal ε( f ) = A 52 δ( f f m ), wth A 7 and A 52 beng the ampltudes for a 7 and 52 db SPL snusodal sgnal, respectvely. Usng (3) and(7), ths leads to the expresson 1 = L eff C s ĥ om ( fm ) 2 γ ( fm ) 2 A 2 52 ( ) ĥ om fm 2 ( ) γ fm 2. (13) A C a When (12) s substtuted nto (13), an expresson s obtaned where C s s the only unknown. A numercal soluton to ths equaton can be found usng, for example, the bsecton method (cf. [34]). A sutable choce for f m would be f m = 1 khz, snce t s n the mddle of the audtory range. Ths calbraton at 1 khz does not sgnfcantly reduce the accuracy of the model at other frequences. On the one hand the ncorporaton of a threshold-n-quet curve preflter provdes the proper frequency dependence of thresholds n quet. On the other hand, JNDs do not dffer much across frequency both n the model predctons and humans. 5. MODEL EVALUATION AND COMPARISON WITH PSYCHOACOUSTICAL DATA To show the valdty of the model, some basc psychoacoustcal data from lstenng experments wll be compared to model predctons. We wll consder two cases, namely snusods masked by nose and snusods masked by snusods. Maskng of snusods has been measured n several experments for both (whte) nose maskers [12, 35] and for snusodal maskers [36]. Fgure 2a shows maskng curves predcted by the model for a whte nose masker wth a spectrum level of 3 db/hz for a long duraton sgnal (sold lne) and a 2 mllsecond sgnal (dashed lne) wth correspondng lstenng test data represented by crcles [12]andastersks[35], respectvely. Fgure 2b shows the predcted maskng curve (sold lne) for a 1 khz 5 db SPL snusodal masker along wth correspondng measured maskng data [36]. The model predctons are well n lne wth data for both snusodal and nose maskers, despte the fact that no tonalty detector was ncluded n the model such as s conventonally needed n maskng models for audo codng (e.g., [1]). Only at lower frequences, there s a dscrepancy between the data for the nose masker and the predctons by the model. The reason for ths dscrepancy may be that n psychoacoustcal studes, runnng nose generators are used to generate the masker sgnal rather than a sngle nose realsaton, as t s done n audo codng applcatons. The latter case has, accordng to several studes, a lower maskng strength [37]. Ths dfference n maskng strength s due to the nherent masker power fluctuatons when a runnng nose s presented, whch depends nversely on the product of tme and bandwdth seen at the output of an audtory flter. The narrower the audtory flter (.e., the lower ts centre frequency), the larger these fluctuatons wll be and the larger the dfference s expected to be.

7 1298 EURASIP Journal on Appled Sgnal Processng Masked threshold (db SPL) Frequency (Hz) 1 4 Masked threshold (db SPL) Number of components Masked threshold (db SPL) (a) Frequency (Hz) (b) Fgure 2: (a) Maskng curves predcted by the model for a whte nose masker wth a spectrum level of 3 db/hz for a long duraton sgnal (sold lne) and a 2- mllsecond sgnal (dashed lne) wth correspondng lstenng test data represented by the crcles [12]andastersks[35], respectvely. (b) Maskng curves for a 1 khz 5 db SPL snusodal masker. The dashed lne s the threshold n quet. Crcles show data from [36]. As can be seen n Fgure 2, the relatvely weaker maskng power of a snusodal sgnal s predcted well by the model wthout the need for explct assumptons about the tonalty of the masker such as those ncluded n, for example, the ISO MPEG model [1]. Indeed, n the case of a nose masker (Fgure 2a), the masker power wthn the crtcal band centred around 1 khz (bandwdth 132 Hz) s approxmately 51.2 db SPL, whereas the snusodal masker (Fgure 2b) has a power of 5 db SPL. Nevertheless, predcted detecton thresholds are consderably lower for the snusodal masker (35 db SPL) than for the nose masker (45 db SPL). The reason why the model s able to predct these data well s that for the tonal masker, the dstortonto-masker rato s constant over a wde range of audtory flters. Due to the addton of wthn-channel dstorton detectabltes, the total dstorton detectablty wll be relatvely large. In contrast, for a nose masker, only the flter centred on the dstorton component wll contrbute to the total dstorton detectablty because the off-frequency flters have a very low dstorton-to-masker rato. Therefore, the wdeband nose masker wll have stronger maskng effect. Note that for narrowband nose sgnals, the predcted maskng power, n lne wth the argumentaton for a snusodal masker, wll also be weak. Ths, however, seems to be too conservatve [38]. 1 4 Fgure 3: Masked thresholds predcted by the model (sold lne) and psychoacoustcal data (crcles) [25]. Masked thresholds are expressed n db SPL per component. A specfc assumpton n ths model s the ntegraton of dstorton detectabltes over a wde range of audtory flters. Ths should allow the model to predct correctly the threshold dfference between narrowband dstorton sgnals and more wdeband dstorton sgnals. For ths purpose an experment s consdered where a complex of tones had to be detected n the presence of maskng nose [25]. The tone complex conssted of equal-level snusodal components wth a frequency spacng of 1 Hz centred around 4 Hz. The masker was a 2 khz nose sgnal wth an overall level of 8 db SPL. The number of components n the complex was vared from one up to 41. The latter case corresponds to a bandwdth of 4 Hz, whch mples that the tone complex covers more than one crtcal band. Equaton (9)was used to derve masked thresholds. As can be seen n Fgure 3, there s a good correspondence between the model predctons and the data from [25]. Therefore, t seems that the choce of the lnear addton that was made n (6) dd not lead to large dscrepances between psychoacoustcal data and model predctons. To conclude ths secton, a comparson s made between predctons of the MPEG-1 Layer I [1] and the model presented n ths study whch ncorporates spectral ntegraton n maskng. The MPEG model s one of a famly of models used n audo codng that are based on spectral-spreadng functons to model spectral maskng. When the maskng of a narrowband dstorton sgnal s consdered, t s assumed that the audtory flter that s spectrally centred on ths dstorton sgnal determnes whether the dstorton s audble or not. When the energy rato between dstorton sgnal and maskng sgnal as seen at the output of ths audtory flter s smaller than a certan crteron value, the dstorton s naudble. In ths manner the maxmum allowable dstorton sgnal level at each frequency can be determned whch consttutes the maskng curve. An effcent mplementaton for calculatng ths maskng curve s a convoluton between the masker spectrum and a spreadng functon both represented on a Bark scale. The Bark scale s a perceptually motvated frequency scale smlar to the ERB-rate scale [39]. The spectral ntegraton model presented here does not consder only a sngle audtory flter to contrbute to the detecton of dstortons, but potentally a whole range of

8 Perceptual Model for Snusodal Audo Codng 1299 Level (db) Level (db) Frequency (Hz) (a) Frequency (Hz) (b) Fgure 4: Masked thresholds predcted by the spectral ntegraton model (dashed lne) and the ISO MPEG model (sold lne). The maskng spectrum (dotted lne) s for (a) a 1 khz snusodal sgnal and (b) a short segment of a harpschord sgnal. flters. Ths can have a strong mpact on the predcted maskng curves. Fgure 4a shows the maskng curves for a snusodal masker at 1 khz for the MPEG model (sold lne) and the spectral ntegraton model (dashed lne). The spectrum of the snusodal sgnal s also plotted (dotted lne), but scaled down for vsual clarty. As can be seen, there s a reasonable match between both models, showng some dfferences at the tals. In Fgure 4b, n a smlar way the maskng curves are shown but now resultng from a complex spectrum (part of a harpschord sgnal). It can be seen that the maskng curves dffer systematcally showng much smoother maskng curves for the spectral ntegraton model as compared to the MPEG model. For the spectral ntegraton model maskng curves are consderably hgher n spectral valleys. Ths effect s a drect consequence of the spectral ntegraton assumpton that was adopted n our model (cf. (6)). In the spectral valleys of the masker, dstorton sgnals can only be detected usng the audtory flter centred on the dstorton whch wll lead to relatvely hgh masked thresholds. Ths s so because off-frequency flters wll be domnated by the masker spectrum. However, detecton of dstorton sgnals at the spectral peaks of the masker s medated by a range of audtory flters centred around the peak, resultng n relatvely low masked thresholds. In ths case the off-frequency flters wll reveal smlar dstorton-to-masker ratos as the on-frequency flter. Thus, n the model proposed here, detecton dfferences between peaks and troughs are smaller, resultng n smoother maskng curves as compared to those observed n a spreadng-based model such as the ISO MPEG model The smoothenng effect s observed systematcally n complex sgnal spectra typcally encountered n practcal stuatons and represents the man dfference between the spectral ntegraton model presented here and exstng spreadngbased models. 6. APPLICATION TO SINUSOIDAL MODELLING Snusodal modellng has proven to be an effcent technque for the purpose of codng speech sgnals [4]. More recently, t has been shown that ths method can also be exploted for low-rate audo codng, for example, [41, 42, 43]. To account for the tme-varyng nature of the sgnal, the snusodal analyss/synthess s done on a segment-by-segment bass, wth each segment beng modelled as a sum of snusods. The snusodal parameters have been selected wth a number of methods, ncludng spectral peak-pckng [44], analyss-bysynthess [41, 43], andsubspace-basedmethods [42]. In ths secton we descrbe an algorthm for selectng snusodal components usng the psychoacoustcal model descrbed n the prevous secton. The algorthm s based on the matchng pursut algorthm [45], a partcular analyssby-synthess method. Matchng pursut approxmates a sgnal by a fnte expanson nto elements (functons) chosen from a redundant dctonary. In the example of snusodal modellng, one can thnk of such functons as (complex) exponentals or as real snusodal functons. Matchng pursut s a greedy, teratve algorthm whch searches the dctonary for the functon that best matches the sgnal and subtracts ths functon (properly scaled) to form a resdual sgnal to be approxmated n the next teraton. In order to determne whch s the best matchng functon or dctonary element at each teraton, we need to formalse the problem. To do so, let D = (g ξ ) ξ Γ be a complete dctonary, that s, a set of elements ndexed by ξ Γ,where Γ s an arbtrary ndex set. As an example, consder a dctonary consstng of complex exponentals g ξ = e 2πξ( ). In ths case, the ndex set Γ s gven by Γ = [, 1). Obvously, the ndexng parameter ξ s nothng more than the frequency of the complex exponental. Gven a dctonary D, the best matchng functon can be found by, for each and every functon, computng the best approxmaton and selectng that functon whose correspondng approxmaton s closest to the orgnal sgnal. In order to facltate the followng dscusson, we assume wthout loss of generalty that g ξ =1forallξ. Gvena partcular functon g ξ, the best possble approxmaton of the sgnal x s obtaned by the orthogonal projecton of x onto the subspace spanned by g ξ (see Fgure 5). Ths projecton s gven by x, g ξ g ξ.hence,wecandecomposex as x = x, g ξ g ξ + Rx, (14) where Rx s the resdual sgnal after subtractng the projecton x, g ξ g ξ. The orthogonalty of Rx and g ξ mples that x 2 = x, gξ 2 + Rx 2. (15)

9 13 EURASIP Journal on Appled Sgnal Processng g ξ x Rx x, g ξ g ξ span(g ξ ) Fgure 5: Orthogonal projecton of x onto span(g ξ ). We can do ths decomposton for each and every dctonary element and the best matchng one s then found by selectng the element g ξ for whch Rx s mnmal, or, equvalently, for whch x, g ξ s maxmal. A precse mathematcal formulaton of ths phrase s ξ = arg sup x, gξ. (16) ξ Γ It must be noted that the matchng pursut algorthm s only optmal for a partcular teraton. If we subtract the approxmaton to form a resdual sgnal and approxmate ths resdual n a smlar way as we approxmated the orgnal sgnal, then the two dctonary elements thus obtaned are not jontly optmal; t s n general possble to fnd two dfferent elements whch together form a better approxmaton. Ths s a drect consequence of the greedy nature of the algorthm. The two dctonary elements whch together are optmal could be obtaned by projectng the sgnal x onto all possble two-dmensonal subspaces. Ths, however, s n general very computatonally complex. An alternatve soluton to ths problem s to apply, after each teraton, a Newton optmsaton step [46]. To account for human audtory percepton, the untnorm dctonary elements can be scaled [43],whch s equvalent to scalng the nner products n (16). We wll refer to ths method as the weghted matchng pursut (WMP) algorthm. Whle ths method performs well, t can be shown that t does not provde a consstent selecton measure for elements of fnte tme support [47]. Rather than scalng the dctonary elements, we ntroduce a matchng pursut algorthm where psychoacoustcal propertes are accounted for by a norm on the sgnal space. We wll refer to ths method as psychoacoustcal matchng pursut (PAMP). As mentoned n Secton 3 (see (11)), the perceptual dstorton can be expressed as D = ε( f ) 2 v f 2 = â( f ) ε( f ) 2, (17) ( f ) f where â = v 2. It follows from (1) that â( f ) = C s L eff ĥ om ( f ) 2 γ ( f ) 2 NM + C a. (18) Perceptual dstorton Number of snusods Fgure 6: Perceptual dstorton assocated wth the resdual sgnal after snusodal modellng as a functon of the number of snusodal components that were extracted. By nspecton of (18), we conclude that â s real and postve so that, n fact, the perceptual dstorton measure (17) defnes anorm x 2 = f Ths norm s nduced by the nner product x, y = f â( f ) x( f ) 2. (19) â( f ) x( f )ŷ ( f ), (2) facltatng the use of the dstorton measure n selectng the perceptually best matchng dctonary element n a matchng pursut algorthm. In Fgure 6, the perceptual dstorton assocated wth the resdual sgnal s shown as a functon of the number of real-valued snusods that have been extracted for a short segment of a harpschord excerpt (cf. (11)). As canbe seen the perceptually most relevant components are selected frst, resultng n a fast reducton of the perceptual dstorton for the frst components. For a detaled descrpton the reader s referred to [47, 48]. The fact that the dstorton detectablty defnes a norm on the underlyng sgnal space s mportant, snce t allows for ncorporatng psychoacoustcs n optmsaton algorthms. Indeed, rather than mnmsng the commonly used l 2 -norm, we can mnmse the perceptually relevant norm gven by (19). Examples nclude ratedstorton optmsaton [32], lnear predctve codng [49], and subspace-based modellng technques [5]. 7. COMPARISON WITH THE ISO MPEG MODEL IN A LISTENING TEST In ths secton we assess the performance of the proposed perceptual model n the context of snusodal parameter estmaton. The PAMP method for estmatng perceptually relevant snusods reles on the weghtng functon â whch, by defnton, s the nverse of the maskng curve. Equaton (18) descrbes how to compute the maskng curve for the proposed perceptual model. We compare the use of the proposed perceptual model n PAMP to the stuaton where the maskng curve s computed usng the MPEG-1 Layer I-II (ISO/IEC ) psychoacoustcal model [1]. There are several reasons for comparson wth the MPEG psychoacoustc model; the model provdes a well-known

10 Perceptual Model for Snusodal Audo Codng 131 reference and because of ts frequent applcaton, t s stll a de facto state-of-the-art model. Usng the MPEG-1 psychoacoustc model maskng curve drectly n the PAMP algorthm for snusodal extracton s not reasonable because the MPEG-1 psychoacoustc model was developed to predct the maskng curve n the case of nose maskees (dstorton sgnals). It predcts for every frequency bn how much dstorton can be added wthn the crtcal band centred around the frequency bn. Ths predcton s, however, too conservatve n the case that dstortons are snusodal n nature snce n ths case the dstorton energy s not spread over a complete crtcal band but s concentrated n one frequency bn only. Hence, we can adapt the MPEG-1 model by scalng the maskng functon wth the crtcal bandwdth such that the model now predcts the detecton thresholds n the case of snusodal dstorton. The net effect of ths compensaton procedure s an ncrease of the maskng curve at hgh frequences by about 1 db, thereby de-emphaszng hgh-frequency regons durng snusodal estmaton. In fact, ths maskng power ncrease at hgher frequences reduces the gap between the maskng curves between the ISO MPEG model and the proposed model (cf. Fgure 4) By applyng ths modfcaton to the ISO MPEG model, and by extendng the FFT order to the sze of the PAMP dctonary, t s suted to be used n the PAMP method. The dctonary elements n our mplementaton of the PAMP method were real-valued snusodal functons wndowed wth a Hannng wndow, dentcal to the wndow used n the analyss-synthess procedure descrbed below. In the followng, we present results obtaned by lstenng tests wth audo sgnals. The sgnals are mono, sampled at 44.1 khz, where each sample s represented by 16 bts. The test excerpts are Carl Orff, Castanet, Célne Don, Harpschord Solo, contemporary pop musc, and Suzanne Vega. The excerpts were segmented nto fxed-length frames of 124 samples (correspondng to 23.2 mllseconds) wth an overlap of 5% between consecutve frames usng a Hannng wndow. For each sgnal frame, a fxed number of perceptually relevant snusods per frame were extracted usng the PAMP method descrbed above, where the perceptual weghtng functons â were generated from maskng curve derved from the proposed perceptual model (see (18)) and the modfed MPEG model descrbed above, respectvely. For the MPEG model we made use of the recommendatons of MPEG Layer II, snce these support nput frame lengths of 124 samples. The maskng curves were calculated from the Hannng-wndowed orgnal sgnal contaned wthn the same frame that s beng modelled usng the PAMP method. Fnally, modelled frames were syntheszed from the estmated snusodal parameters and concatenated to form modelled test excerpts, usng a Hannng wndow-based overlapadd procedure. To evaluate the performance of the proposed method, we used a subjectve lstenng test procedure whch s somewhat comparable to the MUSHRA test (multstmulus test wth hdden reference and anchors) [51]. For each test excerpt, lsteners were asked to rank 6 dfferent versons: 4 excerpts modelled usng the modfed MPEG maskng curve and fxed Table 1: Scores used n subjectve test. Score Equvalent 5 Best 4 Good 3 Medum 2 Poor 1 Poorest model orders (.e., the number of snusodal components per segment) of K = 2, 25, 3, and K = 35, and one excerpt modelled usng the proposed perceptual model wth K = 25. In addton, to have a low-qualty reference sgnal, an excerpt modelled wth K = 3, but usng the unmodfed MPEG maskng curve was ncluded. As a reference, the lsteners had the orgnal excerpt avalable as well, whch was dentfed to the subjects. Unlke the MUSHRA test, no hdden reference and no anchors were presented to the lsteners. The test excerpts were presented n a parallel way, usng the nteractve benchmarkng tool descrbed n [52] as an nterface to the lsteners. For each excerpt, lsteners were requested to rank the dfferent modelled sgnals on a scale from 1 5 (n steps of.1) as outlned n Table 1. The lsteners were nstructed to use the complete scale such that the poorest-qualty excerpt was rated wth 1 and the hghestqualty excerpt wth 5. The excerpts were presented through hgh-qualty headphones (Beyer-Dynamc DT99 PRO) n a quet room, and the lsteners could lsten to each sgnal verson as often as needed to determne the rankng. A total of 12 lsteners partcpated n the lstenng test, of whch 6 lsteners worked n the area of acoustc sgnal processng and had prevously partcpated n such tests. The authors dd not partcpate n the test. Fgure 7 shows the overall scores of the lstenng test, averaged across all lsteners and excerpts. The crcles represent the medan score, and the error bars depct 25 and 75 percent ranges of the total response dstrbutons. As can be seen, the excerpts generated wth the proposed perceptual model (SCAS@25) show better average subjectve performance than any of the excerpts based on the MPEG psychoacoustc model, except for the MPEG case usng a fxed model order of 35 (MPEG@35). As expected, the MPEG-based excerpts have decreasng qualty scores for decreasng model order. Furthermore, the low-qualty anchor (MPEG@3nt,.e., the MPEG model wthout spectral tlt modfcaton) receved the lowest-qualty score on average. The statstcal dfference between the qualty scores was analysed usng a pared t-test usng a sgnfcance level of p<.1, and by workng on the score dfferences between the proposed perceptual model and each of the MPEG-based methods. The H hypothess was that the mean of such dfference dstrbuton was zero (µ = ), whle the alternatve hypothess H 1 was that µ >. The statstcal analyss supports the qualty orderng suggested by Fgure 7. In partcular, there s a statstcally sgnfcant mprovement n usng the proposed perceptual model (SCAS@25) over any of the MPEG-based methods except for MPEG@35 whch performs better than SCAS@25 (p < ). In fact, the model presented here

11 132 EURASIP Journal on Appled Sgnal Processng Best 6 5 More specfcally, the model presented here leads to a reducton of more than 2% n terms of number of snusods needed to represent sgnals at a gven qualty level. Good 4 ACKNOWLEDGMENTS The authors would lke to thank Ncolle H. van Schjndel, Gerard Hotho, and Jeroen Breebaart and the revewers for ther helpful comments on ths manuscrpt. Furthermore, the authors thank the partcpants n the lstenng test. The research was supported by Phlps Research, the Technology Foundaton STW, Appled Scence Dvson of NWO, the Technology Programme of the Dutch Mnstry of Economc Affars, and the EU project ARDOR, IST Medum Poor Poorest SCAS@25 MPEG@35 MPEG@3 Fgure 7: Subjectve test results averaged across all lsteners and excerpts. leads to a reducton of more than 2% n terms of number of snusods needed to represent sgnals at a gven qualty level. As mentoned already n Secton 5 the most relevant dfference between the proposed model and the ISO MPEG model s the ncorporaton of spectral ntegraton propertes n the proposed model. Ths leads to systematcally smoother maskng curves such as predcted by our model for complex masker spectra (cf. Fgure 4). The effect of ths s that fewer snusodal components are used for modellng spectral valleys of a sgnal wth the proposed perceptual model as compared to the ISO MPEG model. We thnk that ths dfference accounts for the mprovement n modellng effcency that we observed n the lstenng tests and we expect that smlar mprovements would have been observed when our approach was compared to other perceptual models that are based on the spectral-spreadng approach such as those used n the ISO MPEG model. 8. CONCLUSIONS In ths paper we presented a psychoacoustcal model that s suted for predctng masked thresholds for snusodal dstortons. The model reles on sgnal detecton theory and ncorporates more recent nsghts about spectral and temporal ntegraton n audtory maskng. We showed that, as a consequence, the model s able to predct dstorton detectabltes. In fact, the dstorton detectablty defnes a (perceptually relevant) norm on the underlyng sgnal space whch s benefcal for optmsaton algorthms such as rate-dstorton optmsaton or lnear predctve codng. The model proves to be very sutable for applcaton n the context of snusodal modellng, although t s also applcable n other audo codng contexts such as transform codng. A comparatve lstenng test usng a snusodal analyss method called psychoacoustcal matchng pursut showed a clear preference for the model presented here over the ISO MPEG model [1]. MPEG@25 MPEG@2 MPEG@3nt REFERENCES [1] IISO/MPEG Commttee, Codng of movng pctures and assocated audo for dgtal storage meda at up to about 1.5 Mbt/s - part 3: Audo, 1993, ISO/IEC [2] T. Yoshda, The rewrtable mndsc system, Proc. IEEE, vol. 82, no. 1, pp , [3] A. Hoogendoorn, Dgtal compact cassette, Proc. IEEE, vol. 82, no. 1, pp , [4] T. Panter and A. Spanas, Perceptual codng of dgtal audo, Proc. IEEE, vol. 88, no. 4, pp , 2. [5] K. N. Hamdy, M. Al, and A. H. Tewfk, Low bt rate hgh qualty audo codng wth combned harmonc and wavelet representaton, n Proc. IEEE Int. Conf. Acoustcs, Speech, Sgnal Processng (ICASSP 96), vol. 2, pp , Atlanta, Ga, USA, May [6]S.N.Levne,Audo representatons for data compresson and compressed doman processng, Ph.D. thess, Stanford Unversty, Stanford, Calf, USA, [7] H. Purnhagen and N. Mene, HILN the MPEG-4 parametrc audo codng tools, n Proc. IEEE Int. Symp. Crcuts and Systems (ISCAS ), vol. 2, pp , Geneva, Swtzerland, May 2. [8] W. Oomen, E. Schujers, B. den Brnker, and J. Breebaart, Advances n parametrc codng for hgh-qualty audo, n Proc. 114th AES Conventon, Amsterdam, The Netherlands, March 23, preprnt [9] F.P.Myburg,Desgn of a scalable parametrc audo coder,ph.d. thess, Technsche Unverstet Endhoven, Endhoven, The Netherlands, 24. [1] H. S. Malvar, Sgnal Processng wth Lapped Transforms, Artech House, Boston, Mass, USA, [11] P. P. Vadyanathan, Multrate Systems and Flter Banks, Prentce Hall Sgnal Processng Seres, Prentce Hall, Englewood Clffs, NJ, USA, [12] J. E. Hawkns and S. S. Stevens, The maskng of pure tones andofspeechbywhtenose, Journal of the Acoustcal Socety of Amerca, vol. 22, pp. 6 13, 195. [13] T. S. Verma, A perceptually based audo sgnal model wth applcaton to scalable audo codng, Ph.D. thess, Stanford Unversty, Stanford, Claf, USA, [14] R. L. Wegel and C. E. Lane, The audtory maskng of one pure tone by another and ts probable relaton to the dynamcs of the nner ear, Phys. Rev., vol. 23, pp , [15] H. Fletcher, Audtory patterns, Revews of Modern Physcs, vol. 12, no. 1, pp , 194. [16] P. M. Sellck, R. Patuzz, and B. M. Johnstone, Measurements of BM moton n the gunea pg usng Mössbauer technque, Journal of the Acoustcal Socety of Amerca, vol. 72, pp , 1982.

12 Perceptual Model for Snusodal Audo Codng 133 [17] J. P. Egan and H. W. Hake, On the maskng pattern of a smple audtory stmulus, Journal of the Acoustcal Socety of Amerca, vol. 22, pp , 195. [18] K. G. Yates, I. M. Wnter, and D. Robertson, Baslar membrane nonlnearty determnes audtory nerve rate-ntensty functons and cochlear dynamc range, Hearng Research, vol. 45, no. 3, pp , 199. [19] R. D. Patterson, Audtory fltershapes derved wth nose stmul, Journal of the Acoustcal Socety of Amerca, vol. 59, pp , [2] M. van der Hejden and A. Kohlrausch, The role of envelope fluctuatons n spectral maskng, Journal of the Acoustcal Socety of Amerca, vol. 97, no. 3, pp , [21] M. van der Hejden and A. Kohlrausch, The role of dstorton products n maskng by sngle bands of nose, Journal of the Acoustcal Socety of Amerca, vol. 98, no. 6, pp , [22] J. P. Egan, W. A. Lndner, and D. McFadden, Maskng-level dfferences and the form of the psychometrc functon, Percepton and Psychophyscs, vol. 6, pp , [23] D. M. Green and J. A. Swets, Sgnal Detecton Theory and Psychophyscs, Kreger, New York, NY, USA, [24] S. Buus, E. Schorer, M. Florentne, and E. Zwcker, Decson rules n detecton of smple and complex tones, Journal of the Acoustcal Socety of Amerca, vol. 8, no. 6, pp , [25] A. Langhans and A. Kohlrausch, Spectral ntegraton of broadband sgnals n dotc and dchotc maskng experments, Journal of the Acoustcal Socety of Amerca, vol. 91, no. 1, pp , [26] S. van de Par, A. Kohlrausch, J. Breebaart, and M. McKnney, Dscrmnaton of dfferent temporal envelope structures of dotoc and dchotc targets sgnals wthn dotc wde-band nose, n Proc. 13th Internatonal Symposum on Hearng,pp , Dourdan, France, August 23. [27] G. van den Brnk, Detecton of tone pulse of varous duratons n nose of varous bandwdths, Journal of the Acoustcal Socety of Amerca, vol. 36, pp , [28] S. van de Par and A. Kohlrausch, Applcaton of a spectrally ntegratng audtory flterbank model to audo codng, n Fortschrtte der Akustk, Plenarvorträge der 28. Deutschen Jahrestagung für Akustk, DAGA-2, pp , Bochum, Germany, 22. [29] T. Dau, D. Püschel, and A. Kohlrausch, A quanttatve model of the effectve sgnal processng n the audtory system. I. Model structure, Journal of the Acoustcal Socety of Amerca, vol. 99, no. 6, pp , [3] R. D. Patterson, The sound of a snusod; spectral models, Journal of the Acoustcal Socety of Amerca, vol. 96, no. 3, pp , [31] B. R. Glasberg and B. C. J. Moore, Dervaton of audtory flter shapes from notched-nose data, Hearng Research, vol. 47, no. 1-2, pp , 199. [32] R. Heusdens, J. Jensen, W. B. Klejn, V. Kot, O. Namut, S. van de Par, N. H. van Schjndel, and R. Vafn, Snusodal codng of audo and speech, n preparaton for Journal of the Audo Engneerng Socety, 25. [33] B. C. J. Moore, An Introducton to the Psychology of Hearng, Academc Press, London, UK, 3rd edton, [34] G. Charestan, R. Heusdens, and S. van de Par, A gammatone based psychoacoustcal modelng approach for speech and audo codng, n Proc. ProRISC/IEEE: Workshop on Crcuts, Systems and Sgnal Processng, pp , Veldhoven, The Netherlands, November 21. [35] A. J. M. Houtsma, Hawkns and Stevens revsted at low frequences, Journal of the Acoustcal Socety of Amerca, vol. 13, no. 5, pp , [36] E. Zwcker and A. Jaroszewsk, Inverse frequency dependence of smultaneous tone-on-tone maskng patterns at low levels, Journal of the Acoustcal Socety of Amerca, vol. 71, pp , [37] A. Langhans and A. Kohlrausch, Dfferences n audtory performance between monaural and dotc condtons. I. Masked thresholdsnfrozennose, Journal of the Acoustcal Socety of Amerca, vol. 91, pp , [38] S. van de Par and A. Kohlrausch, Dependence of bnaural maskng level dfferences on center frequency, masker bandwdth and nteraural parameters, Journal of the Acoustcal Socety of Amerca, vol. 16, pp , [39] E. Zwcker and H. Fastl, Psychoacoustcs Facts and Models, Sprnger, Berln, Germany, 2nd edton, [4] R. J. McAulay and T. F. Quater, Snusodal codng, n Speech Codng and Syntess, W.B.KlejnandK.K.Palwal, Eds., chapter 4, pp , Elsever Scence B. V., Amsterdam, The Netherlands, [41] M. Goodwn, Matchng pursut wth damped snusods, n Proc. IEEE Int. Conf. Acoustcs, Speech, Sgnal Processng (ICASSP 97), vol. 3, pp , Munch, Germany, Aprl [42] J. Neuwenhujse, R. Heusdens, and E. F. Deprettere, Robust exponental modelng of audo sgnals, n Proc. IEEE Int. Conf. Acoustcs, Speech, Sgnal Processng (ICASSP 98), vol. 6, pp , Seattle, Wash, USA, May [43] T. S. Verma and T. H. Y. Meng, Snusodal modelng usng frame-based perceptually weghted matchng pursuts, n Proc. IEEE Int. Conf. Acoustcs, Speech, Sgnal Processng (ICASSP 99), vol. 2, pp , Phoenx, Arz, USA, May [44] R. J. McAulay and T. F. Quater, Speech analyss/synthess based on a snusodal representaton, IEEE Trans. Acoust., Speech, Sgnal Processng, vol. 34, no. 4, pp , [45] S. G. Mallat and Z. Zhang, Matchng pursuts wth tmefrequency dctonares, IEEE Trans. Sgnal Processng, vol. 41, no. 12, pp , [46] K. Vos and R. Heusdens, Rate-dstorton optmal exponental modelng of audo and speech sgnals, n Proc. 21st Symposum on Informaton Theory n the Benelux, pp , Wassenaar, The Netherlands, May 2. [47] R. Heusdens, R. Vafn, and W. B. Klejn, Snusodal modelng usng psychoacoustc-adaptve matchng pursuts, IEEE Sgnal Processng Lett., vol. 9, no. 8, pp , 2. [48] R. Heusdens and S. van de Par, Rate-dstorton optmal snusodal modelng of audo and speech usng psychoacoustcal matchng pursuts, n Proc. IEEE Int. Conf. Acoustcs, Speech, Sgnal Processng (ICASSP 2), vol. 2, pp , Orlando, Fla, USA, May 22. [49] R. C. Hendrks, R. Heusdens, and J. Jensen, Perceptual lnear predctve nose modellng for snusod-plus-nose audo codng, n Proc. IEEE Int. Conf. Acoustcs, Speech, Sgnal Processng (ICASSP 4), vol. 4, pp , Montreal, Quebec, Canada, May 24. [5] J.Jensen,R.Heusdens,andS.H.Jensen, Aperceptualsubspace approach for modelng of speech and audo, IEEE Trans. Speech Audo Processng, vol. 12, no. 2, pp , 24. [51] ITU, ITU-R BS Method for subjectve assessment of ntermedate qualty level of codng systems, 21. [52] O. A. Namut, Audo codec Benchmark manual,departmentof Medamatcs, Delft Unversty of Technology, January 23.

13 134 EURASIP Journal on Appled Sgnal Processng Steven van de Par studed physcs at the Endhoven Unversty of Technology (TU/e), and receved hs Ph.D. degree n 1998 from the Insttute for Percepton Research on a topc related to bnaural hearng. As a Postdoctoral Researcher at the same nsttute, he studed audtory-vsual nteracton and he was a Guest Researcher at the Unversty of Connectcut Health Centre. In the begnnng of 2 he joned Phlps Research, Endhoven. Man felds of expertse are audtory and multsensory percepton and low-bt-rate audo codng. He publshed varous papers on bnaural detecton, audtory-vsual synchrony percepton, and audo-codng-related topcs. He partcpated n several projects on low-bt-rate audo codng based on snusodal technques and s presently partcpatng n the EU Adaptve Rate-Dstorton Optmzed Audo coder (ARDOR) project. Armn Kohlrausch studed physcs at the Unversty of Göttngen, Germany, and specalzed n acoustcs. He receved hs M.S. degree n 198 and hs Ph.D. degree n 1984, both n perceptual aspects of sound. From 1985 untl 199 he worked at the Thrd Physcal Insttute, Unversty of Göttngen, and was responsble for research and teachng n the felds psychoacoustcs and room acoustcs. In 1991 he joned the Phlps Research Laboratores, Endhoven, and worked n the Speech and Hearng Group, Insttute for Percepton Research (IPO). Snce 1998, he has combned hs work at Phlps Research Laboratores wth a Professor poston for multsensory percepton at the TU/e. In 24 he was apponted a Research Fellow of Phlps Research. He s a member of a great number of scentfc socetes, both n Europe and the USA. Snce 1998 he has been a Fellow of the Acoustcal Socety of Amerca and serves currently as an Assocate Edtor for the Journal of the Acoustcal Socety of Amerca, coverng the areas of bnaural and spatal hearng. Hs man scentfc nterests are n the expermental study and modellng of audtory and multsensory percepton n humans and the transfer of ths knowledge to ndustral meda applcatons. Jesper Jensen receved the M.S. and Ph.D. degrees from Aalborg Unversty, Aalborg, Denmark, n 1996 and 2, respectvely, both n electrcal engneerng. From 1996 to 21, he was wth the Center for PersonKommunkaton (CPK), Aalborg Unversty, as a Researcher, Ph.D. student, and Assstant Research Professor. In 1999, he was a Vstng Researcher at the Center for Spoken Language Research, Unversty of Colorado at Boulder. Currently, he s a Postdoctoral Researcher at Delft Unversty of Technology, Delft, the Netherlands. Hs man research nterests are n dgtal speech and audo sgnal processng, ncludng codng, synthess, and enhancement. Søren Holdt Jensen receved the M.S. degree n electrcal engneerng from Aalborg Unversty, Denmark, n 1988, and the Ph.D. degree from the Techncal Unversty of Denmark, n He has been wth the Telecommuncatons Laboratory of Telecom Denmark, the Electroncs Insttute of the Techncal Unversty of Denmark, the Scentfc Computng Group of the Dansh Computng Center for Research and Educaton (UNI-C), the Electrcal Engneerng Department of Katholeke Unverstet Leuven, Belgum, the Center for PersonKommunkaton (CPK) of Aalborg Unversty, and s currently an Assocate Professor n the Department of Communcaton Technology, Aalborg Unversty. Hs research actvtes are n dgtal sgnal processng, communcaton sgnal processng, and speech and audo processng. He s a Member of the Edtoral Board of EURASIP Journal on Appled Sgnal Processng, and a former Charman of the IEEE Denmark Secton and the IEEE Denmark Secton s Sgnal Processng Chapter. Rchard Heusdens s an Assocate Professor n the Department of Medamatcs, Delft Unversty of Technology. He receved hs M.S. and Ph.D. degrees from the Delft Unversty of Technology, the Netherlands, n 1992 and 1997, respectvely. In the sprng of 1992 he joned the Dgtal Sgnal Processng Group, Phlps Research Laboratores, Endhoven, the Netherlands. He has worked on varous topcs n the feld of sgnal processng, such as mage/vdeo compresson and VLSI archtectures for mage-processng algorthms. In 1997, he joned the Crcuts and Systems Group, Delft Unversty of Technology, where he was a Postdoctoral Researcher. In 2, he moved to the Informaton and Communcaton Theory (ICT) Group where he became an Assstant Professor, responsble for the audo and speech processng actvtes wthn the ICT Group. Snce 22, he has been an Assocate Professor. Research projects he s nvolved n cover subjects such as audo and speech codng, speech enhancement, and dgtal watermarkng of audo.