ADVANCES IN PARAMETRIC CODING FOR HIGH-QUALITY AUDIO

Transcription

1 ADVANCES IN PARAMETRIC CODING FOR HIGH-QUALITY AUDIO E.G.P. Schuijes, A.W.J. Oomen Philips Digital Systems Laboatoies Glaslaan (SFJ 7) 566 LW Eindhoven, The Nethelands A.C. den Binke, A.J. Geits Philips Reseach Laboatoies Pof. Holstlaan 4 (WY8) 5656 AA Eindhoven, The Nethelands bet.den.binke@philips.com andy.geits@philips.com ABSTRACT Compaed to taditional wavefom coding standads that employ subband o tansfom coding, paametic coding is egaded as a technique that allows even futhe eduction of bit ates. In esponse to a call fo poposals by MPEG-4 in Januay, Philips submitted a paametic code tagetting at a high quality level. Cuently, this code is impoved in the couse of the MPEG-4 Extension standadisation pocess. Recently, two enhancements have been made inceasing the oveall pefomance of the code. These consist of an impoved model fo the noise object and a paametic steeo extension.. INTRODUCTION The coding pefomance of taditional wavefom coding schemes seems to be satuating. In ode to futhe impove coding efficiency, paametic audio coding is egaded as a pomising candidate. Although a numbe of poposals have been intoduced that combine wavefom coding fo the lowe fequency egion and paametic coding fo the highe fequency egion (see e.g. [, ]), we advocate high quality audio coding using a full paametic epesentation. In the context of MPEG-4 Extension, Philips has poposed a paametic coding scheme that is based on the notion that any audio signal can be decomposed into thee objects: tansients, sinusoids and noise. A complete desciption of this coding scheme has aleady been pesented in []. Theefoe, only a summay of the geneal concept of the paametic coding scheme is included in Section. In Section two quality impovement poposals that have been accepted within MPEG ae pesented. Futhemoe, the cuent status in MPEG-4 Extension is discussed in Section 4. Finally, a new poposal to impove the quality futhe, specifically fo the citical items, is pesented.. PARAMETRIC MODEL FOR HQ AUDIO Within the field of paametic audio coding the model that is applied detemines to a high degee the oveall quality that can be achieved. Theefoe, the objects that have been chosen to descibe the audio signal eflect attibutes which ae well known fom auditoy peception and the physics of natual audio signals. Accoding to typical pattens that can be obseved in spectogams of audio signals (see Figue ), the following thee objects ae defined:. Tansients; tansients epesent the non-stationay pat of the audio signal. They ae chaacteised by a fast change in signal powe o amplitude. Theefoe, modelling tansients using quasi-stationay pattens poves to be an inefficient appoach.. Sinusoids; sinusoids ae the highly pedictable components within an audio signal. They ae clealy defined in fequency and typically last fo a long time. Hence it is assumed that these spectal tajectoies can be modelled accuately using sinusoids.. Noise; noise epesents the stochastic pat of the audio signal. In natue, noise-like souces ae often encounteed, e.g. the ustle of the wind o unvoiced speech. The peception of such noise-like signals clealy diffes fom tonal signals. The above objects ae all coded and quantised using peceptual citeia []... Tansients Tansients can be oughly catagoised in two types. The fist one is chaacteised as a shot bust of enegy, the second type can be descibed as a sudden change of signal level, i.e., a tansition. Accoding to this classification two types of tansients have been defined, being a Meixne tansient [4] and a step tansient. The Meixne tansient, coesponding to the desciption of a bust of enegy, is chaacteised by the following paametes:. position, the stating position of the tansient;. envelope paametes, two paametes descibing the Meixne function;. sinusoidal paametes (fequency, amplitude and phase), descibing the wavefom undeneath the envelope. The discete-time Meixne function is defined as g(n) = ( ξ ) b/ (b)n ξ n, () n! with b >, < ξ < and n =,,,.... Futhemoe, the Pochhamme symbol (b) n denotes the n-tems poduct (b) n = b (b + )... (b + n ). This desciption contains a fast attack (associated with b) and an exponential decay (associated with ξ). This coesponds well to envelopes as found in natual audio signals. The step tansient coesponds to the desciption of a sudden change in signal powe level. It is descibed meely by its position. As MPCA-

2 x Time (samples) Figue : Envelope matching fo tansient phenomena. Top plot: example of a signal bust and the estimated Meixne envelope (dashed line). Bottom plot: example of a change in enegy level and the estimated tansition position (dashed line). Sampling fequency fo both signals is 44. khz. Figue : Spectogams fo Castanets, Hapsichod and Heavy Metal (fom top to bottom), illustating the thee objects, viz., tansients (vetical lines in Castanets), sinusoids (hoizontal lines in Hapsichod) and noise (no clea time-fequency localization in Heavy Metal). The intensity is given in a gey scale, the dake aeas indicating highe intensities. such it does not descibe a signal by itself. It only influences the way the othe objects (sinusoids and noise) ae synthesised. Figue shows examples of both tansient types... Sinusoids Fo stationay segments, we use the following signal model: with s(t) = I(t) X i= A i (t) cos(φ i (t)) + n(t), () Z t Φ i (t) = φ s,i + ω i (τ)dτ, t s,i () whee the subscipt i denotes the i th sinusoid, A i (t) epesents the (slowly vaying) amplitude, Φ i (t) epesents the phase function with stat phase φ s,i and ω i(t) epesents the slowly vaying fequency. I(t) denotes the numbe of sinusoids at time t and n(t) is a (coloued) noise signal. The equations above ae at the vey basis of ou paametic model. It is not pactical to extact paametes on a sample by sample basis. If the functions A i(t) and ω i(t) ae indeed slowly vaying functions of time it is also not necessay to do so. In a pactical appoach the sinusoidal paametes ae estimated on a fame by fame basis, which coesponds to sampling the functions A i (t), ω i(t) and Φ i(t) at a specific update ate. If the signal model of equation is valid, tacks that span multiple fames can be fomed by a linking mechanism (see Figue ). The majo pat of the bitate savings stems fom the fact that fo these tacks only small innovations need to be coded. Futhemoe, if the assumption of slowly vaying fequencies ove time is tue, the phase infomation becomes edundant. Given a cetain fequency and a phase belonging to a fame k and the fequency of fame k +, belonging to the same tack, the phase fo fame k + can be pedicted. In ode to comply with this assumption the update ate of the sinusoids has been chosen elatively high (aound 8 ms)... Noise Fo the peception of noise-like signals it is not necessay to match the oiginal wavefom. It is sufficient to match the spectal envelope as well as the tempoal one. As such it is also descibed in the paametic model. The spectal envelope is coded by means of an ARMA (Auto Regessive Moving Aveage) model (see [5]). The ARMA model is an efficient desciption and is able to captue spectal peaks as well as valleys. In the decode this model is excited by unit-vaiance white noise. The tempoal envelope is coded by means of a sequence of socalled noise gains pe fame. These paametes descibe the gain that has to be applied to the ARMA model in ode to match the powe of the output signal of this model to the actually measued powe of the encoded signal.. IMPROVEMENTS Since the intoduction of the paametic coding scheme, as descibed in the pevious section, the model has been extended to futhe impove the quality. The fist impovement consists of a tempoal envelope desciption fo the noise object eplacing the MPCA-

3 6 f (Hz) k- k fame k Time (samples) Figue : Tacking of fequencies. Fequencies ae estimated pe fame, and linked ove fames theeby foming sinusoidal tacks. Stats of tacks ae called biths (indicated by asteisks) and endings of tacks ae called deaths (indicated by cicles). Continuations ae indicated by dots. noise gains. The second impovement adds a paametic steeo desciption to the coding scheme... Noise model In the cuent model, the tempoal envelope is descibed by noise gains. These gains ae detemined using 4. ms segments with 5% ovelap. In the decode, an envelope is geneated by intepolation of the gains using 4. ms Hanning windows. This model has some limitations as can be seen in Figue 4. The tempoal envelope cannot be modelled accuately enough by the noise gains. The peaks in the envelope of the signal ae smeaed due to the segment size and the 5% ovelap. Going to smalle segment sizes will incease the bit ate and also the uncetainty of the measued gains. In ode to model the tempoal envelope moe accuately, the gain measuement using fixed shot segments should be avoided. The model should be capable of accuately descibing peaks in the tempoal envelope, since these ae impotant featues in the signal. Theefoe, an all-pole model is poposed to descibe the tempoal envelope. The segment size to identify the envelope is inceased to aound 4.5 ms. By using long segments, moe coefficients can be spent to descibe the peceptual impotant pats of the tempoal envelope. The impovement in modelling peaks in the envelope is shown in Figue 4. In the encode, the all-pole model is estimated by fist tansfoming the segment to the fequency domain. By applying the well-known LPC technique in the fequency domain, a set of pediction coefficients is obtained [6]. Given the set of pediction coefficients, a gain is calculated such that the envelope matches the oveall gain of the segment. In Figue 5, the geneation of a tempoally-shaped noise signal is shown. The pediction coefficients β = [β, β,, β Kt ] and Figue 4: Tempoal envelope. Dotted line: audio signal, top solid line: estimated envelope using the gains, bottom solid line: estimated envelope using all-pole modelling. the gain g ae etieved fom the bit steam. Fom β, a tempoal envelope e(n) is constucted by: e(n) = P K t k= β k exp( jπnk/n), (4) whee n =,,, N. A white noise sequence of length N is scaled by envelope g e(n). The successive segments ae ove-lap added and esult in a noise signal. The window in the ovelapadd pocedue consist of thee pieces: a Hanning window fade-in, a constant pat and a Hanning window fade-out. The signal is input to the ARMA filte. β EG g NG OLA Figue 5: Tempoally-shaped noise signal geneato consisting of an envelope geneato (EG), a noise geneato (NG) and an ovelap-add module (OLA). The input ae the gain paametes g and the pediction coefficients β. The pediction coefficients β ae tansfomed to the time-domain equivalents of LSFs and quantised using a conventional scala quantisation of 8 bits. Then they ae entopy coded. The gains g ae quantised on a db scale using 7 bits and they ae encoded diffeentially ove time. The update ate of the paametes is.7 ms. An aveage bit ate of appoximately. kbit/s fo the MPEG excepts is achieved fo encoding the tempoal envelope with the ode K t equal to 5. MPCA-

4 .. Paametic steeo Thus fa the paametic coding scheme only suppoted dual mono coding without exploiting inte-channel coelation and ielevancy. In wavefom codes as standadised in MPEG and MPEG, intechannel coelation and ielevancy is exploited using well-known techniques such as Mid/Side and Intensity steeo coding. Howeve, in the context of paametic coding it is not obvious how to use these techniques efficiently. In line with the idea of paameteisation, the model has been extended with a paametic steeo scheme, based on the wok of Beebaat, van de Pa and Kohlausch [7]. l Non-unifom filteing Non-unifom filteing Paamete extaction IID ITD ICC... Encode Figue 7: Extaction of steeo paametes. In Figue 6, the basic opeation of the paametic steeo encode is illustated. Based on the left l and ight time domain channels a meged channel m is deived. Futhemoe, duing the meging a numbe of steeo paametes, descibing the elationship between l and, ae extacted. The meged signal is encoded using the (mono) paametic encode descibed in Section, esulting in the thee objects tansients, sinusoids and noise, which ae quantised and coded. The steeo paametes ae also quantised and coded and multiplexed into a bit-steam togethe with the (quantised and coded) paametes fom the (mono) paametic encode. bit- steam Bit-steam demux Decoding Decoding Paametic decode m' Steeo econstuction steeo paametes l' ' l Paametic steeo extaction m steeo paametes Paametic code Quantisation and Coding Quantisation and Coding Figue 6: Block diagam of paameteic steeo encode. Bit-steam fomatte bit- steam Fo a numbe of fequency bands, the paametes defining the steeo image ae extacted. The fequency bands ae non-unifom in bandwidth. Fo each band, fo a total of maximally 4 bands, thee paametes ae extacted as illustated in Fig. 7:. The intechannel intensity diffeence, o IID, defined as the atio of intensities of the band-limited signals.. The intechannel time diffeence, o ITD, defined as the intechannel delay of the band-limited signals.. The intechannel coss-coelation, o ICC, defined as the (dis)similaity of the left and ight band-limited signal. In geneal the update ate of the steeo paametes does not need to be as high as that of the monaual code. An update ate of appoximately ms povides a sufficiently high quality epesentation of the steeo image fo most of the geneal audio mateial. Figue 8: Block diagam of paameteic steeo decode. the steeo paametes econstuct the steeo left (l ) and ight ( ) signal. Figue 9 depicts the block-diagam of the econstuction of the left and ight signal. Fom the monaual time domain signal m a decoelated signal d is calculated. Both the monaual time domain signal m and the decoelated signal d ae tansfomed to the fequency domain. Then the fequency domain paametes ae pocessed with the IID, ITD and ICC paametes esulting in fequency domain epesentations fo the left and ight channel. These ae tansfomed back to the time domain. Ovelap-add is used to combine adjacent fames. m' D d Time to fequency tansfom Time to fequency tansfom Paamete pocessing Fequency to time tansfom Fequency to time tansfom l' '... Decode IID ITD ICC At the decode side, as illustated in Figue 8, the bit-steam is de-multiplexed to mono and steeo paametes. Both sets of paametes ae decoded. The paametic decode decodes the mono paametes esulting in the signal m. This signal m togethe with Figue 9: Reconstuction of the steeo signal. MPCA-4

5 ... Tansient pocessing It is known fom the field of auditoy peception that tansients ae stong cues fo the binaual pocessing of sound (see e.g. [8]). Theefoe, the time esolution aound tansients has been inceased by means of applying an exta shot steeo window with its own set of paametes. Futhemoe, this shot tansient window is cented aound the exact tansient position as encoded within the monaual steam (see Figue ). Within the monaual pat of the bit-steam the tansient position is aleady encoded. Theefoe, only an index pointing towads the monaual fame containing the tansient position, which is used fo binaual pocessing, is given. windowing monaual signal paametic steeo windowing tansient position Figue : Windowing fo monaual and paametic steeo signal. The fame the tansient belongs to is maked light gey. Within the steeo pat of the bit-steam an index is placed to this fame...4. Scalability In the bit-steam syntax the steeo paametes ae spead ove a steeo base laye and a steeo extension laye in a scalable fashion (see Figue ). mono steeo base laye steeo extension laye Figue : Bit-steam scalability. Fist of all, the steeo base laye contains values epesenting the whole fequency ange, i.e., a single IID, ITD and ICC paamete coveing the whole fequency ange. The steeo extension laye contains paamete values fo bands of the complete fequency ange, i.e., a set of IID, ITD and ICC paametes togethe coveing the whole fequency ange. The paametes fom this extension laye ae coded diffeentially with espect to the paametes in the base laye. In this way, full scalability is achieved in encode as well as decode. Decoding only the monaual content of the bitsteam delives of couse a monaual signal only. Decoding the monaual content plus the base laye povides a low quality steeo image at low computational cost. If duing the decoding pocess the extension laye is taken into account, a high quality steeo image is obtained. So, depending on the amount of pocessing powe o channel capacity at hand, the decode automatically switches to lowe quality decoding of the steeo image. At the encode side, the same easoning applies. The encode can decide to encode only the base laye to obtain a decease in computational complexity o a bit-ate saving. 4. MPEG STANDARDISATION STATUS MPEG-4 is pogessing its task to poduce new coding standads that outpefom o extend existing MPEG-4 coding technology. In esponse to a Call fo Poposals (CfP) [9] issued in Januay, Philips has submitted a coding scheme on high-quality paametic audio coding. This coding scheme stated off in the Woking Daft (WD) phase, in Decembe as Refeence Model (RM) of MPEG-4 Extension. At the same time the fist impovement poposals, evaluated in coe expeiments, wee accepted which esulted in RM. The standadisation time schedule defined fo MPEG-4 Extension is given in Table. Standadisation phase Date WD (Woking Daft) Decembe CD (Committee Daft) Decembe FCD (Final Committee Daft) July FDIS (Final Daft Intenational Standad) Decembe Table : Standadisation time schedule fo MPEG-4 Extension In the Woking Daft (WD) phase impovement poposals can be submitted. Then, acceptance is based on the appoval of the audio sub-goup. Afte the Committee Daft (CD) peiod is effective, changes need fomal appoval of National Bodies. In July at the Klagenfut MPEG meeting, the two new poposals elated to tempoal noise envelope and paametic steeo coding, both descibed in Section, wee submitted. Duing that meeting, both coe expeiment poposals wee consideed to be complete. The check phase of the paametic steeo poposal consisted of a subjective listening test compaing the RM encode in dual mono mode at kbit/s vesus the RM encode in paametic steeo mode at kbit/s. Table shows the appoximate bit-ates of the steeo base and extension laye fo the MPEG test items. As a consequence of these bit-ates the monaual channel could spend up to appoximately 6 to kbit/s wheeas fo the case of dual mono encoding each channel could only spend about 6 kbit/s. Thus, the paametic steeo coding scheme is much moe efficient than the dual mono coding scheme. Note that fo a low quality steeo image, as descibed by the base laye only, the additional bit-ate is negligible. Pat of bit-steam Bit-ate (kbit/s) Steeo base laye. -.4 Steeo extension laye Steeo total.7-6. Table : Appoximate bit-ates fo both steeo layes fo the MPEG-4 test items at a total bit-ate of kbit/s. MPCA-5

6 5. ONGOING DEVELOPMENTS At the MPEG meeting of Octobe, a new spectal noise model was poposed. The RM vesion suppoted an ARMA model including an all-pole (LPC) model. It was found, howeve, that the ARMA model was not capable of poviding the necessay spectal detail in the low-fequency egion. That means that noise was smeaed ove too boad bands in the synthesise leading to the impession of too much low-fequent noise. This effect can be educed by applying high-pass filteing but then the impession of the decoded signal became synthetic, a metallic-like sound is intoduced which is most notably in speech signals. A good balance between too much noise and a synthetic sound impession could not be eached. In ode to be able to povide the necessay detail in the lowfequency ange of the spectum, it was decided to use a Laguee model instead of the ARMA model []. The noise synthesis filte is depicted in Fig. and has tansfe function H(z) accoding to whee /H(z) = A (z) KX k= A (z) = p λ z z λ, A(z) = λ + z z λ, α k {A(z)} k, (5) and λ is a paamete which can be tuned in accodance with an auditoy elevant fequency scale []. The paametes α k of these filtes can be estimated using input data windowing yielding stable synthesis filtes []. Using a mapping, the filte coefficients can be quantised and tansmitted as LARs o LSFs []. + x ˆx α + α. αk A A. Figue : Laguee-based noise synthesis model. The filte A is a fist-ode filte, the filtes A ae fist-ode allpass sections. The input is a white noise signal, the output x is a coloued noise signal. The impovement obtained by eplacement of the ARMA model by the Laguee model was veified at the MPEG meeting. The esults of the infomal listening test ae shown in Fig.. Even fo A the sub-optimal testing conditions, the quality of all coded excepts was maintained o impoved. Consequently, the efeence model will be pomoted to RM. MOS Suzanne Vega Geman Male English Female Tumpet Ochesta Laguee vs ARMA Cont. Pop Figue : Infomal subjective test esults fo RM including Laguee noise modelling compaed to RM using ARMA modeling at 4 kbit/s mono. The y-axis epesents the Mean Opinion Scoe (MOS) with 95% confidence intevals. A positive scoe indicates a pefeence of the Laguee model ove the ARMA model. Hapsichod Castanets 6. CONCLUSIONS Paametic audio coding is gaining inteest and is showing clea pogess in its pefomance. Cuently, a paametic coding scheme tagetting at high quality audio is unde standadisation in the context of MPEG-4 Extension. A numbe of ecently intoduced methods have futhe boosted the coding efficiency. The addition of paametic steeo is egaded as a majo step fowad. Not only does it povide a new way of steeo coding, but it also tuns out to be a highly efficient epesentation of steeo signals. We feel that the paametic model desciption is eaching a stable situation. In ode to exploit the full potential of paametic coding, futhe quality impovements have to be found mainly in the encode. 7. REFERENCES [] M. Dietz, L. Liljeyd, K. Kjöling and O. Kunz, Spectal Band Replication, a novel appoach in audio coding, Pepint 555, th AES Convention, Munich, - May. [] O. Kunz, Enhancing MPEG-4 AAC by Spectal Band Replication, Technical Sessions Poceedings of Wokshop and Exhibition on MPEG-4 (WEMP4), pp. 4 44, San Jose Faimont (USA), 5-7 June,. [] A.C. den Binke, E.G.P. Schuijes and A.W.J. Oomen, Paametic Coding fo High-Quality Audio, Pepint 5554, th AES Convention, Munich, - May. Pitchpipe Bagpipes Glockenspiel Plucked Stings Mean MPCA-6

7 [4] A.C. den Binke, Meixne-like functions having a ational z-tansfom, Int. J. Cicuit Theoy Appl., :7 46, 995. [5] A.C. den Binke and A.W.J. Oomen, Fast ARMA modelling of powe spectal density functions, In Poc. EU- SIPCO, Tenth Euopean Signal Pocess. Conf., pp. 9, Tampee (SF), 5-8 Sept.. [6] J. Hee and J.D. Johnston, Enhancing the pefomance of peceptual audio codes by using tempoal noise shaping, Pepint 484, st AES Convention, Los Angeles, 8- Novembe 996. [7] J. Beebaat, S. van de Pa and A. Kohlausch, Binaual pocessing model based on contalateal inhibition I. Model setup, J. Acoust. Soc. Am., :74 88, [8] R.G. Klumpp and H.R. Eady, Some measuements of inteaual time diffeence thesholds, J. Acoust. Soc. Am., 8:859 86, 956. [9] Audio Subgoup, Call fo poposals fo new tools fo audio coding, ISO/IEC JTC/SC9/WG N794,. [] V. Voitishchuk, A.C. den Binke and S.J.L. van Eijndhoven, Altenatives fo waped linea pedictos, In Poc. th PoRISC Wokshop, pp. 7 7, Veldhoven (NL), 9- Novembe. [] J.O. Smith and J.S. Abel, Bak and ERB bilinea tansfom, IEEE Tans. Speech Audio Pocess., 7:697 78, 999. [] A.C. den Binke, Stability of linea pedictive stuctues using IIR filtes, In Poc. th PoRISC Wokshop, pp. 7, Veldhoven (NL), 9- Novembe. [] A.C. den Binke and F. Riea-Palou, Quantisation and intepolation of laguee pediction coefficients, In Poc. th PoRISC Wokshop on Cicuits, Systems and Signal Pocessing, Veldhoven (NL), 8-9 Novembe. MPCA-7