This document is downloaded from DR-NTU, Nanyang Technological University Library, Singapore.

Transcription

1 This doument is downloaded from DR-NTU Nanyang Tehnologial University Library Singapore. Title Primary-Ambient Extration Using Ambient Spetrum Estimation for Immersive Spatial Audio Reprodution Author(s) He Jianjun; Gan Woon-Seng; Tan Ee-Leng Citation He J. Gan W.-S. & Tan E.-L. (). Primary-Ambient Extration Using Ambient Spetrum Estimation for Immersive Spatial Audio Reprodution. IEEE/ACM Transations on Audio Speeh and Language Proessing 3(9) Date -8 URL Rights IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses in any urrent or future media inluding reprinting/republishing this material for advertising or promotional purposes reating new olletive works for resale or redistribution to servers or lists or reuse of any opyrighted omponent of this work in other works. The published version is available at: [

2 Primary-Ambient Extration Using Ambient Spetrum Estimation for Immersive Spatial Audio Reprodution Jianjun He Student Member IEEE Woon-Seng Gan Senior Member IEEE and Ee-Leng Tan Abstrat The diversity of today s playbak systems requires a flexible effiient and immersive reprodution of sound senes in digital media. Spatial audio reprodution based on primary-ambient extration (PAE) fulfills this objetive where aurate extration of primary and ambient omponents from sound mixtures in hannel-based audio is ruial. Severe extration error was found in existing PAE approahes when dealing with sound mixtures that ontain a relatively strong ambient omponent a ommonly enountered ase in the sound senes of digital media. In this paper we propose a novel ambient spetrum estimation (ASE) framework to improve the performane of PAE. The ASE framework exploits the equal magnitude of the unorrelated ambient omponents in two hannels of a stereo signal and reformulates the PAE problem into the problem of estimating either ambient phase or magnitude. In partiular we take advantage of the sparse harateristi of the primary omponents to derive sparse solutions for ASE based PAE together with an approximate solution that an signifiantly redue the omputational ost. Our objetive and subjetive experimental results demonstrate that the proposed ASE approahes signifiantly outperform existing approahes espeially when the ambient omponent is relatively strong. Index Terms Primary-ambient extration (PAE) spatial audio ambient spetrum estimation (ASE) sparsity omputational effiieny. S I. INTRODUCTION PATIAL audio reprodution of digital media (suh as movies and video games) has gained signifiant popularity over the reent years []. The reprodution methods generally differ in the formats of audio ontent. Despite the growing interest in objet-based audio formats [] suh as Dolby Atmos [] DTS multi-dimensional audio (DTS: X) [3] most existing digital media ontent is still in hannel-based formats (suh as stereo and multihannel signals). The hannel-based audio is usually speifi in its playbak onfiguration and it does not support flexible playbak onfigurations in domesti or This work is supported by the Singapore Ministry of Eduation Aademi Researh Fund Tier- under researh grant MOE-T--4. Jianjun He and Woon-Seng Gan are with Digital Signal Proessing Lab Shool of Eletrial and Eletroni Engineering Nanyang Tehnologial University Singapore Ee-Leng Tan is urrently with Beijing Sesame World Co. Ltd China. ( jhe7@e.ntu.edu.sg ewsgan@ntu.edu.sg joseph@sesame-world.om). personal listening irumstanes []. Considering the wide diversity of today s playbak systems [4] it beomes neessary to proess audio signals suh that the reprodution of the audio ontent is not only ompatible with various playbak systems Depending on the atual playbak system the hallenges in spatial audio reprodution an be broadly ategorized into two main types: loudspeaker playbak and headphone playbak [7]. The hallenge in loudspeaker playbak deals with the mismath of loudspeaker playbak systems in home theater appliations where the number of loudspeakers [8] or even the type of loudspeakers [9]-[] between the intended loudspeaker system (based on the audio ontent) and the atual loudspeaker system is different. Conventional tehniques to solve this hallenge are often referred to as audio remixing (i.e. downmix and upmix) for example Left only Right only (LoRo) Left total Right total (LtRt) matrix-based mixing surround sound systems et [8] []-[4]. These audio remixing tehniques basially ompute the loudspeaker signals as the weighted sums of the input signals. For headphone playbak the hallenge arises when the audio ontent is not tailored for headphone playbak (usually intended for loudspeaker playbak). Virtualization is often regarded as the tehnique to solve this hallenge [] where virtualization of loudspeakers is ahieved by binaural rendering i.e. onvolving the hannel-based signals with head-related impulse responses (HRIRs) of the orresponding loudspeaker positions. These onventional tehniques in spatial audio reprodution are apable of solving the ompatibility issue but the spatial quality of the reprodued sound sene is usually limited [] [6]-[8]. To improve the spatial quality of the sound reprodution the MPEG audio standardization group proposed MPEG Surround and related tehniques whih typially address the multihannel and binaural audio reprodution problem based on human pereption [9]-[]. In the synthesis these tehniques usually employ the one-hannel downmixed signal and the subband spatial ues whih better suit the reprodution of the distint diretional soure signals as ompared to the diffuse signals [9] []. To further improve the quality of the reprodued sound sene the pereption of the sound senes is onsidered as a ombination of the foreground sound and bakground sound [3] whih are often referred to as primary (or diret) and ambient (or diffuse) omponents respetively [4]-[7]. The primary omponents onsist of point-like diretional sound soures whereas the ambient omponents are made up of diffuse environmental sound suh as the reverberation

3 Input PAE Primary omponents Ambient omponents Primary rendering Ambient rendering Spatial attributes Postproessing Any playbak system Fig.. Blok diagram of PAE based spatial audio reprodution []. applause or nature sound like waterfall [] [8]. Due to the pereptual differenes between the primary and ambient omponents different rendering shemes should be applied to the primary and ambient omponents for optimal spatial audio reprodution of sound senes [] [9]. However the existing hannel-based audio formats provide only the mixed signals [3] whih neessitate the proess of extrating primary and ambient omponents from the mixed signals. This extration proess is usually known as the primary-ambient extration (PAE). As a spatial audio proessing tool [8] [6] [7] [9] [6] [9] PAE an also be inorporated into spatial audio oding systems suh as spatial audio sene oding [4] [3] and diretional audio oding [3]. Essentially PAE serves as a front-end to failitate flexible effiient and immersive spatial audio reprodution. First by deomposing the primary and ambient omponents of the sound sene PAE enables the sound reprodution format to be independent of the input format hene inreasing the flexibility of spatial audio reprodution [3] [33]. Seond PAE based reprodution of sound senes does not require the individual sound objets as in objet-based format (whih is the most flexible) but is able to rereate pereptually similar sound senes hene maintaining the effiieny of spatial audio reprodution []. Last but not least PAE extrats the two key omponents of the sound senes namely diretional and diffuse sound omponents. These omponents are highly useful in rereating an immersive listening experiene of the sound sene [4] [34]-[36]. Figure illustrates the PAE based spatial audio reprodution system where the primary and ambient omponents undergo different rendering shemes []. The rendering shemes differ for loudspeaker or headphone playbak [8] [34] [37]. For loudspeaker playbak the primary omponents are reprodued using vetor base amplitude panning (VBAP) [3] or vetor base intensity panning [38] [39] to reprodue the aurate diretion of the sound soures. The ambient omponents on the other hand are further deorrelated and distributed to all the loudspeaker hannels to reate an envelopment effet of the sound environment [4] [4]. For headphone playbak the onventional virtualization that simply applies binaural rendering to the mixed hannel-based signals is problemati [6] [7]. PAE based virtualization resolves this problem by applying binaural rendering to the extrated primary omponents reating aurate virtual sound soures in the desired diretions [7] for headphone playbak [6] [4]. Similar to the loudspeaker playbak ase the ambient omponents are deorrelated using artifiial reverberation [9] [4] [8] [9] to reate a more natural sound environment. Numerous approahes are applied to solve PAE with stereo and multihannel input signals [] [4]. In this paper we fous on stereo input signals sine they are still one of the most widely used formats and the PAE approahes for stereo signals an be extended to deal with multihannel signals [] [4]-[44]. For the basi signal model the (stereo) mixed signal is onsidered as the sum of the primary and ambient omponents. The primary and ambient omponents are mainly disriminated by their inter-hannel ross-orrelations i.e. the primary and ambient omponents are onsidered to be orrelated and unorrelated respetively []. Based on this model time-frequeny masking approahes were introdued where the mask is obtained as a nonlinear funtion of the inter-hannel oherene of the input signal [8] or derived based on the harateristi that ambient omponents have equal level in the two hannels of the stereo signal [4] [4]. Further investigation of the differenes between two hannels of the stereo signals has led to several types of linear estimation based approahes [46] inluding prinipal omponent analysis (PCA) based approahes [] [] [4]-[3] and least-squares based approahes [4] [46] [4]. These linear estimation based approahes extrat the primary and ambient omponents using different performane-related riteria [46]. To deal with omplex input signals that do not satisfy the basi stereo signal model other PAE approahes onsider signal model lassifiation [] time/phase differenes in primary omponents [] [3] [3] [6] non-negative matrix fatorization [7] independent omponent analysis [8] et. Due to the nature of summing input signals diretly [46] the aforementioned PAE approahes often have diffiulty in removing unorrelated ambient omponent in the extrated primary and ambient omponents. The extration error in these PAE approahes is more severe when the ambient omponent is relatively strong ompared to the primary omponent [46] as often enountered in digital media ontent inluding busy sound senes with many disrete sound soures that ontribute to the environment as well as strong reverberation indoor environment. In [9] we proposed an ambient phase estimation (APE) framework to improve the performane of PAE. The APE framework exploits the equal-magnitude harateristi of unorrelated ambient omponents in the mixed signals of digital media ontent and was solved by pursuing the sparsity of the primary omponents [6] (this approah is known as APES [9]). However due to the trigonometri operations required in the estimation of the ambient phase the omputational ost of APES is still too high. In this paper we re-onsider the PAE problem from a higher level i.e. from the perspetive of ambient spetrum estimation (ASE). Besides APE a new formulation referred to as ambient magnitude estimation (AME) is derived and solved using the same riterion as in APES. Furthermore an approximate solution to the ASE problem shall be disussed so as to further redue the omputational ost. A omparative analysis on the objetive performane of the proposed ASE approahes and existing PAE approahes in terms of exation error and omputational effiieny is onduted with our simulations. To perform an in-depth

4 evaluation of these PAE approahes the performane measures proposed in [46] are adopted. However the alulation of these performane measures in [46] is only appliable for PAE approahes with analyti solutions. Therefore we propose a novel tehnique to ompute these measures for PAE approahes without analyti solutions as is the ase with the proposed ASE approahes. Moreover statistial variations are introdued to the ambient magnitudes to examine the robustness of the proposed ASE approahes. Furthermore subjetive listening tests are onduted to omplement the objetive evaluation. The remainder of this paper is strutured as follows. In Setion II we review the basi stereo signal model. The ambient spetrum estimation framework for PAE inluding APE and AME is formulated in Setion III. This is followed by the proposed solutions for ASE in Setion IV. Setion V explains the alulation of the performane measures whih are used to evaluate the PAE approahes in the experiments in Setion VI. Finally Setion VII onludes this paper. II. STEREO SIGNAL MODEL In spatial audio PAE is often onsidered in time-frequeny domain [7] [] [] [8] [3] [4] [4]. It is generally assumed that there is only one dominant diretional soure (a.k.a. primary omponent) in eah frequeny band of the input signal. PAE is independently arried out on eah frequeny band of eah frame (onsisting of N short frames) of the input signal [] [8] [33] [4] [4]. We denote the stereo signal in time-frequeny domain at time index n and frequeny bin index. Hene the l as X n l where the hannel index stereo signal at subband b that onsists of bins from to b lb l is expressed as n b X n l X n l X n l X b b b [38]. The stereo signal model is expressed as: X n b P n b A n b () where P and A are the primary and ambient omponents in the th hannel of the stereo signal respetively. Sine the frequeny band of the input signal is generally used in the analysis of PAE approahes the indies nb are omitted for brevity. The stereo signal model assumes that the primary and ambient omponents in the two hannels of the stereo signals are orrelated and unorrelated respetively. Correlated primary omponent an be haraterized by time and amplitude differenes between the two hannels [6]. For this paper we shall only onsider the primary omponent to be amplitude panned that is P k P where k is referred to as the primary panning fator [] [4] [4]. For this paper we assume k suh that the hannel ontaining the stronger diretional primary omponent is hannel. This amplitude panned primary omponent is ommonly found in stereo reordings using oinident tehniques and sound mixes using onventional amplitude panning tehniques [3]. Considering that only the mixed signal is given as input and no prior information is available it is neessary to estimate k using orrelations [46] or histograms of amplitude differenes [6]. T For example based on the autoorrelations of the two hannels r r and ross-orrelations between the two hannels r r r r r we an estimate k as k. r r For an ambient omponent that is made up of environmental sound it is usually onsidered to be unorrelated with the primary omponent [3] [63] [64] as well as being balaned in two hannels in terms of signal power. To quantify the power differene between the primary and ambient omponents the primary power ratio is defined as the ratio of total primary power to total signal power in two hannels: P X. Previous study revealed that the performane of PAE is highly dependent on where lower generally indiates inferior overall extration performane [46]. Using the method desribed in [46] we omputed for many movie and gaming traks (e.g. Avatar Brave Battlefield 3 BioShok Infinite) and found that the perentage for the ases with over half of the time frames having relative strong ambient power (i.e..7 ) is around 7% in these digital media ontent examples. Sine high ourrene of strong ambient power ase degrades the overall performane of PAE a PAE approah that also performs well in the presene of strong ambient power is desired. III. AMBIENT SPECTRUM ESTIMATION The diffuseness of ambient omponents usually leads to low ross-orrelation between the two hannels of the ambient omponents in the stereo signal. During the mixing proess the sound engineers synthesize the ambient omponent using various deorrelation tehniques suh as introduing delay [6] all-pass filtering [66]-[68] artifiial reverberation [] and binaural artifiial reverberation [69]. These deorrelation tehniques often maintain the magnitude of ambient omponents in the two hannels of the stereo signal. As suh we an express the spetrum of ambient omponents as A A W () where denotes element-wise Hadamard produt A A A is the equal magnitude of the ambient omponents and the element in the bin (n l) of W n l e j n l where nl is the bin (n l) of W is θ and θ A is the vetor of phase samples (in radians) of the ambient omponents. Following these disussions we shall derive the ASE framework for PAE in two ways: ambient phase estimation [] and ambient magnitude estimation. A. Ambient Phase Estimation Considering the panning of the primary omponent P kp the primary omponent in () an be anelled out and we arrive at X kx A ka. (3) By substituting () into (3) we have

5 where. k. k A X X W W (4) represents the element-wise division. Beause ambient magnitude A is real and non-negative we derive the relation between the phases of the two ambient omponents as (refer to Appendix A for detailed derivation) θ θ arsin k sin θ θ () θ X k X Furthermore by substituting (4) and where. () into () we have A X kx. W kw W (6) P X X X. W W W. Sine k k X and k an be diretly omputed from the input [46] W W are the only unknown variables on the right hand side of the expressions in (6). In other words the primary and ambient omponents are determined by W W whih are solely related to the phases of the ambient omponents. Therefore we reformulate the PAE problem into an ambient phase estimation (APE) problem. Based on the relation between θ and θ stated in () only one ambient phase θ needs to be estimated. B. Ambient Magnitude Estimation To reformulate the PAE problem as an ambient magnitude estimation problem we rewrite () for every time-frequeny bin as: ' X kx P ka X P A. (7) Consider these bin-wise spetra stated in (7) as vetors in omplex plane (represented by an arrow on top) we an express their geometri relations in Fig. as X X OB B B OC C C ' Re Im Re Im P OP P P Re Im k A PB A PC. Let r denote the magnitude of the ambient omponent i.e. r A A Then we have PC r PB kr. Therefore. by drawing two irles with their origins at B and C we an find their intersetion point P (selet one point when there are two intersetion points) whih orresponds to the spetrum of the primary omponent and leads to the solution for the extrated primary and ambient omponents. For any estimate of ambient magnitude r ˆ the oordinates of point P shall satisfy The solution of B PRe BRe PIm B Im krˆ PRe CRe PIm C Im rˆ. P RePIm for (9) is given by: C B k rˆ B C C Re Re Re Re Im Im ˆP Re B BC C B k rˆ B C C BC Im Im Im Im Re Re ˆ Im (8) (9) () C (C Re C Im ) X P Im ka A P (P Re P Im ) O ( ) ' X where the Eulidean distane between the points B and C Re Re Im Im BC C B C B and ˆ B (B Re B Im ) Fig.. Geometri representation of (7) in omplex plane in AME. k r k rˆ BC BC. Based on (8) the spetra of the primary and ambient omponents an then be derived as: ˆ ˆ ˆ ˆ P PRe jp Im P k P ˆ ˆ Re jp Im () ˆ ˆ ˆ ˆ A X P jp A X k Pˆ jp ˆ. Re Im Re Im Therefore the PAE problem beomes the problem of determining r i.e. ambient magnitude estimation. The approah to determine r and selet one of the two solutions in () will be disussed in Setion IV. It an be inferred from Fig. that determining the ambient magnitude is equivalent to determine the ambient phase as either of them will lead to the other. Therefore we onlude that APE and AME are equivalent and they are olletively termed as ambient spetrum estimation. The blok diagram of the ASE based PAE is illustrated in Fig. 3. We argue that in theory by aurately obtaining the spetra of ambient omponents it is possible to ahieve perfet extration (i.e. error-free) of the primary and ambient omponents using the formulation of ASE whih is not possible with existing PAE approahes as a onsequene of residue error from the unorrelated ambient omponent [46]. IV. AMBIENT SPECTRUM ESTIMATION WITH A SPARSITY CONSTRAINT The proposed ambient spetrum estimation framework an greatly simplify the PAE problem into an estimation problem with only one unknown parameter per time-frequeny bin. To estimate these parameters we shall exploit other harateristis of the primary and ambient omponents that have not been used in previous derivation. One of the most important harateristis of sound soure signals is sparsity whih has Re

6 Input signal x x T-F STFT Transform Selet ASE APE AME Find θ Find r Compute Pˆ Pˆ Aˆ Aˆ Inverse T-F Transform Extrated primary and ambient omponents pˆ pˆ aˆ aˆ Fig. 3. Blok diagram of ASE based PAE. been widely used as a ritial riterion in finding optimal solutions in many audio and musi signal proessing appliations [6]. In PAE sine the primary omponents are essentially diretional sound soures they an be onsidered to be sparse in the time-frequeny domain [6]. Therefore we estimate the ambient phase or magnitude spetrum by restriting that the extrated primary omponent is sparse. We refer to these approahes as ambient spetrum estimation with a sparsity onstraint (ASES). By applying the sparsity onstraint in APE and AME ASES an be divided into two approahes namely APES and AMES. A. Ambient Phase Estimation with a Sparsity Constraint With a sparsity onstraint the ambient phase estimation problem an be expressed as follows: ˆ* θ arg min P ˆ () θˆ where ˆP is the -norm of the primary omponent whih is equal to the sum of the magnitudes of the primary omponent over the time-frequeny bins. Sine the objetive funtion in () is not onvex onvex optimization tehniques are inappliable. Heuristi methods like simulated annealing [7] require optimization to be performed for all the phase variables and hene are ineffiient in solving APES [9]. On this note a more effiient method referred to as disrete searhing (DS) to estimate ambient phase was proposed in [9]. DS is proposed based on the following two observations. First the magnitude of the primary omponent at one time-frequeny bin is solely determined by the phase of the ambient omponent at the same time-frequeny bin and hene the estimation in () an be independently performed for eah time-frequeny bin. Seond the phase variable is bounded to and high preision of the estimated phase may not be neessary. Thus the optimal phase estimates an be seleted from an array of disrete phase ˆ d d D d D with D values where being the total number of phase values to be onsidered. In general the value of D affets the extration and the omputational performane of APES using DS [9]. Following () and (6) a total number of D estimates of the primary omponents an be omputed. The estimated phase then orresponds to the minimum of magnitudes of the primary ˆ* ˆ * * d where d arg min Pˆ d omponent i.e. d D Finally the extrated primary and ambient omponents are omputed using (6). It shall be noted that in DS a suffiient ondition of the sparsity onstraint was employed in solving the APES problem in (). B. Ambient Magnitude Estimation with a Sparsity Constraint Similarly to APES that is solved using the sparsity onstraint the ambient magnitude estimation problem an be expressed as: * rˆ arg min P ˆ (3) ˆ r where ˆr is the estimated ambient magnitude of all the time-frequeny bins. As no onstraints are plaed on the ambient magnitude spetra among the time-frequeny bins in one frame the estimation of ambient magnitude an also be onsidered to be independent for every time-frequeny bin. Therefore the estimation of ambient magnitude an be obtained individually for every time-frequeny bin by minimizing the primary magnitude under the AMES framework. To derive the solution for AMES we follow the geometri relation illustrated in Fig.. To ensure the existene of intersetion point P the following onstraint PC PB BC PB PC (4) has to be satisfied whih leads to: r r r () lb BC BC where r lb rub k. k k When k = there is no physial upper bound from (4). Based on the objetive of minimizing the magnitude of primary omponent we an atually enfore an approximate upper bound for k = for example let r OB OC k. Thus the ambient ub magnitude is bounded and the same numerial method DS (as used in APES) is employed to estimate r in AMES. Consider an array of disrete ambient magnitude values d d rˆ d rlb rub where d D with D D D being the total number of ambient magnitude estimates onsidered. For eah magnitude estimate rˆ d we selet the one P ˆ Re P ˆ Im of two solutions from () whih gives the smaller primary magnitude. After derivation we an unify the solution for the seleted P ˆ ReP ˆ Im as ub

7 Table I: Computational ost of APES AMES and APEX (for every time-frequeny bin) Operation Square root Addition Multipliation Division Comparison Trigonometri operation APES D D+8 D+3 4D+6 D- 7D+6 AMES D+ D+3 4D+4 9D+3 D- APEX D: number of phase or magnitude estimates in disrete searhing ˆP B C C B k rˆ d Re Re Re Re Re d ˆP BC B C d sgn B C B C B Im Im Re Im Im Re BC C B k rˆ d C BC Im Im Im Im Im d B C d sgn B C B C BC Re Re Re Im Im Re (6) where sgn(x) is the sign of x. The estimated magnitude of the primary omponent is obtained as ˆ ˆ ˆ P d P d P d (7) Re Im The estimated ambient magnitude then orresponds to the minimum of the primary omponent magnitude i.e. ˆ * ˆ * * r r d where d =arg min Pˆ d. Finally the d D extrated primary and ambient omponents are omputed using (). C. Computational Cost of APES and AMES In this subsetion we ompare the omputational ost of APES and AMES as shown in Table I. In general both AMES and APES are quite omputational extensive. AMES requires more operations whih inlude square root addition multipliation and division but requires no trigonometri operations. By ontrast APES requires 7D+6 times of trigonometri operations for every time-frequeny bin. The omputational effiieny of these two approahes is affeted by the implementation of these operations. D. An Approximate solution: APEX To obtain a more effiient approah for ambient spetrum estimation we derive an approximate solution in this subsetion. For every time-frequeny bin we an rewrite () for the two hannels as: ospa X P A P A PA k P A k P A os X P A P A os PA P A P A os PA (8) where PA PA are the phase differenes between the spetra of the primary and ambient omponents in hannel and respetively. From (8) we an obtain that os PA ospa k P A k P X X. (9) Solving (9) for P we arrive at (). From () when k > the minimization of P an be approximately ahieved by minimizing X k ospa ospa (onsidering that X in most ases sine k ) whih leads to PA PA. Aording to the relation between the two ambient phases in () we an infer that it is impossible to always ahieve both PA and PA at the same time. Clearly sine k a better approximate solution would be taking PA. On the other hand when k = one approximate solution to minimize P would be letting. These onstraints an be applied in either APE or AME framework. Here applying the onstraints in APE is more straightforward and we shall obtain the approximate phase estimation as: ˆ * X k θ. () X X k As the phase (or the phase differene) of the input signals is employed in () we refer to this approximate solution as APEX. As shown in Table I APEX requires the lowest omputational ost and is signifiantly more effiient than either APES or AMES. The performane of these approahes will be evaluated in the following setions. V. EVALUATION OF PAE An evaluation framework for PAE was initially proposed in [46]. In general we are onerned with the extration auray and spatial auray in PAE. The overall extration auray of PAE is quantified by error-to-signal ratio (ESR in db) of the extrated primary and ambient omponents where lower ESR indiates better extration of these omponents. The ESR for the primary and ambient omponents are omputed as pˆ p ESR P log p () aˆ a ESR A log. a

8 The extration error an be further deomposed into three omponents namely the distortion interferene and leakage (refer to [46] for the explanation of these three error omponents). Corresponding performane measures of these error omponents an be omputed diretly for PAE approahes with analyti solutions. As there is no analyti solution for these ASE approahes we need to find alternative ways to ompute these measures. In this setion we propose a novel optimization tehnique to estimate these performane measures. First we onsider the extrated primary omponent in time domain p ˆ. Sine the true primary omponents in two hannels are ompletely orrelated no interferene is inurred [46]. Thus we an express pˆ as pˆ p Leak Dist (3) where the leakage is Leak w w os PA ospa ospa ospa A k A k k X X k P X X X X k p p p a a and the P P distortion is Dist p. To ompute the measures we need to estimate wp w P first. Considering that p a and a are inter-unorrelated we propose the following way to estimate w w with P P * * w w ˆ w w P P P P arg min p p a a (4) wp wp Thus we an ompute the measures leakage-to-signal ratio (LSR) and distortion-to-signal ratio (DSR) for the primary omponents as * * wp w a Pa LSR P log p () * * ˆ wp w p p a Pa DSR P log. p Seond we express a ˆ in a similar way as aˆ a Leak a Intfa Dista (6) where the leakage is Leaka w p and the interferene A A Intfa w a originates from the unorrelated ambient omponent. The two weight parameters wa wa an be estimated as w w * * A A aˆ a wa p wa a wa wa arg min A ospa ospa PA PA 4 A sin sin (7) Thus we ompute the measures LSR interferene-to-signal ratio (ISR) and (DSR) for the ambient omponents using * wa p LSR A log a * w Aa ISR A log a aˆ a wa p wa a DSR A log. a k. (8) Previous experiene on evaluating linear estimation based PAE approahes suh as PCA and least-squares suggests that these parameters wp wp wa wa are bounded to [- ] hene we an employ a simple numerial searhing method similar to DS to determine the optimal estimates of these parameters using a ertain preision [46]. As audio signals from digital media are quite non-stationary these measures shall be omputed for every frame and an be averaged to obtain the overall performane for the whole trak. On the other hand spatial auray is measured using the inter-hannel ues. For primary omponents the auray of the sound loalization is mainly evaluated using inter-hannel time and level differenes (i.e. ICTD and ICLD). In this paper there is no ICTD involved in the basi mixing model for stereo input signals and the ICLD is essentially determined by the estimation of k whih is ommon between the proposed approahes and the existing linear estimation based approahes suh as PCA [46]. For these two reasons spatial auray is not evaluated for primary omponent extration but is foused on the extration of ambient omponents. The spatial auray of the ambient omponent is evaluated in terms of its diffuseness as quantified by inter-hannel ross-orrelation oeffiient (ICC from to ) and the ICLD (in db). It is lear that a more diffuse ambient omponent requires both ICC and ICLD to be loser to. VI. EXPERIMENTAL RESULTS AND DISCUSSIONS In this setion we shall present a omprehensive objetive and subjetive evaluation of the proposed ASE approahes and two existing PAE approahes namely PCA [] and time-frequeny masking [4]. In these experiments the searhing method of APES or AMES is DS with D =. Based on the performane measures introdued in Setion V we shall ompare the overall extration error performane the speifi error performane inluding leakage distortion and interferene as well as the spatial auray of the ambient omponents. Additionally we will also ompare the effiieny of these PAE approahes in terms of the omputation time based on our simulation. To examine the robustness of these ()

9 ESR P - (a) k = 4 APES AMES APEX PCA Masking ESR P - (b) k = ESR P - () k = Primary power ratio Primary power ratio Primary power ratio (d) k = 4 (e) k = (f) k = ESR A - ESR A - ESR A Primary power ratio Primary power ratio Primary power ratio Fig. 4. Comparison of the ESR of (a-) extrated primary omponents and (d-f) extrated ambient omponents with respet to different k values using APES AMES APEX PCA [] and Masking [4]. PAE approahes we evaluate the proposed approahes using synthesized mixed signal with unequal ambient magnitude in two hannels. Lastly subjetive listening tests were onduted to examine the pereptual timbre and spatial quality of different PAE approahes. The stereo mixed signals employed in the experiments are synthesized in the following way. One frame (496 samples sampling rate: 44. khz) of speeh signal is seleted as the primary omponent whih is amplitude panned k 4. A wave to hannel with a panning fator lapping sound reorded at the beah is seleted as the ambient omponent whih is deorrelated using all-pass filters with random phase [68]. The stereo signal is obtained by mixing the primary and ambient omponents based on different values ranging from to with an interval of.. First we ompare the overall performane of the three ASE approahes with two other PAE approahes in the literature namely PCA [] and Masking [4]. For the proposed ASE approahes FFT size is set as 496 while for Masking the best setting for FFT size is found as 64. The ESR of these approahes with respet to different values of and k is illustrated in Fig. 4. Our observations of the ESR performane are as follows: ) Generally the performane of all these PAE approahes varies with. As inreases ESR P dereases while ESR A inreases (exept ESR A of PCA). Considering primary omponents to be more important in most appliations it beomes apparent that the two representative existing approahes annot perform well when is low. ) Primary panning fator k is the other fator that affets the ESR performane of these PAE approahes exept PCA. For the Masking approah the influene of k is insignifiant for most ases exept ESR P at very low and ESR A at very high. By ontrast the ASE approahes are more sensitive to k. The ESR of APES and AMES are lower at higher k espeially when is high. For APEX the performane varies between k > and k = whih was implied in (). 3) Irrespetive of and k APES and AMES perform quite similar. Both APES and AMES outperform existing approahes at lower i.e. from <.8 when k = { 4} to <. when k =. APEX an be onsidered as an approximate solution to APES or AMES for k > and when k = it beomes idential to PCA (this an also be verified theoretially). Seond we look into the speifi error performane of ASE approahes at k =. Note that there are some slight variations in these error measures for lose values whih is due to the inauray in the estimation of speifi error omponents. Nevertheless we an observe the following trends. As shown in Fig. (a) and (b) we found that the performane improvement of ASE approahes in extrating primary omponents lies in the redution of the ambient leakage though at the ost of introduing more distortion. For ambient omponent extration PCA and Masking yield the least amount of leakage and interferene respetively. Note that the little amount of leakage in PCA and interferene in Masking are atually due to the estimation error sine none of them theoretially exist in the extrated ambient omponents. Nevertheless the ASE approahes yields moderate amount of these errors whih results in a better overall performane. Third we examine the spatial auray of PAE in terms of the diffuseness of the extrated ambient omponents. As shown

10 (a) (b) - LSR P - - DSR P APES AMES APEX PCA Masking Primary power ratio Primary power ratio () (d) (e) - LSR A DSR A - - ISR A Primary power ratio Primary power ratio Primary power ratio Fig.. Comparison of the speifi error performane of (a-b) LSR and DSR in the extrated primary omponents and (-e) LSR DSR and ISR in the extrated ambient omponents using APES AMES APEX PCA and Masking. in Fig. 6(a)-() the lowest and highest ICC are ahieved with true ambient omponents and ambient omponents extrated by PCA respetively. The ASE approahes outperform the existing approahes and are more effetive in extrating diffuse ambient omponents at higher k and lower. For ICLD of the extrated ambient omponents as shown in Fig. 6(d)-(f) we observed that all approahes extrat ambient omponents with equal level between the two hannels whereas PCA works only for k =. Fourth we ompare the extration performane as well as the omputation time among these PAE approahes. The simulation was arried out on a PC with i-4 CPU 8 GB RAM 64-bit windows 7 operating system and 64-bit MATLAB 7... Though MATLAB simulations do not provide preise omputation time measurement ompared to the atual implementation we ould still obtain the relative omputation performane among the PAE approahes. The results of omputation time averaged aross all the and k values are summarized in Table II. It is obvious that the three ASE approahes perform better than PCA and Masking on the average. But when we ompare the omputation time among APES AMES and APEX we found that AMES is around x faster than APES but is still far away from the omputation time of the existing approahes. The APEX whih estimates the ambient phase diretly using the phase of the input signals is over 4x faster as ompared to AMES and beomes quite lose to the Masking approah and hene an be onsidered as a good alternative ASE approah for PAE. Furthermore in order to ahieve real-time performane (in frame-based proessing) the proessing time must be less than 496/44. = 9.88 (ms). It is lear that APEX together with PCA and Masking satisfies this real-time onstraint. Fifth we study the robustness of the proposed ASE approahes using experiments with the input signals ontaining unequal ambient magnitudes in the two hannels. To quantify the violation of the assumption of equal ambient magnitude we introdue an inter-hannel variation fator v that denotes the range of variation of the ambient magnitude in one hannel as ompared to the other hannel. Let us denote the ambient magnitude in the two hannels as r r. The variation of v log r r db. ambient magnitude is expressed as In the ideal ase we always have v. To allow variation we onsider v as a random variable with mean equal to and variane as. In this experiment we onsider two types of distributions for the variation namely normal distribution and uniform distribution and examine the performane of these PAE approahes with respet to different variane of variations i.e. at. and k =. We run the experiment times and illustrate the averaged performane in terms of ESR and ICC in Figs. 7 and 8. We observed that as the variane of the variation inreases the ESR performane of proposed ASE approahes beomes worse though ICC was not affeted muh. The ASE approahes are more robust to ambient magnitude variations under normal distribution ompared to uniform distribution. Compared to PCA and Masking the proposed approahes are still better with the variane of variation up to db. Therefore we onlude that the three ASE approahes are in general robust to ambient magnitude variations. Lastly subjetive tests were arried out to evaluate the pereptual performane of these PAE approahes. A total of 7

11 . (a) k = 4. (b) k =. () k = ICC.6 ICC.6 ICC Primary power ratio (d) k = Primary power ratio (e) k = Primary power ratio (f) k = ICLD - - True APES AMES APEX PCA Masking ICLD - ICLD Primary power ratio Primary power ratio Primary power ratio Fig. 6. Comparison of the diffuseness of the extrated ambient omponents in terms of (a)-() ICC and (d)-(f) ICLD using APES AMES APEX PCA and Masking. Table II: Average ESR ICC and omputation time of PAE approahes Method APES AMES APEX PCA [] Masking [4] ESR P ESR A ICC of ambient omponents Computation time (ms) subjets ( males and two females) who were all between -3 years old partiipated in the listening tests. None of the subjets reported any hearing issues. The tests were onduted in a quiet listening room at Nanyang Tehnologial University Singapore. An Audio Tehnia MTH-A3 headphone was used. The stimuli used in this test were synthesized using amplitude panned (k = ) primary omponents (speeh musi and bee sound) and deorrelated ambient omponents (forest anteen and waterfall sound) based on two values of primary power ratio (.3.7 ) for the duration of -4 seonds. Both the extration auray and spatial auray were examined. The testing proedure was based on MUSHRA [7] [7] where a more speifi anhor (i.e. the mixture) is used instead of the low-passed anhor aording to reent revision of MUSHRA as disussed in [7]. The MATLAB GUI was modified based on the one used in [73]. Subjets were asked to listen to the lean referene sound and tested sounds obtained from different PAE approahes and give a sore of - as the response where and 8- represent a bad poor fair good and exellent quality respetively. Finally we analyzed the subjets responses for the hidden referene (lean primary or ambient omponents) mixture and three PAE approahes namely Masking [4] PCA [] and APEX. Note that APEX is seleted as the representative of ASE approahes beause APES and AMES exhibit very similar extration results. The box plots of the subjetive sores of the extration and spatial auray for the tested PAE approahes are illustrated in Figs. 9. Note that for eah PAE approah we ombine the subjetive sores of different test stimuli and different values of primary power ratio so as to represent the overall performane of these PAE approahes. Despite the relatively large variations among the subjetive sores that are probably due to the different sales employed by the subjets and the differenes among the stimuli we observe the following trends. On one hand we observed that APES outperforms the other PAE approahes in extrating aurate primary omponents as shown in Fig. 9(a). In Fig. 9(b) APEX though slightly worse off than PCA still produes onsiderable auray in ambient extration. The good pereptual performane of ambient omponents extrated from PCA lies in the very low amount of primary leakage as shown in Fig. (). On the other hand we found that the spatial performane were also affeted by the undesired leakage signals as ompared to the lean referene as found in the mixtures whih preserve the same spatial quality as the referene but were rated lower than the referene. With respet to the diffuseness of the ambient omponents APEX performs the best while PCA performs quite poorly. On this note we find PCA sarifies on the diffuseness of the extrated ambient omponents for the sake of a better pereptual extration performane. A further analysis of the ANOVA results shows that the p-values are extremely small whih reveals that the differenes among the performane of these PAE approahes are signifiant. To sum up the subjetive evaluation results the proposed ASE approahes yield the best performane in terms

12 - (a) -. (b) () ESR P ESR A ICC True APES AMES APEX PCA Masking Variane of variation Variane of variation Variane of variation Fig. 7. Comparison of the performane of PAE approahes in the presene of normally distributed variations in the ambient magnitudes in two hannels (with. k = ): (a) ESR P (b) ESR A () ICC of ambient omponents. (a) (b) () ESR P True APES AMES APEX PCA Masking ESR A Variane of variation Variane of variation Variane of variation Fig. 8. Comparison of the performane of PAE approahes in the presene of uniformly distributed variations in the ambient magnitudes in two hannels (with. k = ): (a) ESR P (b) ESR A () ICC of ambient omponents. ICC of extration and spatial auray whih is onsistent with our objetive evaluation results. For the purpose of reproduible researh the soure ode and some of the proessed traks used in our experiments an be found in [74]. Despite the improved performane of the proposed ASE approah as shown in these simulations and experiments there are a few issues need to be arefully onsidered to generalize the results to more omplex audio signals in digital media. One of them is the time-frequeny transform. The proposed PAE approahes as well as the existing PAE approahes were proposed based on a basi stereo signal model. How to obtain the most appropriate time-frequeny representation so that all the assumptions in this signal model are satisfied are extremely important to ensure a good PAE performane. Seondly though only the sparsity onstraint is used in this paper other onstraints an also be employed to improve the performane of ambient spetrum estimation espeially for the ase with k lose to. Some of these onstraints inlude the orrelation of the ambient omponents the independene between primary and ambient omponents et. Thirdly probabilisti approahes shall be developed to model the ambient magnitude variations better. Last but not least extending the PAE approahes from stereo to multihannel signals (e.g..) is also of great pratial value. One idea is to apply PAE to the downmixed signals [43] seleted pairs [44] or even every pair of the multihannel signals [4]. However a more omprehensive study on these extensions of PAE approahes needs to be arried out. VII. CONCLUSIONS In this paper we presented a novel formulation of the PAE problem in the time-frequeny domain. By taking advantage of equal magnitude of ambient omponent in two hannels the PAE problem is reformulated as an ambient spetrum estimation problem. The ASE framework an be onsidered in two ways namely ambient phase estimation and ambient magnitude estimation. The novel ASE formulation provides a promising way to solve PAE in the sense that the optimal solution leads to perfet primary and ambient extration whih is unahievable with existing PAE approahes. In this paper ASE is solved based on the sparsity of the primary omponents resulting in two approahes APES and AMES. To thoroughly evaluate the performane of extration error we proposed an optimization method to ompute the leakage distortion and interferene of the extration error for PAE approahes without analytial solutions. Based on our experiments we observed signifiant performane improvement of the proposed approahes over existing approahes. The improvement on error redution is

13 (a) Extration auray of primary omponents (b) Extration auray of ambient omponents () Diffuseness auary of ambient omponents Subjetive sore Subjetive sore Subjetive sore Referene Mixture Masking PCA APEX Referene Mixture Masking PCA APEX Referene Mixture Masking PCA APEX Fig. 9. Subjetive performane for (a) the extration auray of primary omponents (b) the extration auray of ambient omponents and () diffuseness auray of ambient omponents. around 3-6 db on average and up to - db for lower whih is mainly due to the lower residual error from the unorrelated ambient omponents. Moreover the ASE approahes perform better for mixed signals having heavily panned primary omponents than those having slightly panned primary omponents. In terms of the spatial auray the ASE approahes extrat more diffuse ambient omponents. When it omes to the omputational effiieny of these PAE approahes we found that AMES is an order of magnitude faster than APES under the same setting in MATLAB simulation but is still not as effiient as existing approahes. For this purpose we have also derived an approximate solution APEX and verified its effetiveness as well as the effiieny in our simulation. Besides the ideal situation where the ambient magnitudes are equal in two hannels the robustness of these ASE approahes was also examined by introduing statistial variations to the ambient magnitudes in the two hannels of the stereo signal. It was found that the proposed approahes an still yield better results with the variane of variations up to db. The objetive performane of the proposed ASE approahes was also validated in our subjetive tests. Future work inludes the study on the use of estimation riteria other than the sparsity of the primary omponent [7] time-frequeny transform in PAE and handling more omplex stereo and multihannel signals using ASE. Appendix A Derivation of the relation between θ and θ in () We show the derivation for the relation between the ambient phases in two hannels. First we rewrite W kw os θ k os θ j sin θ k sin θ. Sine A is real we have the following relation: sin θ. os θ sin θ ksin θ. os θ kos θ whih an be further rewritten as sin θ θ k sin θ θ. (9) Two solutions arise when solving for θ : θ θ θ θ (3) where k θθ Then we have sin = k sin θθ = arsin sin and... and os = k sin θ θ. Based on the other ondition that ambient magnitude A is nonnegative the imagery (or real) part of W kw must have the same sign as the imagery (or real) part of X k. X Next we examine the two solutions for this ondition. We take the first solution θ and rewrite the ratio of imagery part of W kw to the imagery part of X kx as Im Im W kw θ X kx θ sin θ k sin θ θ sin θ θ sin θ k sin θ sin θ osθ θ k os θ θ k θ θ (3) os os. Therefore the sign of the imagery part of W kw is different from the sign of imagery part of X kx resulting in negative values for ambient magnitude A. Therefore the first solution in (3) is inadmissible. Similarly we take the seond solution θ and derive the ratio of imagery part of W kwto the imagery part of X kx as Im W kw θ X kx θ Im (3) os θ θ k os θ θ. Therefore the sign of the imagery part of W kw is the same from the sign of imagery part of X kx ensuring nonnegative values in ambient magnitude A. Hene we an onlude that based on the seond solution the relation between the ambient phases in two hannels is θ θ arsin k sin θ θ.

14 REFERENCES [] ITU Report ITU-R BS.9-4: Multihannel sound tehnology in home and broadasting appliations. [] Dolby (4 Jul. ) Dolby Atmos next-generation audio for inema issue 3 [Online] Available: [3] J. M. Jot and Z. Fejzo Beyond surround sound - reation oding and reprodution of 3-D audio soundtraks in 3st Audio Eng. So. Conv. New York NY Ot.. [4] J. Herre J. Hilpert A. Kuntz and J. Plogsties MPEG-H audio the new standard for universal spatial/3d audio oding J. Audio Eng. So. vol. 6 no. pp De. 3. [] F. Rumsey Spatial quality evaluation for reprodued sound: terminology meaning and a sene-based paradigm J. Audio Eng. So. vol. no. 9 pp Sep.. [6] F. Rumsey Spatial audio: eighty years after Blumlein J. Audio Eng. So. vol. 9 no. / pp. 7 6 Jan./Feb.. [7] F. Rumsey Spatial audio proessing: upmix downmix shake it all about J. Audio Eng. So. vol. 6 no. 6 pp Jun. 3. [8] F. Rumsey Spatial Audio. Oxford UK: Foal Press. [9] W. S. Gan E. L. Tan and S. M. Kuo Audio projetion: diretional sound and its appliation in immersive ommuniation IEEE Signal Proess. Mag. vol. 8 no. pp. 437 Jan.. [] E. L. Tan and W. S. Gan Reprodution of immersive sound using diretional and onventional loudspeakers J. Aoust. So. Am. vol. 3 no. 4 pp. 3-3 Apr.. [] E. L. Tan W. S. Gan and C. H. Chen Spatial sound reprodution using onventional and parametri loudspeakers in Pro. APSIPA ASC Hollywood CA. [] M. R. Bai and G. Y. Shih Upmixing and downmixing two-hannel stereo audio for onsumer eletronis IEEE Trans. Consumer Eletron. vol. 3 no. 3 pp. -9 Aug. 7. [3] M. A. Gerzon Optimal reprodution matriies for multispeaker stereo J. Audio Eng. So. vol. 4 no. 7/8 pp Jul./Aug. 99. [4] Re. ITU-R BS.77 Multi-Channel Stereophoni Sound System with or without Aompanying Piture ITU 993 Available: [] D. R. Begault 3-D Sound for Virtual Reality and Multimedia Cambridge MA: Aademi Press 994. [6] J. Breebaart and E. Shuijers Phantom materialization: a novel method to enhane stereo audio reprodution on headphones IEEE Trans. Audio Speeh Lang. Proess. vol.6 no. 8 pp. 3 Nov. 8. [7] M. M. Goodwin and J. M. Jot Binaural 3-D audio rendering based on spatial audio sene oding in Pro. 3rd Audio Eng. So. Conv. New York 7. [8] S. K. Zielinski and F. Rumsey Effets of down-mix algorithms on quality of surround sound J. Audio Eng. So. vol. no. 9 pp Sep. 3. [9] J. Breebaart and C. Faller Spatial audio proessing: MPEG surround and other appliations. Chihester UK: John Wiley & Sons 7. [] C. Faller and F. Baumgarte Binaural ue oding part II: shemes and appliations IEEE Trans. Speeh Audio Proess. vol. no. 6 pp. 3 Nov. 3. [] C. Faller Coding of spatial audio ompatible with different playbak formats in 7th Audio Eng. So. Conv. San Franiso CA Ot. 4. [] M. M. Goodwin and J. M. Jot Primary-ambient signal deomposition and vetor-based loalization for spatial audio oding and enhanement in Pro. ICASSP Hawaii 7 pp. 9-. [3] N. Stefanakis and A. Mouhtaris Foreground suppression for apturing and reprodution of rowded aousti environments in Pro. ICASSP Brisbane Australia pp.. [4] M. M. Goodwin and J. M. Jot Spatial audio sene oding in Pro. th Audio Eng. So. Conv. San Franiso 8. [] J. He E. L. Tan and W. S. Gan A study on the frequeny-domain primary-ambient extration for stereo audio signals in Pro. ICASSP Florene Italy 4 pp [6] K. Sunder J. He E. L. Tan and W. S. Gan Natural sound rendering for headphones: integration of signal proessing tehniques IEEE Signal Proessing Magazine vol. 3. no. pp. -3 Mar.. [7] K. Kowalzyk O. Thiergart M. Taseska G.. Del Galdo V. Pulkki and E. A. P. Habets Parametri spatial sound proessing IEEE Signal Proess. Magazine vol. 3 no. Mar pp [8] C. Avendano and J. M. Jot A frequeny-domain approah to multihannel upmix J. Audio Eng. So. vol. no. 7/8 pp Jul./Aug. 4. [9] F. Menzer and C. Faller Stereo-to-binaural onversion using interaural oherene mathing in Pro. 8th Audio Eng. So. Conv. London UK. [3] T. Holman Surround sound up and running nd ed. MA: Foal Press 8. [3] J. M. Jot J. Merimaa M. M. Goodwin A. Krishnaswamy and J. Larohe Spatial audio sene oding in a universal two-hannel 3-D stereo format in 3rd Audio Eng. So. Conv. New York NY Ot. 7. [3] V. Pulkki Spatial sound reprodution with diretional audio oding J. Audio Eng. So. vol. no. 6 pp. 36 Jun. 7. [33] F. Rumsey Time-frequeny proessing for spatial audio J. Audio Eng. So. vol. 8 no. 7/8 pp Jul./Aug.. [34] S. Y. Park S. Lee and D. Youn Robust representation of spatial sound in stereo-to-multihannel upmix in Pro. 8th Audio Eng. So. Conv. London UK. [3] J. Usher and J. Benesty Enhanement of spatial sound quality: a new reverberation-extration audio upmixer IEEE Trans. Audio Speeh Lang. Proess. vol. no. 7 pp. 4 Sep. 7. [36] C. Faller Matrix surround revisited in Audio Eng. So. 3 th Int. Conf. Saariselka Finland Mar. 7. [37] C. Faller and J. Breebaart Binaural reprodution of stereo signals using upmixing and diffuse rendering in Pro. 3th Audio Eng. So. Conv. New York. [38] M. M. Goodwin and J. M. Jot Analysis and synthesis for universal spatial audio oding in st Audio Eng. So. Conv. San Franiso CA Ot. 6. [39] J. M. Jot V. Larher and J. M. Pernaux A Comparative Study of 3-D Audio Enoding and Rendering Tehniques in Pro. 6th Audio Eng. So. Int. Conf. Rovaniemi Finland 999. [4] C. Faller Multiple-loudspeaker playbak of stereo signals J. Audio Eng. So. vol. 4 no. pp. -64 Nov. 6. [4] T. Lee Y. Baek Y. C. Park and D. H. Youn Stereo upmix-based binaural auralization for mobile devies IEEE Trans. Consum. Eletron. vol. 6 no. 3 pp.4-49 Aug. 4. [4] J. Thompson B. Smith A. Warner and J. M. Jot Diret-diffuse deomposition of multihannel signals using a system of pair-wise orrelations in Pro. 33rd Audio Eng. So. Conv. San Franiso. [43] A. Walther and C. Faller Diret-ambient deomposition and upmix of surround signals in Pro. IWASPAA New Paltz NY Ot. pp [44] H. Chung S. B. Chon and S. Kim Flexible audio rendering for arbitrary input and output layouts in Pro. 37 th AES Conv. Los Angeles CA Ot. 4. [4] J. Merimaa M. M. Goodwin J. M. Jot Correlation-based ambiene extration from stereo reordings in 3rd Audio Eng. So. Conv. New York Ot. 7. [46] J. He E. L. Tan and W. S. Gan Linear estimation based primary-ambient extration for stereo audio signals IEEE/ACM Trans. Audio Speeh Lang. Proess. vol. no. pp. 7 Feb. 4. [47] M. Goodwin Geometri signal deompositions for spatial audio enhanement in Pro. ICASSP Las Vegas 8 pp [48] R. Irwan and R. M. Aarts Two-to-five hannel sound proessing J. Audio Eng. So. vol. no. pp Nov.. [49] Y. H. Baek S. W. Jeon Y. C. Park and S. Lee Effiient primary-ambient deomposition algorithm for audio upmix in Pro. 33rd Audio Eng. So. Conv. San Franiso. [] M. Briand D. Virette and N. Martin Parametri representation of multihannel audio based on prinipal omponent analysis in Pro. th Audio Eng. So. Conv. Paris 6. [] J. Se-Woon H. Dongil S. Jeongil P. Young-Cheol and Y. Dae-Hee Enhanement of prinipal to ambient energy ratio for PCA-based parametri audio oding in Pro. ICASSP Dallas pp [] D. Shi R. Hu W. Tu X. Zheng J. Jiang and S. Wang Enhaned prinipal omponent using polar oordinate PCA for stereo audio oding in Pro. ICME Melbourne Australia pp [3] J. He E. L. Tan and W. S. Gan Time-shifted prinipal omponent analysis based ue extration for stereo audio signals in Pro. ICASSP Vanouver Canada 3 pp [4] C. Uhle and E. A. P. Habets Diret-ambient deomposition using parametri wiener filtering with spatial ue ontrol in Pro. ICASSP Brisbane Australia pp

15 [] A. Härmä Classifiation of time-frequeny regions in stereo audio J. Audio Eng. So. vol. 9 no. pp Ot.. [6] J. He and W. S. Gan Multi-shift prinipal omponent analysis based primary omponent extration for spatial audio reprodution in Pro. ICASSP Brisbane Australia pp [7] C. Uhle A. Walther O. Hellmuth and J. Herre Ambiene separation from mono reordings using non-negative matrix fatorization in Pro. 3th Audio Eng. So. Int. Conf. Saariselka Finland 7. [8] H. Sawada S. Araki R. Mukai and S. Makino Blind extration of dominant target soures using ICA and time frequeny masking IEEE Trans. Audio Speeh Lang. Proess. vol. 4 no. 6 pp Nov. 6. [9] J. He W. S. Gan and E. L. Tan Primary-ambient extration using ambient phase estimation with a sparsity onstraint IEEE Signal Proess. Letters vol. no. 8 pp. 7-3 Aug.. [6] M. Plumbley T. Blumensath L. Daudet R. Gribonval and M. E. Davies Sparse representation in audio and musi: from oding to soure separation Pro. IEEE vol. 98 no. 6 pp Jun.. [6] J. Blauert Spatial hearing: the psyhophysis of human sound loalization. Cambridge MA: MIT Press 997. [6] O. Yilmaz and S. Rihard Blind separation of speeh mixtures via time-frequeny masking IEEE Trans. Sig. Proess. vol. no. 7 pp Jul. 4. [63] K. Kinoshita M. Delroix T. Nakatani and M. Miyoshi Suppression of late reverberation effet on speeh signal using long-term multiple step linear predition IEEE Trans. Audio Speeh Lang. Proess. vol. 7 no. 4 pp May. 9. [64] E. A. P. Habets S. Gannot and I. Cohen Late reverberant spetral variane estimation based on a statistial model IEEE Signal Proess. Letters vol. 6 no. 9 pp Sep. 9. [6] F. Rumsey Controlled subjetive assessments of two-to-five hannel surround sound proessing algorithms J. Audio Eng. So. vol. 47 no. 7/8 pp Jul./Aug [66] M. Shroeder An Artifiial Stereophoni Effet Obtained from a Single Audio Signal J. Audio Eng. So. vol. 6 no. pp Feb [67] G. Potard I. Burnett Deorrelation tehniques for the rendering of apparent sound soure width in 3D audio displays in Pro. DAFx 4 Naples Italy Ot. 4. [68] G. Kendall The deorrelation of audio signals and its impat on spatial imagery Computer Musi Journal vol. 9 no. 4 pp [69] F. Menzer and C. Faller Binaural reverberation using a modified Jot reverberator with frequeny-dependent interaural oherene mathing in 6th Audio Eng. So. Conv. Munih Germany May 9. [7] P. J. V. Laarhoven and E. H. Aarts Simulated annealing Netherlands: Springer 987. [7] ITU ITU-R Reommendation BS.34-: Method for the subjetive assessment of intermediate quality levels of oding systems 3. [7] J. Liebetrau F. Nagel N. Zaharov K. Watanabe C. Colomes P. Crum T. Sporer and A. Mason Revision of Re. ITU-R BS. 34 in Pro. 37th AES onvention LA Ot 4. [73] V. Emiya E. Vinent N. Harlander and V. Hohmann subjetive and objetive quality assessment of audio soure separation IEEE Trans. Audio Speeh Lang. Proess. vol. 9 no. 7 pp. 467 Sept.. [74] J. He. ( Jan. 4). Ambient Spetrum estimation ASE [online]. Available: [7] E. Vinent N. Bertin R. Gribonval and F. Bimbot From blind to guided audio soure separation IEEE Signal Proess. Magazine vol. 3 no. 3 pp Jianjun He (S ) reeived his B.ENG. degree in automation from Nanjing University of Posts and Teleommuniations China in and is urrently pursuing his Ph.D. degree in eletrial and eletroni engineering at Nanyang Tehnologial University (NTU) Singapore. In he was working as a general assistant in Nanjing International Center of Entrepreneurs (NICE) building platforms for start-ups from oversea Chinese sholars in Jiangning Distrit Nanjing China. Sine he has been a projet offier with Shool of Eletrial and Eletroni Engineering in NTU. His Ph.D. work has been published in IEEE Signal Proessing Magazine IEEE/ACM Transations on Audio Speeh and Language Proessing (TASLP) IEEE Signal Proessing Letters and ICASSP et. He has been an ative reviewer for various Journals and onferenes inluding IEEE TASLP Journal of Audio Engineering Soiety et. Aiming at improving humans listening his researh interests inlude audio and aousti signal proessing 3D audio (spatial audio) psyhoaoustis ative noise ontrol soure separation and emerging audio and speeh appliations. Currently He is a student member of the IEEE and Signal Proessing Soiety (SPS) a member of APSIPA and an affiliate member of IEEE SPS audio and aousti tehnial ommittee. Ee-Leng Tan reeived his BEng (st Class Hons) and PhD degrees in Eletrial and Eletroni Engineering from Nanyang Tehnologial University in 3 and respetively. His researh interests inlude image/audio proessing and real-time digital signal proessing. To date his work has been awarded three patents in Japan Singapore and US. He urrently holds the position of a Chief Siene Offier leading the researh and development of the tehnologial ompany Beijing Sesame World Co. Ltd. Conurrently Dr Tan onsults as the tehnial advisor for several start-ups. Woon-Seng Gan (M 93-SM ) reeived his BEng (st Class Hons) and PhD degrees both in Eletrial and Eletroni Engineering from the University of Strathlyde UK in 989 and 993 respetively. He is urrently an Assoiate Professor in the Shool of Eletrial and Eletroni Engineering in Nanyang Tehnologial University. His researh interests span a wide and related areas of adaptive signal proessing ative noise ontrol and spatial audio. He has published more than international refereed journals and onferenes and has granted seven Singapore/US patents. He had o-authored three books on Digital Signal Proessors: Arhitetures Implementations and Appliations (Prentie Hall ) Embedded Signal Proessing with the Miro Signal Arhiteture (Wiley-IEEE 7) and Subband Adaptive Filtering: Theory and Implementation (John Wiley 9). He is urrently a Fellow of the Audio Engineering Soiety(AES) a Fellow of the Institute of Engineering and Tehnology(IET) a Senior Member of the IEEE and a Professional Engineer of Singapore. He is also an Assoiate Tehnial Editor of the Journal of Audio Engineering Soiety (JAES); Assoiate Editor of the IEEE Transations on Audio Speeh and Language Proessing (ASLP); Editorial member of the Asia Paifi Signal and Information Proessing Assoiation (APSIPA) Transations on Signal and Information Proessing; and Assoiate Editor of the EURASIP Journal on Audio Speeh and Musi Proessing. He is urrently a member of the Board of Governor of APSIPA.