EUSIPCO

Transcription

1 EUSIPCO STABLE TIME-FREQUENCY CONTOURS FOR SPARSE SIGNAL REPRESENTATION Yoonseo Lim 1, Brr Shinn-Cunninghm 2, nd Timothy J. Grdner 3 Dept. Cognitive nd Neurl Systems 1, Biomedil Eng. 2, Biology 3, Boston University Boston, MA 221 USA ABSTRACT Mny signls nnot e resolved in time nd frequeny with single time-sle of nlysis nd multi-nd representtions re needed tht n dpt to the lol signl ontent. Using newly developed ontour-sed representtion of signls, we show tht effiient multi-nd representtions rise when long-rnge, struturlly stle shpes re enhned reltive to kground. For the exmples provided here, resolution in time nd frequeny is distriuted dptively so tht eh omponent of signl is represented in its most prsimonious form. The resulting representtion is hrterized y simple shpes in the timefrequeny plne. Index Terms Time-frequeny nlysis, dptive filtering, ressignment nd sprse representtion 1. INTRODUCTION Sprse time-frequeny methods typilly serh for liner deomposition of signls through miniml numer of ditionry elements [1]. The ditionry elements re drwn from n over-omplete set, whih my e defined -priori or dpted to speifi stimulus lss [2]. Numerous itertive ssemly proesses re effetive, ut roust methods for single-pss dptive time-frequeny representtions remin elusive, though numerous promising diretions hve een proposed [3]-[]. The strting point of these nd other timefrequeny representtions is the preltion of the timefrequeny plne into isolted toms of energy with no intrinsi ssoitions mong them. From this sis, the struture of long-rnge shpes in time nd frequeny nnot esily guide dptive lgorithms, lthough mny signls re nturlly represented y oherent long-rnge forms, suh s ontours. Reently, generl time-frequeny method ws desried whose elementry units re ontours of vrying shpes. These shpes fully represent ny signl, ut the detils of the shpes depend on the signl ontent nd on the time-sles of nlysis. Eh ontour in the representtion is oherent ojet - omponent of the signl whose oundries re defined y region of the Gor trnsform tht ontins no nlyti zeros [7]. The ontours n e interpreted s the miniml oherent units of the signl from the perspetive of the nlyti Gor trnsform. Their sles re typilly muh lrger thn the resolution limit of the nlysis. Using this ontour representtion, prior study demonstrted how mesures of ontour omplexity ould e used to optimize the time-sle of nlysis, on verge, for n entire signl [7]. The impliit ssumption in tht work ws tht prsimonious representtions would involve ontours of low urvture. The present work is motivted y the desire to define more generl priniple for dptive time-frequeny nlysis sed on time-frequeny ontours. The priniple is s follows: when signl omponent is nlyzed in its own nturl time-sle, then the ontours tht represent the omponent re struturlly stle - the detils of the shpes do not hnge with smll vritions in the prmeters of nlysis. This hypothesis does not presume tht ontours should e simple in form, ut only tht they e struturlly stle. A proess tht enhnes struturlly stle shpes provides sprse multi-sle representtion of omplex signls. In the following, we outline the theory, nd provide few exmples. 2. BACKGROUND The ontour desription of sound [7] is sed on generliztion of the ressignment proess []. This involves the Gor trnsform, ( χ ) nd the ssoited trnsform ( η) sed on window shpe tht is the derivtive of gussin: ( ) = e t τ χ t,ω η( t,ω ) = 1 σ 2 t ( )2 2σ t 2e ( ) x( τ )dτ iω t τ iφ t,ω = χ ( t,ω ) e ( ) (1) ( τ t)e ( t τ )2 2σ t 2e iω ( t τ ) x ( τ )dτ (2) These trnsforms re pplied to the ousti signl of interest, x(t), whih is funtion of time ( t), to produe representtion tht is funtion of oth time nd frequeny ( ω ). In this expression, σ t defines the time-sle of the nlysis window, therefore the resolution of the nlysis. Contours edges re equivlent to the fixed points of the time-frequeny ressignment proess, sujet to the onstrint tht ressignment moves long fixed ngle. By 1

2 -4 o o 4 o 9.ms 2ms M i. M j M j M i 7 Fig. 1. The struturl stility of ontour shpes n guide n dptive time-frequeny nlysis. In this exmple, the signl omponents (two frequeny sweeps) re seprle when 2ms filters re used in the Gor trnsform. Contours re lulted for three time sles (rows) nd three ngles (olumns.) At the optimum time-sle (middle row), ontour shpes re roust to vritions in the ngle of nlysis. Contour energy is drwn from nd illustrted in hot olor sle. nother definition, these points re sttionry phse points for the resynthesis integrl tht produes the originl signl from the Gor trnsform [7]. Contour edges re defined y: I ((η χ)e iθ ) = (3) where θ defines ontour preferene ngle in the timefrequeny plne nd I( f ) is the imginry omponent of f. Intuitively, (η χ)e iθ is n pproximtion to the derivtive of the Gor trnsform long speifi ngle( θ ) in the time-frequeny plne - losely relted to heuristi expressions for spetrl derivtives sed on multi-tper spetrl nlysis[9],[1]. The points tht stisfy (3) form extended losed loops in the time-frequeny plne tht follow the ridges, vlleys nd sddle points of χ. To divide the ontours into oherent units, ontours re segmented whenever they ross zeros of the Gor trnsform [7]. It is possile to nlytilly define wveform for eh ontour suh tht the sum of ll wveforms equls the originl signl [7]. In ll imges shown here, the olor sle for eh ontour is equl to the lol vlue of χ. 3. PARSING COMPLEX SOUNDS USING MULTIPLE TIME-SCALES For every time-sle nd ngle of nlysis, distint ojetsed deomposition exists. Every hoie of time-sle σ t nd ngle θ genertes its own ontour representtion nd ssoited territories - n over-omplete fmily of vlid ontour representtions, eh of whih fully ptures the signl ontent (Fig. 1 in [7]). The omplexity nd struturl stility of the ontour shpes depend on how well the ngle nd time-sle prmeters re mthed to the signl ontent. Fig. 1 illustrtes ontour shpes derived for simple signl, nlyzed with multiple hoies of time-sle nd ngle. The signl onsists of two losely sped, prllel frequeny sweeps. In this figure, rows represent nlysis in different time-sles nd olumns nlysis in different ngles. 4ms 1 Fig. 2. Quntifying the struturl stility of ontours. () Contours lulted for frgment of white noise with superimposed 7kHz tone. Contours in red re lulted for rnge of ngles, t single time sle. Blue dots re mxim of the signl. Green dots re minim. The ontours tht trk the 7kHz tone re tightly undled together - inditing lol struturl stility of ontour shpe ross vritions in the nlysis prmeters. In (), two ontours re extrted from the region mrked y the white squre in (). The territories elonging to the two ontours re shded in gry-sle, nd their overlp in white. Although ontour sets from eh time-sle nd ngle produe omplete representtion of the signl, time-sle of 2 ms, for this signl, yields the simplest ontours nd the most oherent long-rnge form. The underlying priniple is simple: t the optiml time-sle, eh omponent is sped y more thn the resolution of the time-frequeny unertinty: in time nd in frequeny. Therefore, t this time-sle, the signl omponents re seprle in the timefrequeny plne. How n one utomtilly selet from the over-omplete ontour sets representtion of omplex signls where eh suomponent of signl is represented in its own nturl time-sle nd ngle? An erlier pulition suggested seleting ontours with the simplest shpes [7]. Here we suggest more generl riterion - selet struturlly stle ontours, regrdless of their shpes. When signl omponent is nlyzed in its own nturl time-sle nd ngle, then the long-rnge ontours tht represent the signl re struturlly stle - the detils of the shpes do not hnge with smll vritions in the prmeters of nlysis. Returning to Fig. 1, for exmple, one n oserve tht t the optiml time-sle, ontour shpes for the hosen signl do not depend sensitively on prmeter θ. For this simple signl, the ontours in the middle row of Fig. 1 re the struturlly stle ontours. Fig. 2 nd 3 illustrte how the struturl stility of ontours n highlight tonl signl emedded in noise. The nlysis revels quiver of similrly shped ontours tht trk the tonl omponent of the sound for n ngle ner θ. To quntify the struturl stility of ontour, we (1) lulte set of ontours for rnge of prmeters, σ t nd θ. (2) Crete sprse timefrequeny mtrix, M i representing thikened representtions of eh ontour. The mtrix for ontour is zero everywhere unless the pixel flls within neighorhood of the ontour defined y the resolution of the underlying Gor trnsform used to generte the ontour ( Δt = σ t,δf = 1 σ t ). (3) Define onsensus sore for eh ontour 2

3 9 24 στ :. mse S i = mx M i M j j 1 Fig. 3. Consensus sores enhne ontours tht follow signl rther thn noise. The undle of ontours tht trk the emedded 7kHz sinusoid stnd out from the noise in this nlysis. { } (4) This mximiztion involves mny sprse mtrix multiplitions (the omputtion is qudrti in the numer of ontours). In words- eh ontour is ssigned sore defined y its mximl overlp with ny other ontour. In Fig. 2, the light gry pixels represent the mtrix for the red ontour, the drker pixels the mtrix for the lue ontour, nd their overlp M i M j is the re of the white pixels. We ll this the onsensus sore of the ontour. This onsensus sore is not normlized y ontour length, so the soring system fvors ontours tht re oth long nd highly overlpping with some neighoring ontour. In Fig. 3, the ontours of Fig. 2 re reolored ording to their onsensus sores, proess tht highlights the signl region ontining the sinusoid. Speifilly, if the oordintes of i th ontour re represented y mtrix, C i in disrete pproximtion to χ, then Fig 3 is onsensus imge defined y CI = S i C i () Fig. 3 ws lulted using set of ontours defined in single time-sle nd mny ngles; the more generl pproh used in the susequent figures omines ontours ross vritions in oth time-sle nd ngle. A simple exmple illustrting this multi-nd pproh n e found in Fig. 4, whih demonstrtes the nlysis of lik nd tone emedded in noise. The onsensus imges in this figure re produed y the pointwise histogrm of ll ontours, weighted y their individul onsensus sores. For this signl, the onsensus imge (Fig. 4) urtely trks oth the lik nd the tone sine eh omponent is represented using informtion in its own nturl time-sle. It must e emphsized tht no -priori informtion ws pplied to this figure. The onsensus ontour nlysis lso works for omplex signls. Roughly speking, s long s signl omponents re lolly sped y distnes in time nd frequeny greter thn the spred of the time-frequeny unertinty (for some time-sle), the method will highlight these omponents y emphsizing ontours drwn from the pproprite time-sles. Fig. demonstrtes how the onsensus opertion n redue the representtion of ontours of low struturl 1 στ : 2 mse d stility in omplex signl, reveling prsimonious signl representtion. Fig. shows the stndrd spetrogrm of ird song. Fig. shows the olletion of ( C i ) of the sme ird song, where ontours στ :. mse 3 7 Fig. 4. Consensus highlights signl in noise. The nlyzed signl is frgment of white noise with n emedded sinusoid t 7kHz, nd lik t t=ms. () Contours weighted y onsensus highlight the signl omponents with high temporl preision for the lik nd high frequeny preision for the sinusoid. () Lol mplitudes of the Gor trnsform. In pnels () nd (d) individul ontours re shown for 2ms nd.ms timesle respetively. (Red, θ = π / 2 : Blk, θ = ). The lk ontour in pnel () trks the sinusoid, while the red ontour in pnel (d) trks the lik. The two ontours tht trk the signl omponents re struturlly stle nd stnd out reltive to noise in the onsensus-weighted imge in pnel (). ll ontours re lulted over nrrow rnge of relevnt time-sles. Even though the time-sles re lredy mthed to zer finh song, the summed imge is visully dense with signl omponents multiply represented in different ngles nd time-sles. Fig. shows the CI (Eq. ) for tht sme song, whih highlights the struturlly stle fetures of the dt. Fig. d further weights this imge y the lol spetrogrm power, S i C i χ i where χ i is the Gor trnsform used to lulte i th ontour. Fig. pplies the sme onsensus enhnement to the nlysis of humn speeh smple. In this nlysis, onsensus ontours t low ngles trk some of the the formnts, while onsensus ontours t steep ngles trk the glottl pulses. 4. COMPUTATIONAL METHODS AND RESYNTHESIS The nlysis desried here uses the Disrete Gor trnsform (24 frequeny ins, Signl smpling rte 2 khz or 4kHz) with window overlp of 23 smples. All mtrixes used in ontour lultions hve the sme resolution. For resynthesis, we do not synthesize ext wveforms for eh ontour s desried previously [7], ut use n overlpped inverse FFT for eh olumn of the timefrequeny onsensus imge. The purpose of the onsensus representtion is not to extly represent the originl signl ut to pture the slient fetures of signl s prsimoniously s possile - sprse pproximtion to the originl signl. The ury of the resynthesis n sle 3

4 Stndrd speeh sonogrm d Consensus ontours Extrting onsonnts nd formnts Fig.. Consensus proess n revel prsimonious representtion of signl. Input signl is short syllle of zer finh song. () Stndrd spetrogrm of signl nlyzed for time-sles.3 ~ 2.2 ms, () Colletion of ll ontours () Consensus imge of the signl weighted y onsensus sore (d) Consensus imge of signl weighted y onsensus sore nd lol power from spetrogrm. smoothly from high qulity pereptul mth to ompt, lower qulity representtions of sound, depending on the utoff in ontour onsensus sores. The onsensus imge from Fig. inorporting ontours from ll ngles provides firly omplete pereptul resynthesis of the speeh smple. Resynthesis sed on Fig. remins intelligile, sine mny of the signl formnts re ptured y this popultion of ontours. 1. Time (se) The system my rek down soon Fig.. Spetrogrm nd ontour representtion of humn speeh. () Spetrogrm of humn speeh sentene, The system my rek down soon. Gor trnsform lulted with σ t = 3 ms. () Top soring onsensus ontours lulted for time-sles 1-4.ms. () Highlighting the shllow- ngle onsensus ontours drwn from 2-4.ms time-sle. Mny of these low-ngle ontours follow the formnts, or vol trt resonnes essentil to the pereption of speeh. The onsensus imges ( CI in Fig. ) omine qulittively different forms of informtion, nd for some pplitions, these should e kept seprte. Speifilly, the onsensus sore, Si for single ontour inludes informtion out oth the struturl stility of form nd length of the ontour. Furthermore, the onsensus imge ( CI in Fig. ) is influened y ontour density t eh point in time nd frequeny. The imges tht pper to e most useful dd yet one more feture - the lol weighting of onsensus ontours y the spetrogrm power (Fig. d) For ny quntittive nlysis, sttistil understnding of the reltive ontriutions of these fetures will e needed.. CONSENSUS PROVIDES AN OBJECT-BASED SIGNAL ENHANCEMENT In the onsensus proess, ontours re never sudivided. If ontour ontriutes to the finl representtion, it does so in its entirety, even if some time-frequeny points long the ontour re not in onsensus with some other ontour. Fig. 7 revels how the notion of ontour onsensus differs from simple mesure of ontour density. For doule hirp signl, ontours re lulted in time-sles in the rnge of 2-1ms. In Fig. 7 nd 7 the time-frequeny points of highest pixelwise overlp fll etween the two sweeps. Fig. 7 ontins the result of the ontour-sed onsensus (Eq. ). The tke home messge from this figures is tht pointwise mesures of ontour density n fil to extrt prsimonious representtions. Any proess of thresholding the imges in Fig. 7 or 7 will fil to disover the prsimonious representtion in Fig. 7. To hieve the gins of the ontour-sed nlysis, the method must mplify stle ontours rther thn just stle pixels. This ontour or ojet-sed time-frequeny priniple ws sent from prior definition of ross-ndwidth onsensus [11]. The overlp of ressigned pixels in multi-nd nlysis produes figure similr to Fig. 7. Ressignment lone does not provide the gins of n ojet-sed timefrequeny nlysis, sine ressignment does not link together ssoited points in the time-frequeny plne. The onsensus sore depends on the grnulrity of the prmeter spe in time-sle (σ t ) nd ngle (θ ) tht is explored. The sore lso depends on the numer of frequeny hnnels, nd the temporl overlp or step-size in the disrete pproximtion to the nlyti Gor trnsform (spetrogrm). A prinipled pproh to this nlysis ould ompute ontour sore distriutions in noise, nd then selet signl ontours sed on their likelihood in this kground distriution. However, this noise distriution must e reomputed for the ext prmeter settings used in eh nlysis. Not only does the grnulrity of the prmeter serh ffet the results, ut soring proess sed on onsensus sores requires wkwrd deisions suh s how to rnk vertil ontour tht trks lik reltive to horizontl ontour tht trks tone (The hoie of disretiztion for the Gor trnsform influenes this reltive weighting sine it impts ontour length whih is folded into the onsensus sores).. LIMITATIONS 4

5 Fig. 7. Pixel-sed onsensus mesures fil to extrt prsimonious representtion. () Overlp of unweighted ontours lulted in timesles 2-1ms. Pixels of highest overlp fll etween the two sweeps. () Overlp of the sme ontours, weighted y the lol power of the Gor Trnsform. () Top soring ontours lulted y the onsensus mesure define here. The onsensus representtion is lossy (unless ll ontours re tken), nd wht is lost depends on the detils of omplex soring proess. Clulting ontour sets requires little time eyond the lultion of the disrete spetrogrm, ut the present method repets this lultion over two dimensionl prmeter spe nd then dds to this soring proessing involving sprse mtrix multiplition tht is qudrti in the numer of ontours. A signifint dvne would emody the priniple of the ontour stility nlysis in simpler proess tht ws less omputtionlly intensive. 7. DISCUSSION The sis of the ontour representtion is the oservtion tht ny signl n e represented s olletion of ontours in the time-frequeny plne with ssoited simple wveforms. For spetrlly dense signls suh s white noise, lol ontour shpes hnge quikly with smll hnges in the time-sle or ngle of nlysis. However, signl omponents tht re seprle from the kground n produe ontour shpes tht re stle to hnges in the prmeters of nlysis. By emphsizing long, struturlly stle ontours, prsimonious signl representtions n e found where seprte omponents re represented in their own nturl time-sles nd ngles of nlysis. The result is not just n dptive time-frequeny nlysis, ut provides n elementry form of strem segregtion sine ontours tht survive the winnowing proess n e ressemled element y element to pture hosen fetures of the originl signl. The vertil nd horizontl omponents of the signl in Fig. 4 n e seprtely resynthesized y tking only onsensus ontours from vertil or horizontl ngles. Similrly, mny of the formnts of the speeh smple re seprted from glottl pulses y tking only the low-ngle onsensus ontours (Fig. ). The method desried here is rooted in the nlyti struture of the Gor trnsform, ut the priniples will generlize to other trnsforms suh s the hirplet trnsform [12]. Struturlly stle forms re found when eh omponent of signl is represented in its own nturl timesle nd ngle. This priniple n guide n dptive timefrequeny nlysis, though quntittive enhmrks remin to e exmined.. ACKNOWLEDGMENT The uthors would like to thnk nonymous reviewers for omments on this mnusript. This work is supported in prt y CELEST, Ntionl Siene Foundtion Siene of Lerning Center (NSF OMA-397), nd y Creer Awrd t the Sientifi Interfe to T.G. from the Burroughs Wellome Fund nd Smith fmily wrd to T.G. 9. REFERENCES [1] S. G. Mllt nd Z. Zhng, Mthing pursuits with time-frequeny ditionries, Signl Proessing, IEEE Trnstions on, vol. 41, no. 12, pp , [2] M. S. Lewiki, Effiient oding of nturl sounds, Nture Neurosiene, vol., no. 4, pp. 3 33, 22. [3] R. R. Coifmn nd M. V. Wikerhuser, Entropy-sed lgorithms for est sis seletion, Informtion Theory, IEEE Trnstions on, vol. 3, no. 2, pp , [4] D. Jones nd R. G. Brniuk, A simple sheme for dpting time-frequeny representtions, Signl Proessing, IEEE Trnstions on, vol. 42, no. 12, pp , [] D. Jones nd T. Prks, A high resolution dt-dptive time-frequeny representtion, Aoustis, Speeh nd Signl Proessing, IEEE Trnstions on, vol. 3, no. 12, pp , 199. [] D. Rudoy, P. Bsu, nd P. J. Wolfe, Superposition Frmes for Adptive Time-Frequeny Anlysis nd Fst Reonstrution, Signl Proessing, IEEE Trnstions on, vol., no., pp , 21. [7] Y. Lim, B. Shinn-Cunninghm, nd T. J. Grdner, Sprse ontour representtions of sound, IEEE Signl Proessing Letters, vol. 19, no. 1, pp. 4 7, Ot [] F. Auger nd P. Flndrin, Improving the redility of time-frequeny nd time-sle representtions y the ressignment method, Signl Proessing, IEEE Trnstions on, vol. 43, no., pp. 1 19, 199. [9] P. Mitr nd H. Bokil, Oserved Brin Dynmis. Oxford University Press, USA, 27. [1] D. J. Thomson, Spetrum estimtion nd hrmoni nlysis, presented t the Proeedings of the IEEE, 192, vol. 7, no. 9, pp [11] T. J. Grdner nd M. O. Mgnso, Sprse timefrequeny representtions, Proeedings of the Ntionl Ademy of Sienes of the United Sttes of Ameri, vol. 13, no. 1, pp , 2. [12] M. Aoi, Y. Lim, U. Eden nd T. J. Grdner, Ojetsed spetro-temporl nlysis of uditory signls, Computtionl nd Systems Neurosiene (COSYNE), Slt Lke City, 213