Extraction and Representation of Features, Spring Lecture 5: Speech and Audio Analysis. Zheng-Hua Tan

Transcription

1 Extractio ad Rrstatio of Faturs, Srig 0 Lctur 5: Sch ad Audio Aalysis Zhg-Hua Ta Multimdia Iformatio ad Sigal Procssig Dartmt of Elctroic Systms Aalborg Uivrsity, Dmar t@s.aau.d Extractio of Faturs, V, Zhg-Hua Ta Fatur xtractio A scial form of dimsioality rductio, usd wh th iut data is Too larg to b stord or rocssd Rdudat much data, but ot much iformatio Data is trasformd ito a comact rrstatio a st of faturs. Sch ad audio sigals hav a lot i commo. Extractio of Faturs, V, Zhg-Hua Ta

2 Sch aalysis Most alicatios of sch rocssig must xloit th rortis of sch sigals Sch Aalysis: th rocss of xtractig such rortis from a sch sigal. sch sch aalysis DSP rrstatio of sch alicatios:.g. itch, formats, boudary dtctio, sch & sar rcogitio Extractio of Faturs, V, Zhg-Hua Ta 3 Sch aalysis: Short-tim aalysis Short-tim sch aalysis Tim-domai rocssig Frqucy-domai sctral rocssig Liar rdictiv codig LPC aalysis Cstral aalysis Filtr ba aalysis Extractio of Faturs, V, Zhg-Hua Ta 4

3 Prortis of sch sigals Sch is a tim-varyig sigal: xcitatio itch amlitud Extractio of Faturs, V, Zhg-Hua Ta 5 Short-tim rocssig solutio Assumig that sch has o-tim-varyig rortis fixd xcitatio ad vocal tract withi short itrvals Procssig short sgmts frams of th sch sigal ach tim f x, m x m w m Extractio of Faturs, V, Zhg-Hua Ta 6 3

4 Fram-by-fram rocssig Frams oft ovrla o aothr Th fram-basd aalysis yilds a timvaryig squc as a w rrstatio of th sch sigal samls at 8000/sc vctors at 00/sc Extractio of Faturs, V, Zhg-Hua Ta 7 Widows Rctagular widow w[ ], 0 N Hammig widow w[ ] cos, 0 N N Extractio of Faturs, V, Zhg-Hua Ta 8 4

5 Choic of widow Widow ty Badwidth of Hammig widow is about twic th badwidth of Rctagular Attuatio of mor tha 40dB for Hammig as comard with 4 db for Rctagular, outsid assbad Widow duratio - N Icras N = dcras widow badwidth N should b largr tha a itch riod, but smallr tha a soud duratio Extractio of Faturs, V, Zhg-Hua Ta 9 Sch aalysis: Tim-domai Short-tim sch aalysis Tim-domai sch rocssig Frqucy-domai sctral rocssig Liar rdictiv codig LPC aalysis Cstral aalysis Filtr ba aalysis Extractio of Faturs, V, Zhg-Hua Ta 0 5

6 Tim-domai aramtrs Short-tim rgy Short-tim avrag magitud Short-tim ro crossig rat Short-tim autocorrlatio Short-tim avrag magitud diffrc Extractio of Faturs, V, Zhg-Hua Ta Short-tim rgy Th log trm rgy dfiitio is ot usful for tim-varyig sigals m E x m Short-tim rgy of wightd sigal aroud is dfid as E m [ x m w m] Extractio of Faturs, V, Zhg-Hua Ta 6

7 Examls of short-tim rgy It ca b usd to dtctio voicd/uvoicd/silc Effcts of widow ty, duratio N badwidth ad why? Uttrd by a mal sar. Two lots covrg as N icrass. Extractio of Faturs, V, Zhg-Hua Ta 3 Short-tim magitud Lss ssitiv to larg sigal lvls as comard to rgy whr x trms is usd. M x m w m m Extractio of Faturs, V, Zhg-Hua Ta 4 7

8 Short-tim avrag ro-crossig rat A ro-crossig occurs if succssiv samls hav diffrt algbraic sigs. It is a masur of th frqucy. Dfiitio whr ad m Z sg[ x m] sg[ x m ] w m x 0 sg[ x ] x 0 0 N w N 0 othrwis Extractio of Faturs, V, Zhg-Hua Ta 5 Zro-crossig rat distributios A histogram of avrag ro-crossig rats avragd ovr 0 msc for both voicd ad uvoicd sch I diffrt frqucy bads 80//0ms=4H Extractio of Faturs, V, Zhg-Hua Ta 6 8

9 Examl of ro-crossig rat Although th ro-crossig rat varis cosidrably, th voicd ad uvoicd rgios ar quit romit. Extractio of Faturs, V, Zhg-Hua Ta 7 Short-tim autocorrlatio fuctio Th autocorrlatio fuctio m x m x m Th short-tim autocorrlatio fuctio R x m w m x m w m m Extractio of Faturs, V, Zhg-Hua Ta 8 9

10 Alicatios Boudary dtctio short-tim rgy ro crossig rat Pitch stimatio short-tim autocorrlatio fuctio Extractio of Faturs, V, Zhg-Hua Ta 9 Sch aalysis: Frqucy-domai Short-tim sch aalysis Tim-domai sch rocssig Frqucy-domai sctral rocssig Liar rdictiv codig LPC aalysis Cstral aalysis Filtr ba aalysis Extractio of Faturs, V, Zhg-Hua Ta 0 0

11 Extractio of Faturs, V, Zhg-Hua Ta Discrt-tim Fourir trasform Covolutio ad multilicatio duality: dw X x x X jw jw jw jw ] [ ] [ ] [ ]* [ ] [ jw jw jw H X Y h x y d X W Y w x y w j j jw ] [ ] [ ] [ Extractio of Faturs, V, Zhg-Hua Ta Short-tim Fourir trasform It is motivatd by th d for a sctral rrstatio to rflct th tim-varyig rortis of th sch wavform m jwm jw m x m w X ] [ ] [

12 Sctra Extractio of Faturs, V, Zhg-Hua Ta 3 Sctra of voicd/uvoicd souds Extractio of Faturs, V, Zhg-Hua Ta 4

13 Sctrogram Sctrogram two-dimsioal wavform amlitud/tim is covrtd ito a thr-dimsioal attr amlitud/frqucy/tim Widbad sctrogram: aalyd o 5ms sctios of wavform with a st of ms voicd rgios with vrtical striatios du to th riodicity of th tim wavform ach vrtical li rrsts a uls of vocal folds whil uvoicd rgios ar solid/radom, or sowy Narrowbad sctrogram: o 50ms itch for voicd itrvals i horiotal lis Extractio of Faturs, V, Zhg-Hua Ta 5 Wid- ad arrow-bad sctrograms wavform F3 F Widbad sctrogram F arrowbad sctrogram Extractio of Faturs, V, Zhg-Hua Ta 6 3

14 Sch aalysis: LPC aalysis Short-tim sch aalysis Tim-domai sch rocssig Frqucy-domai sctral rocssig Liar rdictiv codig LPC aalysis Cstral aalysis Filtr ba aalysis Extractio of Faturs, V, Zhg-Hua Ta 7 Discrt-tim filtr modl for sch Its hilosohy is rlatd to th sch modl i which sch is modlld as th outut of a liar, timvaryig systm xcitd by ithr quasi-riodic ulss or radom ois. Th LPC rovids a robust ad accurat mthod for stimatig th aramtrs of th tim-varyig systm. H Extractio of Faturs, V, Zhg-Hua Ta 8 4

15 5 Extractio of Faturs, V, Zhg-Hua Ta 9 LPC aalysis For fficit codig, sch sigals ar oft modlld usig aramtrs of th vocal tract sha that grats thm. Pol-ro modl idal durig a statioary fram All-ol modl siml: a mattr of aalytical cssity q l l l a b G U S H a G U S H 0 multil ols o ro a a Extractio of Faturs, V, Zhg-Hua Ta 30 All-ol modl th LPC modl whr u is a ormalisd xcitatio ad G is th gai of th xcitatio a G U S H U a G S GU a S S Gu s a s

16 Th LPC modl Aftr xcludig th xcitatio trm, a giv sch saml at tim, s, ca b aroximatd as a liar combiatio of th ast sch samls: ~ s s s... s whr th cofficits,,..., ar assumd costat ovr th sch fram. LPC aalysis: to dtrmi a st of rdictor cofficits { } dirctly from th sch sigal. Extractio of Faturs, V, Zhg-Hua Ta 3 LPC aalysis quatios Widowd sch: Error of liar rdictor s s x s w s sˆ Mthod: miimis ma-squard rdictio rror Short-tim avrag rdictio rror E m m [ s m s m m ] Extractio of Faturs, V, Zhg-Hua Ta 3 6

17 LPC aalysis quatios cot d Fid such that is miimal E Rsultig i Dfi covariac Th E 0 for i,,..., i m m E m [ s m s m s m i s m ˆ s m Extractio of Faturs, V, Zhg-Hua Ta 33 m s m i s i, m i s m ˆ i, i,0 i,,..., m ] m Short-tim LP aalysis To solv th followig quatio for th otimum rdictor cofficits th ˆ s i,0 aˆ i, i,,..., w hav to comut i, ad th solv th rsultig st of quatios. Extractio of Faturs, V, Zhg-Hua Ta 34 7

18 Sch aalysis: Cstral aalysis Short-tim sch aalysis Tim-domai sch rocssig Frqucy-domai sctral rocssig Liar rdictiv codig LPC aalysis Cstral aalysis Filtr ba aalysis Extractio of Faturs, V, Zhg-Hua Ta 35 Homomorhic sch rocssig Agai, sch is modlld as th outut of a liar, timvaryig systm liar tim-ivariat LTI i short sg. xcitd by ithr quasi-riodic ulss or radom ois. Th roblm of sch aalysis is to stimat th aramtrs of th sch modl ad to masur thir variatios with tim. Sic th xcitatio ad imuls rsos of a LTI systm ar combid i a covolutioal mar, th roblm of sch aalysis ca also b viwd as a roblm i saratig th comots of a covolutio, calld dcovolutio. y[ ] x[ ]* h[ ] Extractio of Faturs, V, Zhg-Hua Ta 36 8

19 Homomorhic dcovolutio Covrts a covolutio ito a sum y x * h yˆ xˆ hˆ Caoic form for systm for homomorhic dcovolutio * * D * [ ] L[ ] D - * [ ] x x ˆ y ˆ y x * x x xˆ yˆ ˆ y y y ˆ Extractio of Faturs, V, Zhg-Hua Ta 37 Th charactristic systm Th charactristic systm for homomorhic dcovolutio D * [ ] * Z[ ] log[ ] Z - [ ] x X X ˆ xˆ Extractio of Faturs, V, Zhg-Hua Ta 38 9

20 Cstral aalysis Obsrvatio: taig logarithm of X, th x[ ] x[ ]* x[ ] X X X log{ X } log{ X i.., Xˆ Xˆ Xˆ xˆ [ ] xˆ [ ] ˆ x[ ] } log{ X } So, th two covolvd sigals ar additiv. Extractio of Faturs, V, Zhg-Hua Ta 39 Comlx cstrum ad ral cstrum Ral cstrum is th v art of c [] xˆ [ ] ˆ jw xˆ[ ] X log{ X c[ ] log X jw jw jw dw } jw jw dw dw comlx cstrum cstrum cstrum was coid by rvrsig th first syllabl i th word sctrum. Extractio of Faturs, V, Zhg-Hua Ta 40 0

21 Ml-frqucy cstral cofficic MFCC Extractio of Faturs, V, Zhg-Hua Ta 4 Ml-frqucy filtr ba Extractio of Faturs, V, Zhg-Hua Ta 4

22 Sch aalysis: Cstral aalysis Short-tim sch aalysis Tim-domai sch rocssig Frqucy-domai sctral rocssig Liar rdictiv codig LPC aalysis Cstral aalysis Filtr ba aalysis Extractio of Faturs, V, Zhg-Hua Ta 43 Gammato filtrs Gammato is a widly usd modl of auditory filtrs. Imuls rsos th roduct of a gamma distributio ad siusoidal to: whr f is th frqucy, φ is th has of th carrir to, a is th amlitud, is th filtr's ordr, b is th filtr's badwidth, ad t is tim. Extractio of Faturs, V, Zhg-Hua Ta 44

23 Gammato filtrs Extractio of Faturs, V, Zhg-Hua Ta 45 Summary Short-tim sch aalysis Tim-domai rocssig Frqucy-domai sctral rocssig Liar rdictiv codig LPC aalysis Cstral aalysis Filtr ba aalysis Extractio of Faturs, V, Zhg-Hua Ta 46 3