A Robust Feature Extraction Algorithm for Audio Fingerprinting

Transcription

1 A Robust Feature Extracton Algorthm for Audo Fngerprntng Janpng Chen 1, Tejun Huang 2 1 Insttute of Computng Technology, Chnese Academy of Scences, Bejng , Chna 2 Key Laboratory of Machne Percepton(Mnstry of Educaton), Pekng Unversty, Bejng , Chna {jpchen, tjhuang}@jdl.ac.cn Abstract. In ths paper, we present a new feature extracton algorthm whch can generate robust and relable feature n a fngerprnt system. Ths algorthm s referred to as weghted ASF (WASF). The feature n our algorthm s extracted based on a MPEG-7 descrptor-audo Spectrum Flatness (ASF) and Human Audtory System (HAS). It also apples several effectve flters to mprove the feature robustness and uses another MPEG-7 descrptor: Audo Sgnature (AS) to reduce the feature dmenson and ncrease the feature compactness. The smooth flter bank can effcently resst the nose dstorton n addton to some other common dstortons such as samplng rate change and ampltude normalzaton, whle the frst order nverse flter can effectvely resst the speed-change dstorton wth 90.1% dscrmnaton for the 5% speed acceleraton dstorton. Ths algorthm s tested under several audo dstortons: samplng rate change, nose addton, data compresson and speedchange and so on. For these dstortons, the WASF algorthm can get dscrmnaton more than 90%. The MFCC feature and another MPEG-7 descrptor-audo spectrum Centrod (ASC) are also consdered. Keywords: Audo fngerprntng, weghted ASF, Audo Spectrum Flatness, flter bank, nverse flter. 1 Introducton The ncreasng number of audo resources, especally n the network, and the ntensty of Intellgent Property (IP) protecton has ncreased the nterest n technques for automatc audo dentfcaton. There are two man approaches: watermarkng and fngerprntng. In the last few years, the fngerprntng technque has brought much more attenton. The audo fngerprntng technque can be used n many applcatons [1], such as fle sharng servces, broadcast montorng and so on. In dgtal rghts management (DRM) system [2], the fngerprntng technque s also urgently requred for the protecton of Intellgent Property of the owner of meda rghts. In general, a fngerprntng system needs to have the followng propertes: robustness, relablty, compactness and scalablty. The robustness ndcates that the fngerprntng system can resst varous common audo dstortons. The relablty

2 ndcates the fngerprntng system should gve contnuous rght results over a wde varety of nputs. The compactness ndcates the fngerprntng data should be small and need small storage. The scalablty ndcates the system can be not only run n large devces but also n resource-constraned devces. Recently, there are some researches on ths topc. In [1], Hatsma and Kalker calculate the energy dfference of the nter-frame and ntra-frame and convert t to bt value and then use a sequence of bts to form an audo fngerprnt. In [2], the square root of the mean energy across the tme concatenatng the standard devaton of the RMS power s used to form a fngerprnt. The MPEG-7 audo descrptors-audo Spectrum Flatness and Audo Sgnature are used to form the fngerprnt n [3]. And n [4], a two-layer OPCA technque s used to generate the nose-resstant fngerprntng. In [5], the normalzed spectral sub-band moments has been used to generate an effcent fngerprnt. Computer vson and mage process methods are also ntroduced nto the audo process n [6] [7]. For these algorthms, they are mostly amed to several dstortons and don t effcently resst the speed-change dstorton. The speedchange dstorton s referred to n [8]; t s based on the work of [1]. In ths paper, we use the weghted MPEG-7 descrptor: Audo Spectrum Flatness [3] [9] to generate our audo feature because the perceptual feature computed usng ASF can effcently characterze the audos and be robust to a varety of audo dstortons. Otherwse, we use many effectve flters to reduce dstortons, especally the nose and speed-change, and make use of two ear process functons n Human Audtory System (HAS) [10] to enhance the property of the audo data. In order to compact the fngerprnt, we use MPEG-7 descrptor -Audo Sgnature. Ths descrptor can effcently compact the data and mantan the feature robustness. The rest of ths paper s organzed as follows. After ths ntroducton, secton 2 descrbes the proposed fngerprntng extracton algorthm n detal. Secton 3 shows the expermental results. Fnally, the concluson of the work and the acknowledgment are gven n secton 4 and secton 5 respectvely. 2 Proposed Audo Fngerprntng Algorthm In ths secton, we descrbe the fngerprntng extracton algorthm of ths system. The framework s shown n Fg.1. Ths framework can be parttoned nto three parts: front-process, feature computaton and end-process. Fg.1- the framework of the feature extracton algorthm

3 2.1 Front-Process Ths step contans pre-process, framng, tme frequency transformaton and data flterng. A stereo waveform should be converted nto a mono waveform n the preprocess phase because ths proposed algorthm s amed to the mono waveforms. In order to extract robust feature from the dynamc audo data, a framng wndow should be appled to the audo waveform to obtan relatvely statc audo clps. There are some optonal wndows, such as rectangular wndow, hannng wndow, hammng wndow and blackman wndow. In our algorthm, we select hammng wndow to 2 frame the audo H ( ) cos( ), where N s the number of samples n N 1 each wndow frame such that 0 <N. We have tested several frame lengths and found that longer frame length can gve more perceptual nformaton but take more tme. In our method, we set each frame length 90ms and nter-frame overlap rate 2/3. In ths way, we can reduce the dscontnuty of the data. Usually, the overlap rate should be set larger than 1/2 to get better contnuty. Then, we apply the Dscrete Cosne Transform for each frame to generate the frequency spectrum. After the transformaton, a normalzaton process s needed; t s the combnaton of two methods as follows: Y (, X (, (1) Y(, mn Z (, max mn (2) Where X(, s the j th sample data of the th frame, and s the mean and standard devaton of the th frame respectvely, mn and max s the mnmum and maxmum data of Y(,. These two functons make the audo data from dfferent audo clps n the same range [0, 1]. In order to reduce the nose dstorton effcently, whatever whte nose or Gaussan nose, we use a smooth flter bank shown n Fg.2 to flter the data. Ths flter bank s composed of three smooth flters: a 3-pont mean flter, a 5-pont Gaussan flter and a 3-pont hammng fler. Y(, =X(, H 1 H 2 H 3 (3) Fg.2-smooth flter bank

4 From our experment, we fnd ths smooth flter bank s effcent to whte and Gaussan nose n our weghted ASF algorthm. Fg.3 shows the result of an audo segment wth 20% Gaussan nose addton dstorton and processed by the smooth flter bank. (a) Fg.3-(a) segment wth 20% Gaussan nose addton (b) audo after smooth flter bank process From Fg.3, we can see the segment wth 20% Gaussan nose addton has been smoothed and the man perceptual property s mantaned after the process of smooth flter bank. Of course, the more the number of the smooth flters n the flter bank, the smoother the audo frequency spectrum, but more local perceptual nformaton wll be weakened. Therefore, three smooth flters are enough. After the smooth flterng, we should apply the HAS ear functons. Accordng to the HAS, the functons n the nner ear and mddle ear are respectvely shown as: ) Outer ear: (b) A db ( f khz ) 2.184( f 2 f ( 3.3) 1000 f 3.6 (4) ) 6.5e 0.001( ) ) Mddle ear: A ( f ) / 20 W ( f ) 10 db (5) Where f s the frequency of each sample data n Hz. In addton, there s a scalng factor for each sample data: G L p / (6) L KN A F max ( fc) ( NF 1) 4

5 Where K s the energy compensaton coeffcent and s relatve to the wndow functon used when framng the audo, A max s the maxmum ampltude of the sample data, L p s set to 92db, and N F s the number of samples n a frame and ( f c ) vares from 0.84 to 1. So for each sample data, we get a weght as follows: WS(f)=G L W(f) (7) If the samplng rate of an audo s khz, the weght curve of a clp wth 0.09s length s shown n Fg.4: Fg.4-the weght curve generated by the HAS ear functons From the Fg.4, we can see the weght ncreases nonlnearly n the frequency range about 250Hz-2000Hz. Ths weghted operaton can enhance the perceptual property of ths senstve frequency range. For the speed-change dstorton, t causes msalgnment both n the tme doman and the frequency doman [8]. Common methods cannot effcently resst ths dstorton. To resst the speed-change dstorton, we should consder the dstortons n the two domans. We fnd the all-zero frst order nverse flter s effcent to ths dstorton n our algorthm and ts z transformaton s as follows: A(z)=1+a 1 z -1 (8) Ths nverse flter can flatten the frequency response and get a good effect on sgnal-to-quantzaton-nose rato versus frequency [11]. In our experment, we set the

6 frst-order coeffcent a 1 to Ths functon can get a good result for the speedchange dstorton. However, t can weaken the effcency of the nose dstorton process. Therefore, an addtonal operaton should be appled to each audo frame to avod the dstorton possbly brought by the nverse flter. Pror to the use of nverse flter, we use a hammng-lke wndow functon to generate a weght functon ncreased by degrees, whch s represented below: 2 w ( cos( 2( N 1) 2 )), 0 N (9) Ths functon can reduce the nfluence of the data n the low frequency range and mantan the tone-lke property of the processed data. It s a better choce to balance the performance of nose addton process aganst speed-change process and stll get a good result both n the case of these two dstortons. Ths process of the hammng-lke wndow weght functon and the nverse flter can brng about some nose to the resultng audo data, so a smooth flter bank should be appled to the audo data to reduce ths dstorton. In ths smooth flter bank, we don t use the mean flter but a hammng flter, and then ths flter bank contans two 3-pont hammng flters. 2.2 Feature Computaton After the front-process, we begn to use the weghted ASF descrptor to compute the audo feature. To obtan a robust feature, we should get the most senstve part of the frequency spectrum. In our experment, we select the frequency range n 250 Hz-2000 Hz to extract the audo feature. Then the frequency spectrum of each frame s parttoned nto bands n a logarthmc spacng and these bands are not overlapped. The number of bands n each frame s defned as follows: log hfre lofre bandnum 2 ( / ) (10) octavere soluton Where hfre and lofre are the upper and lower frequency lmts of each frame, respectvely and octaveresoluton represents the logarthmc frequency resoluton wth the recommended range of 1/16 to 8 octaves, and we set t 1/4 n our algorthm. In order to reduce the data and adapt the frequency resoluton to log band, power spectrum coeffcents above the frequency of 1 khz are grouped and the average value s taken as a new value. The groupng s defned n the followng way: between frequency 1 khz and 2 khz, two consecutve power spectrum coeffcents are grouped. Then we use the weghted ASF descrptor to compute the features for each frequency band. The weghted flatness measure s defned as the rato of the

7 geometrc and the arthmetc mean of the weghted power spectrum coeffcents and shown below: WASF n n1 0 n1 1 n 0 w P w P (11) w s the weght of each power spectrum coeffcent. In our experment, we set ths weght as follows: w P n 1 Pk k0 (12) In ths way, we get a weghted ASF feature vector for each audo frame WASF,,..., ], where s the ndex of a frame and [, 0,1, N1 N s the band number of the frame. 2.3 End-Process The feature generated from one frame s not enough to dentfy a whole audo clp, so M feature vectors generated from the part 2.2 are ntegrated to compose a feature block for dentfyng an audo clp. In our algorthm, we set M to 198 whch s about 6- second length. Then we get a WASF matrx: WASF. M 0,0 1,0 1,0. 0,1 1,1 M 1,1 0, N 1 1, N 1. M 1, N 1 For the feature matrx WASF, we subtract the mean of each row m (0 <M) to make the mean of each frame feature zero n order to mantan the consstency of each frame feature. For these M frames, the resultng feature s huge. In order to reduce the data, dmenson reducton technque should be appled. We consder the MPEG-7 descrptor: Audo Sgnature. Ths descrptor uses a scalng factor to condense the audo date. Accordng to [9], ths condensaton wll not weaken the perceptual property of the audo data. Ths scalng factor s also called decmaton factor df. In

8 our experment, we set ths decmaton factor 24. Then n the WASF feature matrx, b m/ df, m s the number of frames. Then the number of blocks n the tme axs s we can get a feature matrx wth the dmenson as b N. Let S be the resultng feature matrx, then the arthmetc mean of each block s calculated as the new element n matrx S and denoted by: 1 S ( k, df df 1 0, j, 0 j<n, 0 k<b (13) Where k s the row ndex of matrx S. In the end, a normalzaton process wth functon (2) wll be appled to S, and then we obtan the fnal resultng feature whch s denoted as fngerprnt. 3 Expermental Results To evaluate the performance of the proposed algorthm, we prepared 203 musc audos, contanng pop, rock, pano, flute, country musc and so on. These source audos are all parameterzed wth Hz samplng rate, mono and 16 bts/sample. For each source audo, we make several dstortons respectvely as follows: (a) 2s slence addton, (b) 80% ampltude normalzaton, (c) samplng rate 22050Hz, (d) samplng rate 32000Hz, (e) samplng rate 44100Hz, (f) mp3 compactness, (g) 20% whte nose addton, (h) 25% Gaussan nose addton, () 20% Gaussan nose addton, ( 5% speed acceleraton, (k) free dstorton. In our experment, we get a 6-second clp begnnng from the locaton of 10s n each audo as our test clp. In ths way, we get 2233 test clps and 203 source clps n all. In addton to the weghted ASF algorthm, we also test the followng algorthms: (1) Audo Spectrum Centrod (2) MFCC We use the Eucldean dstance to match the two comparng features. We set the fngerprnt of the source clp and a dstorted clp as S and D respectvely, and then the dstance between two frames from the source and dstorted clps respectvely s defned as: ds tan ce 1 n n 1 j0 ( S(, D(, ) 2 (14) Where n s the number of frames and s the row ndex of the fngerprnt matrx. Then

9 the dstance of S and D s defned as: 1 Ds tan ce( D, S) b b 1 0 ds tan ce (15) The experment result s shown n Table.1. Methods Table.1-Experment results ASC (%) MFCC (%) WASF (%) Dstorton (a) (b) (c) (d) (e) (f) (g) (h) () ( (k) From Table.1, we can see the weghted ASF descrptor has good performance to varous dstortons. Especally, the WASF algorthm has 90.1% dscrmnaton to speed-change dstorton whle the other two algorthms have a lower dscrmnaton less than 50%. It also has 97.5% dscrmnaton to 20% Gaussan nose addton dstorton and 99.5% dscrmnaton to 20% whte nose addton and 25% Gaussan nose addton. In addton to these dstortons, the proposed algorthm has very hgh recognton rate. Of course, the dscrmnaton of the mp3 compresson dstorton of weghted ASF s a lttle lower than that of the MFCC algorthm. On the whole, however, the WASF method can effcently resst a varety of dstortons. 4 Concluson For a good fngerprntng system, the extracted feature should be robust to varous dstortons and have a good relablty property. In ths paper, the proposed algorthm weghted ASF s amed for ths purpose. From the experment results, we can see that the proposed algorthm has over 90% dscrmnaton rate to the ten dstortons. Contrary to other algorthms, ths proposed algorthm has better performance to many dstortons than that of other algorthms. The next work we wll do s to apply much

10 more test clps. The sze of the resultng fngerprnt s another ssue we should pay attenton to. 5 Acknowledgment Ths work was supported by the Natonal Key Technology R&D Program [2006BAH02A10 and 2006BAH02A13] of Chna. References 1. J.Hatsma and T.Kalker, A hghly robust audo fngerprntng system, Proc. Int. Conf. Musc Informaton Retreval, Vdya Venkatachalam, Luca Cazzant, Navdeep Dhllon and Maxwell Wells, Automatc Identfcaton of Sound Recordngs, Sgnal Processng Magazne, IEEE, M.Sert, B.Baykal and A.Yazc, A Robust and Tme-Effcent Fngerprntng Model for Muscal Audo, IEEE Tenth Internatonal Symposum, Chrstopher J.C.Burges, John C.Platt and Soumya Jana, Dstorton Dscrmnant Analyss for Audo Fngerprntng, IEEE Transacton on speech and processngs, Vol.11, No.3, May, Jn S.Seo, Mnho Jn, Sunl Lee, DalWon Jang, Seungjae Lee and Chang D.Yoo, Audo Fngerprntng Based on Normalzed Spectral Subband Moments, Sgnal Processng Letters, IEEE, Yan Ke, Derek Hoem, Rahul Sukthankar. Computer Vson for Musc Identfcaton. Processngs of Computer Vson pattern Recognton, Shumeet Baluja, Mchele Covell. Audo Fngerprntng: Combnng Computer Vson & Data Stream Processng. ICASSP, J.Hatsma and T.Kalker, Speed-change resstant audo fngerprntng usng auto-correlaton, Acoustcs, Speech and Sgnal Processng, ISO/IEC FDIS :2001(E), Informaton Technology-Multmeda Content Descrpton Interface-Part 4: Audo. 10. P.Kabal, An Examnaton and Interpretaton of ITU-R BS.1387: Perceptual Evaluaton of Audo Qualty, McGll Unversty, John D.Markel, Dgtal Inverse Flterng-A New Tool for Formant Trajectory Estmaton, Audo and Electroacoustcs, IEEE transactons, June, 1972.