Speaker Diary Compilation by Dependent Combination of Audio Coefficients

Transcription

1 HASAN ALGOTIR_KADHI et al: SPEAKER DIARY COPILATION BY DEPENDENT COBINATION O... Speaker Diary Copilation by Dependent Cobination o Audio Coeicients Hasan Algotir_Kadhi 1,,*, Lok Woo 1 and Satna Dlay 1 1 Electrical and Electronic Engineering School, Newcastle University, Newcastle-Upon-Tyne, UK Electrical Engineering Departent, College o Engineering, Al-ustansiriyah University, Baghdad, Iraq Abstract The paper describes a novel ethod that iprovises the procedure or supervised speaker diary copilation. The procedure supposes that the database o the speakers is available. Initially, the database and observation signal o the speakers, are prepared. The audio eatures have been extracted ro the database and the observation signal. Instead o using one o el requency Cepstral Coeicient, Perceptual Linear Prediction, or Power Noralized Cepstral Coeicients, a cobination o all o the have been used. The cobination or o these eatures is independent, i.e. they are concatenated in the eature atrix. The coparison between eatures o observation signal and statistical properties o database eatures, has been ade. A coparing procedure is used to ake the decision o the logical ask or coparison. Both o botto-up and top-down scenarios collaborate to coplete the last decisions successully. Diary copilation Error Rate test denotes that cobination o eatures has less errors than any one alone. Keywords - Speaker diary copliation; Segentation; Clustering; el eature Cepstral Coeicient; Power Noralized Cepstral Coeicient; Perceptual Linear Predictive. I. INTRODUCTION Speaker Diary copilation is speech signal processing dealing with the dialogue conversation aong persons. Although ore than one speaker participate on the dialogue, the dialogue orat is the talking o only one person alone at any speciic tie. This description is the ideal orat o a dialogue, but real orat o a dialogue contains an overlapping periods o ixture speech between ore than one talkers. In addition to dialogue and ixture speech, the conversations ay contain background noise and unwanted signals (e.g. the coughing) [1] []. ixture speech is dierent orat o conversation, where, ore than one speaker are talking siultaneously at speciic tie. Speech separation is audio and speech signal processing which deals with ixture speech. Alost, speech separation has dierent approaches to overcoe ixture proble and restore the isolated speech o each speaker [3]. When the input o diary copilation or separation is only the observation signal, the processing is unsupervised achine Learning L. They are called speaker diary copilation and Blind Speech Separation respectively. But soetie, a useul inoration can be evolved during the unsupervised L processing, then i this inoration supports the job, the L becoes sei-supervised L. In contrast, when the database and/or inoration about speech/ speaker are available initially with the observation signal, the processing is called Supervised L [4], and the above processing are called Supervised Speaker Diary copilation and Inored Speech Separation [5] respectively. ig. 1 illustrates the typical block diagra o supervised speaker diary copilation. The inputs are the speaker s database in addition to the observation dialogue speech signal (ig. line 1). The outputs (ig. lines, 3 & 4) are the isolated speech signals o each speaker alone A, B or C in addition to the errors. ig. shows illustrative tie doain o ideal inputs and outputs speech signals o speaker diary copilation process. The conversations (dialogue) are arbitrary conigured ro 3 speakers. Each one is speaking alone, either A, B or C. Speaker diary copilation consist o two ain processing, Speaker Segentation and Clustering. The task o the segentation is the inding o the switching instances ro any speaker to next speaker. or ig. 1 exaple, the inding o these instances (ro A to B, ro B to C, ro C to B, ro B to A, ro A to C and ro C to A) produces 7 segents o speech signal. The task o clustering is the assignent o the 1st, the 5th & the 7th segents to the cluster A; the assignent o the nd & the 4th segents to the cluster B; and the assignent o the 3rd & the 6th segents to the cluster C. The clusters A, B and C belong to the speakers A, B and C respectively [6]. or the speaker segentation, initially, the input signal is prepared by reoving unwanted signals, then the audio eatures (coeicients) o the input signal are extracted. PLP, CC, LCC, PNCC etc. are well-known algoriths or extracting the eatures. The atheatics can describe the statistical odel o these eatures e.g. Gaussian ixture odel and/or describe the chances o transitions ro speciic state to other using Hidden arkov odel H. Input dialog speech signal; A or B or C is talking igure 1. Database and/or useul Inoration Speaker Diary copilation process by Speaker Segentation then Clustering Typical block diagra o supervised Speaker Diary copilaiton. A B C DOI /IJSSST.a ISSN: x online, print

2 H ALGOTIR_KADHI et al: SPEAKER DIARY COPILATION BY DEPENDENT COBINATION O... Decade by decade, ore o algoriths have been trialed, successully, or the extracting o audio coeicients (eatures). The tasks o these eatures was expanding to solve probles o speech recognition, speaker identiication and veriication, speaker diary copilation...etc. [9]. igure. Ideal Speaker Diary copilation, the 1st line is the input observation signal (dialog conversation); the nd, the 3rd and the 4th lines are the outputs signals (speech o the persons A, B & C respectively). or the clustering process, the segents are conigured hierarchal, then starting ro the top or the botto o this coniguration. Iteratively, the process divides the segents to clusters, where nuber o clusters equal nuber o speakers. eedback loop is necessary to peror the iteration. Speaker diary copilation has dierent application on ultiedia, broadcasting, banking, conerencing etc. [7]. Recently, any sotware or speaker diary copilation are available e.g. ALIZE [8]. Soe o these sotware are ree open source. II. EXTRACTION O AUDIO EATURES Because o liitation o the tie doain representation o the audio signal on its processing, other representation o such signal had been eployed. Via the classical STT, requency doain was the irst choice. STT is an all-pass liner-scaled ilter-bank based on the apping o tie doain to requency doain, atheatically. Several decades ago, coeicient-doain was suggested to transit audio by the sending o the corresponding conidents o the signal instead o the signal itsel. In addition to the traditional linear-scaled STST ilter-bank, other nonlinear-scaled had been utilized. Those scales are, experientally, ound according to the natural behavior o cochlea o huan ear. There are three aous rescaling. The irst is the Critical-Band requency Integration, Equivalent Rectangular Bandwidth ERB. The ERB orula is: ERB( ) 4.7(4.37 1) where is the center o linear-scaled ilter-bank (by Hz), and ERB () is its corresponding nonlinear ERB scale. The second scale is the Triangular requency Integration. elscale. The el-scale orula is: ( ) 595log10(1 / 700) () The third scale is Gaatone requency Integration, Bark-scale. The Bark-scale orula is: B( ) /(1960 ) (3) Since the atheatical orula o these nonlinear ilterbanks cannot be derived, experientally, the easureents are the alternatives. Since the characteristics o the huan hearing are non-standard and dierent ro listener to other listeners, all the above scales have their other orulas. igure 3. ERB, el & Bark requency doain scaling. ost o those are based on the above nonlinear scales. The el-requency Cepstral Coeicients CC uses triangular ilters those centers located according to elscale. ERB is used or Perceptual Linear Predictive PLP to extract its eatures. The RASTA ilter is located inside PLP processing to robust its ability to peror this task, which is called RASTA-PLP. Power Noralized Cepstral Coeicient PNCC is the last algorith that extract the audio eatures. Bark-scale is used or PNCC. There are several other ethods or extraction the audio eatures, but the above three have been used in this paper. or speciic DSP ields, these algorith can use linear scale instead o the nonlinear successully. Vertically, Table I suaries the tie-doain sequences o the C, the RAST-PLP and the PNCC [10]. TABLE I. el ERB requency (Hz) Bark PROCEDURES O CC, RASTA-PLP AND PNCC CC RASTA-PLP PNCC Pre-ephasis ilter Pre-ephasis ilter STT STT STT Square o agnitude Square o agnitude Square o agnitude Triangular requency Critical-band Gaatone integration el scale requency requency integration integration ERBscale Bark scale Nonlinear Copression RASTA iltering Tie-requency Noralization Nonlinear Expansion ean-power Noralization Logarithic Nonlinearity Power-unction Nonlinearity ( ) 1/3 Power-unction Nonlinearity ( ) 1/15 DCT IDCT DCT LPC-Based Cepstral Recursion ean Noralization ean Noralization ean Noralization Output the eatures Output the eatures Output the eatures DOI /IJSSST.a ISSN: x online, print

3 H ALGOTIR_KADHI et al: SPEAKER DIARY COPILATION BY DEPENDENT COBINATION O... III. PROCESSING O DATABASE Standard TIIT library o speech has been used or the ipleenting o the paper process [11]. Approxiately, the database o the library is 0 inutes speaking or each eale and ale. This tie is enough or covering the required statistical properties o the speaker. Each speech signal is divided to overlapped raes. The overlapping between the adjacent raes is /3 o any rae period. The rae is scaled by haing window. The period o each rae is 3 sec, so the increent rae is sec i.e. 1/3, approxiately sec is chosen, because the L o the syste is speaker dependent, and the speech keeps its statistical stationary characteristics or the short ties, approxiately, equal 10 sec. The sapling requency is 8000 saples/sec, so each rae consists o 56 saples and the increent is 86 saples. Each speech ile contains N t o speech saples. Nuber o overlapped raes N is N t /86. To reove the unwanted unvoiced speech signals, pitch detection algorith is used e.g. YAAPT [1]. The CC, the RASTA-PLP and the PNCC eatures (coeicients) o these speech signals, are extracted. The procedure is done rae by rae; with 13 eatures per rae. Total eatures o the CC, the RASTA-PLP or the PNCC are conigured as 13-by-N atrix. or, the eatures atrices o the CC, the RASTA-PLP and PNCC are, R, and P respectively and each atrix is 13-by-N. or, the eatures atrices o the CC, the RASTA-PLP and PNCC are, R, and P respectively and each atrix is 13-by-N. or any atrix, the irst row is the collection o the irst eature sequentially rae by rae. The irst colun is contains the 13 eatures o the irst rae and so on or the other rows and coluns. The above database is adequate or the requireents o supervised L and speaker dependent process. According to our raing or the TIIT recorded speech, each speaker ( or ) signal can extracts about 1,000 set o eatures. Each atrix (o those 6 atrices) is 131,000. Statistically, each eature has 1,000 trials. To prepare the statistical characteristics o the eatures, one by one, Histogra o each one is calculated at irst. Histogra does not have air balance in the coparison between any two patterns. Per unit histogra, which is the Probability Density unction (PD) has this balance perectly. The results are, 13 PDs per speaker or CC, PLP and PNCC. eature-by-eature, pattern o both speakers are prepared. The inal PDs are shown the in ig. 3 4 & 5 or the best and the worst cases o the 13 eatures. The best eature or the C is the 1st, or the PLP is the 7th and or the PNC is the 1th. The worst eatures or the C is the 5th, or the PLP is the 11th and or the PNC is the 1st. These are according to the Euclidean Distance (ED, ED R and ED P or CC, RASTA-PLPC and PNCC respectively) between the patterns o the eatures o and [13]: ED (4) ED ED R P R P R P R P R P. is the absolute-value operator. Neither C, PLP nor PNC has the perect ability to extracts the optial audio eatures because the ED o each eature is luctuating. But, cooperatively, ED o the, alost, has distances those indicates that the overlapped area between & patterns are acceptable. Traditionally, the cooperation between C, RASTA-PLP and PNC can be expressed by concatenation atrix or PN o their atrices [14]. The condition or C, PLP and PNC concatenation is the diensionality o their atrices. This condition can be exist by choosing equal nuber o eatures per rae or each o the. In addition to that, period o the increental raes are equal or all o the (we suppose that all o the have the sae s ). This atrix alignent is independent cobination and it does not have the ability to create any interactions between, R and P. In order to create this interactivity, dependent cobination between the is tested in this paper. Several atheatical relationships between these atrices have been trialed. According to the output results, the best is the addition between these atrices: RP R P RP R P RP R P Obviously, the ED o this cobination is not the best or all the 13 eatures, but it has the ability to avoid the worst cases (large aount o overlapping). In ig. 4 1 eatures have acceptable ED copared with the ED o each one alone o CC, PLP & PNCC. The exception is the 5 th eature, and the reason o this exceptionality is the 5 th eature o each one o CC, PLP & PNCC are bad ED. ig. 5 & 6 illustrate the situation o the best o the alone and the best eature o the dependent cobination. The ig. 5 & 6 also, shows the worst cases according to the ED. C PLP C+PLP+PNC igure 4. Eucledian distances between PD patterns o, &, o 13 eatures (C, PLP, PNC and Cobination). PNC (5) (6) DOI /IJSSST.a ISSN: x online, print

4 H ALGOTIR_KADHI et al: SPEAKER DIARY COPILATION BY DEPENDENT COBINATION O... ED between & EDR EDP EDRP igure 5. The best eature or the C is the 1st, or the PLP is the 7th, or the PNC is the 1th and or the cobined eatures is the 8th. ED between & EDR EDP EDRP igure 6. The worst eatures or the C is the 5th, or the PLP is the 11th, or the PNC is the 1st and or the cobinded eatures is the 5th. IV. PROCESSING O OBSERVATION SIGNAL A. Segentation ro TIIT standard library [11], or the sae speakers &, observation signal is prepared. The signal is a conversation between the. It is inutes o a dialogue speech. The sae procedure o the database processing is used, i.e. sae sapling requency, large raes, Haing windowing, increental period etc. YAAPT also reoves the unwanted signals. or the segentation process, the sae previous algorith (o the database) has been used. The eatures o the observation signal is extracted using the CC, the RASTA-PLP & PNCC. or each ethod, 13 eatures is extracted ro each rae. The dependent cobination ors the vector o each rae because the process o clustering is ipleented rae by rae. Each vector is the result o addition o the 3 vectors o three above ethods. Paraeters o observation signal processing should be the sae o the paraeters o the database processing (saples per rae, ilter per rae, eatures per rae... etc.). B. Clustering and Scenarios The second part o speaker diary copilation is the clustering process. This process assigns each segent to speciic speaker. Nuber o clusters is equal nuber o the speakers who are involving with the conversation observation signal. Hierarchal Scenarios is used in this paper to peror the clustering task. There are two scenarios, Top-Down and Botto-Up. Top-down scenario supposes that all the segents belong to one speaker then eliinates the undesired segents one-by-one till reaches to the steady state. eedback loop is essential to peror this task. Iterations o this loop continue till the constraints be existed. Botto-up scenario supposes that each segent belongs to one speaker then erges the adapted segents together and produces one cluster. One-by-one till reaches to the steady state, the scenario collects clusters equal to nuber o speakers. eedback loop is essential to peror this task also using the sae steps, where, iterations o this loop continue till the constraints be existed. ig. 7 illustrates the basics o these scenarios. Instead o the segents, the clustering (this paper) is collecting the raes because they do not need the segent processing and the switching ties between any rae and its adjacent are known. This paper uses the botto-up scenario, so the process try to erge the adjacent raes. Each rae has two types o neighbors, adjacent rae and non-adjacent. Except the irst and the last raes, each rae has two adjacent raes at Let Hand Side LHS and Right Hand Side RHS. Also, each rae has large nuber o non-adjacent raes at LHS and RHS. Adjacent raes have ore eects on the correction o the erging decision than the non-adjacent. But, liited nuber o the adjacent (two) reduces the ability or aking the decision. In contrast, non-adjacent raes are ore than two but their abilities or correction are poor. The coproised solution or this case is the choosing o the double adjacent raes in addition to several eective non-adjacent [15]. igure 7. Typical diagra o the Hierarchal Scenarios o clustering. DOI /IJSSST.a ISSN: x online, print

5 H ALGOTIR_KADHI et al: SPEAKER DIARY COPILATION BY DEPENDENT COBINATION O... V. IPLEENTATION AND TEST The ollowing procedure is the overall ipleentation. ro TIIT [11], speech database o & is prepared. Each speech ile is about 0 inutes o speaking. Arbitrary, the dialogue input observation signal also prepared ro this library. The signal is 8 inutes. The speech coniguration o this signal is inutes or then inutes or then inutes or and then inutes or. This signal is the upper line in ig 8. Each speech signal is divided to raes o signal. Duration o each rae is 3 sec, i.e. 56 saples per rae (sapling requency is 8000 Hz). The overlapping between each adjacent raes is 170 saples (67%). The increental period is sec ( = 86 saples). Each rae is scaled by Haing window. Using the YAAPT algorith [1], unwanted signals are reoved ro the three speech signals, &. or each signal, the ethods CC, RASTA- PLP then PNCC extracts the audio eatures, ethod by ethod. 13 eatures per each rae is extracted or each ethod. The resulting eatures o C, RASTA-PLP then PNC are oratted as 9 atrices, R, & P or ;, R, & P or ; and, R, & P or. Using the equation 6 & 7, RP, RP & RP are calculated. In order to deine the statistical chances o RP & RP each one alone, Histogra o both o the is calculated or each eature o the 13. To avoid the possibilities o unair, two alignents are done. The irst is the inding o the distance between the axiu and the iniu values o each atrix eleents. This distance represents the resolution o the histogra. The second alignent is the per-uniting o the histogra, i.e. the deining o the PD o each eature. The right curves in ig 5 & 6 are the PD o the best and the worst o these cobined eatures. The audio eatures o the observation signal is located inside the 13-by-N atrix RP, where, N is nuber o raes o this signal. The raes are arranged rae beside it s the ollowing-in-tie rae. This coniguration akes that each rae has relationship with its RHS & LHS adjacent neighbor raes. Because the tie o each rae is sec, this relationship is strongly enough [15]. We suppose that this relationship o the non-adjacent raes is less than the adjacent. This reduction will be increased when the distance between the raes becoe larger. These details are useul or the raes clustering by the down-up scenario. or each rae, its relationships with the neighbor (adjacent and non-adjacent) raes, deterine the identity o this rae, or. or each rae, the best nuber o neighbor raes are 0 to 30, i.e. 10 to 15 or RHS and 10 to 30 or LHS. or the optiized results, iterative loop is perored according to the suitable constraints. The asking o this procedure is logical ( or ), and the resulting are two speech signals, the irst is plus the error ( signal) and the second is plus the error ( signal). The nd & the 3 rd lines in ig. 8 are the outputs when CC is used; the 4 th & the 5 th lines are the outputs when RASTA-PLP is used; the 6 th & the 7 th lines are outputs when PNCC is used and the 8 th & the 9 th are the outputs when dependent cobined eatures is used. Subjectively, the cobined-dependent-eatures ethod iproves the results positively. The listener evaluate this odiication and they answered those are an iproveent. or the clear investigation, objective test is calculated. The well-known Diary copilation Error Rate DER [16] is chosen to evaluate and copare this procedure with the tradition procedures: dur( ).(ax( N re ( ), Nhyp( )) Ncorrect( )) 1 DER (10) dur( ). N 1 re where is total nuber o speaker raes. The observation and outputs speech have the sae speakers. N re() and N hyp () are the nuber o the person who is speaking in the rae. N correct is the nuber o persons who are speaking in. The calculated DER or the our ethods is tabulated in the TABLE II. The results indicate, clearly, that there are better output diary speech signals. TABLE II. PERCENTAGE DIARY COPILAITON ERROR RATE O C, PLP & PNC, COPAIRED WITH THE COBINED-DEPENDENT EATURES ETHOD. DER CC PLP PNCC COBINED-DEPENDENT 7.% 5.7% 1.5% 3.1% 1.% 0.6% 1.3%.0% ALL 4.5% 15.3% 7.1%.7% VI. CONCLUSIONS The described procedure is based on the interactive dependent cobination o the CC, the PLP, the PNCC odiies these algoriths, positively, when they are using each one alone or they are cobined independently. The subjective tests tell us about this iproveent, in addition to the objective test DER which is calculated, and its results obvious this iproveent. This procedure is checked successully or two speakers only with dierent genders. or the audio and speech DSP, each L & PR algorith alone has positive & negative properties, but the dependentcobined or(s) o the, uch ties, can reduce(s) their negative eects and/or expand their positive eects. The dependent cobination, soeties also, is better than the traditional concertation independent cobination. VII. UTURE WORKS In this paper, the inner addition between the atrices o audio eatures is used to create the dependency between the dierent eatures. In the uture, there are other dierent atheatical operations which can be investigated to check their adaptively or the L & PR dependency. DOI /IJSSST.a ISSN: x online, print

6 H ALGOTIR_KADHI et al: SPEAKER DIARY COPILATION BY DEPENDENT COBINATION O... REERENCES [1]. T. Knox, "Speaker Diary copilaiton: Current Liitations and New Directions," PhD desertaion, University o Caliornia, Berkeley/ Electrical Engineering and Coputer Sciences, 013. [] X. A. iro, S. Bozonnet, N. Evans, C. redouille, G. riedland and O. Vinyals, "Speaker diary copilaiton: A review o recent research," Audio, Speech, and Language Processing, IEEE Transactions on, vol. 0, no., pp , 01. [3] Cooke,., and Te-Won Lee. "Speech separation challenge, 006." [4]. ohri, A. Rostaizadeh and A. Talwalkar, oundations o achine learning, IT press, 01. [5] A. Liutkus, J.-L. Durrieu, L. Daudet and G. Richard, "An overview o inored audio source separation," in Iage Analysis or ultiedia Interactive Services (WIAIS), th International Workshop on, 013. [6] S. E. Tranter, D. Reynolds and others, "An overview o autoatic speaker diary copilaiton systes," Audio, Speech, and Language Processing, IEEE Transactions on, vol. 14, no. 5, pp , 006. [7]. Rouvier, G. Dupuy, P. Gay, E. Khoury, T. erlin and S. eignier, "An open-source state-o-the-art toolbox or broadcast news diary copilaiton," IDIAP Resraech Report 013. [8] J.-. Bonastre, N. Scheer, D. atrou, C. redouille, A. Larcher, A. Preti, G. Pouchoulin, N. W. Evans, B. G. auve and J. S. ason, "ALIZE/spkdet: a state-o-the-art open source sotware or speaker recognition.," in Odyssey, 008. [9] J. O. Sith III and J. S. Abel, "Bark and ERB bilinear transors," Speech and Audio Processing, IEEE Transactions on, vol. 7, no. 6, pp , [10] C. Ki and R.. Stern, "Power-noralized cepstral coeicients (PNCC) or robust speech recognition," in Acoustics, Speech and Signal Processing (ICASSP), 01 IEEE International Conerence on, 01. [11] J. S. Garoolo, L.. Lael, W.. isher, J. G. iscus and D. S. Pallett, "DARPA TIIT acoustic-phonetic continous speech corpus CD-RO. NIST speech disc 1-1.1," NASA STI/Recon Technical Report N, vol. 93, p. 7403, [1] S. A. Zahorian and H. Hu, "A spectral/teporal ethod or robust undaental requency tracking," The Journal o the Acoustical Society o Aerica, vol. 13, no. 6, pp , 008. [13].. Deza and E. Deza, Encyclopedia o distances, Springer, 009. [14] V. Vapnik, The nature o statistical learning theory, Springer Science \& Business edia, 013. [15] N. Evans, S. Bozonnet, D. Wang, C. redouille and R. Troncy, "A coparative study o botto-up and top-down approaches to speaker diary copilaiton," Audio, Speech, and Language Processing, IEEE Transactions on, vol. 0, no., pp , 01. [16] igure 8. The 1st line is the input signal. The nd & the 3rd are & o CC. The 4th & the 5th are & o PLP. The 6th & the 7th are & o PNCC. The 8th & 9th are & o dependent cobined o the. DOI /IJSSST.a ISSN: x online, print