SPEAKER IDENTIFICATION USING MODULAR RECURRENT NEURAL NETWORKS. M W Mak. The Hong Kong Polytechnic University

Size: px

Start display at page:

Download "SPEAKER IDENTIFICATION USING MODULAR RECURRENT NEURAL NETWORKS. M W Mak. The Hong Kong Polytechnic University"

Anis Harvey
6 years ago
Views:

1 SPEAKER IDENTIFICATION USING MODULAR RECURRENT NEURAL NETWORKS M W Ma The Hong Kong Polyechnic Universiy ABSTRACT This paper demonsraes a speaer idenificaion sysem based on recurren neural newors rained wih he Real-ime Recurren Learning algorihm (RTRL). A series of speaer idenificaion experimens based on isolaed digis has been conduced. The daabase conains four uerances of en digis spoen by en speaers over a period of nine monhs. The resuls sugges ha recurren newors can encode saic and dynamic feaures of speech signals. They also show ha he proposed sysem ouperforms he radiional speaer idenificaion sysems in which Bacpropagaion newors are used. However, his paper demonsraes experimenally ha he oupus of he RTRL newors are highly dependen on he iniial porion of he inpu sequences. Removing he firs few vecors from he inpu sequences will lead o a subsanial reducion in idenificaion accuracy.. INTRODUCTION Speaer recogniion can be divided ino speaer verificaion and speaer idenificaion. The former is o verify wheher an unnown voice maches wih he voiced of he claimed ideniy. The laer is o idenify an unnown voice from a se of nown voices. Speaer verificaion sysems are mainly applied o securiy access conrol while speaer idenificaion sysems are mainly used in criminal invesigaion []. Several researchers have aemped o use feedforward neural newors in speaer recogniion. For example, Oglesby and Mason [,3] repored several speaer idenificaion experimens based on Bacpropagaion newors. Ma e al. [4,5] have compared he performance of Muli-layer Perceprons and Radial Basis Funcion newors in a series of speaer idenificaion experimens. They found ha Radial Basis Funcion newors are more robus han Muli-layer Perceprons in speaer idenificaion. Ambiairajah e al. [6] proposed a hybrid Muli-layer Percepions (MLP)-Radial Basis Funcion (RBF) model for speaer verificaion. The weighs of an MLP predicor are used as inpus o an RBF classifier. The hybrid model is designed o operae in he ime domain alone wihou any ime warping procedure. Therefore, pre-processing of inpu speech signals is unnecessary. Ambiairajah e al. demonsraed ha he hybrid MLP-RBF model is reliable and robus. Recenly, Farrell e al. [7] evaluaed various classifiers for ex-independen speaer recogniion and a new classifier called he Modified Neural Tree Newor (MNTN) was proposed. The MNTN is a hierarchical classifier ha combines he characerisics of decision rees and feedforward newors. The classifiers were evaluaed on a subse of he TIMIT daabase consising of 38 speaers. Farrell e al. found ha he performance of he MNTN is beer han ha of he full-search V classifiers. The approaches menioned above are mainly based on feedforward newors. Therefore, he dynamic characerisics of speech signals, which may conain speaer dependen informaion, are ignored. Previous researches [8,9] have shown ha dynamic feaures can improve he performance of auomaic speaer recogniion sysems. The dynamic feaures are usually exraced from a sequence of feaure vecors using eiher emporal differencing or regression analysis. The dynamic feaures are appended o he saic feaures o form he combined feaure vecors. The paern classifier reas he combined feaure vecors as independen saic paerns. However, long-erm dependency of he speech daa is ignored. This is because simple differencing produces a poinwise esimaion of he rae of change of feaures, whereas he regression approach gives an average insananeous gradien only. To incorporae long-erm emporal informaion, his paper proposes an idenificaion sysem based on recurren newors. The recurren newors inheri he mapping capabiliy of feedforward newors and, a he same ime, capure he dynamic feaures of speech signals. We can noe ha he exracion of dynamic feaures becomes he as of he paern classifier raher han he pre-processor.. RECURRENT NETWORK APPROACH Feedforward newors have difficuly in modeling emporal signals or dynamic sysems. This is because hey require a buffer o hold he recen inpu samples. To model emporal signals or dynamic sysems, several newor archiecures and learning algorihms incorporaing feedbac loops have been proposed. For example, William and Zipser [0] proposed he Realime Recurren Learning (RTRL) algorihm ha can be run on-line so ha learning occurs while he inpu sequence is presened. Therefore, i can deal wih sequences of arbirary lengh. The RTRL algorihm is used o rain he recurren newors in he curren sudy. Hereafer, we denoe he fully conneced recurren

2 newors rained wih he RTRL algorihm as RTRL newors. An RTRL newor (Fig. ) comprises inpu nodes, processing nodes, feedforward and recurren connecions. Some of he processing nodes are assigned as oupu nodes. The oupu of every processing node is conneced o all processing nodes including iself, i.e., fully conneced recurren newor. The oupu of a processing node a he curren ime sep depends on he inpu signals and feedbac signals in he previous ime sep. William and Zipser showed ha his ype of recurren newors can be rained by updaing heir weighs in every processing cycle, i.e., real-ime learning. Le he parameers of an RTRL newor, as shown in Fig., be defined as follows: x p () = signal applied o inpu node p a ime sep. y () = acual oupu of processing node a ime sep. d () = arge oupu of processing node a ime sep. s () = acivaion of processing node a ime sep. w p = weigh connecing inpu node p o processing node. v q = weigh connecing processing node q o processing node. = number of processing nodes. P = number of inpu nodes. Since he acivaion of a processing node is he weighed sum of he curren inpu and feedbac signals, we have P+ s = w x + v y p p q q p= q= where w,p+ is he bias and x P+ =. () The oupu of he processing node a he nex ime sep is y ( + ) = f ( s ) () where f ( ) is a sigmoid funcion. Denoe T() as he se of indices such ha he processing nodes have arge values (eachers) a ime sep, and le e () be he insananeous error beween he arge oupu and he acual oupu of processing node. We have e d y T = 0 oherwise. (3) Hence, he oal insananeous squared error a ime sep is E = { d y }. T (4) The weighs are updaed according o he gradien descen rules and w v E = α i, j P + w = α T e E = α v w = α e i, j v T ( ) where α is he learning rae. By using () and (), we obain and wih (5) (6) ( + ) q ( ) = f ( s ) x j δ i + vq (7) w q= w ( + ) q ( ) = f ( s ) y j δ i + vq (8) v q= v i ( 0) ( 0) = = = 0 and δ i = (9) w v 0 i. Equaions () o (9) compleely define he RTRL algorihm. RTRL newors are paricularly suiable for () inpu/oupu sequence associaion, () sequence generaion, and (3) sequence recogniion. RTRL newors have been successfully applied o process conrol [] and channel equalizaion []. These researches demonsrae ha he inernal saes of he RTRL newors encode he dynamic characerisics of inpu sequences. 3. EXPERIMENTS AND RESULTS 3. Speech Daabase and Feaure Vecors The speech daabase consiss of four uerances of en digis ( 0 o 9 ) spoen by en speaers (five male and five female). The daa were colleced in an ordinary office environmen over a period of nine monhs. For each uerance, he analogy speech signals wen hrough he following seps o obain a sequence of -h order Cepsrum coefficiens. ) The speech signals were band limied by a 50 Hz o 3.5 Hz bandpass filer. ) The filered speech signals were sampled a 8 Hz by a 4-bi A/D converer.

3 3) The silen porions were removed manually using a waveform edior. 4) The resuling speech signals were pre-emphasized by a filer wih ransfer funcion H(z)=-0.95z -. A Hamming window wih a window lengh of 8 ms was applied. For each frame, he window advanced 4 ms. 5) LPC analysis was performed on each frame and a sequence of -h order Cepsral vecors was obained. The sequences of Cepsral vecors, derived from all of he uerances of all speaers, form he feaure vecors for he RTRL newors. 3. Speaer Idenificaion Sysem Fig. shows he proposed idenificaion sysem. The sysem conains as many modules as he number of speaers. Each module is an RTRL newor modeling he characerisics of a single speaer. For a sysem wih en speaers, here will be en RTRL newors. Each newor was rained in such a way ha i will produce an increasing oupu when a sequence of Cepsral vecors associaed wih ha speaer is applied; oherwise, i will produce a decreasing oupu. There are several reasons behind his modular approach and a recen sudy [3] has explained and demonsraed is advanages. Firsly, i is easier o learn simple problems separaely han o learn a sophisicaed problem ha consiues several simple problems. Secondly, he chance ha he newor converges o he opimum soluion is higher because he oupu of each individual newor will no inerfere each oher. 3.3 Idenificaion Experimens and Performance Evaluaion The idenificaion experimens are ex-dependen in he sense ha he raining se and es se use he same conex. However, he proposed sysem was esed by presening isolaed digi sequences and he sysem may no be able o now he digi sequences in advance. Therefore, he as acled by he proposed sysem is more difficul han ha of he radiional ex-dependen sysems where he conexs of he digi sequences are imporan for idenificaion. Each RTRL newor was rained independenly wih sequences of -h order Cepsral vecors derived from he firs and second uerances of en digis spoen by en speaers. These vecors form he raining se. The es se was derived from he hird and fourh uerances of he en digis spoen by he same speaers. Where he raining vecors were derived from he speaer associaed wih he newor o be rained, an increasing ramp arge funcion was applied. Oherwise, a decreasing ramp arge funcion was applied (see Fig. ). Therefore, he arge funcions are n d( n ) = 0. 5 ± 0.45 N n (0) where n =,...,N n, and N n is he number of Cepsral vecors in he n-h spoen digi. For each newor, he number of processing nodes,, and he learning rae, α, were se o and 0.05, respecively. Each newor was rained unil he mean squared error beween he arge oupu and acual oupu no longer decrease or 00 epochs have been reached. In his sudy, one epoch is defined as he presenaion of all Cepsral vecors in he raining se. During he learning process, he learning rae was reduced by 0% if he error ceases o decrease for 0 epochs. I was found ha he mean squared error can be furher reduced by decreasing he learning rae during he learning process. The idea is ha we can locae he valley of he error surface quicly by using a large learning rae during he iniial learning phase. However, when approaching he minimum of he error surface, a small learning rae is used o preven he weigh vecor from oscillaing above he poin of minimum. Fig. 3 shows he arge oupu and acual oupu when he hird uerances of he digi hree (spoen by en speaers) were presened o a newor modeling speaer. The resuls show ha he acual oupu follows he arge oupu. Therefore, for each es digi, only one ou of en newors will produce an increasing oupu, while all he oher newors will produce a decreasing oupu. In Fig. 4, en sequences of he digi four, spoen by en differen speaers, were presened o he en RTRL newors. For each speaer indicaed in he Cepsral sequence from axis, a sequence of Cepsral vecors was presened o he en newors. The Newor oupu axis indicaes he oupu of each newor subjeced o hese sequences. Fig. 4 demonsraes ha, for each digi, only he newor associaed wih he speaer who speas ha digi produces an increasing oupu. Therefore, he speaer can be idenified correcly wih high confidence. When an unnown Cepsral sequence is applied o he sysem, he slope of he acual oupu produced by each newor is evaluaed using firs-order linear regression. We can, herefore, idenify an unnown speaer by selecing he newor wih he larges slope. In he idenificaion experimen, m (m 0) randomly seleced digis were fed o each RTRL newor successively. The classificaion is correc if he newor ha gives he larges slope is he speaer s own newor. The process was repeaed for all possible combinaions of he m digis ou of 0 digis. By averaging he number The number of processing nodes was found empirically in a pilo experimen in which an RTRL newor, wih processing nodes varied beween 5 o 5, was rained. I was found ha he newor requires a leas processing nodes o learn he sequences of Cepsral vecors.

4 of correc classificaions, he idenificaion accuracy, I a (m), is obained. Therefore, we have I ( a m ) = Number of correc classificaions 00%. () 0 m Moreover, he idenificaion confidence, I c (m), is defined as he average difference beween he wo larges slopes among he en newors when m randomly seleced digis are applied. The higher he idenificaion confidence, he more confiden ha he sysem idenifies he speaer correcly. Table shows he idenificaion accuracy and idenificaion confidence for varies number of random digis per es. No. of digis per es (m) Idenificaion accuracy in %, I a (m) Training se Tes se Idenificaion confidence, I c (m) 00 Training se Tes se Average TABLE - Idenificaion accuracy and idenificaion confidence for differen number of random digis per es. Table demonsraes ha he idenificaion accuracy grows as he number of digis per es increases. This is because he duraion of he speech segmens used in each es increases wih he number of digis. However, here is no improvemen in idenificaion confidence when longer speech segmens were used. Moreover, he idenificaion confidences for he es se are significanly less han ha of he raining se. Table compares he performance of he RTRL based sysem wih ha of he radiional one [4,5] in which feedforward newors were used. The number of random digis per es was se o five in boh cases, i.e., m=5. Boh sysems used he same raining se and es se. Therefore, direc comparison is possible. No. of hidden nodes per newor No. of free parameers per newor Table shows ha he RTRL newors ouperform he Bacpropagaion (BP) newors in erms of idenificaion accuracy, bu heir performance is slighly poorer han ha of he Radial Basis Funcion (RBF) newors. Anoher ineresing poin is ha, for he given as, he number of free parameers (weighs) of he RTRL newors is far less han ha of he BP and RBF newors. This implies ha he archiecure of he RTRL newors enables hem o encode he saic and dynamic speaer characerisics wih smaller number of weighs. 3.4 Sensiiviy of RTRL Newors BP RBF RTRL Idenificaion accuracy 73.5% 96.9% 94.4% TABLE - Comparison of Bacpropagaion newors, Radial Basis Funcion newors, and RTRL newors in speaer idenificaion. As he RTRL newors learn he emporal srucure of he sequence raher han he individual vecors, heir oupu rajecory will depend on he iniial porion of he sequence. In order words, if he iniial porion of a sequence conains several vecors ha are no useful for he as o be acled, he oupu rajecory may deviae from he arge rajecory significanly. The following experimens demonsrae his phenomenon. A spoen digi 6 was segmened manually such ha here is a 30 ms of silen inerval before he sar of he speech signals. A Cepsral sequence was derived by using he mehod described in secion 3.. The iniial vecor of he sequence was removed. This is equivalen o removing 4 ms of speech signals because he Hamming window advanced 4 ms for each frame. The resuling sequence was fed o an RTRL newor ha modeled he speaer who spoe he digi, and he slope of he oupu was recorded. Then, anoher iniial vecor of he sequence was removed and he process repeaed. Ideally, he slope of he oupu rajecory is posiive irrespecive of he lengh of he silence inerval. Fig. 5 illusraes he variaions of he slope wih respec o he duraion of he speech signals o be removed. The resuls show ha he slope aains he maximum afer 8 ms ( frames) of speech signals has been removed. This corresponds approximaely o he beginning poin of he phoneme /s/ in he digi six. Therefore, he oupu rajecory depends on he duraion of he silence inerval preceding he speech signals.

5 Fig. 5 also shows ha he slope decreases rapidly o negaive values afer 70 ms (5 frames) of speech signals has been removed. This corresponds approximaely o he phoneme /i/ in he digi six. Therefore, if we remove he iniial par of he non-silence region of speech signals, he RTRL newor canno generae he desired oupu rajecory. To verify he impac of his beginning poin sensiiviy on he speaer idenificaion sysem, he above experimen was exended o en newors and he whole es se was used. A Speaer idenificaion experimen as described in secion 3.3 was conduced. However, he idenificaion accuracy was recorded when he number of removed vecors in each Cepsral sequence was increased from zero o five. The resuls presened in Table 3 were obained by using five random digis per es. Table 3 indicaes ha he idenificaion accuracy decreases significanly afer wo iniial vecors have been removed. As menioned in secion 3. he Cepsral sequences in he es se were derived from handsegmened speech signals wih all silen regions removed. Therefore, removing any vecors from he sequences is equivalen o removing he vecors ha are significan for he newors o produce he correc oupu rajecories. No. of iniial vecors removed The implicaion of hese experimens is ha we mus segmen he speech signals carefully, or he auomaic segmenaion uni mus be very accurae. This is one of he disadvanages of using RTRL newors in speaer idenificaion. Anoher disadvanage is ha he run-ime complexiy of RTRL newors is O(n 4 ), where n is he number of processing nodes. Therefore, he raining ime exends subsanially as he number of speaers increases. 4. CONCLUSION Idenificaion accuracy, I a (5) 94.4% 94.8% 84.4% 77.4% 73.8% 7.0% % reducion in idenificaion accuracy 0.0% -0.4% 0.6% 8.0%.8% 4.8% TABLE 3 - Sensiiviy of he idenificaion sysem wih respec o he number of vecors in each inpu sequence o be removed. A series of speaer idenificaion experimens, using isolaed digis as raining se and es se, has been conduced. The resuls indicae ha recurren neural newors rained wih he Real-ime Recurren Learning (RTRL) algorihm ouperform he radiional neural based speaer idenificaion sysems in which Bacpropagaion (BP) newors were used. The idenificaion accuracy based on five es digis is 94.4% for RTRL newors while i is 73.5% for BP newors. However, he idenificaion accuracy of he RTRL newors is slighly poorer han ha of he Radial Basis Funcion newors (94.4% vs. 96.9%). Moreover, i is found ha he RTRL newors are very sensiive o he iniial porion of he inpu sequences. Removing some significan vecors in he sequences or adding some irrelevan vecors a he beginning of he sequences may cause incorrec oupu rajecories. 5. ACKNOWLEDGMENT This wor was suppored by The Hong Kong Polyechnic Universiy Gran No. 35/ REFERENCES. French, J.P, 993, Developmen in forensic speaer idenificaion, Acousic Bullein, Sep./Oc., Oglesby, J. and Mason, T.S, 989, Speaer recogniion wih a neural classifier, Proc. of he Firs IEE In. Conf. on Arificial Neural Newors, Oglesby, J. and Mason, T.S, 990, Opimisaion of neural models for speaer idenificaion, Proc. ICASSP, Ma, M.W., Allen, W.G. and Sexon, G.G., 994, Speaer idenificaion using muli-layer perceprons and radial basis funcion newors, Neurocompuing, 6 (), Ma, M.W., Allen, W.G. and Sexon, G.G., 993, Speaer idenificaion using radial basis funcions, The 3rd IEE In. Conf. on Arificial Neural Newors, Ambiairajah, E., Keane, M., Kelly, A. Kilmarin, L. and Taersall, G., 993, Predicive models for speaer verificaion, Speech Communicaion, 3, Farrell, K.R., Mammone, R.J. and Assaleh, K.T., 994, Speaer recogniion using neural newors and convenional classifiers, IEEE Trans. on Speech and Audio Processing, (), Mason, J.S. and Zhang, X., 99, Velociy and acceleraion feaures in speaer recogniion, ICASSP 9, Soong, F. and Rosenbery, A On he use of insananeous and ransiional specral informaion in speaer recogniion, IEEE Trans. on ASSP, 36, Williams, R.J. and Zipser, D., 989, Experimenal analysis of he real-ime recurren learning algorihm, Connecion Science, (), Cafolis, T., 993, A mehod for improving he real-ime recurren learning algorihm, Neural Newors, 6, Kechriois, G., Zervas, E. and Manolaos, E.S., 994, Using recurren neural newors for adapive communicaion channel equalizaion, IEEE Trans. on Neural Newors, 5(), Anand, R. Mehrora, K. Mohan, C.K. and Rana, S., 995, Efficien classificaion for muliclass problems using

6 Newor oupu modular neural newors, IEEE Trans. on Neural Newors, 6(), 7-4. Oupu of an RTRLN in module i Targe funcion: if inpu is derived from speaer i Targe funcion: if inpu is NOT derived from speaer i Time y () v q w p Oupu node Speaer 0 Speaer 9 x x x Inpu Vecor: h order Cepsrum coefficiens Fig. - Archiecure of an RTRL newor in module i and he oupu arge funcion. Cepsral sequence from Speaer 8 Speaer 7 Speaer 6 Speaer 5 Speaer 4 Speaer 3 Speaer Speaer Newor number 0 Speaer ID Decision Uni RTRL Module RTRL Module i RTRL Module N Fig. 4 - Oupus of en RTRL newors in respond o en Cepsral sequences derived from he digi four spoen by en speaers. -h order Cepsrum Coefficiens ime Inpu Fig. - Speaer idenificaion sysem using RTRL newors 0.04 Targe oupu Acual oupu Linear regression on acual oupu RTRL oupu Speaer Speaer Speaer 3 Speaer 4 Speaer 5 Speaer 6 Speaer 7 Speaer 8 Speaer 9 Speaer 0 Slope of oupu rajecory Frame number (ime uni) Fig. 3 - Oupu of an RTRL newor in responds o en Cepsral sequences of he digi hree spoen by en speaers Lengh of removed speech signals (ms) Fig. 5 - The variaion of he slope of he oupu rajecory wih respec o he lengh of speech signals o be removed.

Memorandum on Impulse Winding Tester

Memorandum on Impulse Winding Teser. Esimaion of Inducance by Impulse Response When he volage response is observed afer connecing an elecric charge sored up in he capaciy C o he coil L (including he inside