SCORE-BASED ADAPTIVE TRAINING FOR P300 SPELLER BRAIN-COMPUTER INTERFACE Yuan Zou, Omid Dehzangi, Roozbeh Jafari

Transcription

1 SCORE-BASED ADAPTIVE TRAINING FOR P300 SPELLER BRAIN-COMPUTER INTERFACE Yuan Zou, Omd Dehzang, Roozbeh Jafar Department of Electrcal Engneerng, Unversty of Texas at Dallas, Rchardson, TX 75080, USA ABSTRACT The prmary am of a Bran-Computer Interface (BCI) s to provde communcaton capabltes through bran sgnals recorded from the scalp for those wth bran dsorders to be able to nteract wth the outsde world. In order to properly decode the electroencephalographc (EEG) bran sgnals, the BCI needs to adapt to the subject va calbraton to ensure stable performance. One of the major challenges n realzaton of the EEG sgnals s the long calbraton tme requred snce they show sgnfcant varatons between recordng sessons even for the same subject wthn the same expermental condton. Ths paper proposes a score-based adaptve tranng algorthm that maxmally utlzes relevant nformaton from pror recordng sessons and sgnfcantly shortens the calbraton tme. Also the proposed method s sutable to develop real-tme, wearable, and low-power BCI embedded devces. The BCI developed n ths work s based on the P300 word speller applcaton ntroduced by Farwell and Donchn n The expermental results show that by employng few letters for calbraton, the proposed adaptve tranng algorthm can acheve 100% classfcaton accuracy. Index Terms BCI, EEG sgnal, P300 speller, Calbraton, Adaptve tranng 1. INTRODUCTION Encouraged by new understandng of bran functon over the past two decades, many researchers have explored BCI technology as a new communcaton/control channel for those wth severe neuromuscular dsorders [1, 2]. The goal s to provde basc communcaton capabltes through the EEG bran sgnals recorded from mplanted electrodes on ther scalp so that they can express ther wshes, operate word processng programs, or even control neuroprostheses. Durng the recent years, many research efforts have been made to mprove the performance of BCI technology by ntroducng advanced and computatonally expensve machne learnng [3], sgnal processng [4], and classfcaton technques [5]. Therefore, hgh accuracy s acheved at the expense of ncreased computatonal complexty. However, n real-tme and long-term recordng applcatons, t s hghly desrable to consder smpler and more effcent mathematcal models to reduce the computatonal tme and power consumpton whle mantanng adequate classfcaton accuracy. Wth recent advances n embedded systems and sgnal processng technques, there s a major nterest n developng real-tme, wearable, and low-power BCIs wth embedded systems [6]. One major lmtaton n BCI applcatons, especally the EEG-based BCIs, s the requrement for long calbraton and tranng sessons (e.g., more than one hour) to collect suffcent tranng sgnals for constructng specfc features and classfers. Ths tme-consumng calbraton process s necessary for each new recordng sesson and even for the same subjects that are beyond novces wthn the same expermental envronment. Two man reasons the EEG patterns vary strongly from one sesson to another are 1) subjects have dfferent psychologcal precondtons and 2) electrode couplng condtons vary durng dfferent recordng sessons (between electrodes and subject s scalp). There are several prevous studes on reducng the calbraton tme n the BCI technology and dfferent approaches are proposed. In [7] and [8], subject transfer algorthms were proposed to shorten the calbraton tme for motor magery BCI applcatons. Subject transfer s accomplshed by constructng a prototype of spatal flters from other subjects and adaptng the prototype to the new subject. Also n [9], a smlar algorthm based on Support Vector Machne (SVM) was ntroduced for P300 speller BCI applcaton. However, due to large ntersubject varablty, subject transfer algorthms requre EEG data from multple subjects n order to generate a robust prototype spatal flter, but those data may not be avalable for personal BCI devces. Another strategy s to focus on sesson transfer nstead. In [10] and [11], an algorthm was proposed to skp the calbraton process targeted towards long-term BCI users. It s acheved by generalzng a common spatal flter across sessons estmated va tranng data from pror sessons of the same subject and clusterng of pror spatal flters. In [12], a method was presented that allowed reducng the calbraton tme for both long-term and novel users. Ther approach s based on an ensemble of pror classfers that are transferred to the current sesson. Both approaches requre a large number of hstorc sessons be avalable and nvolve ntensve tranng. Further approaches for reducng calbraton processng tme, especally for the P300 speller BCI applcaton, are threshold-based adaptve tranng [13] and sem-supervsed learnng [14]. In [13], an adaptve tranng procedure was presented that amed to estmate the amount of data needed for calbraton based on two threshold values. The lmtaton of ths approach s that the adaptve tranng s hghly senstve to the value of these thresholds. In [14], t was suggested to ntally use a BCI wth few labeled tranng samples, and then to ncrementally adapt t wth unlabeled onlne data usng an teratve sem-supervsed learnng algorthm. However, the unsupervsed learnng algorthm needs a large amount of unlabeled data n order to acheve a robust BCI wth a satsfactory performance. In ths paper, we nvestgate the problem of reducng the requred calbraton tme for the P300 speller BCI applcaton [2]. We present a score-based adaptve tranng method to maxmally extract relevant data from pror recordng sessons /13/$ IEEE 1143 ICASSP 2013

2 To acheve ths goal, mnmal new recorded data are combned wth the complementary data from pror sessons n a two-stage procedure. Frst, the low qualty recorded data caused by artfacts or dstracton are rejected based on a smlarty score so that the qualty of data for further processng s guaranteed. Then, a log-lkelhood score-based elmnaton algorthm s desgned to select the most complementary data from pror sessons to tran the classfer. Our proposed method sgnfcantly shortens the calbraton tme and at the same tme, t does not nvolve hgh computatonal sgnal processng such as Indepent Component Analyss (ICA). Also, the EEG sgnals n our experments are recorded from a dry electrode system whch s more convenent to wear compared to gel-based electrodes used n prevous studes. Therefore, t s sutable for long-term recordng applcatons and s easy to mplement n real-tme, wearable, and low-power BCI embedded devces. The expermental results show that by usng a few letters durng the calbraton procedure, the proposed adaptve tranng algorthm acheves 100% accuracy. Even wth no calbraton data provded, the proposed algorthm can acheve 79% accuracy. The paper s organzed as follows. Secton 2 descrbes the P300 speller expermental settng. Secton 3 proposes the motvaton and prncple of our proposed score-based adaptve tranng method. Then, secton 4 explans the expermental results and conclusons are presented n secton 5. Fnally, secton 6 shows the relaton to pror works 2. TASK AND DATA ACQUISITION The BCI applcaton nvestgated n ths paper s the P300 speller ntroduced n [2]. It enables users to spell a word from a 6 6 matrx that ncludes all the alphabet letters as well as other useful symbols (Fg. 1). The rows or columns ntensfy sequentally n a random order. To spell a word, the subjects are nstructed to focus on the letter they wsh to communcate by countng the number of tmes t ntensfes. In response, a P300 evoked potental s elcted n the bran whch s a postve deflecton n the EEG after 300ms [15]. By dentfyng ths P300 pattern, t s possble to nfer the atted letter. Sx healthy subjects wth no prevous experence wth the P300 speller partcpate n the experment. EEG data are acqured usng g.usbamp amplfer (g.tec Medcal Engneerng GmbH, Austra) and the BCI software platform BCI2000 [16] n a P300 speller scenaro. Sgnals are recorded at 256 Hz samplng rate from 8 g.hasara actve dry electrodes from Fz, Cz, Pz, P3, P4, PO7, PO8, Oz and referenced at rght mastod. For each subject, two to fve sessons of data was recorded. In each sesson, the subject was nstructed to choose between letters. For each letter, the ntensfcaton lasts for 250 ms followed by a 125 ms blank nterval. Twelve ntensfcatons make up one epoch whch covers all the rows and columns. 15 epochs are carred out for each letter. Thus for one letter, there are =180 ntensfcatons. The task of the P300 speller s to dentfy the subject s desred letter based on the EEG data collected durng the 180 ntensfcatons. 3. THE PROPOSED ALGORITHM Our proposed approach s a two-stage score-based adaptve tranng algorthm. In the frst stage, the low qualty recorded data are rejected based on a smlarty score as the preprocessng Fg1. P300 speller matrx wth one row ntensfed procedure. In the second stage, an adaptve data selecton approach, based on log-lkelhood score, s presented n the classfer tranng procedure. 3.1 Preprocessng: tral rejecton The raw EEG data consst of useful trals carryng dscrmnatve nformaton to detect the target letter. They also nclude low qualty recorded data caused by some artfacts (e.g., eye movement) or due to the subject losng attenton, etc. Each tral s composed of all data samples between ms from the begnnng of target letter ntensfcaton. In order to acheve better classfcaton performance, the bad trals need to be rejected n the preprocessng stage to mprove the data qualty. The most common method used for artfact removal s ICA [17]. However, ICA s not a good preprocessng canddate for realtme applcatons due to the hgh computaton requrement. In ths paper, we present a template matchng algorthm based on a smlarty score for tral rejecton n the preprocessng stage that s straghtforward, easy to mplement and at the same tme leads to hgh performance. The template s defned based on the average of k target trals n whch the P300 pattern s most lkely present (k=30 used n ths paper), (1) where s the template for the electrode channel h, and s the -th target tral. Then, we calculate the smlarty score between each sngle tral and the template as below, ) )) where s the tral ndex, j = {1, 2,, n} s the sample ndex n each tral, and s the smlarty score between the -th tral and the channel h. Assumng that each recorded sample s normalzed to the nterval [0, 1], the above measure maps the smlartes between the tral -th and the h-th template to a real number n the nterval [0,1]. Then, we obtan a rankng accordng to the score, that rejects the trals correspondng to the λ lowest scores for each channel. Therefore, the trals correspondng to the eye movement, muscle artfacts, or dstracton are rejected to guarantee the qualty of data for further processng Score-based adaptve data selecton A major lmtaton n EEG-based BCIs s the requrement for collectng suffcent tranng data at the begnnng of every sesson (.e. calbraton) for constructng robust features and classfers. One way to resolve ths ssue s to employ prevously recorded data to adapt wth the new sesson. Even f we obtan data from multple pror sessons, t mght not be useful for ths purpose due to the nter-sesson varablty. Therefore, we need to desgn a mechansm to select the most relevant data from (2) 1144

3 pror sessons and combne wth mnmal, currently-avalable data n the new sesson. In ths secton, we propose a scorebased adaptve data selecton algorthm based on the backward elmnaton procedure [18], descrbed n Algorthm 1. In ths algorthm, the functon [ )] { )} returns max, the value of for whch f() s maxmum, and the maxmum value s denoted by f( max ). The functon, tran&test(d_tranng, D_Testng) s conventonally supposed to return the accuracy obtaned by tranng the classfer on D_Tranng data and testng on D_Testng data, Accuracy φ = TP φ FP φ (3) where TP φ and FP φ are the set of true postve and false postve for the letter φ, respectvely. However, the accuracy measure may be too coarse to capture the dscrmnatve nformaton among dfferent letters. Therefore n ths paper, we defne the functon tran&test(d_tranng, D_Testng) to return core φ, as the soft accuracy measure for the test target letter φ wthn all rows and columns. In ths way, the scores are obtaned by Stepwse Lnear Dscrmnant Analyss (SLDA) tranng on D_Tranng data and testng on D_Testng data based on the approach n [6]. s the orgnal score of the nput x for the row/column p defned by, Sc p ep x Sx (4) 1 where s the decson score calculated by SLDA classfer for epoch j. ep s the total number of epochs. In ths paper, ep=15. Assumng the target letter φ s located at row p, and column q, the lkelhood of the letter φ s Sc φ =Sc p +Sc q. As the dstrbuton of the scores for each target letter may be dfferent due to varablty of the bran sgnals, the scores are less compatble across dfferent letters. Hence, score normalzaton s a necessary step to provde consstency over the output scores of the classfer. The log-lkelhood rato score normalzaton for the letter φ s calculated as below, M M 1 llr x Sc x log expsc x Sc jx (5) M 1 1, p j1, jq where M=6 s the number of rows/columns n the matrx, s the log-lkelhood score for the letter φ, and exp(.) s the exponental functon. Therefore, core φ, the soft accuracy measure for the test target letter φ s calculated as, j j Score ˆ max llr x max llr x (6) j1,..., c j1,..., c xt P x FP where c s the number of target letters and TP and FP are the set of true postve and false postve data, respectvely. If the ndex of the wnner class for the nput x s equal to φ (.e. arg max llrjx ), j1,..., c x TP True _ label x x FP True _ label x core s a soft measure for the overall performance of the system that s expected to capture more dscrmnatve nformaton than the Accuracy measure. Usng Algorthm 1, a tranng set D tr = {C 1, C 2,, } ncludng the total letters s frst generated. C s the subset (7) Algorthm 1: Adaptve data selecton algorthm Input: D tr: orgnal tranng data; D dev: The development data Output: S tr: fnal selected subset of tranng data Intalze: baselne score: = tran&test (D tr, D dev); Remanng 0 = D tr ; n=1; N t = ; whle n< do for =1 to N t Remove the data correspondng to the -th letter from D tr = tran&test (D tr C, D dev) ; Obtan a rankng of, fnd the maxmal score value ( ) = ) ; f then Remanng n = Remanng n-1 - ; = ; N t = N t -1; n=n+1; else Break; S tr = Remanng n ; data for the -th letter. Then, t sequentally removes the letters from the current set of letters, n order to maxmze core as the overall performance measure on a development set D dev whch s a porton of avalable labeled data that s not contrbuted n the tranng. After the adaptve data selecton procedure, a new data set S tr s generated to tran the SLDA classfer. To evaluate the performance of the proposed algorthm, a test set, D test, ncludng only the data n the new sesson not contrbuted n the tranng or the development sets are employed. The classfcaton accuracy s calculated usng LDA classfer traned usng S tr. 4.1 Tral Rejecton Results 4. RESULTS To assess the performance of the tral rejecton method, we compare the classfcaton accuracy of the sgnal wth and wthout tral rejecton. Three λ values, representng the number of rejected trals, are used n our experments (.e. λ = 5, 10, 15) as reported n Table 1. Subject #1, 3, 4, 6 can only acheve 100% accuracy after applyng the tral rejecton. Also Subject #2 and 5 acheve 100% accuracy wth fewer epochs than before. As expected, the results prove that the tral rejecton method consderably mproves the qualty of data, and as a result t leads to better performance. Table 1 also shows that λ=10 leads to the best results n our experments. In general, λ can be adaptvely selected by cross valdaton on the tranng data. Table 1 Classfcaton accuracy (n %) of the sgnal wth/wthout tral rejecton (100(n) means achevng 100% accuracy after n epochs) Wthout Wth Tral Rejecton Subject Tral Rejecton λ =5 λ =10 λ =15 # (11) 67 #2 100 (14) (4) 100 (4) # (9) 100 (11) 80 # (4) 100(7) #5 100 (10) 100(10) 100(6) 100(9) # (3) 80 Average (6)

4 4.2 Adaptve Tranng Results In our experments, we assume that data for fve target letters are avalable n the current sesson. The rest of the data n the current sesson s used as the test set to assess the performance of our proposed algorthm. We compare the classfcaton accuracy acheved va the score-based adaptve tranng approach, to the system wth no adaptve tranng (no hstorcal data from pror sessons are taken nto account). The comparson s depcted n Fgure 2 whch shows that f no new data s avalable, the non-adaptve system cannot produce any output, whereas our proposed method already generates stable classfcaton accuracy up to 79%. Usng all fve avalable target letters data, the non-adaptve system acheves the same accuracy that the proposed adaptve tranng method generated wthout any data from the current sesson. Wth the fve target letters data, the proposed adaptve tranng method acheves 91% average accuracy wthn sx subjects shown n Fgure 1. Then, we evaluate the effectveness of the proposed scorebased adaptve data selecton crteron. To do so, we compare the adaptve tranng based on core n Eq. (6) wth the Accuracy crteron n Eq. (3). The results of the comparson between the accuracy-based and the score-based methods are reported n Table 2. The results acheved based on the core value are constantly and consderably better than those based on the Accuracy value wth dfferent number of avalable letters. The last two columns n Table 2 show the averaged classfcaton results wth dfferent number of avalable letters n the new sesson under the accuracy-based and score-based adaptve tranng method. The proposed method acheves more than 80% average accuracy on all the subjects wth only two avalable letters data, and more than 90% average accuracy wth fve letters. The results n Table 2 show that the proposed core measure and the adaptve tranng algorthm are effectve n mprovng the Accuracy measure by capturng more dscrmnatve nformaton durng the tranng process. Fnally, based on the results of the score-based adaptve tranng algorthm, fve letters are enough for the calbraton sesson to obtan good classfcaton accuracy. Compared to the requrement of at least 30 letters recordng n the typcal calbraton perod (30*67.5=2025 seconds), our proposed algorthm dramatcally reduces the calbraton tme (5*67.5s=337.5 seconds). Fg 2 Comparson of the classfcaton accuracy acheved by the nonadaptve method (OD, n dash lne) and the score-based adaptve tranng method (AT, n sold lne) 5. CONCLUSION In ths paper, we present a score-based adaptve tranng algorthm to effcently shorten the calbraton perod, as well as acheve better classfcaton performance. In the proposed approach, the data from several pror sessons are combned wth few currently avalable data n the new sesson. The adaptve tranng process s accomplshed by rejectng the redundant trals based on a smlarty score n the preprocessng stage, and then selectng the most relevant data from pror sessons based on the log-lkelhood score value. The expermental results valdate the good performance of our proposed algorthm. Wth fve avalable letters data n the new sesson, the classfcaton accuracy could acheve 100% (91% on average). Also wth no avalable new data, our proposed method can acheve 79% accuracy (70% on average). Furthermore, n contrast to a nonadaptve method and the accuracy-based adaptve tranng algorthm, our proposed algorthm sgnfcantly outperforms for every subjects under dfferent numbers of avalable letters data. Fnally, the results of the proposed algorthm show that fve letters are enough for calbraton to acheve good classfcaton performance. Table 2 Classfcaton accuracy (n %) acheved by the accuracy-based (Acc) vs. score-based (Score) adaptve tranng method. # of letters avalable n new sesson Subjects Averaged on #1 #2 #3 #4 #5 #6 all 6 subjects Score Acc Score Acc Score Acc Score Acc Score Acc Score Acc Score Acc Average

5 6. REFERENCES [1] J. Wolpaw, N. Brbaumer, D. McFarland, G. Pfurtscheller, T. Vaughan et al., Bran-computer nterfaces for communcaton and control, Clncal neurophysology, vol. 113, no. 6, pp , [2] L. Farwell and E. Donchn, Talkng off the top of your head: toward a mental prosthess utlzng event-related bran potentals, Electroencephalography and clncal Neurophysology, vol. 70, no. 6, pp , [3] K. Muller, M. Tangermann, G. Dornhege, M. Krauledat, G. Curo, and B. Blankertz, Machne learnng for real-tme sngle-tral eeg-analyss: From bran-computer nterfacng to mental state montorng, Journal of neuroscence methods, vol. 167, no. 1, pp , [4] A. Bashashat, M. Fatourech, R. Ward, and G. Brch, A survey of sgnal processng algorthms n bran computer nterfaces based on electrcal bran sgnals, Journal of Neural engneerng, vol. 4, no. 2, p. R32, [5] F. Lotte, M. Congedo, A. Lécuyer, F. Lamarche, B. Arnald et al., A revew of classfcaton algorthms for eeg-based bran computer nterfaces, Journal of neural engneerng, vol. 4, [6] U. Hoffmann, JM. Vesn, T. Ebrahm, K. Dserens, An effcent P300-based bran-computer nterface for dsabled subjects, J Neurosc Methods. 2008;167: [13] B. Rvet, H. Cecott, M. Perrn, E. Maby, and J. Mattout, Adaptve tranng sesson for a p300 speller bran computer nterface, Journal of Physology-Pars, vol. 105, no. 1, pp , [14] Y. L, C. Guan, H. L, and Z. Chn, A self-tranng semsupervsed svm algorthm and ts applcaton n an eegbased bran computer nterface speller system, Pattern Recognton Letters, vol. 29, no. 9, pp , [15] S. Sutton, P. Tuetng, and J. Zubn, Informaton delvery and the sensory evoked potental. Scence, [16] G. Schalk, D. McFarland, T. Hnterberger, N. Brbaumer, and J. Wolpaw, Bc2000: a general-purpose brancomputer nterface (bc) system, Bomedcal Engneerng, IEEE Transactons on, vol. 51, no. 6, pp , [17] Y. Zou, J. Hart, and R. Jafar, Automatc eeg artfact removal based on ca and herarchcal clusterng, n Acoustcs, Speech and Sgnal Processng (ICASSP), 2012 IEEE Internatonal Conference on. IEEE, 2012, pp [18] P. Devjver and J. Kttler, Pattern recognton: A statstcal approach. Prentce/Hall Internatonal, [7] W. Tu and S. Sun, A subject transfer framework for eeg classfcaton, Neurocomputng, [8] F. Lotte and C. Guan, Learnng from other subjects helps reducng bran-computer nterface calbraton tme, n Acoustcs Speech and Sgnal Processng (ICASSP), 2010 IEEE Internatonal Conference on. IEEE, 2010, pp [9] M. Kaper and H. Rtter, Generalzng to new subjects n bran-computer nterfacng, n Engneerng n Medcne and Bology Socety, IEMBS th Annual Internatonal Conference of the IEEE, vol. 2. IEEE, 2004, pp [10]M. Krauledat, M. Tangermann, B. Blankertz, and K. Müller, Towards zero tranng for bran-computer nterfacng, PLoS One, vol. 3, no. 8, p. e2967, [11] M. Krauledat, M. Schröder, B. Blankertz, and K. Müller, Reducng calbraton tme for bran-computer nterfaces: A clusterng approach, Advances n Neural Informaton Processng Systems, vol. 19, pp , [12] S. Fazl, F. Popescu, M. Danóczy, B. Blankertz, K. Müller, and C. Grozea, Subject-ndepent mental state classfcaton n sngle trals, Neural networks, vol. 22, no. 9, pp ,