Proceedings of e 6 WSEAS International Conference on Applied Computer Science, Tenerife, Canary Islands, Spain, December 16-18, 26 291 Identification of saccades in Electrooculograms and eir use as a control tool MARCELINO MARTINEZ, EMILIO SORIA, RAFAEL MAGDALENA, ANTONIO JOSÉ SERRANO LÓPEZ Department of Electronics Engineering University of Valencia C/. Doctor Moliner, 5. 461 Burjassot (Valencia) SPAIN Abstract: - This paper describes e tasks carried out to develop a control tool using e changes detected in gaze, which are captured in e electrooculogram signal. The objective is to use ese changes to control a user interface such as Dasher. A software tool for generating visual stimuli and acquiring e eye signal has been developed. These signals were later processed wi a first derivative-based algorim in order to detect e changes. The optimal parameters for e algorim have been determined, and also e sensitivity (S>97%) and e predictive positive value (+PV>9%) of e detector have also been calculated. The preliminary results are promising, but a study wi a greater number of individuals should be made to check e on-line performance wi longer registers. Key-Words: - electrooculogram, saccadic movements, dasher, acquisition, control 1 Introduction The use of gaze-tracking techniques as control systems is a work at has been increasing in recent years [1],[2]. The main idea is to control devices wi eye (or gaze) movements. These movements are extremely fast, so is greater an e response speed achieved wi a pointer, e.g., a mouse. This work presents an acquisition, processing and electrooculogram (EOG) analysis, as an initial stage to use e EOG signal in an open source, free tool called Dasher [3]. This program enables optimal writing in a text editor at is controlled by on-off signals. The auors propose e use of EOG and a modified version of e Dasher software as e base system to develop control device systems for severely handicapped people. This paper is aimed at e acquisition of EOG signals, and e generation of visual stimuli at will generate e saccadic movements, e auors also propose an algorim to characterize movements of is kind. This control signal could be used in e future as a pointer control in e Dasher software, leading to a fast writing tool for e handicapped, or ey can also be used in several applications, such as alarm systems[4], wheel chair guidance[5], etc. 2 Problem Formulation 2.1 Dasher Dasher (www.inference.phy.cam.ac.uk/dasher/) [3] is a text-entry system at has a novel interface incorporating a language model. Text typing is driven by continuous two-dimensional search and navigation wi a device such as a mouse, touchscreen, rollerball, breaing device, or eye-tracker. Figure 1. Main screen of Dasher. The language model has been trained on example documents, or training corpus, which allows Dasher to predict e probability of each character s occurrence in a given context. The size of e space is allocated for each setter and successive characters according to e predicted probability. A screen shot of Dasher is shown in Figure 1. Dasher is licensed
Proceedings of e 6 WSEAS International Conference on Applied Computer Science, Tenerife, Canary Islands, Spain, December 16-18, 26 292 under GPL and it is available for several operating systems, including Windows, Linux and Pocket PC. 2.2 Electrooculogram acquisition Several practical devices have used eye movement as a communication support. The videooculogram, which detects eye movements from pictorial images of e eyeball, requires a video camera to film eye movements in real time. Eye movement detection using infrared reflectance of e cornea is difficult to use over a long period of time because eyes tend to become dry and fatigued [1]. All of ese different techniques have advantages and drawbacks, and eir use is intrinsically related to e target of research or e specific purpose desired. Eyes control eir movements by e use of six muscles at enable em to perform different kinds of movements: saccades, fixation, and smoo pursuits [6]. In basal conditions, e retina has a bioelectrical potential at is negative wi respect to e cornea. The registering of is potential, called EOG, can be done by using surface electrodes placed around e eye. It is a non-invasive technique and can be used as a marker of eye movements. This technique measures eye movements relative to e head position and is not generally suitable for point of regard measurements unless e head position is also measured [7]. This potential is corrupted wi several sources of noise which makes e amplification stage a critical component. The typical spatial resolution when using surface electrodes is ±1.5º-2º[5]. There are some problems associated wi EOG measurement. Eye blinks and eye-muscle electrical activity contaminate e signal; moreover, ere is a considerable wandering of basal line due to electrode drift. All ese factors are considered as noise overlapping to e target signal and must be eliminated using digital processing techniques as a previous step before signal interpretation. Wi is meod, e final signal can be used as a control signal. Electrooculogram signals have bandwid between DC and 1 Hz. The amplitude ranges from 15 μv to 2 μv and is linearly related to e eye displacement, wi nominal sensitivities of about 2μV/deg. An important problem of is bioelectrical potential is e DC component which may saturate e amplifier. A set of five electrodes Ag/AgCl has been used in e acquisition of e signal, located at e positions shown in Figure 2. These potentials have been amplified using e g.bsamp module from Guger Technologies. This module is an AC-coupled amplifier wi a cut-off frequency of.1 Hz; wi is meod, e typical saturation of DC coupling is avoided. In order to reduce skin impedance, e skin was cleaned wi an abrasive gel, Parker Redux Paste. Every register consists of a horizontal channel (channel H) and a vertical channel (channel V), which should enable e detection of changes in bo directions. Figure 2. Layout of electrodes used in e acquisition of EOG registers. Since we are solely interested in e measurement of abrupt eye movement (saccadic), an analysis of e obtained waveform was made in order to be able to determine which sampling frequency was most suitable and us to determine e antialiasing filter to be used. Finally, we decided to use a sampling rate of 2 Hertz per channel and to limit bandwid by means of a 2nd order analogue Butterwor filter wi a cut-off frequency of about 4 Hertz. The entire amplifying system is powered by rechargeable batteries, and also has galvanic isolation to guarantee e safety of e patient. The amplified signal is e input of a data acquisition card (PCL 711B of Advantech) wi 12 bits of resolution. A program has been developed at enables e user to simultaneously control, e acquisition of e EOG signals and e generation of visual stimuli; e position and e duration of e stimuli are selectable parameters. Figure 3 shows e positions in which visual stimuli appear. Each stimulus lasts 2 seconds. Only one stimulus is visible at a time. 1 7 Stimulus position 2 4 5 6 8 Figure 3. Locations of e visual stimuli (only one will be visible at a time). The visual stimuli are yellow circles on a black background. The distance of e subject to e monitor has been fixed to 5 cm. and a 17-inches screen has been used. The height of e chair has also 3 9
Proceedings of e 6 WSEAS International Conference on Applied Computer Science, Tenerife, Canary Islands, Spain, December 16-18, 26 293 been fixed, so at e line of e eyes is approximately in e centre of e screen. Thus, watching e central point does not imply displacement of e eyeball. Additionally, e patient are asked not to move eir heads during e acquisition. 2.3 Saccadic movements detection The problem, (extremely simplified), is shown in Figure 4; is figure shows an ideal EOG signal, wi saccades and fixations. This signal represents e movement of e eye in only one direction; eier horizontal or vertical. It is important to note at we are not interested in e exact point of e eye gaze but in an estimation of e direction (left, right, up and down). 1 5 5 1 5 1 15 2 25 1 5 Blinks 5 5 1 15 2 25 Figure 5. Example of a type I register which shows e effect of eye blinks, mainly in channel V 1 5 5 2 4 6 8 1 12 14 16 4 2 2 Figure 4. Representation of an ideal EOG signal along wi e desired detection marks. Figure 5 shows a register in which transitions have taken place up-down and later left-right. This figure shows at ey are independent movements. Figure 5 also clearly shows e presence of blinks in channel V, which we can consider to be impulsive noise. 2.2.1 EOG registers Type I registers. In is type of register, ere are transitions at only affect one channel. They are composed of repeated patterns up-down or left-right. Thus, e EOG signal only reflects changes in one channel. Figure 5 shows an example of a type I register. The number of type I registers is 16. Type II registers. In is case, e registers are generated using a sequence wi simultaneous vertical and horizontal transitions. According to e numbering in Figure 3, e sequence follows e locations: [5,1,3,9,7,1,5,1,5,3,5,9,5,7,5,2,6,8,4,2,5, 2,5,6,5,8,5,4,5,2,8,5,6,4,5,1,9,5,3,7,5]. Figure 6 shows an example of a type II register. The number of type II registers is 18. 2 4 6 8 1 12 14 16 Figure 6. Example of a type II register. In is case, ere are simultaneous transitions in bo directions. 3 Problem Solution 3.1 Saccadic eye movement detection We have used a variation of e algorim based on e point-to-point derivative of e signal in order to determine e position of e gaze, among e different techniques described in [6]. The algorim operation is e same for bo H and V channels. A simplified pseudo code is shown in Table I. The main difference wi e originally proposed algorim is at a median filter is first applied to e signal. Since e leng of e filter is similar to a blink, blinks are eliminated. In order to estimate resholds on e derivative, we distinguish between positive and negative slopes because e eye line is not always in e middle of e screen. In order to determine e optimal performance of e algorim, e slope reshold ( slope ) and e median filter leng ( MF leng ) must be selected. The parameter slope is a factor at multiplies e average of e signal derivatives in e transition
Proceedings of e 6 WSEAS International Conference on Applied Computer Science, Tenerife, Canary Islands, Spain, December 16-18, 26 294 points determined by e stimulus pattern. We have used e first 25% of e known transitions for parameter estimation. Filter e original signal wi a median filter of leng MF leng Calculate e point to point velocity of e filtered signal. Estimate resholds of e previous signal in e first transition points, for positive and negative slopes. Check e complete signal looking for slopes at exceed e resholds (is way saccadic movements and fixations can be distinguished. The sign of e slope gives information about e gaze direction). Table I. Pseudocode of e proposed algorim. 3.2 Results Many tests were performed on e free parameters and useful information about e optimal values was obtained. The values given to e free parameters were slope =.2:.2:.9, and MF leng =2:2:14, where MATLAB TM notation was used (InitialValue:Step:FinalValue). The parameters at were used for optimal value selection were e sensitivity and positive predictive value value of e saccadic eye movement detector. The optimum values were: Type I registers: Type II registers: slope =.6; slope =.4; MF leng =.5s; MF leng =.5s; Figures 7 and 8 show e result obtained when applying e algorim to a type I and a type II register, respectively. A dot marks e points wi negative slope, and an asterisk marks e points wi positive slope. The determination of e direction of e movement depends on e channel being considered.. Table II shows e results of algorim sensitivity and positive predictive value (+PV) when applied to all e registers. We have separated H and V channels, and positive (Hp, Vp) and negative (Hn,Vn) transitions. The number of transitions was N I =256 for type I registers and N II =27 for type II registers. 4 Conclusions The first conclusion at we observed from e acquired registers is at e changes in e direction of e gazes are appraised clearly in bo channels, wi e vertical channel being e noisier one due to e presence of impulsive noise generated by involuntary blinks. Significant base line wander is observed in e registers, is is due to e extremely low cut-off frequency of e HP filter at sometimes can saturate e amplifier and present signal values at are different from zero even ough an individual is watching e centre of e screen. This effect is also generated due to an inappropriate positioning of e patient. Register Hp Hn Vp Vn Sensitivity 1. 98.4 98.43 97.5 I (N I =256) +PV 99.21 99.2 85.7 85.11 Sensitivity 96.3 97.6 97.39 97.22 II (N II =27) +PV 87.22 91.9 87.33 81.7 Table II. Results of e saccadic eye movement detection algorim. S: sensitivity(%). +PV: Positive Prediction Value. 1 5 5 1 5 1 15 2 25 1 5 1 2 3 4 5 Figure 7. Results of e saccade detection algorim in a type I register. The proposed algorim is quite simple because it only uses a median filter of leng similar to e duration of a blink, and it uses e derivative for e detection of transitions. Derivative resholds are determined in an initial training stage of e system. The application of e algorim on bo types of register have yielded good detection results in bo channels, alough e horizontal channel always yields better results due to e lower amount of noise. The auors are working on combining e output of e algorim wi e Dasher software, which is
Proceedings of e 6 WSEAS International Conference on Applied Computer Science, Tenerife, Canary Islands, Spain, December 16-18, 26 295 currently limited to text typing software but whose operation can be extended for oer purposes. The final goal is to use e EOG signal as a control signal for a Human-Machine Interface to detect transitions raer an to detect e absolute position of e gaze. Anoer possible use of is signal could be as an alarm signal for individuals wi severe handicaps. 8 6 4 2 2 4 6 8 2 4 6 8 1 12 14 16 Sample 8 6 4 2 2 4 6 (a) 8 2 4 6 8 1 12 14 16 Sample (b) The auors wish to ank Prof. Francisco Alcantud, (Faculty of Psychology at e University of Valencia), for his help during e register acquisition, as well as e volunteers who unselfishly collaborated in e register acquisition. References [1] Hori J., Sakano K, Saitoh Y. Development of a Communication Support Device Controlled by Eye Movements and Voluntary Eye Blink. IEICE Trans Inf Syst 26; E89-D(6):179-7. [2]Tecce JJ, Pok LJ, Consiglio MR, O'Neil JL. Attention impairment in electrooculographic control of computer functions. Int J Psychophysiol 25; 55(2):159-63. [3]Ward DJ. Adaptive Computer Interfaces UK: University of Cambridge, 21. [4]Venkataramanan S., Praanay Prabhat, Shubhodeep Roy Choudhury, Harshal B., Sahambi J.S. Biomedical Instrumentation Based on Electrooculogram (EOG) Signal Processing and Application to a Hospital Alarm System. IEEE Proceedings of e Second International Conference on Intelligent Sensing and Information Processing (ICISIP). 25 [5]Barea R, Boquete L, Mazo M, Lopez E. System for assisted mobility using eye movements based on electrooculography. IEEE Trans Neural Syst Rehabil Eng 22; 1(4):29-18. [6] Salvucci DD, Goldberg JH. Identifying fixations and saccades in eye-tracking protocols. Proceedings of e 2 Symposium on Eye Tracking Research & Applications.ETRA '. New York: ACM Press. [7]Duchowsky A.T. Eye Tracking Meodology: Theory and Practice. Springer-Verlag, 23. Figure 8. Results of e saccade detection algorim in a type II register: (a) Horizontal channel, (b) Vertical channel. Even ough e preliminary results are promising, a study wi a greater number of individuals should be made in order to check e online performance wi longer registers. In is study, resholds should be updated on-line Acknowledgments This study has been partially supported by project GV6/248 of Conselleria d Empresa Universitat i Ciencia de la Generalitat Valencia, and project UV- AE-25 968 from e University of Valencia.