ARTIFACTS REMOVAL AND FEATURE EXTRACTION SCHEME FOR STEADY STATE VISUAL EVOKED POTENTIAL BASED BRAIN COMPUTER INTERFACE G.

Transcription

1 ARTIFACTS REMOVAL AND FEATURE EXTRACTION SCHEME FOR STEADY STATE VISUAL EVOKED POTENTIAL BASED BRAIN COMPUTER INTERFACE A Thesis Submitted by G.SARAVANA KUMAR For the award of the degree of DOCTOR OF PHILOSOPHY In Electronics and Communication Engineering Dr.M.G.R Educational and Research Institute Dr.M.G.R University (Declared U/S 3 of the UGC Act, 1956) Periyar E.V.R. High Road, N.H. 4 Highway, Maduravoyal, Chennai APRIL

2 ARTIFACTS REMOVAL AND FEATURE EXTRACTION SCHEME FOR STEADY STATE VISUAL EVOKED POTENTIAL BASED BRAIN COMPUTER INTERFACE A Thesis Submitted by G.SARAVANA KUMAR For the award of the degree of DOCTOR OF PHILOSOPHY In Electronics and Communication Engineering Dr.M.G.R Educational and Research Institute Dr.M.G.R University (Declared U/S 3 of the UGC Act, 1956) Periyar E.V.R. High Road, N.H. 4 Highway, Maduravoyal, Chennai APRIL

3 DEPARTMENT OF ELECTRONICS AND COMMUNICATION ENGINEERING Dr.M.G.R EDUCATIONAL AND RESEARCH INSTITUTE CHENNAI CERTIFICATE FROM THE SUPERVISOR Certified that the thesis entitled Artifacts Removal and Feature Extraction Scheme for Steady State Visual Evoked Potential Based Brain Computer Interface submitted for the degree of Doctor of Philosophy by Mr. G. Saravana Kumar is the record of research work carried out by him under my guidance and supervision. Certified that this work has not formed the basis for the award of any degree, diploma, associate-ship, fellowship or other titles in this University or any other University or Institution of Higher Learning. (Dr. S. Ravi) Supervisor, Professor & Head Department of E.C.E., Dr. M. G. R. Educational & Research Institute, Dr. M. G. R. University. CHENNAI 71

4 DECLARATION I declare that the thesis entitled Artifacts Removal and Feature Extraction Scheme for Steady State Visual Evoked Potential Based Brain Computer Interface submitted by me for the degree of Doctor of Philosophy is the record carried out by me the materials which are not the results of my own work have been clearly acknowledged. Signature of the Research Scholar 72

5 ACKNOWLEDGEMENT I like to express my sincere thanks to Revered founder Mr. A. C. Shanmugam and President Mr. A. C. S. Arun Kumar, for creating a conducive environment and providing obligatory infrastructure for development and implementation of this work. I express my thanks to Dr. P. Aravindhan, Dean (Research), Dr. M. G. R University, Chennai, for his enthusiastic support and insightful ideas for this work to flourish. I express my sincere thanks to Dr. Uma Rajaram Dean E & T, Dr.M.G.R University, Chennai, for the amicable support provided to carry out this work. I extend my genuine thanks with gratitude to my guide Dr. S. Ravi, Professor and Head, Electronics and Communication Engineering, Dr.M.G.R University, Maduravoyal, Chennai, TamilNadu, for his subtle and considerate approach in shaping, guiding and directing me towards effective culmination of this work. I thank my parents, my wife, and my son who offered unconditional love and has stood by me all along. I am grateful to my in-laws, brother and sisters for their unwavering support. I am thankful to Master R. Saravana Karthick for protecting me from the horrendous stress and strain, through his charming graceful smile and larger than life story narration. G. Saravana Kumar 73

6 ABSTRACT Steady State Visual Evoked Potential (SSVEP) based Brain Computer Interface (BCI) systems allow individuals with motor disabilities to use their brain signals to control and communicate with external devices whenever they intend to. These systems are required to remain inactive during all periods in which users do not intend control referred as No Control (NC) Commands and to identify the user s intentional control (IC) commands. This thesis proposes three schemes related to the design of SSVEP based BCI systems. i. Electrode Montage Scheme and EEG signal Recording ii. Artifact Removal Scheme iii. Feature Extraction For the recorded EEG composite signal the frequency ranges corresponds to stimulus related Visual Evoked Potential (VEP) Components. The resulting Spectrum provides VEP frequency band detection. Using this identified frequency ranges EEG artifacts can be reduced. From the output of a visual stimulation paradigm, i.e., electroencephalogram (EEG), the proposed scheme localizes the presence of delta, theta, alpha, beta frequency ranges using parameters such as Spectral Profile (SP) and Peak Power Frequency (PPF). The time-shift based denoising separation extracts specific features using wavelet transforms, which demarcate stimulus evoked features from various rhythms of EEG. This scheme shifts the waveforms by a series of time delays and subsequently linear combinations are formed by repetitive stimulus presentations. The results demonstrate that Bipolar montage scheme can be used as effective montage scheme for SSVEP based BCI. Potential distribution across various regions of the brain for a specific event represented by single map and tri map supports the effectiveness of feature extraction scheme. The efficiency of the artifact removal scheme is substantiated through comparison of two factors True Positive Rate (TPR) and False Positive Rate (FPR) for SSVEP based BCI systems. Keywords: Brain Computer Interface (BCI), Electroencephalogram (EEG), Wavelet Transform (WT), Steady State Visual Evoked Potential (SSVEP), Artifact Removal 74

7 Table Of Contents List of Tables List of Figures i vii ix Chapter. No. Description Page No. Chapter 1 Chapter 2 1. INTRODUCTION Problem Statement and Proposed Solution Focus of Research Proposed Implementation Objectives of the Research Work Benefits of the Research Work Methodology Experimental Setup Electrode Connection Electrode Placement Conclusion Organization of the Thesis Relevant Works Published Paper Paper Paper Paper LITERATURE SURVEY Review of Literature Survey on Artifact Removal techniques Review of Literature Survey on Brain Computer Interface and Feature Extraction 18 75

8 2.3 Review of Literature Survey on Wavelet Transforms 2.4 Review of Literature Survey on Eye movement And tracking methods 2.5 Review of Literature Survey on Visual Evoked Potentials Conclusion 31 Chapter 3 3. BRAIN COMPUTER INTERFACE AND WAVELETS Need for BCI Definition and Classification Non Invasive BCIs Invasive BCI Design factors for BCIs Existing Systems Brain Response Interface P3 character recognition ERS/ERD Cursor Control SSVEP based BCI Mu rhythm Cursor Control Thought Translation Device An Implanted BCI Common Signals used in BCIs Visual Evoked Potential Definitions and Types P3 Component Steady State Visual Evoked Potential Repetitive Visual Stimuli (RVS) Classification Light Stimuli 41 76

9 Single Visual Stimuli Pattern Reversal Stimuli Stimulation Type Characteristics of Single graphic stimuli Characteristics of Pattern Reversal stimuli Comparison between Various Stimuli Slow Cortical Potentials Wavelets for Feature Extraction Definition and Characteristics Reasons for Opting Wavelets Numerical Implementation of Wavelet Transforms Continuous Wavelet Transform The Mexican Hat Wavelet The Morlet Wavelet The Discrete Wavelet Transform Advantages Implementation Schemes Wavelet mapping to Neuroelectric waveforms Matching Pursuit Disadvantages of Matching Pursuit Technique Wavelet Denoising Algorithm for Feature Extraction Matched Meyer Wavelets Conclusion 53 Chapter 4 4. ELECTRODES AND EXPERIMENTAL SETUP Introduction 54 77

10 4.2 Preparation Requirements Scalp Electrodes Subdermal Electrodes Usage method Clip Electrode Nasopharyngeal Electrode Usage method Sphenoidal Electrode Tympanic Electrode Depth Electrode Usage method Cortical Electrode Subdural Electrode Epidural Electrode Electrode Connection Electrode Placement Scheme EEG amplifiers Conclusion 68 Chapter 5 5. WAVELET ANALYSIS OF COMPOSITE EEG SIGNAL Introduction Feature Extraction Using Wavelets Time Series Features Spatial Features EEG signal model Disadvantages of Fourier Transform in Feature Extraction Energy Spread Calculation HAAR Expansion System Relationship between Segments 73 78

11 5.8 Mathematical model of Wavelet Analysis of EEG signal Wavelet Selection Orthogonal Wavelets Mathematical model of Decomposition Algorithm for EEG Processing Designing a Wavelet Based system Selection and display of EEG waveforms Wavelet Selection, Decomposition Vector Calculation, Energy Calculation Coefficient Extraction from Decomposition Vector and display Coefficient Re construction from Decomposition Structure Conclusion 92 Chapter 6 6. RESULTS AND DISCUSSION Data Collection Quality Measures Singular Spectral Entropy (SSE) Spectral Profile (SP) Power Feature (PF) Desynchronziation Process Single frequency Stimulation Paradigm Bi frequency Stimulation Paradigm Feature Extraction Scheme Detailed Comparison and Conclusion

12 6.6.1 Analysis of Scheme Performance on EEG signal with Artifacts Receiver Operating Characteristics (ROC) analysis Conclusion 128 Chapter 7 7. SUMMARY AND FEATURE SCOPE Summary on Artifact Removal Scheme Summary on Montage Scheme Summary on Feature Extraction Scheme Future Scope REFERENCES

13 List of Tables S. No. Description of Table Page No. 1 Table 5.1 The Energy Spectral Density for a frequency range of Hz for electrode pair C4-P Table 5.2 The Energy Spectral Density for a frequency range of Hz for electrode pair FP2-F Table 5.3 The Energy Spectral Density for a frequency range of Hz for electrode pair P4-O Table 5.4 The Energy Spectral Density for a frequency range of Hz for electrode pair T5-O Table 5.5 The Energy Spectral Density for a frequency range of Hz for electrode pair T6-O Table 6.1 Potential Distribution across various regions of the brain for three events Table 6.2 Frequency Orientation and Power distribution for each electrode for the event EYES OPEN Table 6.3 Frequency Orientation and Power distribution for each electrode for the event EYES CLOSED Table 6.4 Frequency Orientation and Power distribution for each electrode for the event VOLUNTARY MUSCLE 104 MOVEMENT 10 Table 6.5 Spectral Components associated with the event EYES OPEN Table 6.6 Spectral Components associated with the event EYES CLOSED Table 6.7 Spectral Components associated with the event VOLUNTARY MUSCLE MOVEMENT Table 6.8 Stimulating Frequncies used in Bi-frequency Stimulation Paradigm Table 6.9 Alphabets Classification Scheme

14 15 16 Table 6.10 Comparison of the average test results for subjects S1, S2, S3 Table 6.11 Comparing TPR of monopolar and bipolar montages for different FAR

15 A List of Figures S. No. Description of Figure Page No. 1 Fig. 1.1 Distribution of Visual Stimulus 12 2 Fig. 1.2 Schematic Top view of EEG electrodes on the head 12 3 Fig. 3.1 Conventional Block diagram of BCI 33 4 Fig. 3.2 Classification of BCIs 34 5 Fig. 3.3 Schematic Representation of Recursive Pyramidal filters 49 6 Fig. 3.4 Wavelet Decomposition Scheme for Composite EEG signal 50 7 Fig. 3.5 Wavelet Denoising Algorithm for Feature Extraction 52 8 Fig. 4.1 Scalp Electrode 55 9 Fig. 4.2 Sub dermal Electrodes Fig. 4.3 Clip Electrode Fig. 4.4 Nasopharyngeal Electrode Fig. 4.5 Tympanic Electrode Fig. 4.6 Depth Electrode Fig. 4.7 Cortical Electrode Fig. 4.8 Subdural Electrode Fig. 4.9 Epidural Electrode Fig Delineation belt (Front View) Fig Delineation belt (Top View) Fig Delineation belt with central electrode (Top View) Fig Electrode Placement Scheme (Front View) Fig Electrode Placement Scheme (Lateral View) Fig Electrode Placement Scheme (Rear View) Fig Electrode Placement Scheme (Lateral View) Fig Electrode Placement Scheme with Jack Box Fig With Photoic Simulation Fig With Photoic Simulation

16 27 Fig. 5.1 Energy Spectral Density for Electrode Pair C4-P Fig. 5.2 Energy Spectral Density for Electrode Pair FP2-F Fig. 5.3 Energy Spectral Density for Electrode Pair P4-O Fig. 5.4 Energy Spectral Density for Electrode Pair T Fig. 5.5 Energy Spectral Density for Electrode Pair T6-O Fig. 6.1 Recorded EEG corresponding to the event EYES OPEN Fig. 6.2 Recorded EEG corresponding to the event EYES CLOSED Fig. 6.3 Recorded EEG corresponding to the event VOLUNTARY MUSCLE MOVEMENT Fig. 6.4 Single Map and Tri Map Potential Distribution for the event EYES OPEN Fig. 6.5 Single Map and Tri Map Potential Distribution for the event EYES CLOSED Fig. 6.6 Single Map and Tri Map Potential Distribution for the event VOLUNTARY MUSCLE MOVEMENT Fig. 6.7 Progressive Amplitude distribution across brain for the event EYES OPEN Fig. 6.8 Progressive Amplitude distribution across brain for the event EYES CLOSED Fig. 6.9 Progressive Amplitude distribution across brain for the event VOLUNTARY MUSCLE MOVEMENT Fig Progressive Amplitude distribution across brain for the event EYES OPEN for different frequency bands Fig Progressive Amplitude distribution across brain for the event EYES CLOSED for different frequency bands

17 43 Fig Progressive Amplitude distribution across brain for 115 the event VOLUNTARY MUSCLE MOVEMENT for different frequency bands 44 Fig CSA Description of the Electrode Potential for the event EYES OPEN Fig CSA Description of the Electrode Potential for the event EYES CLOSED Fig CSA Description of the Electrode Potential for the event VOLUNTARY MUSCLE MOVEMENT Fig Single frequency stimulation results for SUBJECT S Fig Single frequency stimulation results for SUBJECT S Fig Single frequency stimulation results for SUBJECT S Fig Bi frequency stimulation results for SUBJECT S Fig Bi frequency stimulation results for SUBJECT S Fig Bi frequency stimulation results for SUBJECT S Fig Classification rates for each Stimulation frequency with Gaze time of 5 seconds Fig Classification rates for each Stimulation frequency with Gaze time of 3 seconds Fig Average accuracy of the BCI system for three subjects S1, S2, S Fig ROC plot for SUBJECT S Fig ROC plot for SUBJECT S Fig ROC plot for SUBJECT S

18 CHAPTER 1 INTRODUCTION 1.1 Problem Statement And Proposed Solution Neurologically challenged people require Assistive Devices of various kinds to enhance their quality of life and reduce dependency. Systems such as, pointing devices and speech based interfaces were reported to be effective in specific and well-defined environments. The main requirement for these systems is that they required impulses generated by certain muscle groups. The effectiveness of the system is severely restricted for the patients with high degree of disability. This constraint is redressed by a system which does not entail muscle movements through endorsing electrical activity of brain as the control input for the same. The restoration of function can also be accomplished in different ways such as, augmenting the capabilities of remaining pathways, detouring around the points of damage and providing brain with whole new channels for communication. In the first option, muscles that remain under voluntary control substitute for paralyzed muscles. This option is widely used for word processing systems. The benefit of this system is that the percent of false trigger is low and the necessary system requirements are inexpensive. The techniques that detour around points of damage restore function directly to affected muscles by detouring around breaks in the neural pathways that control them. Technique such as Functional electrical stimulation (FES) belongs to this group. A Brain Computer Interface (BCI) is a communication system that does not depend upon on the brain s normal output pathways of peripheral nerves and muscles. Generally, electroencephalogram (EEG) is a display of brain voltage potentials recorded onto paper over specific time duration. This neurological phenomenon reflects the electrical activity of the cerebral cortex, and can be as a control signal for BCI. The specific features associated with this phenomenon is extracted and mapped into another set of control signals that can be eventually used to control devices. Unfortunately, Artifacts (Undesirable Potentials of non-cerebral origin) can modify the shape of a neurological phenomenon that drives a BCI system and in an unintentional control of the device. Therefore there is a need to avoid reject or remove artifacts from the recordings of brain signals. 86

19 Linear filtering is useful for removing only for those artifacts located in certain frequency bands that do not overlap with those of the neurological phenomena of interest and this method fails when the neurological phenomenon of interest and the EMG and EOG artifacts overlap or lie in the same frequency band. Thus, simple filtering approach cannot remove EMG or EOG artifacts without removing a portion of the neurological phenomenon. In case of EOG artifacts removal, these methods are effective in BCI systems that use features extracted from high-frequency components of the EEG such as Mu and Beta rhythm. For BCI systems that depend on low frequency neurological phenomena these methods are undesirable, as EOG artifacts lie in the same frequency range as that of neurological phenomena. In case of EMG artifacts removal, these methods are not successful for BCI systems that use neurological phenomena with high frequency content (such as Beta rhythms). Alternately, using a linear combination of the EOG contaminated EEG signal and the EOG signal is the most common technique for removing ocular artifacts from EEG signals. The linear combination technique provides non-contaminated EEG signal by subtracting EOG artifacts from EOG artifact contaminated EEG signal, where the EOG artifact is scaled by a factor K. The problem lies with the estimation of K value, and conventionally least square criterion is used to estimate the value of K. The disadvantage associated with this scheme is whether the value of K should be calculated separately for each type of EOG artifact and for the different frequencies of a particular EOG artifact. This direct subtraction operation may also remove part of the EEG signal. The disadvantage increases for EMG artifacts, as they have no reference channels and applying regression using signals from multiple muscle groups require multiple reference channels. Blind signal separation (BSS) techniques have also been reported to separate the EEG signals into components that build the EEG signals. They identify the components that are attributed to artifacts and reconstruct the EEG signal without these components. Among the BSS methods, Independent Component Analysis (ICA) is more widely used. ICA is a method that blindly separates mixtures of independent source signals, forcing the components to be independent. It has widely applied to remove ocular artifacts from EEG signals. The advantage of using BSS methods such as ICA is that, they do not rely 87

20 on the availability of reference artifacts for separating the artifacts from the EOG signals. The disadvantage of ICA, along with other BSS techniques, is that, they usually need prior visual inspection to identify artifact components. Artifact removal is the process of identifying and removing artifacts from brain signals. An artifact-removal method should be able to remove the artifacts by retaining the required neurological phenomenon intact. Common methods reported for removing the artifacts in EEG signals are linear filtering, linear combination and regression, principal component analysis and non linear adaptive filtering. The advantages and disadvantages of existing methods for removing artifacts in EEG signals formulate the problem statement for this research work. Thus, in this research work, all the above reported limitations in the removal of artifacts have been explored and a novel technique based on steady state visual evoked potential (SSVEP) has been presented and a BCI system is developed. 1.2 Focus Of Research The reported limitations in artifacts removal schemes and BCI interface units are addressed in this research through a method referred to as Statistical Coefficient Selection method which proposes a solution based on the statistical analysis of wavelet coefficients. In this scheme, the signals from training data set are decomposed into different frequency bands. The decomposed segments are subjected to Short Wavelet Transform (SWT), to yield Scale and Transmission coefficients. These coefficients can then be used compute various statistical parameters associated with the EEG signal. Using a specific criterion the wavelet coefficients are selected. In Statistical Coefficient Selection method, a wavelet space is mapped to a similar structure space consisting of the energies of coefficients. According to set thresholds the level-based threshold is applied, coefficients with higher values of energy and lower energy variations are selected. 88

21 1.3 Proposed Implementation The proposed scheme is implemented as follows: 1. Using an appropriate mother-wavelet, decompose EEG signal and VEP into N levels 2. From decomposed coefficients for EEG reconstruct N+1 time series (Teeg), N-detail Coefficients level, and 1-approximate coefficient level. 3. From decomposed coefficients for VEP reconstruct N+1 time series (T VEP ), N-detail Coefficients level, and 1-approximate coefficient level. 4. Using T eeg and T VEP, for artifact phenomenon L find contribution level. 5. Find minimum amount of correlation (C min ) present between T_eeg_S_i and T_vep_S_i. Determine amplitude profile 6. Compare the i th segment in the EEG signal and the VEP signal. 7. If the correlation is above the the C min value amplitude match is a profile, artifact is assumed to be detected. 8. Set that segment to zero upon artifact detection or retain the current segment. 9. Add all the levels of T vep to generate the final VEP estimate. 1.4 Objectives Of The Research Work (i) To study the existing artifact removal schemes and their limitations (ii) Propose an efficient artifact removal scheme, tested against EMG artifacts To find and plot (iii) Distribution of potential across brain for the montage scheme used. (iv) Localize the potential distribution into three parts, Left part, Right part and Top part. (v) Calculate Peak Power Frequency (PPF) and the distribution of potentials corresponding to different frequency ranges. (vi) Obtain Frequency spectrum depicting PPF, Spectral Edge Frequency, Mean Power frequency against brain s electrical potential. (vii) Determine Electrical potential progression across the brain against time and frequency for chosen electrode placement scheme. (viii) Compute the Power associated with EEG for two types of reference electrodes and different frequency ranges. (ix) Perform Compressed Spectral Analysis (CSA) and Density Spectral Array analysis (DSA) on EEG signals 89

22 (x) Perform Wavelet transform analysis using db5 wavelet on EEG signal for extracting SS-VEP The above objectives shall improve the artifacts removed in the extracted EEG signal and assist well in the BCI system development compared to conventional reported schemes. 1.5 Benefits Of The Research Work This thesis addresses two issues of importance for the design of BCI systems, (i) Handling artifacts in BCI systems (ii) Development of Schemes for analysis of EEG signal (control signal for BCI system) and specific feature extraction. Artifacts are signals recorded from the scalp, which do not come from cerebral activity. Among various artifacts, ocular artifacts cause large amplitude deflections that obscure cerebral activity while others almost replicate the underlying EEG activity and are difficult to extract. The effectiveness of the proposed artifact removal and feature extraction scheme can be used as foundation for evaluating the performance of the BCI systems, through a metric called Information Transfer Rate (ITR). This metric is proposed based on the similarities between BCI and a communication channel and using Shannon s communication theory. ITR measures the amount of information transferred between two reference points. The output Y of a BCI system is the interpretation of the current state of the brain, and it s a useful tool for performance analysis of a BCI system. The proposed algorithm can detect progression of electrical potential across the brain for various time instants and for different frequency bands. This result has potential to generalize more Intention Control (IC) commands and provides localized potential gradient between adjacent electrodes. This dissertation has explored several issues in designing a BCI. Results from algorithm suggest the following observations: 1. A BCI should be designed for flexibility. Important areas for flexibility include signals processing and user applications. 2. Virtual environments may be used as a stimuli in evoked potential BCI applications 3. Signal recognition only accounts for part of the performance and usability in a BCI system. A trade-off between recognition accuracy and time in order to maximize the throughput for a user. 90

23 4. Compressed Spectral Analysis (CSA) provides distribution and temporal behavior of frequencies as well as the intensity of electrical activity which are useful for assessing damaged neural pathways and opting for effective montage scheme. 5. Density Spectral Analysis (DSA) on EEG signals helps us localize rhythmic artifacts which can be used selecting an effective artifacts removal scheme. The portability of the proposed scheme was tested for various reference electrode placement points such as LINKED_EAR, A1, A2, CZ. 1.6 Methodology The methodology consists of (i) Using an experimental setup to extract the EEG signals. (ii) To apply the proposed novel feature extraction technique using wavelet tool (iii) Remove the artifacts effectively using the SSVEP scheme (iv) Interpret the filtered EEG signal and develop BCI interface Experimental Setup This section describes electrode connection and electrode placement schemes for EEG signal, which is used as control signal for BCI Electrode Connection The electrodes are leads are plugged into a head box or jack box which is connected to the EEG machine by a screened multiway input cable. Sockets corresponding to each electrode site are usually labeled and are arranged according to the system of electrodes placement, which will be discussed in later section. For a routine application, minimum of 26 sockets is a reasonable requirement and there should be a number of sockets equipped with switch attenuator for recording high amplitude signal such as electrocardiogram (ECG). Some head boxes have other futures such as meter for testing electro resistance, preamplifiers and earthing sockets. Within the EEG machine, the electrodes are connected to the input of the amplifiers by means of selector switches most of the switches are now microprocessor based. Electrodes are identified in jack box in accordance with the international system electrode nomenclature. Electrodes are identified by a number or a letter except for middle electrode, which carry the subscript-z. Odd numbered on the right electrodes are labeled accordance to the proximate brain region such as front polar as Fp frontal as F parietal as P occipital 91

24 as O, temporal as A. Additional ground electrodes are designated as A. Additional ground electrodes are placed on the midline forehead or in other relatively neutral area. For safely issue, jack box may be electrically or optically isolated to limit the possibility of passing current to the patient Electrode Placement To maintain a constant relationship between the location of electrodes and underlying cerebral structures, a system of electrode placement is necessary. The system of electrode placement has been approved by International federation of Societies for EEG and Clinical Neuro-physiology (IFSECN). The desirable characteristics of electrode placement include the following: 1. Position of electrode must be determined by the measurement from the standard landmark on the scalp. Measurement should be proportional to the skull size and shape. 2. Adequate coverage of all parts of head should be provided by standard designated positions although all may not be used in a given examination. 3. The electrode placement should be symmetrical in the sagittal plain and more strictly should be located to standard landmarks. In spite of slight right to the left scalp asymmetry in normal persons, it is recommended to place the electrodes in homologous position on the two sides. 4. Electrodes should be spaced equally along anteroposterior and transverse axis of the head in order to ensure equal interelectrode distance in bipolar chain. 5. The electrode position should be easy to determine, convenient to apply and retain. 6. Designation of position of electrodes should be in terms of brain areas, e.g. frontal, parietal, temporal, occipital, etc. rather than numbers for better and meaningful communication. 7. Anatomical studies should be carried out to determine the cortical areas most likely to lie beneath the standard electrode position in average subjects. 92

25 1.7 Conclusion This research work proposes and implements a Artifact removal and feature extraction scheme for BCI, various studies on EEG signals such as with artifacts, with eyes closed, with visual stimulus applied, with hyper ventilation etc., are performed. The proposed scheme exhibits portability to EEG signals that contain influence of mixture of various physiological parameters. Also the various pictorial plots are obtained to localize the presence of Steady-State Visual Evoked Potential (SS-VEP). The artifacts were removed efficiently and the corresponding results are presented in detail. An algorithm was developed to find the power distribution present in the EEG signal and the Peak Power Frequency (PPF) associated with it. A detailed CSA analysis is also done here for effective feature extraction by mapping the corresponding neurological phenomenon that is potentials generated in response to Visual stimulus applied. 93

26 1.8 Organization Of The Thesis 1. Chapter 1 presents problem statement and proposed solution along with the objectives of the research work. 2. Chapter 2 gives a detailed literature survey and the benefits of this research work 3. Chapter 3 narrates the theoretical concepts of Brain Computer Interface, Visual Evoked Potentials and Wavelet Analysis of Neuroelectric waveforms. 4. Chapter 4 describes various types of electrodes used for EEG recording, the experimental setup used for EEG recording. 5. Chapter 5 substantiates the wavelet selection through a detailed mathematical analysis in the context of BCI and pseudo code for computing and reconstructing EEG signal coefficients. 6. Chapter 6 presents the results and discussion 7. The Conclusion and future scope is presented in Chapter Relevant Works Published The Relevant manuscript published is summarized as follows Paper 1 In this paper titled Eye Detection and Tracking using Genetic Algorithm, Masaum Journal of Computing (MJC), August 2009, four issues for eye movement detection and tracking were addressed. First, Active still by three dimensional free human head motion, Second, General interface, Third, Real time processing, and Fourth, feature extraction-parameters of eye appearance. In this work, an eye detection and tracking method as the human-machine interface with eye in active scene, such as mobile robots and ubiquitous computing is developed by considering not only the accuracy, but also the general interface and the realtime processing were also taken into account. Additionally, it is desirable to detect the eye and to acquire information of the eye simultaneously, for the real time processing. Generally, it is better to take an eye region as large as possible by camera, in order to acquire the more detailed information from an image. For instance, the eye-gaze estimation method with a target image of one eye is proposed and it is reported that the accuracy is better than a method, which used both eyes. However this causes the difficulty of eye movement detection and tracking because a little motion of 94

27 the object has a large negative effective on the input image. The proposed system is based on a single template matching technique using Genetic algorithm. After an initial population is generated with random numbers, GA is initiated. The matching process is executed between the template image and a target frame in a fitness function. The generation is increased, till a termination condition of GA is satisfied. In this paper the number of generations is chosen as 100. If the termination criterion is not Satisfied, a new population of the next generation is generated according to the fitness of each individual. After GA process is completed, the results are obtained as image and numerical data. Then new process begins for the next frame. Special attention should be paid to the initialization of the GA population only when frame = 0. At this time, some genetic information of the last GA are inherited to the new GA process. This method is narrated above. By the evolutionary image processing, it is possible to detect and track and also extract its geometric information with high accuracy in real-time. The above process is continued till result is obtained Paper 2 In this paper titled, Artifacts Removal Schemes Using Wavelet Transforms in Brain Computer Interface, CiiT International Journal of Digital Image Processing, September 2010, a method to reduce Electroencephalogram (EEG) artifacts from Visual- Evoked Potentials (VEP) in Brain-Computer Interface Design is presented. For the test composite signal the frequency ranges corresponding to stimulus-related VEP components were located using Cyclo Stationary (CS) analysis based algorithm. The resulting cyclic frequency spectrum provides VEP frequency band detection. Using this identified frequency ranges, low pass or band pass filtering is employed for EEG artifacts reduction. The proposed Statistical Wavelet Denoising Algorithm (WDA) is used to distinguish VEP components and EEG artifacts. The advantage of Cyclo Stationarity Property is that trials are not required to be phase locked. Concatenation of trials which is based on internal similarities introduces Cyclo Stationarity. Mallat s Multi Resolution Analysis (MRA) describes procedure to calculate wavelet coefficients, MRA is based on the consecutive application of high-pass and low-pass filters to the original signal x. At stage N, the detail coefficients and approximation coefficients were calculated. The wavelet function is determined by the general shape of the high pass and low pass filters. This scheme uses coif3 mother 95

28 wavelet with Daubechies wavelets which are family of orthogonal wavelets. The algorithm uses following selection parameters: (i) Wavelet type (ii) Levels of Decomposition (iii) Thresholds. The reliability of this scheme depends on ratio of Average values to Standard Deviation and fixation of stringent limits. This method removes mixed biological artifacts and sporadic artifacts Paper 3 In the Paper titled, Feature Classification Scheme for Steady State Visual Evoked Potential based Brain Computer Interface, International Journal of Advanced Engineering and Applications (IJAEA), January 2011, the following experimental setup to record EEG was proposed. This setup employs a set of nine symbols used as visual stimulation to elicit SS-VEP as shown in Fig. 1. The visual stimulation consists of a matrix of 3 X 3 squares labeled with letters from A to I positioned as shown in Fig. 1. In Fig. 2 the distance between two successive blocks can be made equal to side value of a square for better results. Two different visual stimulation paradigms were presented to the subject namely bi-frequency stimulation and single frequency stimulation. Single frequency stimulation frequency can be found on top each square and bi-frequency stimulation frequencies can be found on bottom of each square. In bi-frequency stimulation paradigm each block flickers at two frequencies whereas in single frequency stimulation paradigm each block flickers at single frequency. The subjects were instructed to gaze each block for five seconds with inter stimulus interval equals to five seconds. Fig.2 shows the electrode montage scheme for the visual stimulation paradigm adopted by BCI. Using silver chloride electrodes EEG signals were picked up at 39 positions of the skull. Positions TP9 and TP10 were used positioning reference electrodes. The two electrodes FP1 and FP2 were chosen to capture possible EOG artifacts and eye blinks and FT9 and FT10, O9 and O10 were montaged to identify jaw and muscle activity. 96

29 Fig. 1.1 Distribution of Visual Stimulus Fig. 1.2 Schematic top view of EEG electrode placement scheme on the head Paper 4 In the paper titled Feature Extraction scheme for Brain-Computer Interface using Wavelet Transform, International Journal of Research and Reviews in Computer Science, February 2011 the following feature extraction scheme was proposed. Wavelet transform forms a general mathematical tool for signal processing with many applications in EEG data analysis. Its basic use includes time-scale signal analysis, signal 97

30 decomposition and signal compression. Wavelet transform coefficients can be organized in a matrix T with its non zero elements forming a triangle structure with each its row corresponding to a separate dilation coefficient m. The set of decomposition coefficients of the wavelet transform is defined in the way formally close to the Fourier transform but owing to the general definition of wavelet functions they can carry different information. Using the orthogonal set of wavelet functions they are moreover closely related to the signal energy. The initial wavelet can be considered as a pass-band filter and in most cases half-band filter covering the normalized frequency band (0.25, 0.5). A wavelet corresponds to a pass-band compression is chosen according to its dilation factor. This general property can be demonstrated for the harmonic wavelet function and the corresponding scaling function can be used for further processing. The set of wavelets define a special filter bank which can be used for signal component analysis and resulting wavelet transform coefficients can be further applied as signal features for its classification. Signal decomposition performed by a pyramidal algorithm is interpreting wavelets as pass-band filters. The basic decomposition of a given column vector extracts approximation coefficients, detail coefficients, horizontal coefficients and vertical coefficients and it also provides the book keeping matrix Resulting wavelet coefficients related to chosen scales can then be used as signal features for its classification. 98

31 . CHAPTER 2 LITERATURE SURVEY This chapter presents a detailed literature survey in the following areas associated with Brain-Computer Interface (BCI). 1. Artifact Removal Techniques 2. Brain Computer Interface and Feature Extraction 3. Wavelet Transform 4. Eye Movement Tracking 5. Visual Evoked Potential 2.1 Review of Literature survey on Artifact Removal Techniques [1] Narrates an extensive review of EOG and EMG artifacts associated with BCI systems and removal methods. These artifacts may change the characteristics of neurological phenomenon or even be misinterpreted as source of control in BCI systems. It also reveals the lack of automated methods of rejection or removal of artifacts in BCI systems. This insufficiency may lead to deterioration of the performance of a particular BCI system during practical applications. It draws this conclusion by reviewing and categorizing two hundred and fifty refereed journals and conference papers, based on the type of neurological phenomenon used and the methods employed for handling EOG and EMG artifacts. 99

32 [2] Proposes the implementation scheme of an automated artifact detection scheme for use in the brainstream platform at the Music Mind Machine (MMM) group of the Radboud University for use in their online system. It describes a formal evaluation of this artifact detection method present in EEG. It concludes that usage of Bipolar EOG instead of unipolar may provide a moderate advantage for eye artifacts detection. It also brings the advantage of online usage with offline methods by using Fieldstrip toolkit. [3] Proposes a combination of blind source separation (BSS) and independent component analysis (ICA) which decomposes the signal into artifacts and specific features. It describes the use of support vector machines (SVMs) for automatic feature classification which is designed for online usage. It also provides a comparative study on three ICA algorithms such as JADE, Infomax and FastICA to select a suitable BSS/ICA method which effectively isolates EMG and EOG artifacts. It describes the filter usage in measurements with online feedback and its efficiency evaluation by using three BCI dataset. [4] Presents a new automated artifacts removal algorithm based on Blind Source Separation (BSS) and Support Vector Machines (SVM) from EEG recordings which can accept a four senitcond delay. It also proposes an online implementation scheme without the need for human supervision. It also narrates the effectiveness of this scheme by testing on independently recorded dataset for BCI motor imagery tasks. [5] Presents a novel scheme for the removal of eye-blink (EB) artifacts from EEG signals based on a space-time-frequency (STF) model of EEGs and robust minimum variance beamformer (RMVB). It describes the impact of providing priori information such as the estimation of the steering vector corresponding to the point source EB artifact. This factor was identified using STF model of EEGs provided by the parallel factor analysis (PARAFAC). It also narrates the use of novel core consistency diagnostic (CORCONDIA) based measure for fixing the number of STF factors to be used. 100

33 [6] investigates the performance of a specific self-paced BCI (SBCI) using two different datasets to determine its suitability for using online such as data contaminated with large amplitude eye movements and data recorded in a session subsequent to the original sessions used to design the system. It explains feature extraction scheme which uses three specific neurological phenomena belongs to a different frequency band. It concludes that eye movement artifacts belong to low frequency band and muscle movement artifacts belongs to high frequency band. [7] Presents a class of soft computing algorithms which employs inexact or approximate calculations which includes fuzzy logic, evolutionary computation, statistical discrimination, support vector machine and Bayesian approaches for EEG processing. It compares linear statistical discriminants with support vector machines to conclude that later exhibits a better classification accuracy. [8] It proposes an artifact removal scheme based on Independent Component Analysis which blindly separates mixture of independent source signals forcing the components to be independent. It concludes that ICA increases the strength of motorrelated signals in the Mu rhythms and explains the applicability of Genetic Algorithms to the optimum feature subset selection problem to optimize performance. It employs two types of EEG database where 70% of each dataset used for training and the rest for test classifiers. It compares the usage of ICA and GA with ICA and linear filtering and the result shows that the former scheme provides better classification accuracy. [9] It proposes a non-linear artifact removal model based on regression developed for multi channel EEG. It concludes that quadratic non-linear model performs better than linear and cubic regression models. It investigates a standard linear regression model implemented using MATLAB tested on data sets corresponding to ten trails. The results were substantiated by well defined four peaks in the EOG signal which indicates possible eye blinks occurred at four instances. It also differentiates the usage of linear methods and non linear methods for artifact removal in EEG signals. Linear methods yield localization of positive peak whereas quadratic non-linear methods produce reasonably good results. [10] Presents an extensive survey obtaining clean EEG signals by artifact schemes such as ICA, Wavelet transforms, linear filtering and Artifact Neural Networks. It 101

34 recommends that usage of ICA where distinguishing large number of artifacts is required; Wavelet transforms are suitable for real-time application by proper selection of threshold function; linear filtering can be used when artifact frequency does not interfere or overlap with each other; Artificial Neural Networks (ANN) is used in development of adaptive methods. [11] Examines the impact of muscle or electromyogenic (EMG) artifact on inferential validity for any electroencephalography investigation in the frequency domain owing to its high amplitude, broad spectrum and sensitivity to psychological processes of interest. It compares two popular correction techniques, General Linear Model (GLM) and ICA. It concludes that GLM based methods represent a sensitive and specific tool for correcting on-going or induced spectral changes. It shows that ICA may not represent a panacea for EMG contamination. [12] Narrates a robust method to automatically eliminate eye-movement and eyeblink artifacts from EEG signals. Independent Component Analysis (ICA) is used to decompose EEG signals into independent components. Moreover, the features of topographies and power spectral densities of those components are extracted to identify eye-movement artifact components, and a support vector machine (SVM) classifier is adopted because it has higher performance than several other classifiers. It shows that feature-extraction methods are unsuitable for identifying eye-blink artifact components, and then a novel peak detection algorithm of independent component (PDAIC) is proposed to identify eye-blink artifact components. [13] Addresses the problem of eye blink artifacts detection in BCI systems by implementing an algebraic approach based on numerical differentiation, which is recently introduced from operational calculus. Single channel EEG systems are very useful in EEG based applications where real time processing, low computational complexity and low cumbersomeness are critical constrains. These include brain-computer interface and biofeedback devices and also some clinical applications such as EEG recording on babies or Alzheimer s disease recognition. The occurrence of an artifact is modeled as an irregularity which appears explicitly in the time (generalized) derivative of the EEG signal as a delay. Manipulating such delay is easy with the operational calculus and it leads to a simple joint detection and localization algorithm. While the algorithm is 102

35 devised based on continuous-time arguments, the final implementation step is fully realized in a discrete-time context, using very classical discrete-time FIR filters. The proposed approach is compared with three other approaches: (1) the very basic threshold approach, (2) the approach that combines the use of median filter, matched filter and nonlinear energy operator (NEO) and (3) the wavelet based approach. 2.2 Review of Literature survey on Brain Computer Interface (BCI) [14] Implements a software system that allows for real time brain signal analysis and machine learning classification of affective and workload states measured with functional near-infrared spectroscopy (fnirs) referred as online fnirs analysis and classification (OFAC). It also compares a previous offline analysis with online analysis to conclude that OFAC s online features through real-time classification of two tasks and interface adaptation exhibits 85% accuracy. This result leads to build adaptive user interfaces using fnirs. [15] Presents a framework based on Common Spatial Patterns (CSP) and Linear Discriminant Analysis (LDA) and Covariate Shift Adaptation method to handle nonstationarity normally found in session-to-session transfers of Brain Computer Interfaces (BCIs). It also proves effectiveness of the framework in adapting the testing sessions without the need for labeling the testing session. It uses electrocorticogram (ECoG) dataset and EEG dataset from BCI Competition III. The results exhibits that the framework compares favorably with those methods used in the BCI Competition revealing the effectiveness of covariate shift adaptation in tackling the non-stationarity in BCI. [16] Investigates the application of different machine learning techniques for classification of mental tasks from electroencephalograph (EEG) signals. It finds performance improvement of brain computer interface (BCI) systems by applying Bayesian graphical network, neural network, Bayesian quadratic, Fisher linear and hidden Markov model classifiers to two known EEG datasets in the BCI field. The Bayesian network classifier is used for the classification of EEG signals. The Bayesian network appears to have a significant accuracy and more consistent classification compared to the other four methods. In addition to classical correct classification accuracy criteria, the 103

36 mutual information is also used to compare the classification results with other BCI groups. [17] Proposes a novel signal acquisition tool for BCIs is near-infrared spectroscopy (NIRS), an optical technique to measure localized cortical brain activity. The benefits of using this non-invasive modality are safety, portability and accessibility. It implements a straightforward custom-built system to investigate the functionality of a fnirs-bci system and describes the construction of the device, the principles of operation and the implementation of a fnirs-bci application, 'Mindswitch' that harnesses motor imagery for control. Mindswitch presents a basic 'on/off' switching option to the user, where selection of either state takes 1 min. The results show that fnirs can support simple BCI functionality and presents fnirs as an accessible and affordable option. [18] Experiments with a number of nonlinear and complexity based feature extraction techniques and implements simulation procedures on BCI datasets. It concludes that this feature classification procedure based on non linear characterization problem represents a pragmatic balance of computational simplicity. It compares the results with the average signal power in Mu rhythms in the range of 10 to 15 Hz and in Beta rhythms in the range of 23 to 28 Hz to classify power features associated with EEG. [19] Describes the status of brain computer or brain machine interface research and focus on non-invasive brain computer interfaces (BCIs) and their clinical utility for direct brain communication in paralysis and motor restoration in stroke. A large gap between the promises ofinvasive animal and human BCI preparations and the clinical reality characterizes this work. BCIs based on electroencephalographic potentials or oscillations are ready to undergo large clinical studies and commercial production as an adjunct or a major assisted communication device for paralysed and locked-in patients. It concludes that a lack of contingencies between goal directed thoughts and intentions may be the performance obstacle for BCI performance. In addition to assisted communication, BCIs consists of operant learning of EEG slow cortical potentials and sensorimotor rhythm were also studied. 104

37 [20] Investigates the classification of mental tasks based on EEG data for BCI in two scenarios offline and online. It evaluates the performance of a number of classifiers using benchmark dataset and proposes a new feature selection method that is suitable for the highly correlated EEG data in off line scenario. In online scenario it studies the performance of the proposed work to play a computer game which the signals are processed in real time. [21] Proposes covert attention to spatial locations in the visual field is a relatively new control signal for brain-computer interfaces. It studies previous EEG research methods to show that trials can be classified by thresholding based on left and right hemisphere alpha power in covert spatial attention paradigms. It reexamines the covert attention paradigm based on MEG measurements and shows that classification performance can be improved by applying sparse logistic regression Classification performance steadily increases as the length of the attention period over which is averaged is increased, although this does not necessarily translate into higher bit rates. [22] Presents a novel brain computer interface (BCI) based on motion-onset visual evoked potentials (mveps). For the BCI application, the brief motion of objects embedded into onscreen virtual buttons is used to evoke mvep that is time locked to the onset of motion. EEG data are used to investigate the spatio-temporal pattern of mvep and N2 and P2 components, with distinct temporo-occipital and parietal topography, respectively, are selected as the salient features of the brain response to the attended targets. [22] Demonstrates the performance of different machine learning algorithms based on classification accuracy using dataset II from BCI Competition III for two subjects. The algorithms used were Bayesian Linear Discriminant Analysis, Linear Support Vector Machine, Fisher Linear Discriminant Analysis, Generalized Anderson s Task linear classifier and Linear Discriminant Analysis. [23] Proposes P300 Speller as an effective paradigm for braincomputer interface (BCI) communication. Using this paradigm, it shows that a simple linear classifier can perform as well as more complex nonlinear classifiers. It studies various methods such as 105

38 Fisher s Linear Discriminant (FLD), Stepwise Linear Discriminant Analysis (SWLDA) and Support Vector Machines (SVM) for training a linear classifier. The results indicate marginal performance differences between classifiers trained using these methods. It also shows that by using an ensemble of linear classifiers trained on independent data, performance can be improved. It evaluates several offline implementations of ensemble SWLDA classifiers for the P300 speller. [24] presents a Brain Computer Interface (BCI) based on steady state visual evoked potentials (SSVEP). It uses a stimulation a box equipped with LEDs (for forward, backward, left and right commands) flicks with different frequencies (10, 11, 12, 13 Hz) to elicit SSVEPs. It also derives eight channels of EEG over visual cortex for the scheme with 3 subjects. It calculates and classifies features by using Minimum Energy and Fast Fourier Transformation with linear discriminant analysis. By computing the change rate (fluctuation of the classification result) and the majority weight was calculated to increase the robustness. [25] implements Passive Brain-Computer Interfaces using brain activity as an additional input, allowing the adaptation of the interface in real time according to the user s mental state. The conventional Brain Computer Interface research is designed for direct use with disabled users whereas this scheme proposes passive BCIs for healthy users. The method employs functional near-infrared spectroscopy (fnirs), a noninvasive brain measurement device, to augment an interface so it uses brain activity measures as an additional input channel. [26] discusses the pitfalls associated with Brain-Computer Interface (BCI) applications in the context of ineffective feature classification schemes. It proposes Virtual reality as an alternative and suitable tool to provide subjects the opportunity for training and testing. It highlights the present studies to suggest asynchronous BCI as viable solution. One way to reduce the probability of misclassification is to achieve control with only two different mental tasks. It presents the system that enables the control of a virtual wheelchair with three commands: move forward, turn left and turn right. 106

39 [27] studies the performances of the Gaussian process classifier (GPC) for three different categories of EEG signals, i.e. steady state visually evoked potential (SSVEP), motor imagery and finger movement EEG data, are investigated. The main purpose of this analysis is to explore the practicability of GPC for EEG signals classification of different tasks. It also concludes the GPC exhibits better performances and the probabilistic output provided by the GPC can also be of great benefit to the decision making for both online and offline EEG analysis. [28] presents the first bibliometric analysis of the BCI literature ( ) compiled by Thomson Reuters s Institute for Scientific Information (ISI) Web of Knowledge. It explores various parameters associated with BCI research, that are: 1) to study the evolutionary growth of BCI literature, 2) to assess if it follows Lotka s law of scientific productivity, 3) to identify authors, groups and countries contributing the most to BCI, 4) to reveal the characteristic of citation for the BCI literature, and finally, 5) to determine the core journals that published substantial portions of the literature on BCI. Results indicate that BCI literature follows a power law growth, has an average author count of 3.9 and an average page count of 7.09 and more than half (52.73%) of the BCI literature is never cited, and 14 papers have been cited more than 100 times. The three most productive authors are leading BCI research groups, in Austria, Germany and the USA. [29] proposes the design method of a Brain-Computer Interface(BCI) system based on SSVEP. It achieves an effective collection and processing of SSVEP information through the combination of analog filtering and digital processing. It extracts the features and maps into control commands to control the peripherals. A novelty is introduced by using digital signal processor (DSP) is this BCI. [30] implements a Brain Computer Interface (BCI) for pattern recognition based on classification method based, Probabilistic Neural Network (PNN) with supervised learning. It applies the recognition rate of training samples to the learning progress of network parameters. The learning vector quantization is employed to group training samples and the Genetic algorithm (GA) is used for training the network s smoothing parameters and hidden central vector for determining hidden neurons. It utilizes the standard dataset I(a) of BCI Competition 2003 results show that this way has the best 107

40 performance of pattern recognition, and the classification accuracy can reach 93.8%, which improves over 5% compared with the best result (88.7%) of the competition. This technology provides an effective way to EEG classification in practical system of BCI. [31] discusses the generation of the P300 wave is with the oddball paradigm, which allows detecting targets selected by the user on a screen. It implements the P300- Speller based on this principle. The detection of the P300 requires efficient signal processing and machine learning techniques. It proposes a calibration step needed for training the models. It presents a scheme to evaluate the optimal number of characters that should be spelt in order to provide a working system with minimum calibration duration. The scheme has been tested on data recorded on 20 healthy subjects with an average accuracy of at least 80%. [32] describes the implementation of a Brain-Computer Interface (BCI) for controlling Internet browsing. The system uses electroencephalographic (EEG) signals to control the computer by evoked potentials through the P300 paradigm. This way, using visual stimulus, the user is able to control the Internet navigation via a virtual mouse and keyboard. The system has been developed under the BCI2000 platform. It also shows the experimental results obtained by different users. [33] presents the development of a brain computer interface as an alternative communication channel to be used in Robotics. It encompasses the implementation of an electroencephalograph (EEG), as well as the development of all computational methods and necessary techniques to identify mental activities. The developed brain computer interface (BCI) is applied to activate the movements of a 120lb mobile robot, associating four different mental activities to robot commands. The interface is based on EEG signal analyses, which extract features that can be classified as specific mental activities. First, a signal preprocessing is performed from the EEG data, filtering noise, using a spatial filter to increase the scalp signal resolution, and extracting relevant features. Then, different classifier models are proposed, evaluated and compared. In one of the implementations, a 91% average hit rate is obtained, with only 1.25% wrong commands after 400 attempts to control the mobile robot. 2.3 Review of Literature survey on Wavelet Transforms 108

41 [34] presents the experimental pilot study to investigate the effects of pulsed electromagnetic field (PEMF) at extremely low frequency (ELF) in response to photoplethysmographic (PPG), electrocardiographic (ECG), electroencephalographic (EEG) activity. The assessment of wavelet transform (WT) as a feature extraction method was used in representing the electrophysiological signals. Considering that classification is often more accurate when the pattern is simplified through representation by important features, the feature extraction and selection play an important role in classifying systems such as neural networks. The PPG, ECG, EEG signals were decomposed into timefrequency representations using discrete wavelet transform (DWT) and the statistical features were calculated to depict their distribution. It also investigates for any possible electrophysiological activity alterations due to ELF PEMF exposure. [35] proposes Model Of experts (ME) is modular neural network architecture for supervised learning. A double-loop Expectation-Maximization (EM) algorithm has been introduced to the ME network structure for detection of epileptic seizure. The detection of epileptiform discharges in the EEG is an important component in the diagnosis of epilepsy. EEG signals were decomposed into the frequency sub-bands using discrete wavelet transform (DWT). Then these sub-band frequencies were used as an input to a ME network with two discrete outputs: normal and epileptic. In order to improve accuracy, the outputs of expert networks were combined according to a set of local weights called the ''gating function''. The invariant transformations of the ME probability density functions include the permutations of the expert labels and the translations of the parameters in the gating functions. The performance of the proposed model was evaluated in terms of classification accuracies and the results confirmed that the proposed ME network structure has some potential in detecting epileptic seizures. The ME network structure achieved accuracy rates which were higher than that of the stand-alone neural network model. [36] proposes an electroencephalogram (EEG) analysis system for single-trial classification of motor imagery (MI) data. Feature extraction in brain-computer interface (BCI) work is an important task that significantly affects the success of brain signal classification. The continuous wavelet transform (CWT) is applied together with Student's two-sample t-statistics for 2D time-scale feature extraction, where features are 109

42 extracted from EEG signals recorded from subjects performing left and right MI. First, utilizing the CWT to construct a 2D time-scale feature, which yields a highly redundant representation of EEG signals in the time-frequency domain, from precise localization of event-related brain desynchronization and synchronization (ERD and ERS) components are obtained. Weighing the 2D time-scale feature with Student's two-sample t-statistics, which represents a time-scale plot of discriminant information between left and right MI. Compared to a conventional 2D time-frequency feature and three well-known timefrequency approaches, the experimental results show that the proposed method provides reliable 2D time-scale features for BCI classification. [37] To curtail the Electrode placement time and accelerate the process it is required to record two channels placed on right and left hemisphere of brain based on active region depending on mental task. Raw EEG signals are contaminated with various kinds of noise such as ocular artifacts (EOG) and high frequency muscle noise etc. This paper is an application of wavelet transform based on Filter bank approach whereby it is possible to extract the noise free sub-band signal from noisy EEG signal. The performance of the Extracted Sub-band Components has been evaluated by taking the Power Spectral density for left hemisphere as well as right hemisphere channel sub-band components. [38] studies various brain computer interface implementation schemes and propose a novel scheme for feature extraction of electroencephalogram (EEG). It was based on wavelet packet decomposition (WPD). The energy of special sub-bands and corresponding coefficients of wavelet packet decomposition were selected as features which have maximal separability according to the Fisher distance criterion. The eigenvector was obtained for classification by combining the effective features from different channels and performance was evaluated by separability and pattern recognition accuracy using the datasets of BCI 2003 Competition. [39] presents FuRIA, a trainable feature extraction algorithm for non-invasive Brain-Computer Interfaces (BCI). FuRIA is based on inverse solutions and on the new concepts of fuzzy Region Of Interest (ROI) and fuzzy frequency band and wavelet transforms. FuRIA can automatically identify the relevant ROI and frequency bands for 110

43 the discrimination of mental states, even for multiclass BCI. Once identified, the activity in these ROI and frequency bands can be used as features for any classifier. [40] proposes a scheme which uses EEG signals collected from 64 channels from 20 subjects in the age group of years for determining discrete emotions (happy, surprise, fear, disgust, and neutral) under audio-visual induction (video/film clips) stimuli. Surface Laplacian filtering is used to preprocess the EEG signals and decomposed into five different EEG frequency bands (delta, theta, alpha, beta, and gamma) using Wavelet Transform (WT). The statistical features are derived from all these five frequency bands are considered for classifying the emotions using two linear classifiers such as K Nearest Neighbor (KNN) and Linear Discriminant Analysis (LDA). It concludes that KNN outperforms LDA. [41] proposes a method to select a wavelet basis for classification. It uses a strategy defined by Wickerhauser and Coifman and proposes a new additive criterion describing the contrast between classes. Its performance is compared with other approaches on simulated signals and on experimental EEG signals for brain computer interface applications. [42] presents Functional Near Infrared Spectroscope (fnirs), a scheme which utilize light in the near-infrared range to determine brain activities. Because near infrared technology allows design of safe, portable, wearable, non-invasive and wireless qualities monitoring systems, fnirs monitoring of brain hemodynamics can be value in helping to understand brain tasks. The scheme presents results of fnirs signal analysis indicating that there exist distinct patterns of hemodynamic responses which recognize brain tasks toward developing a BCI. It uses two different mathematics tools, Wavelets analysis for preprocessing as signal filters and feature extractions and Neural networks for cognition brain tasks as a classification module. [43] evaluates different feature extraction alternatives to detect the event related evoked potential signal on brain computer interfaces, trying to minimize the time employed and the classification error, in terms of sensibility and specificity of the method, looking for alternatives to coherent averaging. The scheme presents various feature extraction schemes which uses discrete dyadic wavelet transform using different 111

44 mother wavelets. It uses a single layer perceptron for feature classification. The results obtained with and without the wavelet decomposition were compared; showing an improvement on the classification rate, the specificity and the sensibility for the feature vectors obtained using some mother wavelets. [44] Implements a BCI scheme which uses a combination of wavelet entropy (WE) and band powers (BP) for feature extraction based on imaginary left and right hand movements. Linear discriminant analysis (LDA) was used for classification and mutual information (MI) was used for evaluation because it take into account the magnitude of the outputs. This algorithm was applied on the data set III of BCI competition 2003 and got good results. The results of the experiment showed that this algorithm was a very good method for feature extraction in BCI system. [45] Compares various artificial neural network schemes to discriminate five mental states. Wavelet Packet Transform (WPT) was used for feature extraction of the relevant frequency bands from raw electroencephalogram (EEG) signals. The five ANN training methods used were Gradient Descent Back Propagation, Levenberg-Marquardt Resilient Back Propagation, Conjugate Learning Gradient Back Propagation and Gradient Descent Back Propagation. 2.4 Review of Literature survey on Eye-movement tracking methods [46] Describes the development and improvement of vision based assistive technology, which was tested in experiments with cerebral palsy users. It explains a scheme which applies human-computer interaction (HCI) knowledge and techniques, such as accessibility and usability, to help people with cerebral palsy or other severe disabilities carry out specific tasks with a computer. Two of the most relevant HCI attributes are usability and accessibility. They have compiled a representative set of computer assistive technology devices commonly used with our disabled target users, individuals with CP. It can be used with great accuracy even when the user has exiguous cephalic motion control. 112

45 [47] Develops a camera based scheme to visually track the tip of the nose or the tip of a finger or some other selected feature of the body and moves the mouse pointer on the screen accordingly. People who are quadriplegic and nonverbal, for example from cerebral palsy or traumatic brain injury or stroke, have limited motions they can make voluntarily. Occasionally the selected subimage creeps along the user s face, for example up and down the nose as the user moves his head. They have tried the system with three teenagers with severe disabilities. [48] Describes the design and development of a low-cost eye tracking-based brain-computer interface system for the rehabilitation of the completely locked-in patient having an intact ocular motor control to serve as an alternative means of communication. The core consists of fully functional rehabilitative system As the locked-in patient focuses on a particular pictogram, the program determines the point of gaze and measures the duration that the patient spends looking at that pictogram. It is worth noting that the system currently consists of 24 pictograms stored in an expandable database. They hold the promise to restore independence and mobility to the completely locked-in patient. [49] Presents a high-speed size and orientation invariant eye-tracking method, which can acquire numerical parameters to represent the size and orientation of the eye is presented. It overcomes these problems and template matching is used with genetic algorithm. It also studies various implementation schemes such as eye-gaze direction and eye-tracking. The contrast color is used as a feature of the iris; therefore, the input data is gray scale values. Moreover, considering general use of this system, a general template should be used [25]. The matching process is executed between the template image and a target frame in a fitness function. It also proposes high-speed and orientation invariant eye-detection and tracking method, which can acquire numerical parameters to represent the size and orientation of the eye. It also concludes that high tolerance for human head movement and real-time processing is needed for high fidelity applications such as eyegaze tracking. 2.5 Review of Literature survey on Visual Evoked Potentials 113

46 [50] describes that Single Visual evoked potentials (VEP) are characterized by very low amplitudes and completely buried in noise. These potentials are elicited by the brain when visual stimulations are successively applied to a subject. In the clinical practice, interpretation of the VEP is based on the visual reading of the latency which corresponds to the time of occurrence of the first major peak their approaches were implemented using MATLAB environment. It also implements a structure called threelayer perceptron for diagnosis purposes and efficient use of the VEP recordings. It also proposes the software PEDiag-2 that performs several tasks among which the extraction of the true VEP and its interpretation. [51] implements SSVEP-based brain-computer interface (BCI) which has potential advantage of high information transfer rate. It also explains the application scheme for an environmental controller used by the motion-disabled. Visual evoked potentials (VEPs) recorded from scalp over visual cortex reflect the visual information processing mechanism in brain. It also describes various channel activities which associates the most significant amplitude of SSVEP. Their signal to noise ratio for different channels is the basis of optimal lead selection. The results demonstrate that optimal lead selection is an effective and reasonable method to improve the applicability of SSVEP-based BCI. [52] concludes that single visual evoked potentials (VEP) are very weak and noisy signals. For this reason, powerful extraction tools are needed to improve their clinical use. The results obtained show that non-linear filtering performs efficient VEP extraction based on just a few single responses. This is very significant in terms of saving time especially when recording data from elders and children. In clinical practice, the traditional method of ensemble averaging is commonly used to extract the meaningful VEP signals from a noisy background. In this paper, we expose two methods for filtering VEP. It also proposes a scheme based on the linear adaptive noise canceller technique. They prove that this non-linear filter produces the best results for VEP extraction from filtering single brain answers as compared with those obtained by the use of linear adaptive noise canceller method and the traditional averaging method used in the clinical practice. 114

47 [53] discusses the implementation of Berlin Brain Computer Interface (BBCI) by the research team from Fraunhofer institute at the Cubit 2006 exhibition in Hanover. With this BCI a disabled person can write words and control the mouse pointer by means of his brain waves. It also mentions the visual evoked potentials are significant voltage fluctuations resulting from visually evoked neural activity. Neurons in the visual cortex response to the flickering stimuli at the frequency of the flickering light. SSVEP is characterized as increment in EEG activity for a stimulus frequency. It concludes that the classification of EEG signals, for two different stimuli can be performed by using the artificial neural networks [54] introduces a system that can help the disabled persons, who have no motor control left for communication besides eye movements, to achieve an acceptable level of communication. The system incorporates a brain-computer interface (BCI) for connecting the brain to the computer. They included several issues are crucial to further development and expanded utilization of the BCI technology. The performance metric is the information transfer rate and concludes a higher performance can be expected when using more visual stimuli and more sophisticated signal processing methods, optimized for each user individually. [55] implements a system based on Visual Evoked Potentials (VEPs) for the diagnosis of optic nerve diseases. In this system visual stimulator, electrodes, surge protection circuit, pre-amplifier, filter network, post amplifier, isolation circuit and A/D converter have been used as building blocks to acquire VEPs. This shows the developed set up is evaluated with that of Cadwell Sierra-II standard electrodiagnostic equipment by recording VEP signals for the same group of adult subjects. It narrates the experimental setup where subjects are prepared in accordance with that of clinical protocol and VEPs are recorded using the developed setup. Diagnosis of optic nerve diseases for the recorded VEP signals is performed on the basis of normative data established for that particular neurodiagnostic laboratory. 2.6 Conclusion The literature survey reveals the two main issues of importance associated with existing BCI systems such as handling artifacts, an effective feature extraction scheme. The survey also presents a view of the various types BCI systems, various types of 115

48 control signals, wavelet transforms as a processing tool, existing artifact removal schemes and feature extraction schemes. CHAPTER 3 BRAIN COMPUTER INTERFACE AND WAVELETS 116

49 3.1 Need For BCI A conservative estimate presents the existence of inherited neuromuscular disorder for 1 in 3500 of the world s population. The neurological disorder such as Amyotropic Lateral Sclerosis (ALS), commonly referred as Lou Gehrig s disease, is a progressive fatal neuromuscular disease that attacks nerve cells and pathways in the brain to the spinal cord. These disorders lead to a compulsive acceptance of reduced quality of life, which results in dependence on caretakers amidst escalating social costs. Neuro prosthetics is an area of neuroscience concerned with neural prostheses using artificial devices to replace the function of impaired nervous system or sensory organs. These assistive technology devices depend on motor activities from specific parts of the body. This limitation paves the way to alternative control paradigms such as Brain-Computer Interfaces (BCI s). 3.2 Definition And Classification A Brain Computer Interface is a communication system that does not depend on the brain s normal output pathways of peripheral nerves and muscles. Presently two approaches are pursued in the design of BCI systems Synchronized and Self-paced [56]. In the synchronized approach, which forms the traditional approach to the design of BCI systems, the user can only perform the control in certain time intervals that are specified by the system. An Self-paced BCI (SBCI) is constantly available for a user to use, The performance of SBCI systems is usually summarized by two measures: 1. The correct detection rate of Intentional Control (IC) commands (denoted as the true positive (TP) rate) 2. The amount of false activations during No Control (NC) periods ( periods for which the user does not wish to exert control). The second feature that distinguishes BCIs is whether they utilize invasive (i.e., intra-cranial) or non-invasive techniques to record the electrical signals from the brain. The first feature that distinguishes BCIs is whether they utilize invasive (i.e. intra-cranial) or non-invasive methods of electrophysiological recordings. BCIs can also be classified based on electrode montage schemes on cortical areas of brain. 3.3 Non-Invasive BCIs Non-invasive systems primarily exploit electroencephalograms (EEGs) to control computer cursors or other devices. This approach has proved useful for helping paralyzed 117

50 or locked in patients develop ways of communication with the external world. However, despite having the great advantage of not exposing the patient to the risks of brain surgery, EEG-based techniques provide communication channels of limited capacity. Their typical transfer rate is currently 5 25 bits. 3.4 Invasive BCIs Invasive BCI approaches are based on recordings from ensembles of single brain cells (also known as single units) or on the activity of multiple neurons (also known as multi-units). Figure 3.1 shows a conventional BCI system in which a person controls a device in an operating environment such as powered wheel chair in a house) through a series of functional components. State feedback BCI Transducer Control Display amp Artifact Processor Feature Generator Feature Translator Control Interface Device Controller Signal Enhancement Feature Extraction Feature Selection/ Dimensionality Reduction Feature Classification Postprocessing. Fig. 3.1 Conventional Block diagram of BCI The building components of a BCI system in Fig. 3.1 have the following tasks The electrodes placed on the head of the user to record the electrical activity of brain(electroencephalography (EEG) signals from the scalp, ElectroCorticoGraphy(ECoG) signals from the brain or neuronal activity recorded using microelectrodes implanted in the brain). Fig. 3.2 describes a detailed account of BCI classification Brain-machine interfaces 118

51 NON-INVASIVE This methodology is based on the recordings of EEGs from the surface of the head. It provides solutions for paralyzed people for simple communications with the outside world. However, neural signals have a limited bandwidth INVASIVE Electrodes are implanted intracranially. ThIs methodology provides neural signals of the best quality and has a high potential for further improvement. At the same time, it carries risk associated with an invasive surgical procedure SINGLE RECORDING SITE Several groups have built BMIs based on neural recordings from a single cortical area. A single-area BMI decodes neuronal activity specific for that area, for example motor commands in M1 or cognitive signals in PP. MULTIPLE SITES RECORDING By recording simultaneously from many areas, this approach takes advantages of distributed processing of information in the brain. Although technically challenging, it is highly promising for both developing BMIs and gaining fundamental knowledge SMALL SAMPLES In certain cases, small groups of neurons are sufficient for providing control signals to a BMI. This design suffers from instability related to variables of neuronal activity and changes in the sampled populations of neurons LEPs BMIs based on decoding LFPs suffer less from biocompatibility issues. Their advantages are that they reflect population effects such as neural oscillations. Their bandwidth is, however, limited. Fig. 3.2 Classification Of BCIs LARGER ENSEMBLES Large neuronal ensembles (hundred and in the future feasibly thousand of cell) provide a stable signal to control a multi-degree-offreedom Immediate users will be mainly those who are totally paralyzed by ALS or brainstem stroke or movement disorders such as severe cerebral palsy that abolish muscle control. These users who lack all muscle control or whose remaining control is easily fatigued or otherwise unreliable. Study on this user group reveals that their incidence of depression is not necessarily higher than that of the general population. Standard quality of life 119

52 measures may not be appropriate for people who are severely paralyzed. Their emotional and psychological well being does not necessarily worsen as motor function declines. People who retain minimal voluntary movement might use hybrid systems that combine BCI-based control with conventional control. BCIs might also serve those whose communication and control capacities are impaired by aphasias, apraxias, or autism. The main objective of BCI design is to effectively catering the communication and control needs of lock-in patients which offers enjoyable, productive and independent lives. 3.5 Design Factors For BCIs The following factors play a crucial role in designing BCIs. Quantum of training: User s lack of conventional communication ability can make it hard to assess his or her cognition or even consciousness and may impede the related potential-based tests designed to assess cortical sensory and cognitive processing in such users and evaluate their capacity for mastering BCI. Assessing Neuronal Damage: The deficits that abolish all voluntary muscle control may also impair user control the signal features used by a BCI. The loss of cortical neurons that can occur with ALS. The loss of cortical neurons that can occur with ALs or the extensive cortical and/or subcortical damage typically associated with severe cerebral palsy may affect generation or control of the rhythms, evoked potentials or neuronal activity used for BCI-based communication. Damage to prefrontal-cortex such as with multiple sclerosis, Parkinson s disease and ALS can impair attention and thereby adversely affect BCI use. Motivational factors: Changes in an individual s physical environment or social interactions can greatly affect the extent of BCI use. Effective BCI application in clinical situations requires careful and continual assessment of quality of life. The need for constantly wearing an electrode cap or constantly confronting a particular visual display can have tremendous influence. Aesthetic aspects of BCIs are also important. The primary emphasis of BCI applications is to address user s satisfaction level. 3.6 Existing Systems The spectrum of BCIs covers simple experimental interfaces meant used for testing the suitability of specific EEG signal to full application used by patients. The fundamental blocks used in the BCI are, hardware used in the BCI, the underlying BCI backend software and the user application. The hardware used should deliver 120

53 performance, portability and reliability. The backend includes software for reading in the EEG signals, scheduling them for processing and processing them into a form that may be used by the user application. The backend software determines the BCI portability, extendibility and flexibility. Several important factors should be considered in the design of the application specified as follows, (i) Time to learn (ii) Performance Speed (iii) Number of errors user can make (iv) Retention ability of users (v) Subjective satisfaction The application designer might want to consider the following general goals as specified by the U.S. Military standard for Human engineering Design criteria. (i) Achieve required performance by operator, control and maintenance personnel (ii) Minimize skill and personnel requirements and training time (iii) Achieve required reliability of personnel equipment combinations (iv) Foster design standardization within and among systems. 3.7 Brain Response Interface Brain Response Interface (BRI) is a system that takes advantage of the fact large chunks of the visual system are devoted to processing information from the foveal region [57]. The BRI uses visually evoked potentials (VEP s) produced in response to brief visual stimuli. These EP s are then used to give a discrete command to pick a certain part of a computer screen. This system is one of the few that have been tested on severely handicapped individuals. Word processing output approaches words/minutes and accuracy approaches 90% with the use of epidural electrodes. This is the only system mentioned that uses implanted electrodes to obtain a larger, less contaminated region. 3.8 P3 Character Recognition A BCI that uses P3 evoked potential uses a 6 x 6 grid containing letters from the alphabet is displayed on the computer monitor and users are asked to select the letters in a word by counting the number of times that a row and column containing the letter flashes. The desired letter flashes twice in every set of twelve flashes. The average response to 121

54 each row and column is computed and the P3 amplitude is measured. Response amplitude is reliably longer for the row and column containing the desired letter. After two training sessions, users are able to communicate at a rate of 2.3 characters/min, with accuracy rates of 95%. This system is currently used in a research setting. A positive aspect of using a longer latency component such as the P3 is that it enables proper gazing from the user. 3.9 ERS/ERD Cursor Control Potentials elicited in response to event-related synchronization / desynchronization (ERS/ERD) can be extracted by placing electrodes placed over sensorimotor cortex. This system automatically rejects epochs with eye and muscle artifact. The user application is a simple screen that allows control of a cursor in either the left or right direction. For a single trial the screen first appears blank, and then a target box is shown on one side of the screen. A cross hair appears to let the user know that he/she must begin trying to move the cursor towards the box. After two training sessions, three out of five student subjects were able to move a cursor right or left with accuracy rates from %. When a third category was added for a classification, performance dropped to a low of 60% in the best case SSVEP based BCI By presenting florescent tubes flashing at Hz through operant conditioning methods, the users can control the amplitude of the steady-state visual evoked potential (SS-VEP). This method of control may be considered as continuous as the amplitude may change in a continuous fashion. Either a horizontal light bar or audio feedback is provides when electrodes located over the occipital cortex measure changes in signal amplitude. If the VEP amplitude is below or above a specific threshold for a specific time period, discrete control outputs are generated. After around six hours of training users may have an accuracy rate of greater than 80% in this application. When the SS- VEP is used as a natural response virtually no training is needed in order to use the system. The main drawback of this system is flicker induced fatigue to users Mu Rhythm Cursor Control 122

55 In this application the users are trained to control mu rhythm. This method of control is continuous as the mu rhythm may be altered in a continuous manner. It can be attenuated by movement and tactile stimulation as well as by imagined movement. A user main task is to move a cursor to up or down on a computer screen. While all users are able to learn this type of biofeedback control, the users that do perform with accuracy greater than or equal to 90%. This BCI system consists of a 64-channel EEG amplifier, two 32-channel A/D converter boards, a TMS320C30 based DSP board and a PC with two monitors. Only a subset of the 64-channels is used for control but the number of channels allows recognition to adjust Thought Translation Device With severely handicapped individuals, the Thought Translation Device (TTD) has the distinction of being the first BCI to enable an individual without any form of motor control to communicate with the external world [57]. The training program may use either auditory or visual feedback. The slow cortical potential is extracted from the regular EEG, filtered, corrected for eye movement artifacts, and fed back to the patient. In the case of auditory feedback the positivity / negativity of a slow cortical potential is represented by pitch. When using visual feedback the target positivity/negativity is represented by a high and low box on the screen. A ball shaped light moves toward or away from the target box depending on a subject s performance. The subject is reinforced for good performance with the appearance of a happy face or a melodic sound sequence An Implanted BCI In the implanted BCI system the cursor is determined by the rate of neural firing. The neural wave shapes are converted to pulses and three pulses are an input to the computer mouse. The first and second pulses control X and Y position of the cursor and a third pulse as a mouse click or enter signal. The patients are trained using software that contains a row of icons representing common phrases or a standard qwerty or alphabetical keyboard Common signals used in BCIs The following signals elicited from different parts of brain in response to various stimuli are used as control signals for BCI. 123

56 (i) Mu and Alpha operator conditioning: The mu rhythm is an 8-12 Hz spontaneous EEG rhythm associated with motor activities and maximally recorded over sensorimotor cortex. The alpha rhythm is in the same frequency band, but is recorded over occipital cortex. The amplitudes of these rhythms may be altered through biofeedback training. (ii) Event related synchronization/ Desynchronization (ERS/ERD): Movement related increases and decreases in specific frequency bands maximally located over sensorimotor cortex. Individuals may be trained through biofeedback to alter amplitude of signals in the appropriate frequency bands. These signals exist even when the individual imagines moving as the movement-related signals are preparatory rather than actual. (iii) Slow cortical Potential Operant Conditioning: Large negative or positive shifts in the EEG signal lasting from 300 ms up to several minutes. Individuals may be trained through biofeedback to produce these shifts. (iv) P3 component of Evoked Potential: A positive shift in the EEG signal lasting from ms after a task relevant stimulus. Maximally located over the central parietal region, this is an inherent response and no training is necessary. (v) Short latency visual evoked potential: To produce the component a response to the presentation of a short visual stimulus is necessary. Maximally located over the occipital region, this is an inherent response and no training is necessary. (vi) Individual Neuron Recordings: Individuals receive implanted electrodes that may obtain responses from local neurons or even encourage neural tissue to grow into the implant. Operant conditioning may be used to achieve control or the natural response of a cell or cell may be used. (vii) Steady State Visual Evoked Potential (SSVEP): A response to a visual stimulus modulated at a specific frequency. The SSVEP is characterized by an increase in EEG activity at the stimulus frequency. Typically the visual stimulus is generated using white fluorescent tubes modulated at around Hz or by another kind of strobe light Visual Evoked Potentials Neurological phenomena are specific features of the brain activity that appear in the brain signals and can be used to control a BCI system. They are time-locked to a physical stimulus or to the cognitive responses of the brain [55]. Neurological phenomena are characterized by their voltage amplitude, their latency which is related to the internal or external stimuli and their spatiotemporal distribution. Their amplitude is usually much smaller than the background EEG signal. 124

57 Definition And Types Evoked Potentials (VEPs) are small changes in the brain signal, generated in response to a visual stimulus such as alphabets display. They display properties whose characteristics depend on the type of visual stimulus. Visual Evoked Potentials (VEPs) originate from neurons in the cortex, outer layer of brain and exhibit oscillating, eventrelated potentials in its nature. These signals are responses to visual stimulants and extensively used in neuropsychological studies. The main components of ERP s are exogenous and endogenous. Physical stimuli are referred as exogenous components. These components include a Negative waveform around 100ms (N1) and a Positive waveform around 200 ms after stimulus onset (P2). Visual Evoked Potentials fall into this category. Endogenous components are components which are influenced by cognitive factors. These potentials reflect the visual information processing mechanism in the brain. With reference to the knowledge of brain electrophysiology, potentials generated corresponding to rapidly repetitive stimulations is referred as steady-state VEP (SS-VEP). Specifically transient VEP (TVEP) is a true transient response to a stimulus when the relevant brain mechanisms are in resting states and independent of previous trials. If the stimulus presentation rate is shorter than the duration of a TVEP, the potentials generated for each stimulus will overlap each other to yield SSVEP, which corresponds to brain s steady state excitability. The analysis of the TVEP is based on temporal methods such as template matching, whereas SSVEP detection is usually performed by frequency analysis such as power spectral density estimation P3 Component A positive wave peaking at around 300ms after task-relevant stimuli is known as P3 component. While the P3 is evoked by many types of paradigms the common factors that influence it are stimulus frequency and task relevance. The P3 is found to be fairly stable in locked-in patients, re-appearing even after severe brain stem injuries. Using a brain cognitive signal like the P3 has the benefit of enabling control through a variety of modalities as the P3 enables discrete control in response to both auditory and visual stimuli. As it is a cognitive component, the P3 has been known to change in response to subject fatigue Steady State Visual Evoked Potential (SS-VEP) 125

58 Eliciting Steady-State Visual Evoked Potential (SS-VEP) enhances the effectiveness of the application by providing selection capability among multiple commands. The visual stimulus that generates oscillatory components in the user s EEG, especially in the signals from the primary visual cortex is referred as Repetitive Visual Stimulus (RVS). SSVEPs can be elicited by repetitive visual stimuli at frequencies in the 1 to 100 Hz range [58]. In SSVEP based BCIs the classification accuracy is primarily influenced by the strength of the SSVEP response, the signal-to-noise ratio (SNR), and the differences in the properties of the stimuli. The signal strength of the SSVEP is contributory factor of system s classification speed. The following factors should be taken into consideration for RVS presentation to the user. (i) Stimulation frequencies (ii) Shape of Stimuli Repetitive Visual Stimuli (RVS) Classification The three main categories of repetitive visual stimuli are widely used in various BCIs. (i) Light stimuli. (ii) Single graphics stimuli. (iii) Pattern reversal stimuli Light Stimuli Light stimuli are rendered using light sources such as LED s, fluorescent lights and Xe-lights which are modulated at a specific frequency. These devices generally driven by dedicated electronic circuitry which enables them to accurately render any illumination sequence or waveform. The intensity (time integrated luminance) of the light stimulus is measured in photopic candela seconds per square meter (cd s m 2 or nits s) because the light luminance changes over time, whereas the background luminance is measured in candela per square meter (cd m 2 or nits) [43]. An important parameter to quantify the stimulus strength is the modulation depth which is defined as (lmax lmin) / (lmax+ lmin), where lmin, lmax are the minimum and maximum luminance, respectively Single Visual Stimuli In this scheme, Geometrical shapes such as rectangle, square or arrow are rendered on a computer screen and appear from and disappear into the background at a 126

59 specified rate. The stimulation rate is reported as the number of full cycles per second, normally simply referred to as the frequency of the stimulus Pattern Reversal Stimuli These stimuli were presented on a computer screen by oscillatory alternation of graphical patterns. The widely used stimuli patterns are Checkerboards and Line Boxes. The Checker boards consist of at least two patterns that are alternated at a specified number of alternations per second. Patterns are usually colored in black and white. A checkerboard stimulus is characterized by the subtended visual angle of each tile (spatial frequency), the number of reversals per second, the mean luminance, the field size, and the pattern contrast. It is worth noting that single graphic stimuli could be viewed as a special case of pattern reversal stimuli where the graphic is the first pattern and the second pattern is the background. An important difference is that single graphic stimuli elicit an SSVEP response at the frequency of one full cycle (i.e. two alternations), whereas real pattern reversal stimuli elicit an SSVEP response at the frequency of one alternation. All repetitive visual stimuli have various properties such as frequency, color, and contrast. Both the type and properties of stimuli affect the elicited SSVEP response Stimulation Type The following factors fix the elicitation response to visual stimuli (i) Type of RVS stimuli used (ii) Frequency range (iii) Geometrical Shapes used (iv) Stimulus frequency used (v) Device used Characteristics of Single Graphic stimuli For low frequency band, (i) Geometrical Shapes used Square, Cube, Arrow (ii) Stimulus frequency used Hz, 6.86 Hz and 10 Hz. (iii) Colors used White / Black For Medium frequency band, (i) Geometrical Shapes used Box, Rectangle (ii) Stimulus frequency used (Hz) 14, 16, (iii) Device used LCD, CRT. 127

60 (iv) Colors used White / Black (v) Bit transfer rate 7.5 For Low and Medium frequency band, (i) Geometrical Shapes used Square, Button, Rectangle, Block (ii) Stimulus frequency used (Hz) 6.5, 7.5, 8.6, 10, 12, 15, 17, (iii) Device used LCD, CRT. (iv) Colours used White / Black, Red, Green, Yellow. (v) Bit transfer rate 27.5 to Characteristics of Pattern Reversal Stimuli For low frequency band, (i) Geometrical Shapes used Checker Board (ii) Stimulus frequency used (in Hz) 5, 5.625, 6.9, 8.5, 10, 12,. (iii) Device used TFT, LCD (iv) Colours used White / black (v) Bit transfer rate 7.5 For Medium frequency band, (i) Stimulus frequency used 17 Hz, 20 Hz (ii) Device used LCD, CRT. (iii) Colours used White / Black (iv) Shape used Checkerboard For low and medium frequency band, (i) Geometrical Shapes used Checkerboard and line boxes (ii) Stimulus frequency used (Hz) 8.8, 9.4, 11.55, 12.5, (iii) Device used CRT (iv) Colours used White / Black. (v) Bit transfer rate 27.5 to Comparison Between Various Stimuli Pattern reversal stimuli can produce a more pronounced SSVEP than single graphic stimuli modulated at the same frequency. The SSVEP response elicited by an LED was larger than that by a rectangle stimulus on a computer screen. The SSVEP response for light stimuli was than that for pattern reversal. This property enables BCIs using LED stimuli exhibit high bit transformation rate than BCIs using computer screens 128

61 as a source for stimuli presentation. The variables responsible for results are luminance, contrast and colour. The power of the SSVEP response is affected by both frequency and colour of the stimuli. BCIs using light stimuli usually require the development of dedicated hardware. LEDs are said to be preferable in practical applications that require more than twenty choices, as its performance rely on displaying various stimuli at different frequencies. The refresh rate R of the monitor that is, the number of times that the monitor redraws the screen per second, is usually lower than 100 Hz. Only frequencies that are not lower than R/2 Hz can be used. Upon not meeting this criterion, frequencies will not elicit SSVEP which sabotages the system throughput Slow Cortical Potentials The slow voltage changes generated in cortex corresponds to lowest frequency features of the scalp recorded EEG. These potential shifts occur over s and are called slow cortical potentials (SCPs). They are classified into Positive SCPs and Negative SCPs. Negative SCPs are typically associated with movement and other functions involving cortical activation, while positive SCPs are usually associated with reduced cortical activation. The most important characteristic of SCP is, elicitation is controllable through proper learning. The principal emphasis has been on developing clinical application of this BCI system. It has been tested extensively in people with late-stage ALS and has proved able to supply basic communication capability. In the standard format, EEG is recorded from electrodes at the vertex referred to linked mastoids. SCPs are extracted by appropriate filtering, corrected for EOG activity, and fed back to the user via visual feedback from a computer screen that shows one choice at the top and one at the bottom. Selection takes 4 s. During a 2 s baseline period, the system measures the user s initial voltage level. In the next two seconds, the user selects the top or bottom choice by decreasing or increasing the voltage level by a criterion amount. The voltage is displayed as vertical movement of a cursor and final selection is indicated in a variety of ways. The BCI can also operate in a mode that gives auditory or tactile feedback. Users train in several sessions/week over weeks or months. When they consistently achieve accuracies of 75%, they are switched to a language support program (LSP). The LSP enables the user to choose a letter or letter combination by a series of two-choice selections. In each selection, the choice is between selecting or not selecting a set of one or more letters. The first two selections choose between the two halves of the alphabet, the next two between the two quarters of the 129

62 selected half, and so on until a single letter is chosen. A backup or erase option is provided. While these rates are low, the LSP has proved useful to and highly valued by people who cannot use conventional augmentative communication technologies. Furthermore, a predictive algorithm that uses the first two letters of a word to select the word from a lexicon that encompasses the user s vocabulary can markedly increase the communication rate. A new protocol provides Internet access to one disabled user. A stand-by mode allows users wearing fixed electrodes to access the system 24 h/day by producing a specific sequence of positive and negative SCPs. This sequence is essentially a key for enabling and disabling BCI Wavelets For Feature Extraction The wavelet transform has emerged over recent years as a powerful timefrequency analysis-coding tool suitable for use in manipulation of neuroelectric waveforms such as EEG and Event Related Potential (ERP) recordings from multiple electrode arrays Definition And Characteristics Time-domain wavelets are simple oscillating amplitude functions of time. Scaling and Translations are the two basic parameters of wavelet representations. The set of all scaled and translated wavelets of the same basic wavelet shape forms a wavelet family. (i) They are localized in both time and frequency. (ii) They have large fluctuating amplitudes during a restricted time period and are Very low amplitude or zero amplitude outside of that time range. Wavelets are Band-limited. They are composed of not one but a relatively limited range of short frequencies (iii) They exhibit sensitivity to full-scale range of neuroelectric waveform structure. (iv) They provide effective time-frequency decomposition of signal over a range of Characteristic frequencies that separates individual signal components. (v) The time or space resolution improves as the scale of a neuro event decreases Reasons For Opting Wavelets The following characteristics play a major role in usage of wavelets in multiple applications. 130

63 (i) Localizing the occurrence of transients and component event in neuroelectric waveforms. (ii) The wavelet representation of a neuroelectric waveform is invertible, i.e., original waveform can be reconstructed from a set of analysis coefficients that capture all of time and frequency information. Wavelet shape can be selected or designed to match the shape of components embedded in ERPs or of events such as single trial ERPs embedded in the EEG. Such wavelets are excellent templates to detect and separate those components and events from the background waveform. The ability to model temporal and spectral properties of specific components of neuroelectric waveforms provides optimum resolution of concerned events. Meyer wavelets were developed which matched closely to the shape of many neuroelectric waveform. Wavelet transforms analysis has now been applied to a wide variety of biomedical signals, including electroencephalogram (EEG), electrocardiogram (ECG), clinical sounds, respiratory patterns, blood pressure trends, and DNA sequence Numerical Implementation Of Wavelet Transforms Wavelet transforms, as they are in use today, come in essentially two distinct varieties or classes; the (CWT) and the discrete wavelet transform (DWT) Continuous Wavelet Transform The CWT is a time-frequency analysis method which has a variable window width, which is related to the scale of observation, a flexibility that allows for the isolation of higher frequency features. CWT is not limited to using sinusoidal analyzing functions. The wavelet, a function of a continuous time signal, x(t), is defined as T(a,b) = 1/ a (3.1) Where Ψ*(t) is a complex conjugate of analyzing wavelet function Ψ (t), a dilation parameter of the wavelet and b is its local parameters. Original signal may be reconstructed using inverse transform X(t)=1/c g ab (t) da db/a 2 (3.2) 131

64 In practice, a fine discretization of the CWT is computed, where usually the b location is discredited at the sampling interval and the scale is discretized at sampling interval and the a scale is discretized logarithmically. 1) Wavelet ridges defined as d( T(a.b) 2 /a)/da=0 (3.3) are used for determining instantaneous frequencies and amplitude of signals components. 2) Wavelet modulus maxima defined as d T(a,b) 2 /db=0 (3.4) are using for locating and characterizing singularities in the signal. These can be easily removed when implementing the modulus maxima method. There are many continuous wavelets however, by far, the most popular are Mexican hat wavelet and the Morlet wavelet The Mexican Hat Wavelet The Mexican Hat Wavelet is the second derivative of a Gaussian function given by Ψ(t)=(1-t 2 ) exp( - t 2 / 2). (3.5) has been in morphological characterization of engineering surface, the interrogational of laser-induced ultrasonic signals used to measure stiffness coefficients in a viscoelastic composite material The Morlet Wavelet The Morlet wavelet is the most complex wavelets used. The complex Morlet is defined as Ψ(t)=1/ 4 exp(w 0 t) exp(- t 2 / 2) Where ω 0 is the central frequency of mother wavelet The Discrete Wavelet Transform The DWT provides a nonredundant, highly efficient wavelet representation that can be implemented with a simple recursive filter scheme. It can produce as many coefficients as there are samples within the original neuroelectric waveform without 132

65 information loss. The DWT permits perfect reconstruction of the original neuroelectric waveform by an inverse filtering operation. The DWT employs a dyadic grid (inter power of two scaling in m and n ) with orthonormal wavelet basis function and exhibits zero redundancy. This kind of discretization of the wavelet has the form Ψ m,n (t)=1/ a m 0 Ψ(t-(nb 0 a m 0 /a n 0 )). (3.5) where the integers m and n control the wavelet dilation and translation respectively Advantages (i) The DWT provides orthogonal basis functions, for representing real functions such as neuroelectric waveforms. This property can be used to construct any neuroelectric waveform by adding together point-to-point in time all of the orthogonal wavelets in the subset, whose individual amplitudes were set. (ii) Computation of DWT coefficients for at starting and ending points of neuroelectric waveforms is sufficient. (iii) The scales and translations in a DWT are discrete and sparse. (iv) DWT can decompose neuroelectric signals into bandpass signals that tend to selectively capture the energy in each of the classic EEG rhythmic bands with a time resolution Implementation Schemes The schematic representation of recursive pyramidal filter scheme and decomposition scheme using DWT for a neuroelectric waveforms is shown in Fig. 3.3 and Fig

66 High pass Filter 2 Wavelet coefficient (for detail function) Neuro electric Signal input Low pass Filter 2 Scaling function coefficient Wavelet coefficient (for detail function) 2 High pass Filter Scaling function coefficient (For low resolution solution) 2 Low pass Filter Detail Coeffic ient Figure 3.3 Schematic representation recursive pyramidal filters Level 2 level 3 (60-64 Hz) Level 1 (48-64 Hz) (56-64 Hz) (56-60 Hz) (48-56 Hz) (48-52 Hz) Gamma (52-56 Hz) 134

67 (32-64 Hz) (40-48 Hz) (44-48 Hz) (32-48 Hz) (40-44 Hz) (36-40 Hz) ( Hz) (32-36 Hz) 1 Beta 2 Beta(16-32 Hz) (24-32 Hz) (28-32 Hz) (24-28 Hz) (0-30 Hz) Beta 1 (16-24 Hz) (20-24 Hz) (16-20 Hz) (12-16 Hz) (8-12 Hz) Theta (4-8 Hz) Delta (0-4 Hz) Standard DWT Custom wavelet Packet decomposition, 1 Neuroelectric waveform Fig. 3.4 Wavelet Decomposition Scheme for Composite EEG signal The output of the low pass filter is the set of DWT coefficients associated with a set of companion functions called scaling functions. The high pass coefficients used to create a component waveform known as detail functions. The core wavelet of a DWT is referred as mother wavelet. DWT uses wavelets as octave harmonic filters, holding the ratio of the center frequency of the wavelet filter to its bandwidth constant across scales. A scheme depicting full decomposition tree for a four level decomposition is shown in 135

68 Figure 3.4. This property known as the constant Q property is useful for many filtering applications to neuroelectric waveforms because the spectra of EEG and ERP waveforms tend to have a structure that approximately follows the constant Q pattern Wavelet Mapping To Neuroelectric Waveforms A best basis decomposition is defined as the pathway through the filtering tree that captures the information in a neuroelectric waveform most efficiently. Wavelet transforms offers flexible options to choose basis functions for analyzing time-varying neuroelectric waveforms. The shapes of the basis functions will resemble shape of the waveforms under consideration. There are two methods for selecting basis functions as follows (i) Matching pursuit technique (ii) Direct design technique provided by matched Meyer wavelets Matching Pursuit Examining the spectral properties of a neuroelectric waveform over segments of different size and location and then select a set of basis functions from a large dictionary of basis functions or atoms that closely match the spectral properties of regions under analysis, is the underlying phenomenon of matching pursuit technique. This technique used to characterize the spectral properties of EMG activity and examining its seprability from scalp recorded EEG Disadvantages Of Matching Pursuit Technique (i) The effectiveness of the technique depends the dictionary that is consulted. (ii) The choice of basis functions is driven entirely by the local surface features of the neuroelectric waveform, not by any consideration of the underlying component structure of the waveform. Fig. 3.5 gives the algorithm for extraction of feature components from recorded electrical activity of brain Wavelet Denoising Algorihm For Feature Extraction 136

69 Start EEG waveforms are time-shifted for 0,1, D-1. Creation of Additional Channels using time shifted data to form a matrix of size [JD, T,N], The matrix gives dimensions of duration and component Along the time dimension, trials are concatenated for data reorganization to form a matrix of size [JD, TN]. Extract row values representing channel/time shift.. Extract column values which correspond to specific time sample/trial. Fig. 3.5 Wavelet for Feature The uses time shifted Denoising Algorithm Extraction proposed algorithm versions of EEG signal under consideration. It forms a matrix which gives dimensions of duration and component size. The trials are concatenated along the time dimension for data reorganization. The component series is obtained by performing PCA. The resultant series is normalized for reorganization. A matrix that describes the dimensions of component, time, trial is formed by data reorganization. A matrix of size[jd,t] is obtained by applying the define contrast function. A rotation matrix of components of size [JD,T,N] is obtained by re-performing PCA. The row values and column values are extracted for A specific feature of size [T,N] is extracted. the specific channel and sample. The resultant 137

70 component series is subjected to component analysis operation and the series is normalized for reorganization. The rotation matrix of specific components is obtained by re-performing component analysis operation and specific features are extracted Matched Meyer Wavelets This technique constructs a member of a flexible class of band-limited wavelets, the Meyer wavelets, whose spectrum matches the spectrum of any band-limited signal as closely as possible in a least square sense. An associated scaling function and high and low pass filters are then derived that can be used to perform a DWT on any neuroelctric waveform. The wavelets used can be classified into physiologically unnatural wavelets and physiologically natural wavelets. These wavelets provide better frequency selectivity than the Haar and Daubechies wavelets.it can be concluded that wavelet analysis offers increased power to resolve transient and scale-specific events in neuroelectric datasets, to precisely filter neuroelectric waveforms for noise reduction, to efficiently store and transmit neuroelectric waveforms and images and to observe and quantify their smallscale structure in time and space Conclusion This Chapter gives a detailed account of evolution of BCI in the present context as assistive devices, Visual Evoked Potential origin, Various type of stimuli which elicit VEP, Types of VEP, Wavelet transform and it s classification, type of waveletes used for neuro electric waveforms processing and brief description wavelet denoising algorithm for feature extraction Introduction CHAPTER 4 ELECTRODES AND EXPERIMENTAL SETUP 138

71 The functionality of electrodes is to capture and record electric potentials generated in the brain. By using conduction paste as an interface between scalp and electrodes can potentials can be transferred to EEG recording equipment from patients by using conduction paste as an interface. A cohesive and stable placement of electrodes can act as a method to reduce movement artifacts associated with EEG signals. Amongst various types of metals such as, platinum, gold, stainless steel, Silver-Silver chloride electrodes are most widely used electrodes in recording DC and very low frequency potentials and they exhibit high fidelity. Depending upon the value of electrode potential and signal of interest the electrodes are broadly classified into two categories: 1. Reversible Electrodes made up of platinum or iridium 2. Non Reversible Electrodes. The characteristics that the electrodes should comply are, 1. Materials with high conductivity. 2. They should form contact through double layer charge with the conduction paste which improves the effective skin coverage. This property makes electrode to act as low frequency filter due to capacitance at double layer. 3. They should be non polarized or reversible. Polarized electrodes have high resistance and capacitance which degrades the quality of EEG. Reversible electrodes do not easily get polarized. 4.2 Preparation requirements For better EEG recording the scalp impedance should be reduced. This is accomplished by scalp rubbing with alcohol or other cleansing agent. Excessive rubbing can be irritating and bleeding can raise the impedance. The sensitivity of patients towards conduction paste should be assessed. For patients sensitive to salt solution or bentonite the proposed alternatives are electrode attachment media such as sodium chloride paste or gel, conducting sponges and other specialized electrodes. The common types of EEG electrodes are Scalp Electrodes 139

72 Figure 4.1 Scalp Electrode The scalp electrodes belong to reversible type as shown in Fig A chloride layer on silver discs will yield scalp electrode with the dimension of 4-10mm in diameter. The resistance of the electrodes are constrained to few hundred Ohms and kept below 4 KΩ for optimal recording Subdermal Electrodes Figure 4.2 Sub dermal electrodes Subdermal electrodes are made from platinum or fine stainless steel as shown in Fig The dimensions are chosen as 10mm length and 0.5mm in diameter. The resistance of these electrodes is in the range between 10KΩ and 15KΩ. This property makes these electrodes to introduce electrode artifacts Usage Method The electrode is inserted into the subdermis through the horny layer of the skin, after cleansing. The insertion is made through the needle hub parallel to the scalp. To 140

73 avoid transmission of infectious and transferable diseases these electrodes are subjected to sterilization. The main disadvantage associated with usage of these electrodes lies with painful recording procedure Clip Electrode Figure 4.3 Clip electrode Clip electrodes are categorized as cup electrodes as shown in Fig These electrodes are used as referential electrodes. They exhibit characteristics similar to scalp electrodes. They are clipped in the ear lobes for referential recordings. Usage of these electrodes introduces movement artifacts in the EEG Nasopharyngeal Electrode 141

74 Fig. 4.4 Nasopharyngeal Electrode For recording inferior temporal or frontal discharges, Nasopharyngeal electrodes are used, which are shown in Fig The structure is made up of flexible insulated wire with an uninsulated 2mm tip, with the length of 10 cm. The nasopharyngeal electrode improve the yield of interictal spikes Usage Method A single nasopharyngeal electrode is inserted in each nostril. The wires are bent into S shape, threaded along the nasopharynx and rotated upward to their position within 2 cm of the anterior mesial surface of temporal lobe. A 1-10 KΩ resistor should be placed in series with silver wire to limit current flow. They have a tendency to generate respiratory motion artifacts Sphenoidal Electrode Sphenoidal electrodes are used to record discharge from anterior parts of the brain. They are thin, straight insulated stainless steel wires with a small uninsulated ball at the tip. The conventional dimensions are 50 mm long and 0.5 mm in diameter. This electrode is introduced through a needle cannula into temporal and masseter muscle at the point between the zygoma and sigmoid notch of mandible. The electrode is lateral to foramen oval at the greater wing of sphenoid. The disadvantages of sphenoidal electrode are infection, transmission of disease and injury to facial and trigeminal nerve Tympanic Electrode 142

75 Fig. 4.5 Tympanic electrode For recording discharges from medial temporal lobe and recording brainstem auditory evoked potentials tympanic electrodes are used shown in Fig These electrodes are thin insulated 7 mm diameter made up of stainless steel, gold or platinum ball. The electrode tip is soaked in a conducting solution are placed on the tympanic membrane. With considerate precautionary measures, injury to tympanic membrane can be avoided Depth Electrodes Fig. 4.6 Depth electrode Depth electrodes are used for detecting and capturing electrical potential from specific points, which are not visible on surface recording and the placement scheme, is fixed by neurosurgeon. These electrodes are made up of thin stainless steel, platinum or gold wires which are insulated with an uninsulated tip. Chlorided silver wires are insulated for depth recording because these irritate the brain as in Fig Usage Method The electrodes are stereotactically implanted. The post implantation time span can be extended as per requirements. The commonly chosen target areas for depth recording are Amygdala and supplementary motor area of frontal lobe. The main advantages of depth recording are low resistance of electrode, high signal fidelity, lack of 143

76 muscle artifact and bypassing the high resistance of skull. This electrode localizes the epileptic form activity. The dis advantages of depth electrode are that all deep structure cannot be evaluated by this method resulting in sampling error and invasiveness of the procedure with inherent risk of hemorrhage, infection, reactive meningitis, oedema and headache Cortical Electrode Fig. 4.7 Cortical electrode The electrodes which are used to record electrical activity of brain on surface level during the neurosurgical procedure are referred as cortical electrodes which results in electrocorticography (ECoG) as shown in Fig This helps to localize epileptic activity during epilepsy surgery. In concurrence with the placement scheme these electrodes are connected to the brain by wicks and these wicks are sterilized by isotonic saline solution soaking. The advantage of these electrodes are adaptability to skull size for effective recording Subdural Electrodes Fig. 4.8 Subdural electrode 144

77 Electrodes which are used localize the seizure activity in functional areas of brain for an awake are referred as Subdural electrodes as in Fig These electrodes are placed under the dura. The targets of recording area such as sensory, motor, speech, reading and cognition are identified by stimulation. To study a large region of cortex, subdural grinds may be assembled in approximately hand size array with up to eight rows and eight columns of electrodes. Subdural electrodes are 3mm discs fabricated from stainless steel or platinum. These electrodes are embedded in a sheath of flexible plastic with center-to-center electrode separation of 1 cm Epidural Electrodes Fig. 4.9 Epidural electrode Epidural electrodes are in single or double row strips as in Fig They are less invasive than subdural grinds and can be placed by burr hole. The disadvantage of epidural electrode is inability to record seizure activity which is attributed to small area of recording. These electrodes do not introduce pain during recordings Electrode Connection To preserve a consistent relationship between location of electrodes and cerebral structure, system of electrode placement scheme approved by International Federation of Societies for EEG and Clinical Neurophysiology (IFSECN) is used in this research scheme. The anterioposterior measurements are based upon the distance between nasion and inion over the vertex in the midline. Five points are marked on this line 1. Frontal pole (Fp) 2. Frontal (F) 145

78 3. Central (C) 4. Parietal (P) 5. Occipital (O) and the lineation scheme and electrode orientations are shown in Fig to Fig

79 Fig Delineation belt ( Front View) Fig Delineation belt (Top View) Fig Delineation belt with Central electrode ( Top View) Electrode Placement Scheme 147

80 1. The first point Fp is 10% of nasion-inion distance from the nasion. 2. The second point F is 20% of this distance back from the point fp and so on in 20% steps back from the central, partial and occipital midline points of nasion and inion and lateral measurements are based on the central coronal plain. 3. The distance is first measured from left to right preauricular points recognized as depression at the root of zygoma just anterior to tragus. 4. The tape should be passed through the predetermined central point at vertex when making this measurement. Ten percent of the distance is then taken from temporal point up from the pre auricular point on either side. 5. The central points are marked at 20% of the distance above the temporal points. The anterior posterior lines are electrode over temporal, frontal to occipital lobe are determined by measuring the distance between Fp midline point through the temporal position of central line and black to mid occipital region. 6. The Fp electrode position is then marked 10% of the distance from the midline and the occipital electrode also positioned at 10% of the distance from the mid line at the back. The interior frontal and posterior temporal position then fall 20% of the distance from Fp and occipital electrode respectively along the line. 7. The remaining mid frontal (F 3, F 4 ) and mid parietal coronal lines respectively equidistant between midline and temporal line of electrodes on either side. 8. To differentiate between homologous positions on left and right side of hemisphere and odd numbers as subscript for right hemisphere. Fp 2, F 4, F 8, F 4, C 4, T 6, P 4, and O 2 become standard position for right hemisphere while FP 1,F 3, F 7, C 3,T 3, T 5,P 3 and O 1 becomes standard position for left hemisphere. Electrodes in midline in frontal, central and parietal regions are designated as F z, C z and P z.. These electrode placement scheme is depicted in Fig to Fig With 21 electrodes and 16 channels there or 1.43 X possible montages. This impractical number necessitates some selected number of montages. Once a montage is selected, it may be programmed into the EEG machine so that it may be executed by a single switch selection. 148

81 Fig Electrode Placement Scheme (Front View) Fig Electrode Placement Scheme (Lateral View) Fig Electrode Placement Scheme (Rear View) Fig Electrode Placement Scheme (Lateral View) 149

82 Fig Electrode placement scheme with Jack box Fig With Photoic Simulation Fig With Photoic Simulation

83 EEG Amplifiers In an current available EEG machines, simple amplifier are located in the jack box instead of differential amplifier are located in the machine itself. From the jack box, the EEG electrode input are connected to montage selector board which is two dimensional arrays of push buttons. Each row contains one button per jack box input. Vertically the array is organized into channels having two rows of button per channel. Each channel s output equal to the voltage of the electrode selected in the row 1 minus the voltage of the electrode selected in row 2. The top input of the pair is sometimes referred to as grid 2 or input 2. One electrode can be an input more than one channel. From the montage selector, the EEG signals go to the amplifiers. The amplifiers of the EEG machine are compound devices and should be distinguished from an amplifier whose only function is to increase voltage. The EEG amplifiers also have filters, voltage dividers, input and output jacks and calibration devices. The amplifier multiplies an input voltage by a constant. The amplifiers are designed to receive input voltages within a certain range referred as dynamic range. Input smaller than this range may be lost in the background noise. Voltage higher than maximum input may be caused damage to the equipment and distortion of signals. Flexible control of the dynamic range is achieved with sensitivity setting of the EEG machine. Sensitivity has the unit of mv/cm or µv/mm and is defined as the amount of voltage required to deflect the recoding pens a given distance. A typical sensitivity of the EEG machine is 7µV/mm resulting in pen deflection of 3-20 mm of typical EEG input voltages. In practice, the sensitivity of EEG machine is adjusted so that the signal of interest produces a pen deflection, which is large enough to be read but not so large so as to collide with nearby channels. The gain of the recordings is important and should be mentioned for proper interpretation. Usually the EEG recordings are obtained at 70 µv/cm. The EEG amplifier itself has a frequency response, which is linear over a wide range of input voltage. In practice, the filter settings are determined by the range of linear frequency response. It is important to choose an amplifier whose frequency response is linear over the expected range of the input voltage so that high or low frequency component is not distorted; therefore, EEG amplifier differs from evoked 151

84 potential amplifier. The montage is known as referential when inputs from each channel are kept isolated from the system ground and a distance electrode is chosen as a common reference for several channels. The term monopolar is inappropriate since a voltage always be recorded between two points. Montages linking nearby electrodes are called bipolar and are less disturbed by noise. Both referential and bipolar montages are used in clinical practice. Differential montages are more sensitive to regional changes in EEG potential. Differential amplifiers cancel out the common mode signal. The common mode amplifier ratio refers to rejection of in phase an amplifying out of phase potential. Good EEG amplifiers have common mode rejection ratio of a least 1000 and may be even as high as 10,000 or 100,000. The reference electrode is placed reasonably close to the recoding electrode, which canceled major nose signals. If the reference electrode is placed at a distant location such as leg then it may act as antenna and pickup signal that may exceed the common mode range of amplifiers. While recording with different amplifiers, the polarity conversion is used as follows: 1. When input 1 is negative with respect to input2, pen deflects upward. When input 1 is positive with respect to input 2, pen deflects downward. 2. While the input 2 is negative with respect to input 1 pen deflects downward, while input 2 is positive with respect to input 1 pen deflect upward. This conversion is responsible for phase reversal of EEG signals by which negative signals can be localized to an electrode demonstrating phase reversal between two channels; e.g. in bipolar montage, channel 1 represent different voltage between Fp 2 and F 8 electrodes and channel 2 demonstrates the different amplitude from F 8 and T 4 electrodes. Since pen deflects down in channel 1, F 8 is negative with respect to Fp 2. As the pen deflects up in channel 2, F 8 is also negative with respect to T 4. At F 8 there is local maximum extra cellular negativity. The electrodes common to the two channels showing phase reversal may be presumed to be near the origin of seizure discharge. Such extracellular negativity occurs during interictal spikes adjacent to the seizure focus as result of sudden influx of sodium and calcium into the depolarizing neurons. Many other EEG events including normal brain activity are the result of local extracellular negativity. 152

85 Extracellular positive potentials cause phase reversal is only applicable to bipolar montages. Output of one amplifier may be used as input of the other. This allows a series of amplifiers each with limited dynamic range to boost EEG signal substantially. As the signal becomes larger than the ambient noise; therefore, a differential amplification may no longer be required. Internal amplifier may hence be single ended, i.e. one active input measured with respect to ground Conclusion The proposed experimental set up was used to record EEG for duration of fifteen minutes for the following events: 1. Eyes Closed 2. Eyes Open 3. Eyes Open with Photoic simulation ON 4. Hyper Ventilation 5. Voluntary Muscle Movements. 153

86 CHAPTER 5 WAVELET ANALYSIS OF COMPOSITE EEG SIGNAL 5.1 Introduction A Wavelet transform converts a periodically sampled, time-domain signal into two dimensions representing time and scale. A scheme had been proposed in this thesis with the following objectives: 1. Characterization of the dynamic behavior of biological signals, such as neuroelectric waveforms to describe spatial distribution of electric potentials and power distribution across various frequency bands. 2. Summarization of EEG signals parameters which enables specific feature extraction. The following mathematical analysis explains the degrees of freedom of wavelet analysis used in this method. The degrees of freedom include: 1. Transform type: This parameter fixes two parameters such as type of decomposition and dynamic behavior of EEG signal. 2. Number of levels: This value is chosen based on the transform type and frequency characteristics of the EEG signal. 3. Wavelet basis function: An effective trade-off has to be maintained between system delay and results in selecting wavelet basis functions. 5.2 Feature Extraction Using Wavelets Features describing the complexity of brains signals can be classified into time series features, spatial features and frequency features [65] Time Series Features Time series features includes the i. Average of the signal (offset) ii. Linear trend of the signal iii. Absolute minimum and maximum values iv. Number and order of local minimum and maximum values 154

87 v. Weight factors describing the matching and positions of predefined patterns and slopes of predefined patterns. Linear trends and offsets are applied to slow wave user training with Locked In Syndrome (LIS) patients. The disadvantage of these features in BCI context is that, direct observation is required in single trial studies Spatial Features The spatial features refer to placement of sensors or electrodes. By estimation of Energy Spectral Density (ESD) of the signal, time series can be described by its spectral characteristics. The ESD can be used to identify the important frequency components that change with psychological activities of the user. For BCI signals the spectral analysis is an important method, as the brain generates the task-dependent activity in relatively small frequency components. Feature Classifies are categorized into Linear and Non Linear. In terms of robustness, linear classifier holds edge over non linear. This advantage is attributed to tuning less number of free parameters, less prone to over-fitting. However, linear systems are ineffective in the presence of artifacts, but this can be managed by using regularization. The influence of the following parameters is limited by regularization: i. Strong Noise ii. Classifier complexity iii. Irregularity of the decision vector. The reliance on non linear classification methods has to be limited as they involve appropriate selection of large number of parameter values. Some BCI designs have reported classification algorithms such as FIR-MLP which uses temporal information of the input data. These techniques are efficient in recognizing time series input data. The dynamic nature of the data drives to opt for techniques which are based on temporal features. The performance of the BCI designs can be improved by choosing these classifiers over static classification techniques. Combination of multiple classifiers may yield better results over a single classifier which will increase the performance of BCI systems. Graz BCI is one of the designs that employ this approach. The contribution by each committee member in classifying the information will determine the overall efficiency of the committee s classification rate. This technique is useful in combining information from several channels and spatial regions. 155

88 Many feature extraction methods have employed psychological task based BCI, such as ESD, frequency band powers, asymmetry ratios scalar and multivariate autoregressive coefficients, eigen values of correlation matrix. This scheme utilizes calculation of ESD and rhythmic bands. ESD are parameters to identify mental tasks associated with left hemisphere. 5.3 EEG Signal Model The EEG signal, representing the neural activity of the brain, which includes rhythmic responses and evoked responses can be defined by Eqn X(t) = X R (t) + X E (t) (5.1) The component X R (t) results from superposition of potentials generated by cerebral activity, X R (t) = M B S A (t). The component X E (t) corresponds to evoked potentials in response to presented stimuli, X E (t) = N B S B (t), where M B and N B are mixing matrices and S A (t), S B (t) are contributive elicited potentials to rhythmic and evoked responses respectively. 5.4 Disadvantages of Fourier Transform in Feature Extraction Assuming VEP as continuous function of a continuous scalar argument y(x), and x, y Є R, which is squarely integrable, i.e., y 2 = R y 2 (x) dx <. (5.2) The series can be described as in equ. (5.3), y j (x) = (5.3) The following two conditions are tested in this work on EEG signals, i. orthogonal property (5.4) ii. Signal existence (5.5) Upon Condition Compliance, the Fourier coefficients display detail frequency spectrums of signal and every coefficient is obtained as R y(x) cos(2πf k x) dx R y(x) sin(2πf k x) dx (5.6) Eqn. 5.6 provides Fourier coefficients for EEG imposed with VEP by integration over the whole signal interval. Formalizing the notion of frequency and space/ time 156

89 uncertainty leads to lower bounds on the product of both. It can be given in the following standard form. Let ( ) = { (x) } = R (x) exp (-j x) dx (5.7) (x) = -1 { ( ) } = R ( ) exp ( j x ) d. (5.8) These Fourier Coefficients destroy all space or time information. These coefficients do not tell anything about the location of nonstationarity or singularity in the signal. 5.5 Energy Spread Calculation The time dynamics of EEG signal can be captured by visualization through fixed size window and Fast Fourier Transform (FFT) analysis over this window yields frequency domain description, the effectiveness of this technique is limited by fixed size window. It can be observed that STFT is exhibiting Heisenberg s uncertainly in providing exact knowledge of either time or frequency. The principle is mathematically described as ½, (5.9) where =, (5.10) =. (5.11) Here and are energy spreads of the signal and its Fourier transform with respect to the zero values of ( ) and (x). 5.6 HAAR Expansion System An effective localization in space domain is a pre requisite for uniform approximation of neuro electric waveforms. This improves the efficiency of the feature extraction schemes. The Haar basis system is the simplest method to accomplish this objective. For an HAAR expansion system, to process composite EEG signal the following steps were employed [69]: 1. A function was defined which acts as the indicator of the specific segment associated with EEG signal. 157

90 2. For a defined range, the selected function defines the segment. 3. The authenticity of the selected segments is verified by subjecting translated or shifted versions the function to orthonormal property. 4. The extracted segment is decimated by a specific indicator function to obtain sub segments. 5. The decimation process is iterated by generalized indicator function to obtain a group of subsegments. 6. The generalized indicator function is dependent on two parameters i. Dilation parameter j. ii. Translation parameter k. 7. It was observed that a larger j provides a finer approximation with smaller segments. 5.7 Relationship between Segments The relationship between the derived segments can be established by the following scheme: 1. A different range is fixed. 2. Corresponding to this range another Indicator function is defined. 3. This Indicator function also yield the segment defined by the range. 4. By testing orthonormal property on the indicator function the trueness of segment is established. 5. It was found that segments yielded by the above selected two indicator functions are related with each other. The disadvantage of HAAR expansion system is found to be that the orthonormal base function is dependent on segment width or dilation parameter j. 5.8 Mathematical model of Wavelet Analysis of EEG signal This section narrates expected signal model produced by the wavelet analysis scheme. The Analysis of composite EEG signal at different scales can be mathematically modeled as, (x) (5.12) (5.13) 158

91 = y, = (x) (x) dx (5.14) = y, = (x) (x) dx. (5.15) α o,k =,β j,k = y, (5.16) The representation in Eqn is composed of two summands. The first one is the basic lower frequency part given by the scale functions or father wavelet 0,k (x). The second one is a high-frequency component or mother wavelet j,k (x). The coefficients, referred as wavelet s spectrum is given in Eqn and Eqn which give power distribution of the signal. 5.9 Wavelet Selection The proposed feature extraction scheme uses a wavelet which satisfies the following criterion [69]: 1. The two functions namely scale and wavelet functions are also referred as father and mother wavelets. These functions perform smoothing and differentiating operations which segregates low frequency components and high frequency components of composite EEG signal. On satisfaction of this condition these functions can be extended to multiscale basis functions. This condition is referred as vanishing moment condition given in Eqn R x i (x) dx =, R x i (x) dx = 0, i = 0,.., m 1. (5.17) 2. The condition for basis function selection is i. Inter Scale Orthonormality: It should exhibit orthonormal property for different values of dilation parameter values and same translation parameter values. ii. Intra Scale Orthonormality: It should exhibit orthonormal property for different values of translation parameter values and same dilation parameter values. j,k (x), j,k (x) = (5.18) j,k (x), j,k (x) =, (5.19) 159

92 The properties in Eqn and Eqn are by nature non-trivial and show a condition that scale and wavelet functions and their translation and dilation give the orthonormal basis. It is important to find the finite support between continuous scale and wavelet functions with supports of minimum size for any given order m. These parameters will fix effective compromise between dilation parameter value and size of support functions. Applying the discussed criterion which describes type of wavelet to be used and decomposition order, Daubechies (db5) wavelet was selected for the research scheme Orthogonal wavelets In practice, finite support and compact wavelets are more popular due to their relations to multi-resolution filter banks. These wavelets have finite impulse response (FIR) wavelet filters. Among these wavelets, the most commonly used wavelets can be categorized into two classes: orthogonal and bi-orthogonal wavelet systems. Orthogonal wavelets decompose signals into well-behaved orthogonal signal spaces. In this case, however, the analysis and synthesis filters are not symmetric, a condition that might be required in some applications like image processing. Bi-orthogonal wavelets are more complicated and are defined based on a pair of scaling and wavelet functions [70]. Due to more flexibility in this case, the analysis and synthesis filters can be forced to be symmetric and hence be useful for applications that demand linear phase filtering. The orthogonal wavelets form a local space-frequency basis that can be interpreted as a tiling that covers the space-frequency space with wavelet space-frequency boxes Mathematical model of decomopostion The father wavelet (x) generates the following linear segments with proper indicator function V 0 = { ( x s ) c s : < }, (5.20) V 1 = { f(x) = y(2x) : y Є V 0 }, (5.21)..... V j = {f(x) = y(2 j x) : y Є V 0 }, (5.22) The calculation of wavelet coefficients assumes that a partition of the area of composite EEG signal on segments [ 2 -j k, 2 -j (k+1) ] is the support of the corresponding 160

93 basis functions where k is translation parameter. A high absolute value coefficient narrates the specific location. This information is local in space and frequency Algorithm For EEG Processing The Wavelet Analysis can be implemented in numerous ways. The effectiveness of these schemes is measured by the ability to provide generalization which enables the user to construct arbitrary transforms. These algorithms are found to be ideal suit for offline analysis and ineffective in standalone processing environment. In this research the different feature extraction modules using DWT are developed Designing A Wavelet Based System Step 1: A module had been constructed to read the EEG signal with specific event. The basic requirement is the signal to be analyzed should be captured without any line noise. The presence of line noise acts as an artifact which deters the quality of the feature extraction scheme. Step 2: The fundamental parameters are initially fixed for wavelet analysis. These include the type of transform, the type of decomposition, the number of levels in the decomposition and the type of wavelet filter. To cater degrees of freedom A trade-off between order of basic functions and system delay is ensured to accomplish good results. Step 3: The wavelet coefficients are manipulated to accomplish feature extraction through discarding coefficients associated with artifacts. This is implemented in section Step 4: A mapping between analysis and synthesis structures is obtained through signal reconstruction. The signal reconstruction is achieved through synthesis structure that matches the analysis structure in an optimal way. This module is implemented in section Selection and display of EEG waveform 161

94 This code acts as a communication interface between user and database. This database is repertoire of EEG waveforms which will be presented to the user. The code interacts with the user by providing browse option and offers a dialog box that lists available waveforms in the folder. It enables the user to select an EEG waveform with the specific extension which will be subjected to various operations for extracting feature. The selected waveform is validated against the same array contents and the result is conveyed to the user. On proper selection the selected waveform is assigned with a storage location. The stored waveform is read using the storage path. The image is converted to gray scale image and displayed. DISPLAY browse for the image ; DISPLAY press any key to continue ; DISPLAY select waveform with extension.bmp READ eeg_image; IF eeg_image_arrays_are_same THEN display proper selection ELSE Display improper selection ; SET image_path = READ(eeg_image); CONVERT eeg_image TO eeg_gray_image; SET x= eeg_gray_image; READ x; SHOW x; Wavelet Selection, Decomposition Vector Calculation, Energy Calculation A Discrete Wavelet domain with symmetric extension ability is chosen as operational platform for this module. The energy associated with wavelet coefficients completely characterizes the signal under analysis. Each level captures the same amount of time, but each level has fewer coefficients by a factor of two. The user selects a wavelet function from a given set of functions. The selected function is assigned with an identifier for global use. A new class of objects using the selected function, are created and displayed. The gray scale version of selected EEG waveform is subjected to decomposition operation. The book keeping matrix gives the size of approximation coefficients and detail coefficients. By using the decomposition vector as input, energy 162

95 levels corresponding to approximation and details coefficients are computed. This segment determines the transform type and type of decomposition is performed. SELECT wavelet_mode; SELECT wavelet_name; ASSIGN type_name = wavelet_name; DISPLAY type_name; // DECOMPOSITION VECTOR CALCULATION for i = 1 to 4 COMPUTE WC[i] =wavelet_decomposition[x]; COMPUTE S[i] = size_wc[i]; CONSTRUCT decomposition_matrix[i] = [WC[i],S[i]]; i = i + 1; end; // ENERGY COMPUTATION for j = 1 to 4 INPUT decomposition_matrix_[i]; j = j + 1; end for k = 1 to 4 COMPUTE energy_approximation_coeff[k] = wave_energy[wc[k],s[k]]; COMPUTE energy_horizontal_coeff[k]=wave_energy[wc[k],s[k]]; COMPUTE energy_vertical_coeff[k]= wave_energy[wc[k],s[k]]; COMPUTE energy_diagonal_coeff[k]= wave_energy[wc[k],s[k]]; k = k + 1; end Construction of Analysis and Synthesis stage requires a performance compromise between structure and the type of decomposition. The efficiency of this construction is measured through amount of latency it introduces between input resource signal and the reconstructed output. The most important parameter in fixing the latency is level of wavelet decomposition. The fundamental step in the decomposition structures splits the two band structure of lowest component. This decomposition format is iterated in which node growth forms a tree, till the decomposed signal is available in proper number of 163

96 bands, designated as j. The number of levels in the decomposition is typically a function of the transform type and the frequency characteristics of the resource image Coefficient Extraction From Decomposition Vector And Display This code is fed with decomposition vector as input. It uses four functions namely approximation_coeff_calc, detail_coeff_hor_calc, detail_coeff_ver_calc, detail_coeff_dia_calc for decomposition. These functions perform extraction operation on decomposition vector and yields approximation coefficients, horizontal coefficients, vertical coefficients and diagonal coefficients. The decomposition level ranges from 1 to 4. A parameter for size ncolors sz is calculated. This code also employs two more functions matrix_rescale and coeff_deci. The matrix_rescale function scales the approximation coefficients, horizontal coefficients, vertical coefficients and diagonal coefficients by a value of ncolors from present value. The coeff_deci function selectively extracts the coefficients where the selection size is derived from the value of sz. for i = 1 to 4 INPUT decomposition_matrix[i]; i = i + 1; end // COEFFICIENT EXTRACTION for j = 1 to 4 COMPUTE CA[j] = approx_coeff_calc[wc[j], s[j]]; COMPUTE CH[j] = detail_coeff_hor_calc[wc[j], s[j]];; COMPUTE CV[j] = detail_coeff_ver_calc[wc[j], s[j]]; COMPUTE CD[j]= detail_coeff_dia_calc[wc[j], s[j]]; j = j + 1; end COMPUTE ncolors; COMPUTE sz; // MATRIX SCALING AND SELECTIVE EXTRACTION for k = 1 to 4 COMPUTE res_ca[k] = matrix_rescale[ca[k], ncolors]; COMPUTE res_ch[k]= matrix_rescale[ch[k], ncolors]; 164

97 COMPUTE res_cv[k]= matrix_rescale[cv[k], ncolors]; COMPUTE res_cd[k]= matrix_rescale[cd[k], ncolors]; k = k + 1; end //SELECTIVE COEFFICIENT EXTRACTION COMPUTE dec_ca[1] = coeff_deci[ca[1], sz/2]; for m = 2 to 4 COMPUTE dec_ch[m]= coeff_deci [CH[m], sz/4]; COMPUTE dec_cv[m]= coeff_deci [CV[m], sz/4]; COMPUTE dec_cd[m]= coeff_deci [CD[m], sz/4]; m = m + 1; end Coefficient Re Construction From Decomposition Structure A mapping between various levels of the analysis banks and synthesis banks which resembles the reverse structure of decomposition tree This code performs two functions, reconstruction of coefficients from decomposition structure and scaling. The decomposition vector [wc,s] is applied as input. The computation level ranges from 1 to 4. The selected wavelet function is used here. A function, reconstruct_coeff is used to reconstruct coefficients by accepting decomposition structure, decomposition level and produces approximation coefficients, horizontal coefficients, vertical coefficients and diagonal coefficients. Using matrix_rescale function the coefficients are scaled to a size of ncolors. The scaled coefficients are assigned with identifiers. for i = 1 to 4 INPUT decomposition_matrix[i]; i = i + 1; end //COEFFICIENT RECONSTRUCTION for j = 1 to 4 COMPUTE recon_ca[j] = reconstruct_coeff[wc[j], s[j]]; COMPUTE recon_ch[j] = reconstruct_coeff [wc[j], s[j]];; COMPUTE recon_cv[j] = reconstruct_coeff [wc[j], s[j]]; COMPUTE recon_cd[j]= reconstruct_coeff [wc[j], s[j]]; 165

98 j = j + 1; end // RECONSTRUCTED COFFICIENTS SCALING for k = 1 to 4 COMPUTE res_ca[k] = matrix_rescale[wc[k],s[k], ncolors]; COMPUTE res_ch[k]= matrix_rescale[wc[k],s[k], ncolors]; COMPUTE res_cv[k]= matrix_rescale[wc[k],s[k], ncolors]; COMPUTE res_cd[k]= matrix_rescale[wc[k],s[k], ncolors]; k = k + 1; end Implementation of this algorithm on the recorded EEG for five electrode pairs montaged in temporal, parietal, occipital, frontopolar areas of the brain along with a comprehensive statistical relationship between the frequency and energy spectral density is shown in Table 5.1, Table 5.2, Table 5.3 and Table 5.4. These statistical values enable to localize maximum energy elicited for a specific stimulus as shown in Fig. 5.1, Fig. 5.2, Fig.5.3 and Fig

99 Fig. 5.1 Energy Spectral Density for electrode pair C4-P4 X Axis : Frequency in Hz and Y Axis : ESD (µv 2 /Hz) 167

100 Fig. 5.2 Energy Spectral Density for electrode pair FP2-F4 X Axis : Frequency in Hz and Y Axis : ESD (µv 2 /Hz) 168

101 Fig. 5.3 Energy Spectral Density for electrode pair P4-O2 X Axis : Frequency in Hz and Y Axis : ESD (µv 2 /Hz) 169

102 Fig. 5.4 Energy Spectral Density for electrode pair T5-O1 X Axis : Frequency in Hz and Y Axis : ESD (µv 2 /Hz) 170

103 Fig. 5.5 Energy Spectral Density for electrode pair T6-O2 X Axis : Frequency in Hz and Y Axis : ESD (µv 2 /Hz) 171

104 Table 5.1 The Energy Spectral Density for a frequency range Hz for electrode pair C4-P4 Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) E E E E E E

105 Table 5.2 The Energy Spectral Density for a frequency range Hz for electrode FP2-F4 Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) E E E E E E E E E-05 70

106 Table 5.3 The Energy Spectral Density for a frequency range Hz for electrode pair P4-O2 Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) E E E E E E E E E E

107 Table 5.4 The Energy Spectral Density for a frequency range Hz for electrode pair T5-O1 Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) E E E E E E E E E E E E E

108 Table 5.5 The Energy Spectral Density for a frequency range Hz for electrode pair T6-O2 Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) Frequency (Hz) ESD (µv 2 /Hz) E E E E E E E E

109 5.14 Conclusion Discrete Wavelet Transform (DWT) is a useful feature extraction tool since it explores the time as well as the frequency information of the signal. Although DWT has been used in a number of synchronized BCI systems, there remain some limitations in its application to feature extraction. BCI that use DWT features have mostly employed only one or two channels. To simultaneously explore the wavelet coefficients of BCIs with more channels and to avoid the problems associated with the resultant large feature space an algorithm is proposed. It consists of four stages that compute energies associated with the composite EEG signal recorded. This scheme was implemented on six pairs of channels C4-P4, FP2-F4, P4-O2, T5- O1, and T6-O2. The energy values support the hypothesis that proper channel selection for any user is necessary to obtain superior performance. Another inference also shows that the selected features are not necessarily located in the standard frequency bands. These results also indicate that a channel elimination methodology could be incorporated into the proposed method to further decrease the number of channels used for the operation of the system. lxxiv

110 CHAPTER 6 RESULTS AND DISCUSSION The results and performance of the proposed scheme is presented in this chapter. The Parameters discussed includes: 1. Data Collection 2. Quality Measures 3. Comparison and Conclusion 6.1 Data Collection The data was collected from a right-handed and able-bodied participant of 37 years old. The EEG signal was recorded from 16 monopolar EEG channels (according to the International system at Frontal, Frontopolar, Central, Temporal, Parietal regions). The signals were then converted to bipolar EEG signals since such electrodes are more likely to generate more discriminant EVP features than monopolar electrodes. The conversion was carried out by calculating the difference between adjacent EEG channels and resulted in the generation of 18 bipolar EEG channel pairs. During the recording session the participant performed following operations i. Eyes open ii. Eyes closed iii. Voluntary muscle movements Epochs of the intentional movements type consisted of data collected over an interval containing the onset of movement (measured as the leg movement activation, opening or closure of eyes). The proposed artifacts removal scheme will differentiate the EEG signal with the presence and absence of artifacts. Each intentional movement session lasted approximately two minutes. The recorded waveforms are shown in Fig. 6.1, Fig. 6.2 and Fig lxxv

111 Fig. 6.1 Recorded EEG corresponding to the event EYES OPEN lxxvi

112 Fig. 6.2 Recorded EEG corresponding to the event EYES CLOSED lxxvii

113 Fig. 6.3 Recorded EEG corresponding to the event VOLUNTARY MOVEMENT lxxviii

114 6.2 Quality measures While a variety of BCI modalities are in common use, this scheme focus on exploiting the modulation of Mu rhythm referred to as the sensory motor rhythm and Beta rhythms. Often described as attenuation of the spectral energy in these bands the associated EEG phenomena reflects the desynchronization of cortical circuits related to the intentional movements. The following metrics were chosen for discussion: 1. Singular Spectral Entropy (SSE) 2. Spectral Profile (SP) 3. Power Feature (PF) The above measures represent a pragmatic balance of computational simplicity and a desire to approach the nonlinear characterization problem from the complexity and statistics based perspectives. The complexity of the signal can be defined as the number of independent though possibly interacting components which are active at a particular time, then the complexity may be characterized through the notion of entropy. A variety of power distributions across the brain for different intentional movements is possible and the amplitude progression for the movements can be obtained Singular Spectral Entropy (SSE) To obtain the singular spectrum of the delay embedding of a time series then to model this spectrum as a probability distribution before calculating the entropy of the singular values and the resulting measure is termed as the singular spectral entropy (SSE). Alpha rhythm is characterized by a frequency 8-13 Hz during wakefulness over posterior region of the head generally with high amplitude over the occipital areas. The amplitude of alpha wave is variable and is blocked or attenuated especially visual or mental effort. The posterior rhythm attains alpha range of 10 Hz, and the amplitude of alpha waves varies in different individuals depending upon stretches of optimal alpha activity. The proposed scheme captures the spatial distribution of alpha waves in posterior half of the head that is occipital, parietal and posterior temporal regions. These distributions exhibits responsiveness of alpha to eye closure and opening and attenuation of mu wave by leg movements as shown in Fig. 6.4, Fig. 6.5 and Fig lxxix

115 COLOR SCHEME Fig Potential distribution across brain Fig Potential distribution across brain (Left view) Fig Potential distribution across brain (Right view) Fig Potential distribution across brain(top view) Fig. 6.4 Single Map and Tri Map Potential Distribution for the event EYES OPEN lxxx

116 COLOR SCHEME Fig Potential distribution across brain Fig Potential distribution across brain (Left view) Fig Potential distribution across brain (Right view) Fig Potential distribution across brain(top view) Fig. 6.5 Single Map and Tri Map Potential Distribution for the event EYES CLOSED lxxxi

117 COLOR SCHEME Fig Potential distribution across brain Fig Potential distribution across brain (Left view) Fig Potential distribution across brain (Right view) Fig Potential distribution across brain(top view) Fig. 6.6 Single Map and Tri Map Potential Distribution for the event VOLUNTARY MUSCLE MOVEMENT lxxxii

118 Table 6.1 shows the Potential distribution across the various regions of the brain in response to the stimuli activated. Table 6.1 Potential distribution across various regions of the brain for three events S.No. Event Name Frontopolar (in µv) Frontal (in µv) Central (in µv) Parietal (in µv) Occiptal (in µv) 1 Eyes Open 0 to 2-2 to 0-2 to 2-2 to 2 0 to 7 2 Eyes Closed -4 to 1-5 to -1-8 to 0 0 to 3 2 to 3 3 Voluntary Muscle Movement 2 to 8 0 to 8-2 to 2-1 to -6-8 The observations are summarized as follows: 1. The occipital lobe is the visual processing center of the mammalian brain containing most of the anatomical region of the visual cortex. A measure of 0 µv to 7 µv across occipital region refers to response of visual cortex to the event EYES OPEN. 2. The parietal lobe integrates sensory information from different modalities, particularly determining spatial sense and navigation. It comprises somatosensory cortex and the dorsal stream of the visual system. A measure of 0 µv to 3 µv across Parietal and Occipital region refers to the immediate response of visual cortex and somatosensory cortex to the event EYES CLOSED. These two events were elicited within a span of 5 seconds. 3. The frontal lobe contains most of the dopamine-sensitive neurons in the cerebral cortex. The dopamine system is associated with tasks, planning, and drive. A measure of 0 µv to 8 µv across frontopolar and frontal regions refers the response elicited by the event Voluntary Muscle Movement Spectral Profile (SP) The intuition behind the SSE feature can equally be applied to the frequency spectrum. This measure can be honed by computing the power spectrum. This measure is used to evaluate the effectiveness of directly using the ordinates of the frequency spectra as a feature vector. To prevent the power of the signal from dominating or even affecting the classification process in any way the extracted spectral components were first normalized to one before incorporation into the feature set. This feature vector is referred as spectral lxxxiii

119 profile (SP). The frequency spectrum and power distribution of the EEG signal captured is shown in Table 6.2, Table 6.3 and Table 6.4. Table 6.2 Frequency orientation and Power distribution for each electrode for the event EYES OPEN ELECTRODE DESIGNATION FP1 FP2 F7 F3 FZ F4 F8 T3 C3 CZ C4 T4 T5 P3 PZ P4 T FREQUENCY SPECTRUM POWER DISTRIBUTION lxxxiv

120 Table 6.3 Frequency orientation and Power distribution for each electrode for the event EYES CLOSED ELECTRODE DESIGNATION FP1 FP2 F7 F3 FZ F4 F8 T3 C3 CZ C4 T4 T5 P3 PZ P4 T FREQUENCY SPECTRUM POWER DISTRIBUTION lxxxv

121 Table 6.4 Frequency orientation and Power distribution for each electrode for the event VOLUNTARY MUSCLE MOVEMENT ELECTRODE DESIGNATION FP1 FP2 F7 F3 FZ F4 F8 T3 C3 CZ C4 T4 T5 P3 PZ P4 T FREQUENCY SPECTRUM POWER DISTRIBUTION lxxxvi

122 6.2.3 Power Feature (PF) The Power Feature (PF) is defined as the spectral power contained in Mu and Beta bands. This represents BCI feature extraction and described through Peak Power Frequency (PPF), Mean Power Frequency (MPF) and Spectral Edge Frequency (SEF) for each montaged electrode in the regular order as shown in Table 6.5, Table 6.6 and Table 6.7. lxxxvii

123 Table 6.5 Spectral Components associated with the event EYES OPEN PPF (Hz) MPF (Hz) SEF (Hz) lxxxviii

124 Table 6.6 Spectral Components associated with the event EYES CLOSED PPF (Hz) MPF (Hz) SEF (Hz) lxxxix

125 Table 6.7 Spectral Components associated with the event VOLUNTARY MUSCLE MOVEMENT PPF (Hz) MPF (Hz) SEF (Hz) xc

126 6.2.4 Desynchronization Process Assuming the desynchronization process accompanying motor visualization reflects the activation of previously dormant neuronal circuits then this might also be accompanied by a detectable increase in the signatures of nonlinear dynamics. The property of linear time series is that, the associated statistics remain constant under time reversal, since a linear process is essentially a combination of sinusoids which are symmetric in time. This fact is exploited to provide a powerful indicator of nonlinearity and temporal asymmetry. This factor exhibits the highest variability with respect to the classes of interest and is shown in Fig. 6.7, Fig. 6.8, Fig. 6.9, Fig. 6.10, Fig and Fig xci

127 00:02:47:953 00:02:47:961 00:02:47:969 COLOR SCHEME 00:02:47:977 00:02:47:984 00:02:47:992 00:02:48:000 00:02:48:008 00:02:48:016 00:00:42:023 00:00:42:031 00:00:42:039 Fig. 6.7 Progressive amplitude distribution across brain for the event EYES OPEN xcii

128 00:02:54:289 00:02:54:297 00:02:54:305 COLOR SCHEME 00:02:54:313 00:02:54:321 00:02:54:329 00:02:54:337 00:02:54:345 00:02:54:353 00:02:54:361 00:02:54:369 00:02:54:377 Fig. 6.8 Progressive amplitude distribution across brain for the event EYES CLOSED xciii

129 COLOR SCHEME 00:04:54:898 00:04:54:906 00:04:54:914 00:04:54:922 00:04:54:930 00:04:54:938 00:04:54:945 00:04:54:953 00:04:54:961 00:04:54:969 00:04:54:977 00:04:54:984 Fig. 6.9 Progressive amplitude distribution across brain for the event VOLUNTARY MUSCLE MOVEMENT xciv

130 0-1 HZ 2-3 HZ 4-5 HZ 6-7 HZ 8-9 HZ HZ HZ HZ HZ HZ HZ 22-23HZ Fig Progressive amplitude distribution across brain for the event EYES OPEN for different frequency bands xcv

131 0-1 HZ 2-3 HZ 4-5 HZ 6-7 HZ 8-9 HZ HZ HZ HZ HZ HZ HZ HZ Fig Progressive amplitude distribution across brain for the event EYES CLOSED for different frequency bands xcvi

132 0-1 HZ 2-3 HZ 4-5 HZ 6-7 HZ 8-9 HZ HZ HZ HZ HZ HZ HZ HZ Fig Progressive amplitude distribution across brain for the event VOLUNTARY MUSCLE MOVEMENT for different frequency bands xcvii

133 Compressed Spectral Analysis employs the basic principle of computing the cerebral activity associated with the brain in different frequency bands (delta, theta, alpha, and beta) using spectral analysis and the relation between these activities is established. A ratio type combination such (delta + theta) / (alpha + beta) is chosen as the indicator for feature extraction. This ratio is improved by using an empirical type of discriminant analysis applied to the recorded EEG signal. The values of the features for each of the channels of EEG are shown in Fig. 6.13, Fig and Fig xcviii

134 Fig CSA description of the Electrode Potential for the event EYES OPEN xcix

135 Fig CSA description of the Electrode Potential for the event EYES CLOSED c

136 Fig CSA description of the Electrode Potential for the event VOLUNTARY MUSCLE MOVEMENT ci

137 FREQUENCY [Hz] FREQUENCY [Hz] FREQUENCY [Hz] 6.3 Single Frequency Stimulation Results Three subjects S1, S2, S3 were presented with a Visual Stimulation Paradigm in two ways, single frequency stimulation and Bi frequency stimulation. Fig.6.16, Fig.6.17 and Fig.6.18 shows the results for single frequency stimulation on presentation of alphabets as visual stimuli. The colors used: indicates stimulating frequency, indicates elicited frequency A B C D E F G H I LETTER Fig.6.16 Single frequency stimulation results for SUBJECT S A B C D E F G H I LETTER Fig.6.17 Single frequency stimulation results for SUBJECT S A B C D E F G H I LETTER Fig.6.18 Single frequency stimulation results for SUBJECT S3 cii

138 ELICITED FREQUENCY [Hz] FREQUENCY [Hz] 6.4 Bi frequency Stimulation Results Table 6.8 Shows the stimulating frequencies used in bi-frequency stimulation paradigm. Table 6.8 Stimulating frequencies used in Bi-frequency Stimulation Paradigm Sl.No Alphabet Stimulating Frequencies [Hz] 1 A B C D E F G H I Fig.6.19, Fig.6.20 and Fig.6.21 shows the results for single frequency stimulation on presentation of alphabets as visual stimuli. The Color used: frequency used. indicates stimulating A B C D E F G H I ALPHABET Fig.6.19 Bi frequency stimulation results for SUBJECT S A B C D E F G H I LETTER Fig.6.20 Bi frequency stimulation results for SUBJECT S2 ciii

139 ELICITED FREQUENCY [Hz] A B C D E F G H I LETTER 6.5 Extraction Scheme Fig.6.21Bi frequency stimulation results for SUBJECT S3 By applying the simple amplitude criteria to the relevant frequencies, this classification scheme was developed. A vector is formed from extracted frequency components. The two highest peaks with their amplitudes associated with stimulating frequencies and their harmonics, are considered for the classification scheme. For these frequencies the maximum value in a ratio of value 0.2Hz is obtained. The highest amplitude elicited may correspond either to the stimulating frequency or its harmonics. Based on these amplitudes and its corresponding frequencies the classification scheme may be summarized as follows in Table 6.9. Table 6.9 Alphabets Classification Scheme Sl.No. CRITERION CLASSIFIED RESULT 1 Max. amplitude corresponds to A 5 Hz or 10 Hz 2 Max. amplitude corresponds to B 7 Hz or 14.2 Hz ( second harmonic) 3 Max. amplitude corresponds to C 7.8 Hz or 15.4 Hz( second harmonic) Or 21.2 Hz (Third Harmonic) 4 Max. amplitude corresponds to D 8.6 Hz 5 Max. amplitude corresponds to E 10.6 Hz 6 Max. amplitude corresponds to F 12.2 Hz 7 Max. amplitude corresponds to G 14.2 Hz 8 Max. amplitude corresponds to H 17 Hz 9 Max. amplitude corresponds to 21.2Hz I civ

140 Fig.6.22 and Fig.6.23 shows the classification rates for alphabets A,B,C,D,E,F,G,H,I for two different gaze times 5 seconds and 3 seconds respectively. In Fig the colors used: corresponds to frequency subject gazed at. Corresponds to recognition rate. In Fig the colors used: corresponds to frequency subject gazed at and Corresponds to recognition rate. Fig shows the average accuracy of BCI system for each of the 3 subjects over the nine characters used for gaze time of five seconds and three seconds. Corresponds to system1 classification rate and corresponds to system 2 recognition rate A B C D E F G H I Fig Classification rates for each stimulation frequency Gaze time 5 seconds A B C D E F G H I Fig Classification rates for each stimulation frequency Gaze time 3 seconds Fig Average accuracy of the BCI system for three subjects S1, S2, S3 The analysis with gaze time of five seconds over three subjects showed an average accuracy of the system of 95.6% and for gaze time of three seconds is 92.7%. cv