Universitatea Politehnica din Bucureşti Facultatea de Automatică şi Calculatoare Departamentul de Automatică şi Ingineria Sistemelor

Transcription

1 Universitatea Politehnica din Bucureşti Facultatea de Automatică şi Calculatoare Departamentul de Automatică şi Ingineria Sistemelor LUCRARE DE LICENŢĂ Interfaţă Creier-Calculator (Brain-Computer Interface) Coordonator Prof.dr.ing. Dumitru Popescu Mdc. Claudine Lecoq Absolvent Roventa Madalin Bucuresti, 2013

2 Contents: Foreword Introduction 1.1. Brain-Computer Interface Paper objectives BCI over time Introductory notions Summary EEG signal processing and classification Types of BCI Electroencephalography Neurophysiological signals used to drive a BCI Evoked signals Spontaneous signals Conclusions EEG signals Preprocessing Feature extraction Fuzzy classification Fuzzy classification. Manual rules Inputs/Outputs Fuzzy Inference System Fuzzy classification. Automatic rules Clustering of training data Generation of the fuzzy rules based on the clustered data Rules Optimization Studying the Use of Fuzzy Inference Systems for Motor Imagery based BCI Studying the use of FIS for a motor imagery based BCI, using artificial data sets Studying the use of FIS for a motor imagery based BCI, using real data sets Conclusions. Improvements Appendix A. 7. References 2 P a g e

3 Foreword This project has been assigned to me by the BCI research team from Polytech Lille. The initial project was to develop, only the fuzzy inference system for the classification part, with both manual and automatic rules, because they wanted to see if you can add apriori knowledge to the fuzzy system with automatic rules. Because I couldn t test my system without the other two components, I decided to build the pre-classification and feature extraction parts, but those are more rudimentary than the core of the BCI, the fuzzy inference system, because of the lack of time. So my paper will focus on the classification part, but I will present also the pre-classification and feature extraction part. 3 P a g e

4 Chapter I. Introduction 1.1. Brain-Computer Interface A BCI is a communication system which enables a person to send commands to an electronic device, only by means of voluntary variations of his brain activity [1][2][3][4][5]. This term refers to a interface that takes signals from the brain and it delivers them to an external piece of hardware. Various types of brain-computer interfaces were developed along the time for different purposes. Some of the earliest BCI were used just for recording signals from the brain. First real BCI applications were neuroprostheses, developed for restoring damaged hearing, sight and movement. Thanks to the remarkable cortical plasticity of the brain, signals from implanted prostheses can, after adaptation, be handled by the brain like natural sensor or effector channels [6]. The ultimate purpose of a direct brain-computer interface is to allow an individual, who suffers from a disease that affects a motor function of his body to have effective control over a hardware component that replaces that motor function, simply by brain control. Such an interface would increase an individual s independence, leading to an improved quality of life. 4 P a g e

5 1.2. Paper Objectives Despite the rapid grown from the 90 s till now of the BCI field, it still remains an uncharted research field. The brain is the most complex organism known to humans, so it s not very easy to transpose functions of the brain in a computerbased machine, but we are on the right track, thanks to numerous achievements already obtained. In this paper I will present the following topics: 1. Designing a fuzzy inference system (FIS) for the development of a braincomputer interface, asynchronous, non-invasive, based on the detection of real or imaginary movements, making distinction between the active and rest state. I chose the fuzzy classification method because fuzzy sets are known to be the most appropriate mathematical method to tackle an uncertain (fuzzy) problem, as the BCI designing is. The effectiveness of fuzzy classifiers have been proved on other pattern recognition problems: hand-writing recognition [7], ElectroMyoGraphy (EMG) classification [8] or even brain research, for EEG monitoring [9][10]. Also, another positive aspect of the fuzzy classification is that all the output data can be easily understood and we can gather knowledge from such a system. In the last period of BCI developing, all the classification methods were based on algorithms that use a training set, so that after, it can recognize a mental state. Of course, from those algorithms, we can only state the mental state, but we cannot see the way the algorithm got to the 5 P a g e

6 conclusion, so that s why we cannot gather knowledge. Those types of algorithms comport like a black box, you can only see the entries and the outputs. A fuzzy system allows you to see the connections that were made to get to a conclusion, so, the data is interpretable, and we can gain insights on the brain dynamics. This represents the feature of data interpretability of fuzzy systems. 2. Designing a whole BCI architecture for the recognition of imagined left and right hand movements from two sets of experimental data: Artificial created data, based on common features responses from individuals. Real data, taken from an Open Vibe scenario BCI over time Since the first experiments of ElectroEncephaloGraphy (EEG) on humans by Hans Berger in 1929 [11], the idea that brain activity could be used as a communication channel has rapidly emerged. Berger s first recording device was rudimentary. It was composed of silver foils attached to the patients head. The results of the tests were very disappointing, but it represented the beginning of the brain signals recording. More sophisticated measuring devices, such as the Siemens double-coil recording galvanometer led to success, so EEG permitted studying the brain activity. However, only in 1973 appeared the first prototype of BCI, created by Dr. Vidal [12]. The boom of the BCI field was very late, in the 90 s. Then, more and more laboratories started to develop studies in this research area. Also, the first BCI competitions arrived, were the researchers involved in BCI field could test their systems and have a perspective what techniques of designing an interface are most efficient throughout the world. Since then, more and more interfaces were proposed especially in the medical field, but also in virtual reality. Nowadays, researchers started the development of another Brain-Computer Interface, which is still at a theoretical level, that aims at the development of artificial intelligence. They state that all the brain functions could be uploaded in a BCI, leading to an existence of a brain without a body. Of course, this represents the climax of such a technology, and humans are far away from this objective. 6 P a g e

7 1 4 0 Y e a r r s Fig.1 A Lippmann electrometer is a device for detecting small rushes of electric current and was invented by Gabriel Lippmann in This device was used in the first practical ECG machine. Fig.2 Pocket BCI - the first commercially available interface brain-computer 1.4. Introductory notions Designing a BCI is a very difficult task. It requires knowledge in computer science, neurology and signal processing. Usually, two phases are required to use a brain-computer interface: An offline phase- represents a training phase which allows the researcher to gather information about the system, allowing him to calibrate it. This phase is required because every individual has different brain activity as a response to a stimulus. An online phase- in this phase, the training is over and the interface is used to recognize different mental states and translates them into commands for an external hardware. An online phase is composed usually from 6 steps, in this order: 1. Brain activity measurement: this step consists of measuring the brain activity (in this case EEG measuring technology). 2. Preprocessing: the EEG signal has a lot of perturbations incorporated in it, so in this step we have to denoise and clean the data to enhance the power of the signals that interests us. 3. Feature extraction: is the step where we extract from the signal only the features that interests us. 7 P a g e

8 4. Classification: the classification step assigns a class to a set of features extracted from the signals [13]. This class corresponds to an identified mental state. 5. Translation into a command: after the class identification, a command representing the class is given to an external hardware, 6. Feedback: this step provides the user a feedback about the mental state that was identified. This aims at helping the user controlling his brain activity [1]. The whole architecture is summarized in Fig.3 Fig.3 Architecture of an online BCI 1.5. Summary %%%%mai am de lucru The first chapter allowed us to make an opinion about what a brain-computer interface may stand for. In the second chapter I will introduce the scenario that is needed to understand what BCI means, more exactly, I will present all the components of a brain-computer interface, starting from acquisition, preprocessing, feature extraction, classification and finishing with command translation. Also, I will present the different types of BCI that can be developed nowadays. This represents the introductory part. After having established the grounds for the topic, the paper will continue by presenting the fuzzy classification part, which represents the third chapter. The fuzzy classifiers are particularly attractive for BCI design, because they are "perfectly suitable to deal with the natural fuzziness of real-life classification problems"[14], as Bezdek has highlighted. The fuzzy classifying part it s also divided in two: the construction of a fuzzy system with hand-made rules and one with automatic rules, based on Chiu s algorithm. This part is divided in two, because fuzzy classifiers are known to be suitable for adding a priori knowledge, under the form of hand-made fuzzy rules, so the best results obtained in the hand-made fuzzy system can be added to the automatic fuzzy inference system. Chapter four of the project consists in designing a whole BCI architecture. In this purpose, first, I specified the methods that I used, and after that their part in the interface. This chapter is also divided in three parts: preprocessing, feature extraction and the classification part. For classification, I used the fuzzy classifiers,, explained more thorough in the second chapter. 8 P a g e

9 Chapter V will summarize the results, conclusions and improvements for our interface. Chapter II. EEG signal processing and classification The first part of the chapter represents an introduction in the BCI world. It details the main types of brain-computer interfaces and gives definitions for phenomens associated to this type of brain research. The second part consists of detailing the fuzzy inference systems and offering a summary about Chiu s classification algorithm Types of BCI 1. Regarding the type of method used to extract brain signals: Invasive BCI: research, mostly, has targeted repairing damaged sight and providing new functionalities to paralyzed people. An invasive BCI is implanted directly in the grey matter during a neurosurgical intervention. The quality of the signal is very good because, it is focused only on the critical areas, where we gather the information, so the signal isn t very noisy. The down part of this method is that every foreign object inside our body is treated like an enemy, so this types of BCI are prone to scar-tissue build-up. This will make the signal weaker over time, and it s possible to disappear completely. Of 9 P a g e

10 course, the main disadvantage is that this procedure carries a very high degree of risk for the patient. Partially invasive BCI: are implanted inside the skull, but not in the grey matter. This will prevent scar-tissue build-up. The signal is worse than in case of an invasive BCI, but better than a non-invasive one, because the skull doesn t deflect the signal. Electrocorticography (ECoG) measures brain activity in a similar way as Electroencephalography (EEG), except that the electrodes are embedded in plastic and are placed beneath the dura mater and above the cortex. Non-invasive BCI: as the name suggested, the electrodes are placed above the skull, not beeing needed of any type of surgery. This type of interface is the most common in our days, because it doesn t carry any risk for the patient, but the signal received is very noisy and deflected from the skull. The extraction method carries the name of Electroencephalography (EEG). This project is based on a noninvasive interface. 2. Regarding the independence degree of the interface from the motor functions of the user: Dependent BCI: this type requires that the user controls some of his motor functions during tests, but for severe paralyzed peoples, who can t control their motor functions, this is not a solution. Independent BCI: this type doesn t require any type of motor control from the patient, so it s suitable for cases where motor control doesn t exists. 3. Regarding the independence degree of the interface from stimulus Synchronous Interfaces: with a synchronous BCI, the user can interact with the targeted application only during specific time periods, imposed by the system [15][16][1]. In this case the user has to execute a movement, when a stimulus appears (visual or acoustic). If the movement is executed in the specified interval, then the user will have a feedback, if not, nothing will happen. So, the main advantage of synchronous interfaces is that we always know when a movement is happening and we can have a good identification of the signal which represents a movement, but the disadvantage is that the user is constraint to perform a motor function only in a specific interval, so it s not a method for long term use. Asynchronous Interfaces: in this case, the user can perform a motor task anytime he wants, and the interface must response. This type is also called a self-paced BCI. The principle that is behind this 10 P a g e

11 interface is that the brain signal is permanently analyzed to determine if the user is executing a motor task or is resting. Of course, if he is performing a task, then the interface must decide what kind of motor task the user is executing, hence the degree of complexity of an asynchronous interface. Designing this type of BCI s represents one of the main goals of this field s research, but it is also one of the most challenging problems. In this paper an asynchronous interface is presented. In this paper, we focus our attention on a non-invasive, independent, asynchronous brain-interface EEG (ElectroEncephalography) The classified signal, in this project is an EEG signal, so it is important to thoroughly understand the principles that are laying at the foundation of Electroencephalography. Electroencephalography measures the electrical activity generated by the brain using electrodes placed on the scalp [17]. The man who invented this technique is Hans Berger in He decided to name this technique electroencephalogram [11]. Signals recorded from EEG have a very weak amplitude, in the order of some micro volts, so the signal must be amplified before processing it. EEG measurements are performed with electrodes attached to the head, in number, from 1 to 256, attached in different areas of the scalp. Electroencephalography (EEG) is the most studied potential non-invasive interface, mainly due to its fine temporal resolution, ease of use, portability and low set-up cost [6], but how every good thing comes with a bad part also, the EEG signal is very susceptible to noise and the use of electroencephalography in a braincomputer interface always comes with an extensive training period for the subject, before users can exploit real results from it. Fig.4 Recordings of brainwaves produced by an electroencephalogram 11 P a g e

12 For example, experiments conducted by Niels Birbaumer at the University of Tübingen in Germany, in the mid-90 s, where he trained paralysed people to selfregulate their slow cortical potentials to such an extent that these signals could be used as a binary signal to control a computer cursor [6]. The process was slow, requiring more than an hour for patients to write 100 characters with the cursor, while training often took many months. A type of parameter that is specially interesting for these paper is oscillatory activity, where research was concentrated on producing technologies that can allow the user to choose the brain signals they found easiest to operate a BCI, including mu and beta rhythm. These oscillations are different in terms of spatial and spectral localization, and are called rhythms. There are 6 different rhytms: 1. Delta rhythm: (1-4 Hz) slow rhythm. 2. Theta rhythm: (4-7 Hz) a more faster rhythm than delta. 3. Alpha rhythm: (8-12 Hz). 4. Mu rhythm: (8-13 Hz) is located in the motor and sensorimotor cortex. This rhythm is activated when a person performs movements. It s one of the two rhythms that are investigated in this paper. 5. Beta rhythm: (13-30 Hz) it s a fast rhythm that detects movements, so this is the second type of rhythm that I used in this paper. 6. Gamma rhythm: (above 30 Hz) associated with cognitive functions. Another parameter in brain-computer interfaces is the method of feedback used, as shown in the P300 signals. The biofeedback method requires learning to control brainwaves so the resulting brain activity can be detected. In this paper, we use the mu and beta rhythm to operate the BCI Neurophysiological signals used to drive a BCI The principle that lies at the basis of any interface is identifying several neurophysiological signals that are specific to a motor activity, in order to associate a command to each of these signals (brain patterns). These signals can be divided into two main categories [18][1]: 1. Evoked signals: are generated by the user as a response at a stimulus, unconsciously. Are called Evoked Potentials(EP) 2. Spontaneous signals: are generated voluntarily by the subject, without receiving any stimulus. In this paper, we will deal with spontaneous signals Evoked signals: 12 P a g e

13 An evoked potential or evoked response is an electrical potential recorded from the nervous system of a human or other animal following presentation of a stimulus, as distinct from spontaneous potentials as detected by EEG, EMG, or other electrophysiological recording method [6]. Among the current BCIs, the systems based on EPs have been studied for a long period since the 1970s [19]. This is because it s easy to configure such a system, it has a high information transfer rate and the training period for the user is reduced. The BCI research has focused on visual evoked potentials (VEP), because it s easier for the user to execute a motor task at the sight of a visual stimulus. VEP VEPs reflect the visual information-processing mechanism in the brain. In 1934, Adrian and Matthew noticed potential changes of the occipital EEG can be observed under stimulation of light. Ciganek developed the first nomenclature for occipital EEG components in During that same year, Hirsch and colleagues recorded a visual evoked potential (VEP) on the occipital lobe (externally and internally), and they discovered amplitudes recorded along the calcarine fissure were the largest. In 1965, Spehlmann used a checkerboard stimulation to describe human VEPs. An attempt to localize structures in the primary visual pathway was completed by Szikla and colleagues. Halliday and colleagues completed the first clinical investigations using VEP by recording delayed VEPs in a patient with retrobulbar neuritis in A wide variety of extensive research to improve procedures and theories has been conducted from the 1970s to today [6]. VEP Stimuli: On a large scale, there are 3 types of stimuli used: the diffuse flashing light and the checkerboard and grating patterns. In BCI research, the most popular is the checkerboard, because of it s usage in P300 experiments. VEP Electrode Placement The placement of the electrodes is very important for getting free artifact results. The International Society for Clinical Electrophysiology of Vision (ISCEV) standards for VEP testing recommend that electrode placement follow the system (Harding, G.F., Odom,1996). The active recording electrodes are placed over the active source, which for visual evoked potentials is the visual (occipital) cortex. A reference electrode is placed over an area unresponsive to visual stimuli and a ground electrode connects a second inactive area to the ground terminal of the equipment. VEP extraction The electrical signal is recorded from the surface of the scalp. The smallamplitude evoked potentials (1-20 µv) are embedded in the larger amplitude potentials. To extract information from the experiment, repeated averaged responses 13 P a g e

14 must be used. The VEP are time-locked to a stimulus, so when to the subject is presented the stimulus, you will know after what period of time the electrical activity in his brain should change. The VEP nomenclature is determined by using capital letters stating whether the peak is positive (P) or negative (N) followed by a number which indicates the average peak latency for that particular wave. For example, P50 is a wave with a positive peak at approximately 50 ms following stimulus onset.[wikipedia] Some specific VEPs are: Sweep visual evoked potential Binocular visual evoked potential Chromatic visual evoked potential Hemi-field visual evoked potential Flash visual evoked potential LED Goggle visual evoked potential Motion visual evoked potential Multifocal visual evoked potential Multi-channel visual evoked potential Multi-frequency visual evoked potential Stereo-elicited visual evoked potential Steady state visually evoked potential According to the knowledge of brain electrophysiology, VEPs corresponding to low stimulus rates are categorized as transient VEP (TVEP), and those corresponding to rapidly repetitive stimulations are categorized as steady-state VEP (SSVEP) [19]. The SSVEP and the P300 potential will be the topic of the following paragraphs. 1.SSVEP: VEP introduce transient responses of the visual system, but using long stimulus trains, a steady response will be produced, which can be displayed after averaging. Steady-state potentials are to be distinguished from transient potentials, because their constituent discrete frequency components remain closely constant in amplitude and phase over a long time period [20]. The SSVEPs have the same fundamental frequency (first harmonic) as the stimulating frequency, but usually they also include higher (Regan 1989) and/or subharmonics (Herrmann 2001). 2.P P a g e

15 It s an extensively studied stimulus in the BCI field, because it appears as a response to meaningful, rare stimuli ( oddball stimuli- ; Donchin & Coles, 1988) For example, if to the subject is presented a series of names, and every 3 seconds the subject names appears, a P3 wave is evoked, due to a meaningful stimulus. P3 is a positive-going wave with a scalp amplitude distribution in which it is largest parietally (at Pz) and smallest frontally (Fz), taking intermediate values centrally (Cz). (Fz, Cz, and Pz are scalp sites along the midline of the head.)[21]. The latency of the P3 wave is msec, so the name P300. The latency depends of the stimulus complexity. The amplitude of P3 is inversely proportional to the rareness of presentation Spontaneous signals: Spontaneous signals (e.g. EEG rhythms over sensorimotor cortex) do not depend for their generation on a stimulation. A BCI system which uses spontaneous signals as data entries generates a control signal at a given interval of time based on the classification of EEG patterns taken from a specific mental activity. The most used spontaneous signals are sensorimotor rhythms Sensorimotor rhythms: Represents brain activity(rhythms) related to motor actions, such as foot, hand movement. They are mainly located in the μ ( 8-13 Hz) and β ( Hz) frequency band and these types of rhythms can be controlled in amplitude by the user, so that s why 2 strategies of controlling this phenomenon have been proposed: Operant conditioning A subject can learn to modify voluntarily the amplitude of his sensorimotor rhythms through a (very) long training known as operant conditioning [22] [23] [24][25]. The user is free to choose the control mental strategy. The feedback represents the most significant part of an operant conditioning based system, because from the feedback, the user can understand what s to change in his mental activity to increase the control accuracy. Generally, a linear combined power of the signals in the μ and β band is used to design the control system. The main drawback of this method consists of a long training period, but after the calibration of the system, very good results have been observed. 15 P a g e

16 Motor imagery For a user, performing motor imagery consists in imagining movements of his own limbs [26][27][16]. The imagining of limbs movements have very good determined spatial, frequential and temporal features. For example, the imagining of a left or right hand movement is associated with an ERD (event-related desynchronization) in the contra-lateral side during the movement and an ERS(event-related synchronization) after the movement over in the μ and β rhythms. Using this types of features, it can be determined the type of mental task the user is trying to imagine. The advantage of such a system is that the user doesn t need an extensive period of training, in some cases it works from the first try, but the complexity of such a system is the main drawback, because it uses techniques of advanced signal processing and machine learning algorithms. Slow cortical potentials: Slow Cortical Potentials (SCP) are very slow variations of the cortical activity, which can last from hundreds of milliseconds to several seconds [28][29]. The training period for the user can last more than for an operant conditioning system, but after that t can provide a more stable signal. Non-motor cognitive tasks: These tasks can produce specific EEG signals variations, and can be used in operating a BCI. These tasks can represent: mental mathematical computations, visual counting, music imagination, etc Conclusions EEG signals: All the signals presented in this paper, so far, have been successfully used in designing brain-computer interfaces, but the problem is that you cannot say that a signal is better than another one in this field, because all of them have their pros and cons. The evoked signals are unnatural, because it requires the use of a stimulus. The spontaneous signals are natural, the user can execute the mental task whenever he wants, but are tiring, because of the extensive training period. However, it has been shown that using high performance signal processing algorithms with machine learning algorithms, the training period can be reduced, and sometimes be removed. So, that s why, the purpose of this paper is to build a 16 P a g e

17 BCI based on spontaneous signals, more specifically motor imagery signals, which are described thoroughly in today s literature. This type of signal represents the entry in our brain-computer interface. The next step consists of signal processing and machine learning algorithms. In other studies conducted, this part represented a black box for the users, but now, with the help of the fuzzy classification system, the user can understand the pathways of his brain and adjust it to the needs of the interface, so he can learn, as well as the machine learning algorithm. The three following sections, preprocessing, feature extraction and classification are dedicated to EEG signal processing. This part represent the core of any brain-computer interface. To improve the results, you must improve this part. However, in a BCI isn t compulsory to have all these three sections, they may emerge or may be missing. For example, the preprocessing and feature extraction part may be considered as one, or you can focus on those two parts and the classification part be considered only as a minimal or maximal threshold. Of course, the optimal solution is considered to be the use of all three components, but this takes the interface to a higher degree of complexity. In this paper, I focused my attention on the sensorimotor rhythms located in μ and β band and the strategy for controlling this phenomena is motor imagery Preprocessing After the acquisition of the signal, this must be de-noised and must enhance the significant information embedded in these signals. The acquired signal is very noisy because of the background brain activity, which has no connection with the experiment, and, also it is very affected by numerous artifacts, like ocular or facial muscular activity. This signal retrieved from the artifacts has a larger amplitude than the useful signal, so it s difficult to retrieve the good information without damaging it. This first step has the purpose to increase the signal-to-noise ratio. The output will be a total different signal than the entry one. The next paragraphs will present the main preprocessing techniques used in BCI field. Simple spatial and temporal filters Temporal filters: -low pass filter -band pass filter 17 P a g e

18 Are used to restrict the frequency to a specific domain, where we know that the neurophysiological signals are located. For example, the BCI s based on sensorimotor rhythms use a band pass filter between 8-30 Hz. This is where the µ and β band is located, according to neurophysiological experts [30]. This filtering method is also useful to eliminate slow variations of the EEG signal and power line interference. The purpose of a temporal filter in a BCI is to reduce the influence of frequencies that are lying outside the area of interest. Such a filtering is generally achieved using Discrete Fourier Transform (DFT) or using Finite Impulse Response (FIR) or Infinite Impulse Response (IIR) filters [31]. Spatial filters Similar to the temporal ones, spatial filters are used to isolate the relevant spatial information, as we concentrate on different regions of the brain, depending on the mental task we would like to execute. This filtering method consists in selecting or weighting the contribution of different electrodes. As it is known, when we focus on hand movements, the regions of interest are the motor and sensorimotor cortex areas, hence the use of electrodes C 3 and C 4 or a weighted contribution of the electrodes surrounding for designing a spatial filter. Also, the electrodes used in SSVEP experiments are O 1 and O 2, that are located over the visual areas [32]. Some common types of spatial filters used are: Surface Laplacian(SL) filter and Common Average Reference(CAR). Common spatial patterns This method is based on the decomposition of the EEG signals into spatial patterns [33][34][35]. These patterns are selected in order to maximize the differences between the classes involved, once the data has been projected onto these patterns. Determining these patterns is performed using a joint diagonalization of the covariance matrices of the EEG signals from each class [31]. Inverse solutions Relevant but much less used preprocessing methods for BCI are inverse solutions. Inverse solutions are methods that attempt to reconstruct the activity in the brain volume by using only scalp measurements and a head model [31][36][37]. Conclusions: 18 P a g e

19 As showed in this section, the preprocessing part is very diversified, but choosing the best method or combination of methods hasn t been identified so far. It depends on your system s needs. The preprocessing part has as entry the EEG signal and as output another signal, that is de-noised, and has it s significant features enhanced, thus a better signal-to noise ratio. This is theoretical, but practical, it s a very difficult job to separate the artifacts and the background noise from the useful signal. Nevertheless, the most appreciated and most common used method, is the spatial filtering method. It is shown that it reduces noise drastically, so it improves the performances of the system, if you are working with a sufficient number of electrodes Feature Extraction In this section we will discuss some of the feature extraction methods that have received more attention in BCI systems. Dealing with EEG signals will produce a high quantity of data. The researchers work with a number of electrodes varying from 1 to 256 and with a sampling frequency from 100 to 1000 Hz. This states the need of a feature extraction method, that selects only the valuable information from data. It is crucial to select only data that is meaningful to the system, otherwise the classification algorithm will have entry data that has no relevance for it, making the decision problem harder, or even impossible. In BCI research, feature extractions methods have been divided into four main categories: Time representation methods Frequency representation methods Hybrid representation methods, that includes also, time and frequency methods Parametric modeling Also, it exists other methods that aren t included in this categories. Time representation methods 19 P a g e

20 This extraction method use as features, temporal variations of the signal. Are perfectly adapted to time specific processes, such as P300 or ERD, notably those triggered by motor imagery [31]. Frequency representation methods Frequency-based features have been widely used in signal processing because of their ease of application, computational speed and direct interpretation of the results. Specifically, about one-third of BCI designs have used power-spectral features. Due to the non-stationary nature of the EEG signals, these features do not provide any time domain information. Hybrid representation methods Mixed time-frequency feature extraction methods have shown that can improve the system s performance. They map the one-dimensional signal into a twodimensional function of time and frequency, and are used to analyze the timevarying spectral content of the signals. Parametric modeling Parametric approaches assume the time series under analysis to be the output of a given linear mathematical model. They require an a priori choice of the structure and order of the signal generation mechanism model [38]. 20 P a g e

21 Fig.5 Feature extraction methods in BCI designs based on sensorimotor activity, VEP, P300, SCP, response to mental tasks, activity of neural cells, and multiple neuromechanisms. Conclusions: Choosing the appropriate method for feature extraction is a crucial step in designing a BCI, but as the BCI field had shown us, there isn t an appropriate method for all the cases, there are only subject-specific methods. Of course, this means that feature extraction methods that can be tuned (e.g., band power features, adapted to frequency bands of the user) are likely to have better results Classification A third step in creating a BCI is the classification part. This part is crucial, because it transforms features into commands. Formally, classification consists of finding the class of a feature vector x, using a mapping f, where f is learnt from a training set T. Class represents the 21 P a g e

22 mental state of the user. The purpose of the learning stage is to provide the algorithm pre classified labeled data (here, vectors of 320 features), from which the algorithm builds the mapping in order to predict the labels of new data [38]. Classifiers are divided into 5 main categories: 1. Linear classifiers -they use linear functions to distinguish classes -the most common used: Linear Discriminant Analysis(LDA) and Support Vector Machines(SVM). -the most popular algorithms for BCI research 2. Neural networks -are an assembly of artificial neurons that can produce nonlinear decision boundaries [Bis96]. -are, with linear classifiers the most used in BCI -the most used neural network technique is MultiLayerPerceptron(MLP). 3. Non linear Bayesian classifiers -are more accurate than linear ones, but aren t that simple and effective. -the most used are Bayesian quadratic and Hidden Markov Model 4. Nearest neighbor classifiers -the principle is to assign a class to a feature vector according to it s nearest neighbor. -the most used techniques are k Nearest Neighbors(KNN) and Mahalanobis distance 5. Combinations of classifiers -commonly, only a single classifier technique is used in an interface, but, recently a new trend occurred: to use multiple classifiers aggregated in some way. -there are a couple of strategies used to combine these types of classifiers: Voting, Boosting, Stacking etc. The fuzzy classifiers, relatively unknown to the BCI world, till a couple of years are starting to be often used in designing brain-computer interfaces, thanks to their interpretability and because the user can see what to modify in his brain activity for a better recognition of his mental states. In the next chapter, the fuzzy classifiers will be presented. Chapter III. Fuzzy classification 22 P a g e

23 In this chapter we discuss the classification part, more precisely, the fuzzy classifiers and there use in a brain-computer interface. Fuzzy systems are known to deal with uncertain (fuzzy) information, as such our brain is, there the choice of using fuzzy sets for motor imagery classification. This chapter will be divided into two sections: 1. Fuzzy classification. Manual rules 2. Fuzzy classification. Automatic rules 3.1. Fuzzy classification. Manual rules I have build a fuzzy classification system using Matlab, more specifically the fuzzy graphical user interface (GUI). This section is based on studies made by neurophysiologists, regarding the event related desynchronization and event related synchronization before, during and after a movement. Fuzzy: A fuzzy system consists of: Inputs Fuzzy Inference System(FIS) Outputs Inputs/Outputs: The inputs for the system are: 23 P a g e

24 Inputs Nr.crt. Variable Range Membership functions Tresneg Nul trespos 1 C 3 µ before [-1 1] Trimf: 2 C 4 µ before [-1 1] Triangular membership function 3 C 3 β before [-1 1] 4 C 4 β before [-1 1] 5 C 3 µ [-1 1] during 6 C 4 µ [-1 1] during 7 C 3 β [-1 1] during 8 C 4 β [-1 1] during 9 C 3 µ after [-1 1] 10 C 4 µ after [-1 1] 11 C 3 β after [-1 1] 12 C 4 β after [-1 1] Definitions: 1. Electrodes 24 P a g e

25 -C 3 = electrode or weighted sum of electrodes from the left side (depends on the input data of the entire BCI) -C 4 = electrode or weighted sum of electrodes from the right side (depends on the input data of the entire BCI) 2. Power stamps - β= beta power band - µ= mu power band 3. Time stamps I ve considered that a significant change appears in the two power bands 2 seconds before the movement and it ends one second after the movement has finished, so a movement can be represented in the interval [-2 1] second. -before= before the movement - µ: between second -2 and -1 - β: between second -1.5 and during= during the movement - µ: between second -1 and 0 - β: between second -0.5 and 0.5 -after= after the movement - µ: between second 0 and 1 - β: between second 0.5 and 1 25 P a g e

26 Fig.6 Input example An input can take values between -1 and 1 so the range is [-1 1]. The range for the input is given by the output of the precedent step in building a BCI: feature extraction. The outputof feature extraction will be a feature vector named pdle. In this case the pdle can only take values between -1 and 1 so, the range of our inputs. In the next chapter, the pdle will be presented in detail. I have chosen 3 triangular membership functions for each entry: -tresneg: if the entry is close to -1 -nul: if the entry isn t close to -1 or 1 -trespos: if the entry is close to 1 I have chosen triangular membership functions because they have thorough boundaries, and, in my opinion I think there are better suitable for this type of experiment, but, of course, it can be possible that a membership function with more soften boundaries, such as a gaussian function can be more suitable. In the future, I will try the rest of the membership functions available to see which are giving the best results. According to neurophysiologists, the pdle has negative values before the movement and positive values after the movement, and in case of a rest period the values for the pdle are around 0, so these 3 membership functions are covering these areas. 26 P a g e

27 Example: If the input C 3 µbefore has the value it means that the entry value for the electrode C 3 in the µ band between second -2 and -1 (before the movement) is tresneg. Outputs The outputs for the system are: Nr.crt. Variable Range Membership functions No Possible Yes 1 Left [0 1] Gaussmf: 2 Repos [0 1] Gaussian membership function 3 Right [0 1] Definitions: Mental state: -left= the user is trying to imagine a left movement -repos= the user is in the rest state -right= the user is trying to imagine a right movement 27 P a g e

28 Fig.7 Output example I have chosen the range between 0 and 1 because a user can be in that mental state (the output will be close to 1), the user isn t in that mental state (the output will be close to 0) or it cannot be determined (the output will be 0.5 or close to 0.5), hence the 3 gaussian membership functions: No, Possible, Yes Fuzzy Inference System (FIS) Represents a way of mapping an input space to an output space using fuzzy logic. A FIS tries to formalize the reasoning process of human language by means of fuzzy logic (that is by building fuzzy IF-THEN rules). For instance: If the service is good, even if the food is not excellent, the tip will be generous The process of fuzzy inference involves: Membership Functions, Logical Operations, and If-Then Rules. Matlab offers 2 possibilities of building a FIS through his GUI: A Mamdani type FIS A Sugeno type FIS 28 P a g e

29 Mamdani vs. Sugeno Mamdani's method is the most commonly used in applications, due to its simple structure of 'min-max' operations. The most fundamental difference between Mamdani-type FIS and Sugenotype FIS is the way the crisp output is generated from the fuzzy inputs. While Mamdani-type FIS uses the technique of defuzzification of a fuzzy output, Sugenotype FIS uses weighted average to compute the crisp output. The expressive power and interpretability of Mamdani output is lost in the Sugeno FIS since the consequents of the rules are not fuzzy. But Sugeno has better processing time since the weighted average replace the time consuming defuzzification process. Due to the interpretable and intuitive nature of the rule base, Mamdani-type FIS is widely used in particular for decision support application [39]. In this paper I have chosen a Mamdani type FIS because of the restrictions that a Sugeno-type FIS is imposing. The characteristics of Mamdani FIS used: -Membership functions: already defined at the inputs/outputs section -Logical Operations: Min/Max -Defuzzification method: Centroid -If-Then rules If-Then rules The rules have been based on the artificial signal. This signal represents the common features of brain activity in power bands µ and β, during a real or imaginary movement for all individuals, according to neurophysiologists experts. 29 P a g e

30 Fig.8 Power band changes determined by movement realized by the dominant hand (ipsilateral=situated on the same part of brain as the hand involved in the movement; contralateral=situated on the opposite side of brain) Differences between left and right movement: ERD= event related desynchronization ERS= event related synchronization Taking in account that the movement starts at sampling time 400, and a second lasts 100 sampling periods, you can see in the picture above that the desynchronization(erd) in the beta and mu band in contralateral appears approximately 2 seconds before the movement, and in the ipsilateral side around 0.5 seconds before the movement. The other main difference between left and right hand movement is that the rebound (ERS) is much better defined in the contralateral region, especially in the beta band where the rebound exceeds the reference value, and it only takes less than a second. 30 P a g e

31 Rule explanation: This rule describes the sampling period from the beginning of the movement: If (C3µ_before is nul) and (C4µ_before is nul) and (C3β_before is nul) and (C4β_before is nul) and (C3µ_during is nul) and (C4µ_during is tresneg) and (C3β_during is nul) and (C4β_during is tresneg) and (C3µ_after is not nul) and (C4µ_after is tresneg) and (C3β_after is tresneg) and (C4β_after is tresneg) then (mvt.g is Yes)(repos is No)(mvt.d is No) (1) Movement -2 sec. 1s -2 sec. +1 sec. A (First rule) When our fuzzy system verifies that the conditions for movement are accomplished, it verifies that: C4µ_during is tresneg = on the contralateral side between second -1 and 0 the µ band is low (it s a ERD). C4β_during is tresneg:= on the contralateral side between second -0.5 and +0.5 the β band is low (it s a ERD). C3µ_after is tresneg = on the contralateral side between second 0 and +1 the µ band is low (it s a ERD). C4µ_after is tresneg = on the ipsilateral side between second 0 and +1 the µ band is low (it s a ERD). C3β_after is tresneg = on the ipsilateral side between second 0.5 and +1 the β band is low (it s a ERD). C4β_after is tresneg = on the contralateral side between second 0.5 and +1 the β band is low (it s a ERD). The rest of the conditions may be null, or may not take them into account. On artificial moves, the results were worse when the other inputs weren t taken into account, so I choose to take them into account. 31 P a g e

32 3.2. Fuzzy classification. Automatic rules The BCI community had raised a new issue regarding the knowledge extracted from the classification algorithms. Most of the algorithms are black-boxes, so you can t gather any new information from them, but with the help of a fuzzy system, it s possible to view the extracted patterns from data and understand bits of how our brain works. In this section I will present a FIS based on Chiu s classification algorithm (CFIS) [40]. CFIS is robust to noise, an important quality when dealing with EEG signals, that are known to be very noisy. According to Chiu, the CFIS is generally more efficient than neural networks, which have been successfully used in BCI research [13]. Another important feature of CFIS is that it s a clustering-based algorithm, a thing that is important when dealing with small training sets. CFIS: Chiu s classification algorithm has 3 steps: I. Clustering of training data II. Generation of the fuzzy rules based on the clustered data III. Rules optimization Clustering of training data Clustering of numerical data forms the basis of many modeling and pattern classification algorithms [40]. The purpose of a clustering algorithm is to find natural groupings of data for pattern recognition. Chiu proposed a substractive algorithm, where initially, each data point is considered to be a cluster center. We consider each data point as a potential cluster center and define a measure of the potential of data point X i as: (1) 32 P a g e

33 , where α=4/r 2 a ;. = Euclidean distance R a = constant, defining the radius of the neighborhood After each potential has been calculated, we select the point with the highest potential as the first cluster center, and then we revise the potential of each data point, according to the formula: (2), where β= 4/r b 2 ; The constant r b is effectively the radius defining the neighborhood which will have measurable reductions in potential. We define r b =1.25 r a, X 1 * be the location of the first cluster center and P 1 * be it s potential value So, the point with the highest potential is chosen as the second cluster center. This process continues until all remaining data potentials fall under some fraction of the first potential chosen P 1. These is the criteria for ending the search of the clustering process. Beside this, other criteria are used for avoiding marginal clusters, as an accept and reject ratio: 33 P a g e

34 [40] This substractive algorithm is provided by Matlab within the fuzzy logic toolbox. 34 P a g e

35 The algorithm is embedded in a function, called subclust, which returns the centers of the clusters. The syntax for the subclust function is [41]: [C, S] = SUBCLUST(X, RADII, XBOUNDS, OPTIONS) Outputs: - returns the cluster centers in the matrix C; each row of C contains the position of a cluster center. - S vector contains the sigma values that specify the range of influence of a cluster center in each of the data dimensions. Inputs: -X= the data that needs to be clustered -RADII= has a value between 0 and 1 and specifies the size of the cluster in each of the data dimensions. -XBOUNDS= a matrix used to normalize X into a unit hyperbox. If XBOUNDS is an empty matrix or not provided, the minimum and maximum data values found in X, are used as defaults. -OPTIONS: specifies a vector for changing the default algorithm parameters 1. OPTIONS (1): The squash factor, is used to multiply the RADII values to determine the neighborhood of a cluster center within which the existence of other cluster centers are discouraged. 2. OPTIONS(2): The accept ratio, sets the potential, as a fraction of the potential of the first cluster center, above which another data point will be accepted as a cluster center. 3. OPTIONS(3): The reject ratio sets the potential, as a fraction of the potential of the first cluster center, below which a data point will be rejected as a cluster center. 4. OPTIONS(4): Displays progress information unless it is set to zero. The default values for the OPTIONS vector are [ ]. 35 P a g e

36 After multiple trials on the artificial data and using the GUI for substractive clustering, I ve considered that the next parameters for the subclust function are suitable to this experiment: [Cluster, S] = subclust ( pdle, 0.11, [ ], [ ] ); Fig.8 Clustered pdle Considering that this is the training phase, we know which data is representative for left, right movement and rest, so the data sets that have been clustered are the classes for each movement and for rest. After this process is over, we can say that this are the most significant values for each mental state considered. For each of these found clusters centers, one rule will be automatically created, based on Chiu s algorithm. This makes the topic of our next section. 36 P a g e

37 Generation of the fuzzy rules based on the clustered data A fuzzy if-then rule is generated for each center of cluster found previously. For a given center j, belonging to class Cl i, the generated fuzzy rule is [31]: if X 1 is A j1 and... and X N is A jn then class is Cl i, where N=dimension of the training data X k = K th element of a feature vector X A jk = a gaussian membership function, which is defined as:, where x jk = the k th element of the vector representing the center of the cluster =standard deviation for the gaussian membership function. More precisely, for each value of a cluster center we build a membership function that has the peak equal to the center s value. This are the membership functions for the entries. For the outputs we keep the same membership functions as in the first case: standard gaussian membership functions. 37 P a g e

38 Membership functions for movement Membership functions for transition from movement to rest Membership functions for rest Fig.9 Membership functions As we stated in the last paragraph of the previous section, the outputs for the substractive algorithm, respectively the inputs for this training phase are the cluster s centers for each of the three classes: left, right movement and rest. So for each of these centers, a rule will be created. Next, I will present the Matlab functions from the fuzzy toolbox, that were useful in this step: addvar(a,'input','c3m_av',[-1 1]): creating an input variable C3m_av for FIS a in the range [-1 1] addmf(a,'input',i,'good','gaussmf',[sigma_patrat,cluster(j,i)]): creating a membership function for input i, named good, gaussian type, with equal distribution sigma_patrat and with the center in cluster(j,i) rulelist=[j j j j j j j j j j j j ]: creating a rule where each input has to belong to membership function j and the first output belongs to membership function 3, the other two, type 1 and the weight for this rule is 1. a = addrule(a,rulelist): add the rule to our a FIS These are the most important functions used in building this training FIS. 38 P a g e

39 Rules optimization After the FIS is build, the optimization part follows. This consists in tuning each membership functions according to gradient descent formulas [40]:, where λ= a positive learning rate defined by the user, c= the class of feature vector X class c µ c,max = the highest degree of fulfillment among all the rules that assign X to assign X to class c = the highest degree of fulfillment among all the rules that do not Only the fuzzy rules corresponding to µ c,max and are optimized. The + sign is used for the rule corresponding to µ c,max and the - sign for. The degree of fulfillment is defined as:,where x= input vector α= 4/r a 2 39 P a g e

40 This gradient descent algorithm is a type of competitive learning algorithm: a winner in the good rule category is reinforced and a winner in the bad rule category is punished. Because only 2 rules are updated each time the algorithm is highly efficient. [40]. The algorithm can be improved by changing the membership functions from a gaussian function with equal standard deviation to a two-sided gaussian function. This may have a flat plateau region and different standard deviations on the left and right sides. Fig.10 Two-sided Gaussian membership function 40 P a g e

41 Chapter IV. Studying the Use of Fuzzy Inference Systems for Motor Imagery based BCI This chapter is divided in two sections: I. Studying the use of FIS for a motor imagery based BCI, using artificial data sets FIS with manual rules FIS with automatic rules II. Studying the use of FIS for a motor imagery based BCI, using real data sets FIS with manual rules FIS with automatic rules 4.1. Studying the use of FIS for a motor imagery based BCI, using artificial data sets This section will present the methods used for the 3 essential steps in BCI: preprocessing, feature extraction and classification The EEG data sets that were used in this experiment are build upon experts opinions about changes in mu and beta power band during specific time intervals and it represents motor imagery experiments. The artificial EEG data is a matrix N X 10, where N is the number of samples and the number of columns represents 10 different signals, from 10 electrodes. The sampling time is 128. A movement or a rest period lasts for 5 seconds so it has 640 samples. The artificial signal has as inputs the number of right and left movements. Between every movement there is a rest period. In Fig. 8 is shown a diagram of the signal with 3 left movements and 2 right. 41 P a g e

42 Rest Left Rest Left mvt. mvt. Rest Left mvt. Rest Right mvt. Rest Right mvt. Rest 640 samples To comport like a real one, you can add noise to the artificial signal. The experiments were made with a percentage of 0.25 additional noise. The maximum percentage is 1. The order of the electrodes is: Left side Right side 1. C3 6. C4 2. FC3 7. FC4 3. CP3 8. CP4 4. C1 9. C2 5. C5 10. C6 Preprocessing part Because the data is composed from 10 different signals, it s almost necessary the use of a Laplacian filter, so for the mu band we used a Laplacian filter and for the beta band an averaging method: Laplacian: s(:,1)=signal(:,1) - (signal(:,2)+signal(:,3) + signal(:,4) +signal(:,5))./4; Laplacian for mu left s(:,2)=signal(:,6)- (signal(:,7)+signal(:,8) + signal(:,9) +signal(:,10))./4; Laplacian for mu right 42 P a g e

43 The coefficients used: Laplacian for mu left Represents 0 FC3 0 C5 C3 C1 0 CP3 0 Laplacian for mu right Represents 0 FC4 0 C2 C4 C6 0 CP4 0 Averaging: s(:,3)=(signal(:,2)+signal(:,3) + signal(:,4) +signal(:,5))./4; averaging for beta left s(:,4)=(signal(:,7)+signal(:,8) + signal(:,9) +signal(:,10))./4; averaging for beta right After the signal passes from 10 characteristics (the number of electrodes) to 4 characteristics(mu and beta band in left and right hemisphere), the signal undergoes a frequential preprocessing. This is achieved by using a bandpass Butterworth filter, of the order of 2, between 8-25 Hz for both of the bands. ordre_filtre = 2; frequences_filtre = [8/une_seconde 25/une_seconde]; [bmu, amu] = butter ( ordre_filtre, frequences_filtre,'bandpass'); ordre_filtre = 2; frequences_filtre = [8/une_seconde 25/une_seconde]; [bbet, abet] = butter (ordre_filtre, frequences_filtre,'bandpass'); 43 P a g e

44 At the end of the preprocessing part the signal is squared, to eliminate small variations. Feature extraction part This section allows us to extract the features of the signal and transform them into the inputs that the fuzzy classification system has as entries. This section has 3 major parts: 1. Averaging the signal every 10 samples, to reduce the unwanted characteristics 2. Setting a mean reference power for the beta and mu band: this is when the user is in the rest mental state. I ve considered that the first 2 seconds are suitable for this, because the signal starts with a 5 second rest period. puissance_signal_reference_mu=signal(1:ceil(2*une_seconde/10),1:2); moyenne_puissance_reference_mu = mean(puissance_signal_reference_mu); puissance_signal_reference_beta=signal(1:ceil(2*une_seconde/10),3:4); moyenne_puissance_reference_beta = mean(puissance_signal_reference_beta); 3. Extracting the suitable periods of time from the signal and making them an average. The time periods are: Mu band Beta band Before During After For mu band: puissance_signal_avant_mu=signal(i-floor(2*une_seconde/10):ifloor(1*une_seconde/10),1:2); puissance_signal_pendant_mu=signal(ifloor(1*une_seconde/10):i+floor(0.*une_seconde/10),1:2); puissance_signal_apres_mu=signal(i+floor(0.*une_seconde/10):i+floor(1*une_seco nde/10),1:2); 44 P a g e

45 moyenne_puissance_avant = mean(puissance_signal_avant_mu); moyenne_puissance_apres = mean(puissance_signal_apres_mu); moyenne_puissance_pendant = mean(puissance_signal_pendant_mu); For beta band: puissance_signal_avant_beta=signal(i-floor(1.5*une_seconde/10):ifloor(0.5*une_seconde/10),3:4); puissance_signal_pendant_beta=signal(ifloor(0.5*une_seconde/10):i+floor(0.5*une_seconde/10),3:4); puissance_signal_apres_beta=signal(i+floor(0.5*une_seconde/10):i+floor(1*une_se conde/10),3:4); moyenne_puissance_avant = mean(puissance_signal_avant_beta); moyenne_puissance_apres = mean(puissance_signal_apres_beta); moyenne_puissance_pendant = mean(puissance_signal_pendant_beta); To obtain percentage values for ERD/ERS, the power within the frequency band of interest in the period after the event is given by A whereas that of the preceding base line or reference period is given by R. ERD or ERS is defined as the percentage of power decrease or increase, respectively, according to the expression [DA2]: ERD%=(A-R)/R*100 For the display of the time course of ERD/ERS, a scale displaying either power changes with 0% in the reference period or relative power with 100% in the reference period is recommended. After this step, a feature vector, most commonly known in publications as pdle is formed. For mu band pdle(a,1:2) = (moyenne_puissance_avant - moyenne_puissance_reference_mu)./moyenne_puissance_reference_mu; pdle(a,5:6) = (moyenne_puissance_pendant - moyenne_puissance_reference_mu)./moyenne_puissance_reference_mu; 45 P a g e

46 pdle(a,9:10) =(moyenne_puissance_apres - moyenne_puissance_reference_mu)./moyenne_puissance_reference_mu; For beta band pdle(a,3:4) = (moyenne_puissance_avant - moyenne_puissance_reference_beta)./moyenne_puissance_reference_beta; pdle(a,7:8) = (moyenne_puissance_pendant - moyenne_puissance_reference_beta)./moyenne_puissance_reference_beta; pdle(a,11:12) =(moyenne_puissance_apres - moyenne_puissance_reference_beta)./moyenne_puissance_reference_beta; pdle represents the output of the feature extraction method. It also is the input for our fuzzy classification system, so the correspondence between the columns of the pdle and the inputs of the FIS is: Columns pdle Inputs FIS 1 C 3 µ before 2 C 4 µ before 3 C 3 β before 4 C 4 β before 5 C 3 µ during 6 C 4 µ during 7 C 3 β during 8 C 4 β during 9 C 3 µ after 10 C 4 µ after 11 C 3 β after 12 C 4 β after 46 P a g e

47 Classification part As we explained in the second chapter, in this paper we are focusing on fuzzy classifications systems, more specifically: FIS with manual rules FIS with automatic rules, based on Chiu s algorithm These 2 types were detailed earlier, so there is no need for further information. Results FIS with manual rules We tested the entire BCI presented so far with the artificial signal. It has to detect 64 samples for movement and rest. Between movements there is a rest period. The results are presented from a fuzzy power point of view. The outputs of the FIS are the 3 mental states involved, in this order: left movement, rest, right movement. It assigns for each sample a degree of membership to the mental states already specified. The degree may vary from 0 to 1, so the highest degree for a sample to be detected as a movement or rest state is 1. If the FIS doesn t have sufficient information to determine the type of mental state, the three coefficients are simple 0.5. I will present the results as a chart, as they are easy to understand. The degree is represented on the Y axes and it s called Fuzzy coefficients and the sampling time is on the X axes Here are the charts representing the results: 47 P a g e

48 1. 5 left movements and 6 right movements: From the chart, the following conclusions can be drawn: The system detects better the rest mental state than the movement It detects movement in samples from left movements and 1 right movement 48 P a g e

49 3. 31 left movements and 22 right movements FIS with automatic rules We tested the BCI with the artificial signal. The parameters for the experiment are the same as in the case of FIS with manual rules. The results are: 49 P a g e

50 1. 6 right movements and 6 left movements right movements and 12 left movements 50 P a g e

51 Conclusions: As it s shown in the figures above, the FIS based on Chiu s algorithm has a better accuracy than the FIS with manual rules. It determines movement in samples from 64, but the fuzzy coefficients for the movements have bigger values than in the first case, with manual rules, so we can find an improvement Studying the use of FIS for a motor imagery based BCI, using artificial data sets data. This section will present the methods used to classify real motor imagery The data is taken from an Open Vibe scenario, Handball which is used to identify left and right movements. Open Vibe is a specialized platform used to perform BCI experiments. Handball experiment uses as entry data an EEG signal from 10 electrodes, taken from left and right side, exactly as the artificial signal. The classifier algorithm used in this scenario is linear discriminant analysis (LDA), which characterizes or separates two or more classes of objects or events. The resulting combination may be used as a linear classifier. The classes are: left and right movement. The rest class doesn t exists in the Handball scenario because the problem would get very complicated. The format of the EEG data used in scenario is Open Vibe (.ov). The format used in BCI experiments conducted with the use of Matlab is General Data Format for biosignals (.gdf), so the first step is to transform the OV data into GDF data. For this I ve built an Open Vibe software which accomplishes this task. The GDF format provides a common coding scheme for events, and supports many useful features (different sampling rates and calibration values for different channels, an automated overflow detection, support of different data types, encoding of filter settings etc.), that are only partly implemented in other formats[42]. Briefly, it addresses the need for Subject specific information (gender, age, impairment, etc) Recording location, identification of recording software, etc. Possibilities for storing the electrode positions in spatial coordinates, electrode impedances, etc. 51 P a g e

52 More efficient encoding of date and time, physical dimensions, filter information Non-equidistant (sparse) sampling[gdf.2.0]. This type of format stores an event table, which consists of all movements performed during the tests, their position and length. This helps us to know exactly when a movement has happened. To use the GDF format with Matlab I ve installed a gdf library which allows the user to load an EEG signal. The syntax is: [signal,header]=sload( name of gdf file.gdf); The signal variable stores the EEG data, which in our case is a 10 columns matrix. The header variable stores the support data for the signal, like the event table. The results are shown below: FIS with manual rules The coefficients obtained demonstrates that the manual FIS doesn t recognize the movements anymore. 52 P a g e

53 FIS with automatic rules This chart represents the fuzzy classification for both the training and the experimental data. The training data is on the left side of the chart and the real experiment data is on the right side. As we can see, the training data is well classified, except for the rest period. The experiment data is not so well classified, but we can see that it exists a movement, but in some cases, when it should be a rest period, the classification algorithm shows that it is a movement. Taken into account all these, the FIS based on Chiu s algorithm is starting to work. It recognizes almost all the movements. 53 P a g e