Automatic Electrical Home Appliance Control and Security for disabled using electroencephalogram based brain-computer interfacing

Transcription

1 Automatic Electrical Home Appliance Control and Security for disabled using electroencephalogram based brain-computer interfacing S. Paul, T. Sultana, M. Tahmid Electrical & Electronic Engineering, Electrical & Electronic Engineering, Computer Science & Engineering School of Science, Engineering and Technology, East Delta University Chittagong, Bangladesh [jishu.astro, tanni.tanin, Abstract The development of Brain Computer Interface (BCI) system helps to utilize Electroencephalography technology providing an effective way of turning human thoughts into actions as well as communication to other physical devices without any help of the traditional muscular pathways. The BCI system incorporated with EEG technology has recently become a wonderful solution to provide direct communication and interaction pathways for old age persons, sick patients, and especially for the severe handicapped person. In this Paper we propose a novel automatic electrical home appliance control system model using BCI. The proposed system will collect brain signals through EEG equipment and process using a microcontroller. The extracted brain thought signals will further be sent via available wireless communication technology to the input of the receiver microcontroller that will directly activate and control the electrical appliances. The system will be incorporated with a security alert subsystem along with the control purpose where the user can activate an instant alarm by his thoughts in case of danger as well. It is expected that the advantage of portability and cheapness of this proposed system compared to the other BCI systems will make it superior and more user-friendly. Index Terms BCI, EEG, Home Automation, Microcontrollers, Wireless, Bispectrum. I. INTRODUCTION Motor imagery represents the result of conscious access to the content of the intention of a movement. It can be defined as a dynamic state during which an individual mentally simulates a given action. This type of phenomenal experience implies that the subject feels him/her self performing the action. It is usually performed unconsciously during movement preparation. But a very interesting fact is that, conscious and unconscious motor preparation share common mechanisms and they are functionally equivalent. As a result, a clear image of an intended action can be present even without the limb being involved[1]. A brain computer interface (BCI) is a direct communication pathway between the brain and an external device. It is also called mind-machine interface (MMI), direct neural interface (DNI), or brainmachine interface(bmi). It is a communication system for controlling a device, e.g. computer, wheelchair or a neuro-prosthesis, by human intensions, which does not depend on the brain s normal output pathways of peripheral nerves and muscles but relies on the detectable signals representing responsive or intentional brain activities. It transforms mental intentions into control commands by analyzing the bioelectrical brain activity. BCIs can help patients totally losing volitional motor ability but having intact cognition and improve their living standards[2]. A successful BCI system very much depends on the following criteria: i. Ability of the extracted features to differentiate the task-oriented brain states, ii. Efficiency of the methods for classifying such features in real-time[3]. For analyzing BCIs, the brain activity of a patient has to be recorded. The traditional Electrode system for acquiring brain signal is International 1-2 system that works with the help of 21 electrodes which are placed on the surface of the human scalp. The usage of common Ag-AgCl small disc metal electrodes followed by proper skin preparation, conductive gel etc. can cause discomfort to the human sample for a long term signal acquisition process. Compared to this metal electrode system, user friendly dry electrodes offer more convenient way to EEG technology[4]. EEG measures voltage fluctuations resulting from ionic current within the neurons of the brain. EEG measures the summation of electrical potentials in the form of electric field generated from millions of neurons having same spatial orientation. The electrical field is mainly developed by currents that flow during synaptic excitation of the dendrites[4]. Advantages of EEG are that it is a very low cost technique, non-invasive and recording procedures are comparatively easier. The proposed Brain-computer interface (BCI) model will provide an alternative method of expressing human thoughts other than the traditional pathways of peripheral nerves or muscles with the help of the communication based on neural activity generated by the brain. So, the objective of this research is to prepare a real time basic Brain Computer Interfaced model that will be able to activate Electric loads along with an alarm especially for the disabled people using the classification of EEG signals.

2 A. General BCI Diagram II. GENERAL SYSTEM REVIEW The General BCI system consists of Signal Acquisition block, Feature extraction followed by analysis of the signals, classification of the signals and interfacing with the real time machine through computer with the help of machine learning algorithm. On the other hand, the newly invented single channel prototype called NeuroSky Mind Wave sensor which is a dry type sensor is able to acquire different brain activities in a non-invasive manner. Compared to the International standard electrodes, brain wave sensor provides less complexity along with increased accuracy. Every NeuroSky product encloses a Think Gear chip which enables the interface between users brain and the load activation unit[6]. This TGAM (Think Gear) module consists of an onboard chip that filters the electrical noise by processing the data sets. Raw brainwaves and the essence (like attention, Excitement and concentration) values are determined[7]. The sensor (Fig 4) performs extraction of raw EEG signal in a non-invasive manner and transmits wirelessly to the processing unit through RF transmitter. Sensor composed of headset, an ear clip and sensor arm[8]. Specifications of such devices are: weight 9g, frequency ranges from GHz and its maximum power is 5mw. Sampling rate of EEG signal is 512Hz. Fig. 1. General BCI system B. EEG Signal Acquisition system The signal acquisition part is performed with the help of either wet electrode system or dry electrode system. International 1-2 system, which is a standardized system for electrode placement[5] uses 21 electrodes that are placed upon the human scalp as Fig 2. Fig. 4. The Neurosky mind wave sensor model C. EEG rhythms and waveform The recorded EEG signals from the scalp having amplitudes ranging from 2 to 1 microvolts and a frequency content ranging from.5 to 3-4 Hz can be conventionally classified into five different frequency bands as the table below: Fig. 2. The 21 Electrode placement of International 1-2 system The numbers 1 and 2 refer to percentages of relative distances between different electrode locations on the skull. The general electrode- skin interface diagram is as the Fig 3. Fig. 3. Skin- Electrode interfacing in the wet electrode system D. EEG Data Analysis Signal processing methods can be divided into two general categories: methods developed for the analysis of spontaneous brain activity and brain potentials which are evoked by various sensory and cognitive stimuli. The alteration of the ongoing

3 EEG due to the stimuli is called event related potential (ERP), in the case of external stimulation called evoked potential (EP). There are mainly three modalities of stimulation: auditory stimuli are single tones of a determined frequency, or clicks with a broadband frequency distribution. Visual stimuli are produced by a single light or by the reversal of a pattern as for example a checkerboard. Somatosensory stimuli are elicited by electrical stimulation of peripheral nerves. The power spectral analysis provides a quantitative measure of the frequency distribution of the EEG at the expense of other details in the EEG such as the amplitude distribution and information relating to the presence of particular EEG patterns. Hence timefrequency signal-processing algorithms such as discrete wavelet transform (DWT) analysis are necessary to address different behavior of the EEG in order to describe it in the time and frequency domain. It should also be emphasized that the DWT is suitable for analysis of non-stationary signals, and this represents a major advantage over spectral analysis. 2) Classification of data in the Microcontroller: The acquired and preprocessed data is considered to be the input to the microcontroller. Here the software part of the system will be in use. A prestored matrix containing the feature vector for the previously stored training data will be used by the machine learning algorithm. The acquired EEG data will be processed according to the algorithm described in the software part to produce the test feature vector. The machine learning algorithm wil be implemented in the microcontroller using the following block diagram for KNN classifier[1]. START Acquire (known) training dataset and Unknown test samples Define class for training dataset III. METHODOLOGY The methodology is divided into two sections. The hardware configuration and the software processing part. First, we explain the proposed hardware model and then show the algorithm used in processing of the brain signal with the simulation results included in the following section. A. Hardware Model 1) Data Acquisition: Unlike the electrocardiogram (ECG), EEG has a very low amplitude (5-5 uv) and their noisy nature make it hard to detect them. Another issue is the DC offset of the signal due to electrode-tissue interface. This DC offset is usually 2-5 mv and about 5 times bigger than the signal. Thus, a very low noise, high input impedance and high CMRR (Common Mode Rejection Ratio) instrumentation amplifier is required to amplify these signals and reject the DC offset[9]. The following block diagram shows the data acquisition system. Yes Calculate distance between training dataset and unknown data samples. Check it for all unknown data Sort out distances and first k distances And corresponding classes The test pattern is announced to be of class J if No. of distances (out of K distances) corresponding to class J is maximum Any Unknown test sample remains? Stop Fig. 6. Block Diagram of the KNN algorithm No 3) Wireless Data Transmission: After the processing of the brain signal, the acquired binary result is transmitted through a bluetooth module to the desired receiver device. In our system, we are currently controlling only one load and there are two instructions, namely ON or OFF. But the system can be upgraded easily for multiple loads at the same time. Fig. 5. Block Diagram of the EEG acquisition system Here, we propose the international 1-2 system of leads as the electrode system. B. Software Model We are proposing a classification algorithm which uses the statistical analysis of the bispectrum of the EEG signal. The block diagram in Fig 8 shows the algorithm used in this system. For a test run, we have used an available dataset which features the time delay between the two datasets. It is to be noted that, a real time BCI based system always faces the problem of time difference between training and testing data and most of the existing EEG classification algorithms fail

4 Fig. 7. Overview of the Proposed System to perform well in this situation. Our proposed algorithm has performed well in this circumstances (checked by simulation). In our system, we will use two imagery motor tasks to detect the corresponding ON or OFF signals. Corresponding to each instruction a 1 or a is generated. Fig. 8. Proposed Algorithm 1) Details of the Dataset: The dataset used in this research is the dataset I of the BCI competition III, it contains data of imagined motor movement of left small finger or tongue. The train set and the test set were recorded from the same subject in two different days with one week in between. In the BCI experiment, a subject had to perform imagined movements of either the left small finger or the tongue. The time series of the electrical brain activity was picked up during these trials using a 8x8 ECoG platinum electrode grid which was placed on the contralateral (right) motor cortex. The grid was assumed to cover the right motor cortex completely, but due to its size (approx. 8x8cm) it partly covered also surrounding cortex areas. All recordings were performed with a sampling rate of 1Hz. Further details about the dataset I in the BCI competition III can be found in [11]. After amplification the recorded potentials were stored as microvolt values. Every trial consisted of either an imagined tongue or an imagined finger movement and was recorded for 3 seconds duration. To avoid visually evoked potentials being reflected by the data, the recording intervals started.5 seconds after the visual cue had ended. The labeled training data from the first session was stored in a file called Competition train.mat. It consists of two parts: Part 1: the brain activity during 278 trials. This part is stored in a 3D matrix named X using the following format: [trials electrode channels samples of time series]. Part 2: the labels of the 278 trials. This part is stored as a vector of -1 / 1 values named Y. The unlabeled test data is also stored in a file called Competition test.mat. It contains 1 trials of brain activity in matrix X (3D format is the same as described above) but it contains no labels Y. 2) Feature Extraction: In this method, we have used bispectrum of the original ECoG signal and extracted some higher order statistical features. The feature vector consists of bispectral higher order statistical features of the whole signal. Bispectrum is a higher order statistical analysis. It is an ideal tool to investigate non-linear interactions. The Fourier transform of the third-order cumulant is called the bispectrum. For a non-gaussian random process x(t), its thirdorder cumulant is defined as C 3x (m, n) = E[x(k).x(k + m).x(k + n)] (1) Where, E is the expectation of the of the multiplication of the process and its two lagged versions. The bispectrum of this process is defined as the 2D Fourier transform of the cumulant, B x (ω 1, ω 2 ) = C 3x (m, n).e [ j2π(mω1+nω2)] m= n= (2) On the other hand, if the process is Gaussian, then its thirdorder cumulant is, C 3x (m, n) = (3) That is, any Gaussian noise in the system is nullified by the bispectral analysis. We can define the brain signal as a sum of non-gaussian random process x(t) and Gaussian noise w(t). z(t) = x(t) + y(t) (4) Then, the third-order cumulant of the signal is, C 3z (m, n) = C 3x (m, n) (5) In this method, we have extracted 4 features from the calculated bispectrum from each channel of a single trial. All of these are statistical features. The following features were considered, a) The sum of the logarithmic amplitudes of the bispectrum, F 1 = log( B x (ω 1, ω 2 ) ) (6) ω 1,ω 2 F b) The sum of the logarithmic amplitudes of the diagonal elements of the bispectrum,

5 F 2 = log( B x (ω, ω) ) (7) ω F c) The 1 st order spectral moment of the amplitudes of the diagonal elements of the bispectrum, F 3 = N k. log( B x (ω k, ω k ) ) (8) k=1 d) The 2 nd order spectral moment of the amplitudes of the diagonal elements of the bispectrum, F 4 = N (k H 3 ) 2. log( B x (ω k, ω k ) ) (9) k=1 So, the feature vector is formed as, F = [F 1 F 2 F 3 F 4 ] (1) 3) Feature Quality: We have verified the qualities of the proposed features. The quality is determined by two parameters: 1. Inter-class Separability, 2. Intra-class Compactness. These two parameters for each of the features are shown here. Inter-class Separability: The following figures show the interclass separabilities for the 4 features extracted from the bispectrum of the signal. 15 x Fig. 9. Inter-class separability of feature F 1 8 x x Fig. 11. Inter-class separability of feature F Fig. 12. Inter-class separability of feature F 4 So from the above figures, it can be easily seen that, the feature values of each channel differ by a significant amount for each task. The red line shows feature values for task-1: movement of left small finger and the green line shows feature values for task-2: movement of the tongue. Intra-class Compactness: The following figures show the compactness of the feature (taking the second feature as example) for each of the two tasks. The black line in the figure represents the mean of the feature values for each channel Fig. 1. Inter-class separability of feature F Fig. 13. Intra-class compactness for task-1

6 Fig. 14. Intra-class compactness for task-2 From the above 2 figures, the compactness of the features can be seen for individual tasks. With the exception of 2 or 3 channels in some of the trials, all the feature values form a compact band for each individual task. 4) Classifier: In our method, we have used KNN classifier to predict the labels of the test set. K-nearest neighbor is a nonparametric method used for classification. Its input consists of the k closest training examples in the feature space. The output is a class membership. An object is classified by a majority vote of its neighbors, with the object being assigned to the class most common among its k nearest neighbors. It is among the simplest of all machine learning algorithms. In KNN, k is a positive integer. The value of k depends upon the data. Generally, larger values of k reduce the effect of noise on the classification, but make boundaries between classes less distinct. Smaller values of k make boundaries distinct and computations easier; but noise will have a higher influence. There are three tie-breaking rules: nearest, random and consensus. In our method we have used the first two rules to check the accuracy. The distance metrics are of five types: Euclidean, Cityblock, Cosine, Correlation and Hamming. upgradable to multiple load capacity without the increase of the number of processing units. Most of the existing systems include the processing unit with the load, hence increasing number of loads becomes tedious and costly. Also, we have designed the processing algorithm keeping in mind the timevarying and random nature of EEG signals. Though some further future expansions are needed, the proposed system surely has the qualities to be an effective and efficient automation system to be useful for the handicapped. REFERENCES [1] Martin Lotze and Ulrike Halsband, Motor imagery, Journal of Physiology - Paris 99 (26) [2] Wolpow JR, Birbaumer N, McFarland DJ, et al. Brain-computer interface for communication and control, Clinical Neurophysiology, 22, 113: [3] S.M. Zhou, John Q. Gan, F. Sepulveda, Classifying mental tasks based on features of higher-order statistics from EEG signals in braincomputer interface, Information Sciences 178 (28) [4] Lukas Maly, WHEELCHAIR CONTROL USING EEG SIGNAL CLASSI- FICATION, MS Thesis Paper, BRNO UNIVERSITY OF TECHNOL- OGY. [5] Jasper, H. H., The ten twenty electrode system of the international federation, Electroencephalography and clinical neurophysiology, 1 (1958), [6] Athanasios Vourvopoulos, Fotis Liarokapis, Evaluation of commercial brain-computer interfaces in real and virtual world environment: A pilot study, Computer and Electrical Engg. 4(214) [7] L.R. Stephygraph, N.A. Kumar, V. Venkatraman Wireless Mobile Robot Control through Human Machine Interface using Brain Signals, International Conference on Smart Technologies and Management for Computing, Communication, Controls, Energy and Materials (ICSTM), India. 6-8 May 215. pp [8] NeuroSky related informations available at [9] Lin ZHU et al. Design of Portable Multi-Channel EEG Signal Acquisition System, Biomedical Engineering and Informatics, 29. BMEI 9. 2nd International Conference on. [1] Neerja S. D., Rupesh S. M., Methods towards invasive human brain computer interfaces, International Journal of Scientific and Research Publications, Volume 3, Issue 6, June 213. [11] Lal TN, Hinterberger T, Widman G Robotic Automation through Speech Recognition, Advances in Neural Information Processing Systems, Cambridge: MIT Press, 25, 17: IV. RESULTS We have used accuracy as our performance parameter. Accuracy is defined as the number of labels predicted by the proposed method, matching the true labels provided by the dataset providers. Though the maximum accuracy falls for some cases, we still get 87% accuracy for the cosine distance metric with nearest rule. Moreover, the optimum value of k is reduced to a very low value of 6. Thus the proposed feature vector can provide a high accuracy with low classification complexity. V. CONCLUSION This research paper showed detailed overview of the general EEG based BCI systems along with a proposed model that will implement the system with the help of the microcontroller. A relatively high performing and real-time efficient classification algorithm is presented here. Moreover our designed and proposed system has the advantage of single processing unit compared to most of the existing systems. Our system is easily