Independent Component Analysis of Simulated EEG. Using a Three-Shell Spherical Head Model 1. Dara Ghahremaniy, Scott Makeigz, Tzyy-Ping Jungyz,

Transcription

1 Independent Component Analysis of Simulated EEG Using a Three-Shell Spherical Head Model 1 Dara Ghahremaniy, Scott Makeigz, Tzyy-Ping Jungyz, Anthony J. Belly, Terrence J. Sejnowskiyx fdara, scott, jung, tony, terryg@salk.edu Institute for Neural Computation Technical Report No. INC-9601 yhoward Hughes Medical Institute Computational Neurobiology Laboratory The Salk Institute, P. O. Box San Diego, CA znaval Health Research Center P.O. Box San Diego, CA Department of Neurosciences School of Medicine University of California San Diego La Jolla, CA xdepartment of Biology University of California San Diego La Jolla, CA This report was supported in part by the Navy Medical Research and Development Command and the Oce of Naval Research, Department of the Navy under work unit ONR.Reimb The views expressed in this article are those of the authors and do not reect the ocial policy or position of the Department of the Navy, Department of Defense, or the U.S. Government. Approved for public release, distribution unlimited.

2 Abstract The Independent Component Analysis (ICA) algorithm 1 is a new information-theoretic approach to the problem of separating multichannel electroencephalographic (EEG) data into independent sources 2. We tested the potential usefulness of the ICA algorithm for EEG source decomposition by applying the algorithm to simulated EEG data. These data were synthesized by projecting 6 known input signals from singleand multiple-dipole sources in a three-shell spherical model head 3 to 6 simulated scalp sensors. In dierent simulations, we (1) altered the relative source strengths, (2) added multiple low-level sources (weak brain sources and sensor noise) to the simulated EEG, and (3) permuted the simulated dipole source locations and orientations. The algorithm successfully and reliably separated the activities of relatively strong sources from the activities of weaker brain sources and sensor noise, regardless of source locations and dipole orientations. These results suggest that the ICA algorithm should be able to separate temporally independent but spatially overlapping EEG activities arising from relatively strong brain and/or non-brain sources, regardless of their spatial distributions.

3 1 Introduction Multichannel electromagnetic recordings from the scalp, including EEG, magnetoencephalographic (MEG), event-related potential (ERP) and event-related eld (ERF) data, have been widely used to study dynamic brain processes involved in perception, memory, selective attention, recognition, and priming. However, the underlying brain processes which produce elds recorded at the scalp are largely undetermined. The most common model for EEG generation assumes that electrodes placed on the scalp surface record the electromagnetic activity of local or distributed cortical neural networks which form eective single- or multiple-dipole sources (Fig 1a) 6. 4; 5; EEG recordings consist of a complex distribution of overlapping source activities, making it dicult to identify the contributing independent sources. The problem of separating sources without a priori knowledge of their number or spatial distribution is known as \blind separation". Most existing techniques for approaching the problem of source separation employ second-order statistical methods (e.g. covariance, cross-correlation, and principle component analysis). 6 The Independent Component Analysis (ICA) algorithm 1 we use is a blind separation technique based on information-maximization which uses higher-order statistical information. The algorithm has been recently shown to produce useful decompositions of EEG data 2, separating identiable EEG components (e.g., alpha waves and steady-state responses 8 ) 7; into individual output channels. However, without prior knowledge of the actual brain sources which contribute to the EEG, it is dicult to verify the algorithm's eectiveness. We assume there 1

4 may be a few strong sources active during a given EEG recording period along with a larger number of relatively weak sources. In addition, low-level sensor noise may contaminate scalp recordings. To determine whether the ICA algorithm can successfully separate relatively strong signals mixed with numerous weaker signals, we performed several simulation experiments. We simulated the activities of 6 brain source signals projected in a three-shell spherical head model 3 by volume conduction to 6 scalp electrodes and applied the ICA algorithm to resulting simulated EEG signals. The simulations allowed us to investigate changes in ICA algorithm performance with variations in source strength, location, and orientation as well as eects of adding simulated weak brain signals and sensor noise to the simulated EEG. 1.1 The ICA algorithm The algorithm is based on an `infomax' neural network 10. It nds, by stochastic 1; 9; gradient ascent, a matrix, W, which maximizes the entropy, 11 H(y), of an ensemble of `sphered' input vectors fx s (t)g, linearly transformed and sigmoidally compressed: u(t) = Wx s (t); y = g(u(t)) (1) The `unmixing' matrix W performs source separation, while the sigmoidal nonlinearity g() provides necessary higher-order statistical information. Initial sphering of the zero-mean input data 12 : x s (t) = Px(t) (2) where P is twice the matrix square root of the inverse of the covariance matrix, used to speed convergence: P = 2hxx T i? 1 2 (3) 2

5 W is then initialized (in our simulations with random values between 0.1 and 1.0), and iteratively adjusted using small batches of data vectors drawn randomly from fx s (t)g without substitution, according to: WT W = (I + ^yu T )W; (4) where is the learning rate, I is the identity matrix, and vector ^y has elements ^y i = (@=@u i ) ln(@y i =@u i ) (5) The (W T W) `natural gradient' term in the update equation 13 avoids matrix inversions and speeds convergence. We use the logistic nonlinearity, g(u i ) = (1 + exp(?u i ))?1, for which ^y i = 1? 2y i. When ICA algorithm is trained on EEG data, the rows of the resultant matrix (WP) are linear spatial lters which, applied to the input data, produce source activity waveforms (WPx(t)). The columns of the inverse weight matrix (WP)?1 represent the projection weights from the ICA algorithm sources to the sensor array. Further details and references about the algorithm appear in 15, other related approaches and background material in 19. 1; 13; 14; 10; 16; 17; 18; 2 Methods An overview of the simulation process is given in Fig The three-shell spherical head model In our simulations, we used a three-shell spherical head model which projects dipoles at 4 xed brain locations onto 6 scalp electrodes. The projection matrix containing the model parameters was precomputed by Anders Dale using an analytic representation for a three-shell spherical head model 20. Electrode positions were vertices of a 3; 3

6 triangulated icosahedron located on the model head sphere. At each of the 4 locations in the head model, we placed 1 to 3 dipoles pointing in dierent directions, giving a total of 7 dipoles. We assigned 5 input signals to single dipoles, and 1 input signal (Fig. 2a) to two bilateral dipoles (Fig. 2b). As shown in Fig. 2, two dipoles with dierent orientations were placed at a single dipole location, and three dipoles with dierent orientations were placed at another location. These choices were expressed via a ((4 3) 6) conguration matrix, C, which assigned 6 source signals to the 7 dipoles according to the conguration described above. The conguration matrix was then multiplied by the (6 (43)) weight matrix, F, which projected the 7 dipoles (at the 4 dipole locations) to each of the 6 selected electrode sites. The resulting matrix product: M = FC (6) was a 6 6 \mixing" matrix specifying the simulated EEG signals as linear combinations of the 6 input sources. Simulation variables were chosen such that this mixing matrix was non-singular. Note that despite the complexity of the head model, the mixing matrix was a linear 66 transformation of the 6 sources, and therefore satised the assumptions of the ICA algorithm. 2.2 Input signals The input signals were six 7.5-sec (79,119-point) segments of acoustic signals consisting of speech signals (\iris" and \zach"), drum tapping sounds (\drum"), a sounding gong (\gong"), a choral excerpt from Handel's Messiah (\handel"), and a keyboard synthesizer sequence (\synth"). Each signal was recorded by the auxiliary microphone of a Sparc-10 workstation 1. Before the simulations, each input was made zero-mean 4

7 and normalized by linear scaling to t within the [-1, 1] interval. 2.3 Source strength adjustment To simulate sources with varied strengths, the vector of input signals, s(t), were scaled relative to one another in steps of -8 db (Fig. 3) using a 66 diagonal attenuation matrix, A. Simulated EEG signals, x(t), were derived from the input signals by multiplying by the attenuation and mixing matrices. x(t)= MAs(t) (7) 2.4 Weak brain sources In some experiments, seven simulated weak brain source signals were added to the simulated EEG. These (\brain noise") sources consisted of uncorrelated random noise with a at distribution in the [-1,1] interval, scaled to -40 db below the strongest input signal (i.e., at the same level as the weakest input signal) (Fig. 3). The 7 brain noise sources were assigned to simulated \diuse" dipoles placed close to each of the 7 brain source dipoles by adding 1% gaussian-distributed noise to the matrix, M, before mixing. The mixed brain noise signals were then added to the simulated EEG. 2.5 Sensor noise To simulate EEG sensor noise, uniformly-distributed white noise was added to each sensor signal at an intensity of -64 db below the mean level of the simulated EEG signals. These noise sources were uncorrelated with each other. 5

8 2.6 ICA algorithm training Training with the ICA algorithm began with an initial learning rate of This was reduced to after the rst training step. Thereafter, a heuristic method was used to reduce or increase the learning rate at each time step according to the net change in weights from the previous step. This change was computed by taking the sum of squares of the changes in weights between the current and previous time steps. Whenever the net weight change was less than 10?7, the learning rate was multiplied by 5/8ths. If the learning rate went below 10?7, it was increased to 4 10?6 and the input data was reshued to avoid overlearning. Training was stopped after 32 steps. All computations were performed using Matlab (version 4.2c) on a Sun 670MP with 64 megabytes of RAM and a 40MHz processor (equivalent to a Sparc 2). 2.7 Performance measures SNR in the ICA algorithm output Our measure of the ICA algorithm's performance was the signal-to-noise ratio (SNR) of each input signal in the output sources. For each input signal, s i (t), we dened: s i (t) = s i (t) 0. in which all input signals except for s i (t) were zeroed out. The output source waveforms for s i (t) were then dened as: 0 1 C A (8) u i (t) = WPMAs i (t) (9) 6

9 The signal level, Sik ICA, of the ith input signal in the kth output source waveform was computed by taking the standard deviation of the kth row of u i (t). The noise level for each input signal in each output source was computed by letting s ic (t) consist of all input signals except s i (t): s i c (t) = 0 s 1 (t). s i?1 (t) 0 s i+1 (t). s n (t) 1 C A (10) These \complementary" signal vectors were passed through the simulated mixing and unmixing processes with brain noise and sensor noise sources added, giving output source waveforms: u i c (t) = WPfMA[s i c (t) + n(t)] + r(t)g (11) where n(t) is the weak brain sources and r(t) is the sensor noise. The noise level, Nik ICA, was dened as the standard deviation of the kth row of u c i(t). Then, the SNR of the ith signal in the ICA algorithm source waveforms was dened as: SNR ICA i = max k=1;;n (20log 10 ( SICA ik N ICA ik )) (12) where n is the number of sources SNR in the simulated EEG The SNR of each input signal in the simulated EEG was computed for comparison with the SNR in the ICA output. The signal level, Sij EEG, for the ith input signal 7

10 in the simulated EEG signal was dened as the standard deviation of the simulated EEG in the jth recording electrode (i.e. in the jth row of x i (t)): x i (t) = MAs i (t) (13) The noise level, Nij EEG, for the ith input signal was dened as the standard deviation of the jth row of the complementary mixed signal matrix: x i c (t) = MA[s i c (t) + n(t)] + r(t) (14) SNR of the ith input signal in the simulated EEG was then dened as: SNR EEG i = max j=1;;m (20log 10 ( SEEG ij N EEG ij )) (15) where m is the number of sensors SNR gain from EEG to ICA algorithm outputs For each input signal, the dierence between its SNR ICA and SNR EEG was dened as the SNR gain, G, resulting from ICA algorithm source separation. G i = SNR ICA i? SNR EEG i (16) 2.8 Four simulation experiments We conducted four simulation experiments to test the ecacy and reliability of the ICA algorithm in performing blind separation of EEG signals. Each experiment consisted of six dierent ICA algorithm trainings: Experiment 1: Without noise sources. To study the eect of dierent initial weights, W, and data presentation orders on the output of the ICA algorithm, 8

11 we trained the algorithm with randomized initial weight matrices and input data presentation orders. Experiment 2: With noise sources. The simulations above were repeated with the 13 noise sources (7 weak brain sources and 6 sensor noise sources) added to the simulated EEG signals to test the source separation performance of the algorithm under realistic conditions. Experiment 3: Varying input signal strength assignments. Performance of the ICA algorithm may depend in part on the statistical distributions of the input signals 12. To test whether dierences in the input signal distributions were responsible for the results of Experiment 2, we circularly permuted the order of assignment of input signals (by rotating the rows of s(t)) to attenuation levels A (eqn. 7). Experiment 4: Varying input signal source assignments. In previous experiments, the assignment of stronger and weaker signals to model brain sources was xed. In this experiment, we varied the attenuated signal assignments to brain sources across ICA algorithm trainings. First, we attenuated the input signals in the same order as in Experiment 1. We then circularly permuted the assignment of the attenuated input signals to brain sources (by rotating the rows of As(t) before multiplying by M in equation 7). 9

12 3 Results 3.1 ICA algorithm performance without low-level sources With simulated weak brain sources and sensor noise sources turned o, the ICA algorithm consistently separated each source into a dierent output channel regardless of dierences in signal amplitudes (Fig. 4) and the algorithm's initial conditions. The results conrmed similar ndings reported for earlier audio simulations 1. Each input signal was separated into a dierent output channel with an SNR ICA of at least 30 db. The SNR gain, G, for the 6 input signals ranged from 21 db to 67 db. Although both the input signal levels and SNR ICA varied widely between signals (ranges of 40 db and 36 db respectively), each input source was separated cleanly into a separate ICA algorithm output channel. This result was highly reproducible; standard deviations of SNR ICA across trainings were all less than 1 db. Most SNR gain occurred during the rst training step. 3.2 Eects of adding low-level sources When the 13 low-level sources were added to the simulated EEG, separation remained strong for the two strongest input sources (SNR ICA > 20 db) (Fig. 5), moderate for the two next-strongest signals (SNR ICA > 8 db), and weak for the weakest two input signals (SNR ICA < -10 db). SNR gains for the 6 brain sources ranged from 12 db to 29 db. Nearly all SNR gain occurred during the rst 5 training steps. 3.3 Eects of varying input signal strength assignments Mean dierences in SNR ICA for the 6 input signals closely followed their relative input amplitudes (Fig. 6). The range of mean SNR ICA values (39 db) was again 10

13 close to the range of input levels (40 db). The SNR gain for the 6 input signals ranged from 13 db to 31 db. Stronger sources appeared in individual ICA output channels while weaker ones (and noise sources) were mixed in remaining channels. 3.4 Eects of varying source assignments For each permutation of signal-to-source assignments, the ICA algorithm gave results comparable to those in Experiment 3. The range of mean SNR ICA (39 db) closely matched the range of input signal strengths (40 db) (Fig. 7). SNR gains ranged from 14 db to 30 db. 4 Conclusion The reported eectiveness of the ICA algorithm in separating multiple linearly-mixed sources 1; 9; 10 was reproduced in our EEG simulations using a three-shell head model with 6 input signals. Previously, performance of the algorithm in the presence of multiple weak brain sources and noise sources had not been systematically investigated. In our experiments, relatively strong simulated EEG signals were successfully and repeatedly separated with SNR gains averaging 22 db. Our results indicate that the performance of the algorithm degrades gracefully in the presence of multiple weak independent sources. 5 Discussion The Independent Component Analysis algorithm appears to be a promising tool for the analysis of highly correlated multichannel EEG signals. Our results suggest that relatively strong brain EEG sources may be eectively separated from weak brain and 11

14 noise signals with SNR gains of 20 db and above. Applications of ICA algorithm to averaged event-related potentials (ERPs) may be particularly promising since response averaging increases the amplitudes of activity, time- and phase-locked to experimental events, relative to the activities of all other spontaneous (i.e. non-phase locked) EEG sources. The number of independent strong brain sources contributing to ERP data may be smaller than the number of EEG channels typically used to record them 15. In that case, most or all of the ERP sources may be separable using the ICA algorithm. This algorithm could be used to compare the time courses and relative strengths of ERP source activations in dierent experimental conditions. Since the algorithm describes what independent sources produce its input data, not where these sources are spatially located, neurophysiological interpretation of the ICA algorithm sources poses a further research challenge. Acknowledgements This report was supported in part by grants to S.M., T-P.J. and T.J.S. from the Oce of Naval Research, and to T.J.S. from the Howard Hughes Medical Institute. The authors wish to thank Anders Dale for supplying the head model parameters. 12

15 References 1. Bell, A.J. & Sejnowski, T.J. An information-maximization approach to blind separation and blind deconvolution, Neural Computation 7, (1995). 2. Makeig, S., Bell, A.J., Jung, T-P. & Sejnowski T.J. Independent component analysis of electroencephalographic data. Advances in Neural Information Processing Systems 8, MIT Press (1996). 3. Dale, A.M. & Sereno, M.I. Improved localization of cortical activity by combining EEG and MEG with MRI cortical surface reconstruction - a linear approach. J. Cogn. Neurosci. 5, (1993). 4. Nunez, P.L. Electric Fields of the Brain. New York: Oxford (1981). 5. Scherg, M. & Von Cramon, D. Evoked dipole source potentials of the human auditory cortex. Electroencephalogr. Clin. Neurophysiol., 65, (1986). 6. Chapman, R.M. & McCrary, J.W. EP component identication and measurement by principal components analysis. Brain and Language 27, (1995). 7. Pantev, C., Elbert, T., Makeig, S., Hampson, S., Eulitz, C. & Hoke, M. Relationship of transient and steady-state auditory evoked elds. Electroencephalogr. Clin. Neurophysiol. 88, (1993). 8. Galambos, R., Makeig, S. & Talmacho P. A 40 Hz auditory potential recorded from the human scalp. Proc. Natl. Acad. Sci. USA 78, (1981). 9. Linsker, R. Local synaptic learning rules suce to maximise mutual information in a linear network. Neural Computation 4, (1992). 13

16 10. Nadal, J-P. & Parga, N. Non-linear neurons in the low noise limit: a factorial code maximises information transfer. Network 5, (1994). 11. Cover, T.M. & Thomas, J.A. Elements of Information Theory, John Wiley (1991). 12. Bell, A.J. & Sejnowski, T.J. Learning the higher-order structure of a natural sound. Network: Computation in Neural Systems 7, 2 (1996). 13. Amari S., Cichocki, A. & Yang, H.H. A new learning algorithm for blind signal separation. In Advances in Neural Information Processing Systems 8, MIT Press (1996). 14. Bell, A.J. & Sejnowski, T.J. Fast blind separation based on information theory. Proc. Intern. Symp. on Nonlinear Theory and Applications, Las Vegas (1995). 15. Makeig, S., Jung T-P., Bell, A.J., Ghahremani, D., and Sejnowski, T. S. Blind separation of event-related brain responses into independent components. Nature. (submitted) 16. Cardoso, J-F. & Laheld, B. Equivalent adaptive source separation. IEEE Trans. Signal Proc. (to appear). 17. Comon, P. Independent component analysis, a new concept? Signal Processing 36, (1994). 18. Jutten, C. & Herault, J. Blind separation of sources, part I: an adaptive algorithm based on neuromimetic architecture. Signal Processing 24, 1-10 (1991). 14

17 19. Karhumen, J., Oja, E., Wang, L., Vigario, R. & Joutsenalo, J. A class of neural networks for independent component analysis. IEEE Trans. Neural Networks (to appear). 20. Kavanagh R.N., Darcey T.M., Lehmann D., and Fender D.H. Evaluation of methods for three-dimensional localization of electrical sources in the human brain. IEEE Trans. Biomed. Eng. 9, 25: (1978). 15

18 EEG waveforms Scalp sensors Overlapping source projections at scalp Dipolar brain sources Source waveforms Figure 1: Schematic illustration of two dipole sources with overlapping projections to the scalp. Activities of each dipole (\source waveforms") are projected to the scalp through three conductive layers (brain, skull, and scalp). The scalp sensors record potentials (\EEG waveforms") which sum activity from both dipoles. 16

19 Input Signals Simulated Brain Sources Simulated Scalp Sensors Output Signals Scaling Mixing Sphering Unmixing iris iris gong gong zach handel a c zach handel synth?? drum?? Brain signal sources Brain noise sources b Sensor noise sources EEG Simulation Figure 2: Schematic overview of the simulations. Input signals were scaled relative to one another (circles under \Scaling") and assigned to single- or multipledipole brain sources (long arrows). One signal (a) (here, \zach") was assigned to a bilateral dipole source (b) simulating, for example, a bitemporal source in the auditory cortices. Other signals (here, for instance, \gong", \synth", and \drum") were assigned to sources modeled as single dipoles with dierent orientations at the same brain location (c). Seven weak brain (or \brain noise") sources (small arrows) were positioned near the seven signal dipoles. The 6 input signal sources and 7 brain noise signals were mixed at the 6 simulated EEG sensors on the scalp surface (semicircles). Uncorrelated low-level \sensor noise" signals (small boxes near sensors) was added to the simulated EEG at each of the scalp sensors. After an initial \sphering" of the simulated EEG data, source separation was performed via the \unmixing" matrix produced by the ICA algorithm. Spatial ltering of the simulated EEG with the sphering and unmixing matrices produced output source signals. Four of these (labeled \iris", \gong", \zach", and \handel") were highly correlated with their respective input signals. Two other ICA algorithm outputs (labeled `??') mixed the remaining two weakest input signals (\synth" and \drum") with the noise signals. (See for an audio presentation of the signals at each stage of the simulation). 17

20 Relative strengths of input signals signal 1 signal 2 signal 3 signal 4 signal 5 signal 6 weak brain signals db Figure 3: Relative strengths of input signals. Input signals were scaled relative to one another in -8 db steps. The 7 weak brain (or \brain noise") sources added to the simulated EEG in Experiments 2-4 (rightmost bar) were scaled to the level of the weakest input signal. 18

21 Training with 6 different initial weights without noise Output SNR (db) 20 0 ICA EEG Training Steps Figure 4: Experiment 1. Output signal-to-noise ratio (SNR) for each input signal in the simulated EEG signals (dashed lines) and during ICA algorithm training. ICA algorithm separation performance was strong and consistent across all sources and multiple training runs. 19

22 40 Training with 6 different initial weights 30 Output SNR (db) EEG ICA Training Steps Figure 5: Experiment 2. When 13 additional low-level sources (7 weak brain sources, 6 sensor noise sources) were added to the simulated EEG, ICA performance in separating the 6 input signals was favorable (> 20 db) for strongest input signals, and poor (< -10 db) for relatively weak inputs. SNR gains (dierence between EEG and nal ICA SNR values) ranged from 12 db to 29 db for the six signals. 20

23 Permutation of source signals BEFORE attenuation Output SNR (db) ICA EEG Training Steps 30 Figure 6: Experiment 3. Blind separation performance by the ICA algorithm for 6 permutations of input signal ordering prior to attenuation (see Section 2.8 of text). The order of signal attenuation was reproduced in the output SNR. The range of output SNR values after 32 training steps (rightmost values) was close to the 40 db range of relative input signal strengths. SNR gains for the 6 sources ranged from 13 db to 31 db. 21

24 40 Permutation of source signals AFTER attenuation 30 Output SNR (db) ICA EEG Training Steps 30 Figure 7: Experiment 4. ICA algorithm performance for six dierent orders of assignments of attenuated input signals to brain sources (see Section 2.8 of text). Again, stronger signals were separated better than weaker signals, and the range of mean output SNRs (39 db) was nearly equal to the input signal scaling range (40 db). SNR gains for the 6 sources ranged from 14 db to 30 db. 22