Integrated Brain-Machine-Body Interfaces

Transcription

1 Integrated Brain-Machine-Body Interfaces Department of Bioengineering Institute for Neural Computation UC San Diego

2 Integrated Systems Neuroengineering Neuromorphic/ Neurosystems Engineering Neural Systems Learning & Adaptation Silicon Microchips Human/Bio Interaction Environment Sensors and Actuators

3 NSF EFRI 2012 Mind, Machines and Motor Control (M3C) Distributed Brain Dynamics of Human Motor Control G. Cauwenberghs, K. Kreutz-Delgado, T.P. Jung, S. Makeig, H. Poizner, T. Sejnowski, F. Broccard, D. Peterson, M. Arnold, A. Akinin, C. Stevenson, J. Menon EEG brain dynamics and Parkinson s METRIC fitness function Q PD markers adaptive control Force feedback EMG, kinetics, gaze EEG MoBI MIMO parameters!" synaptic plasticity force CyberGlove MoCap CNS PNS PNS MoCap MoBI thalamocortical/bg model PNS CNS PNS Neuromorphic emulation of brain dynamics in motor control SRT SRT IFAT Level 1 HiAER Level 1 HiAER IFAT SRT SRT SRT SRT IFAT Level 1 HiAER Level 1 HiAER IFAT SRT SRT Computational modeling Level 2 HiAER SRT SRT Connector

4 Neuromorphic Engineering in silico neural systems design Neuromorphic Engineering g 1 g 0 g 2 g 2 g 2 g 0 g 1 g 0 g 1 Neural Systems Learning & Adaptation VLSI Microchips

5 Silicon Model of Visual Cortical Processing V Bipole cells (diffusive network) g! 1! g! 0! g! 2! g! 2! g! 2! g! 0! g! 1! g! 1! g! 0! V1 LGN 4 6 Complex and hypercomplex cells (lateral inhibition) C! -x! C! +z! C! +y! I! -x! I! 0! I! I! 0! I! +z! I! 0! C! 0! I! +y! -y! I! -z! I! 0! I! 0! I! 0! I! +x! C! +x! 88 transistors/pixel (including photodetector) Optic Nerve C! -y! C! -z! Neural model of boundary contour representation in V1, one orientation shown (Grossberg, Mingolla, and Williamson, 1997) Single-chip focal-plane implementation (Cauwenberghs and Waskiewicz, 1999)

6 Silicon Learning Machines for Embedded Sensor Adaptive Intelligence Large-Margin Kernel Regression Kerneltron: massively parallel support vector machine (SVM) in silicon (JSSC 2007) Class Identification Sensory Features Analog ASP A/D Digital MAP Forward Decoding GiniSVM Sequence Identification Sub-microwatt speaker verification and phoneme recognition (NIPS 2004)

7 Kerneltron: Adiabatic Support Vector Machine Karakiewicz, Genov and Cauwenberghs, 2007 MVM x i SUPPORT VECTORS x INPUT KERNEL K (x,x i ) y = sign(!# i yik( xi, x) + i" S! i SIGN y b) Classification results on MIT CBCL face detection data Karakiewicz, Genov, and Cauwenberghs, VLSI 2006; CICC TMACS / mw! adiabatic resonant clocking conserves charge energy! energy efficiency on par with human brain (10 15 SynOP/S at 15W) capacitive load resonance

8 Sub-Micropower Analog VLSI Adaptive Sequence Decoding Chakrabartty and Cauwenberghs, 2004 j! GiniSVM i! 840 nw power X[n-1] X[n] X[n+1] Forward decoding MAP sequence estimation Biometric verification MVM MVM SUPPORT VECTORS x s 30x24 KERNEL 30x24 s! i1 K(x,x s ) x INPUT Silicon support vector machine (SVM) and forward decoding kernel machine (FDKM) f i1 (x) 14 24x24 P i1 NORMALIZATION 24x24 i FORWARD DECODING 24 " i[n] P " j[n-1]

9 Neuron-Silicon and Brain-Machine Interfaces Neuromorphic Engineering Adaptive Sensory Feature Extraction and Pattern Recognition Neuro Bio Learning & Adaptation Micropower Mixed-Signal VLSI Neurosystems Engineering Biosensors, Neural Prostheses and Brain Interfaces

10 Brain Machine Interfaces and Motor Control The brain s motor commands!! Parietal/frontal cortex Implanted electrodes Electroencephalogram (EEG)! Cortical signals, noninvasive! Low bandwidth (seconds)! Nerve signals Spinal cord electrodes Electromyogram (EMG)! Muscle signals, noninvasive! Higher bandwidth (milliseconds)! translated into motor actions! Machine learning/signal processing! Neuromorphic approaches Central pattern generators (CPGs) Nicolelis, Nature Rev. Neuroscience 4, 417, 2003

11 Wireless Non-Invasive, Orthotic Brain Machine Interfaces Calit2 StarCAVE immersive 3-D virtual reality environment Yu Mike Chi, 2010 TATRC Grand Challenge! Mind-machine interfaces for augmented human-computer interaction! Body sensor networks for mobile health monitoring and augmented situation awareness

12 BioSemi Active2 Scalp EEG Recording State of the art EEG recording! channels! Gel contact electrodes! Tethered to acquisition box! Off-line analysis

13 Envisioned High-Res EEG/ICA Neurotechnology T.J. Sullivan, S.R. Deiss, T.-P. Jung, and G. Cauwenberghs, 2008 Flex Printed Circuit EEG/ICA Silicon Die Dry Electrode RF Wireless Link Integrated EEG/ICA wireless EEG recording system! Scalable towards channels! Dry-contact MEMS electrodes (NCTU, Taiwan)! Wireless, lightweight! Integrated, distributed independent component analysis (ICA)

14 Independent Component Analysis The task of blind source separation (BSS) is to separate and recover independent sources from (instantaneously) mixed sensor observations, where both the sources and mixing matrix are unknown. x 1 x 2 x 3 Source signals Sensor observations Reconstructed source signals s 1 s 2 s(t) N A x(t) M W y(t) Mixing matrix Unmixing matrix N Independent component analysis (ICA) minimizes higher-order statistical dependencies between reconstructed signals to estimate the unmixing matrix. Columns of the unmixing matrix yield the spatial profiles for each of the estimated sources of brain activities, projected onto the scalp map (sensor locations). Inverse methods yield estimates for the location of the centers of each of the dipole sources.

15 EEG Independent Component Analysis! ICA on single-trial EEG array data identifies and localizes sources of brain activity.! ICA can also be used to identify and remove unwanted biopotential signals and other artifacts. EMG muscle activity 60Hz line noise Swartz Center for Computational Neuroscience, UCSD Left: 5 seconds of EEG containing eye movement artifacts. Center: Time courses and scalp maps of 5 independent component processes, extracted from the data by decomposing 3 minutes of 31-channel EEG data from the same session and then applied to the same 5-s data epoch. The scalp maps show the projections of lateral eye movement and eye blink (top 2) and temporal muscle artifacts (bottom 3) to the scalp signals. Right: The same 5 s of data with the five mapped component processes removed from the data [Jung et al., 2000].

16 Envisioned High-Density EEG Embedded in Elastic Fabric Non-contact electrode! No skin/subject preparation! Insulated, embeddable in elastic fabric Fully integrated! On board power, signal processing, wireless transceiver Applications! Brain computer interface! Mobile, health monitoring

17 Wireless Non-Contact Biopotential Sensors Mike Yu Chi and, 2010 EEG alpha and eye blink activity recorded on the occipital lobe over haired skull

18 Capacitive Non-Contact Electrodes Senses biopotential signals without contact! Capacitive signal coupling! No electro-gel! Through clothing and hair Basic idea is well-known! First patent in 1968 (Richardson)! Several groups (Prance) and one company (Quasar) have pursued this Technology still problematic! Noise, interference pickup, artifacts! Circuit complexity, materials, construction, cost! Nothing beyond lab prototype [1] C.J. Harland, T.D. Clark, and R.J. Prance. Electric potential probes - new directions in the remote sensing of the human body. Measurement Science and Technology, 2: , February [2] A. Lopez and P. C. Richardson. Capacitive electrocardiographic and bioelectric electrodes. IEEE Transactions on Biomedical Engineering, 16: , [3] P. Park, P.H. Chou, Y. Bai, R. Matthews, and A. Hibbs. An ultra- wearable, wireless, low power ECG monitoring system. Proc. IEEE International Conference on Complex Medical Engineering, pages , Nov 2006.

19 Challenges in Non-Contact Sensors Capacitive coupling, rather than ohmic contact, between scalp/skin and electrode# active shield electrode skin skull unity gain buffer amplifier Amplifier parasitic input capacitance! Reduces gain as electrode-skin distance changes! Severely degrades CMRR! Increases the effect of amplifier voltage noise Integrates current noise at biopotential signal frequencies! Amplifier input biasing! Large resistance required for adequate low frequency response adds further current noise

20 Non-Contact Sensor Noise

21 Non-Contact Sensor Design Non-contact sensor fabricated on a printed circuit board substrate Sensing Plate! Amplifier! Active Shield! Advantages:! Robust circuit! Inexpensive production Standard 4-layer PCB!! Safe, no sharp edges or fingers, can be made flexible! Very low power (<100µW/sensor)! Strong immunity to external noise Chi and Cauwenberghs, 2010

22 Wearable Wireless EEG/ECG System Y. M. Chi, E. Kang, J. Kang, J. Fang and G. Cauwenberghs, 2010 Prototype non-contact sensor system with 4-channels! Bluetooth wireless telemetry and microsd data storage! Rechargeable battery Mounted in both head and chest harnesses EEG Hand-band! ECG Chest Harness! Electronics!

23 ECG Comparison Simultaneously acquired ECG in laboratory setting! No 60Hz Filter! Y. M. Chi, E. Kang, J. Kang, J. Fang and G. Cauwenberghs, 2010

24 Sample ECG Data Derived 12-lead ECG from 4 electrodes mounted in chest harness! Y. M. Chi, E. Kang, J. Kang, J. Fang and G. Cauwenberghs, 2010

25 ECG Under Motion Sitting! Walking! Running! Jumping! Y. M. Chi, E. Kang, J. Kang, J. Fang and G. Cauwenberghs, 2010

26 Non-Contact EEG Recording over Haired Scalp Y. M. Chi, E. Kang, J. Kang, J. Fang and G. Cauwenberghs, 2010 Easy access to hair-covered areas of the head without gels or slap-contact EEG data available only from the posterior! P300 (Brain-computer control, memory recognition)! SSVP (Brain-computer control)

27 Non-Contact vs. Ag/AgCl Comparison Y. M. Chi, E. Kang, J. Kang, J. Fang and G. Cauwenberghs, 2010 Subject s eyes closed showing alpha wave activity! Full bandwidth, unfiltered, signal show (.5-100Hz)!

28 Wireless Interfaces Digitization Wireless Telemetry Energy and noise efficiency metrics

29 Energy and Noise Efficiency Metrics Noise Efficiency Factor (NEF):! Relative measure of energy cost of a biopotential amplifier, relative to that of an ideal amplifier with same input referred noise power! Thermal noise fundamental limit: NEF = 1! Practical limit for CMRR > 80 db: NEF > 2 (2.3 demonstrated) Energy per Conversion Level Figure of Merit (FoM):! Energy cost of an analog-to-digital converter, per conversion, and divided by the number of quantization levels! State of the art: FoM = ~ 10 fj at 10b and 100ksps Range Efficiency:! Energy per bit, per squared meter of wireless transmission! Depends on target BER and power at the receiver! State of the art: ~ 10 fj/m 2

30 EEG/ECoG/EMG Amplification, Filtering and Quantization Mollazadeh, Murari, Cauwenberghs and Thakor (2009)! Low noise 21nV/!Hz input-referred noise 2.0µVrms over 0.2Hz-8.2kHz! Low power 100µW per channel at 3.3V! Reconfigurable Hz highpass, analog adjustable 140Hz-8.2kHz lowpass, analog adjustable 34dB-94dB gain, digitally selectable! High density 16 channels 3.3mm X 3.3mm in 0.5µm 2P3M CMOS 0.33 sq. mm per channel

31 Implantable Wireless Telemetry and Energy Harvesting Transcutaneous wires limit the application of implantable sensing/actuation technology to neural prostheses! Risk of infection Opening through the skin reduces the body s natural defense against invading microorganisms! Limited mobility Tethered to power source and data logging instrumentation Wireless technology is widely available, however:! Frequency range of radio transmission is limited by the body s absorption spectra and safety considerations Magnetic (inductive) coupling at low frequency, ~1-4 MHz Very low transmitted power requires efficient low-power design Sauer, Stanacevic, Cauwenberghs, and Thakor, 2005

32 Sensor Interface Conditioning Telemetry Sauer, Stanacevic, Cauwenberghs, and Thakor (2005) Implantable probe with biopotential electrodes, VLSI acquisition, microbatteries, and power harvesting telemetry chip. Telemetry Biopotential acquisition Inductor Coil SoS released probe body Electrodes Data Receiver Power Clock Extraction CLK Power Transmitter Rectification Regulation VDD GND Data Modulation Data Encoding Data Power delivery and data transmission over the same inductive link Telemetry chip (1.5mm X 1.5mm)

33 Silicon-on-Sapphire (SOS) Ultra-Wide Band (UWB) RF Transmission Tang, Andreou, and Culurcielo (2009) Pulse-based UWB radio transmitter! operates with sub-milliwatt power at multi-megahertz data rates and at microwatts of power for kilohertz data rates! body area networks and sensor networks Implemented in silicon-on-sapphire (SOS) process! optimizes its operation at high-speed and low-power consumption UWB transmitter antenna UWB transmitter integrated circuit in silion-on-sapphire (SOS)

34 Emerging Technologies Alternatives to EEG Wireless Brain Interfaces! NIR (near infrared spectroscopy)! Miniaturized fmri (functional magnetic resonance imaging)! Miniaturized MEG (magnetoencephalography) Optogenetics! ChR2 optical activation of targeted neurons! NPhR optical inactivation of targeted neurons Others!

35 CMOS Imaging in Awake Behaving Rats Murari, Etienne-Cummings, Cauwenberghs, and Thakor (2010) 180!m Minute 0 Minute 12 Minute 30 Minute 60! First simultaneous behavioral and cortical imaging from untethered, freely-moving rats.

36 Integrated Systems Neuroengineering Neuromorphic/ Neurosystems Engineering Neural Systems Learning & Adaptation Silicon Microchips Human/Bio Interaction Environment Sensors and Actuators