Integrating Human and Computer Vision with EEG Toward the Control of a Prosthetic Arm Eugene Lavely, Geoffrey Meltzner, Rick Thompson

Transcription

1 Integrating Human and Computer Vision with EEG Toward the Control of a Prosthetic Arm Eugene Lavely, Geoffrey Meltzner, Rick Thompson & Brain-Computer interface for hci and games

2 Brain Interface EEG: In 1924, Berger succeeded in recording the first human electroencephalogram 1983 sci-fi classic Brainstorm, starring Christopher Walken and Natalie Wood as scientists that invent The Hat, a device that can record thoughts and project them into the minds of others

3 Paper 1: Brain computer interface (BCI) for control of an upper limb prosthetic Approach: Electrical neural activity data for motor intent estimation Eye gaze direction for target selection Data streams are augmented by computer vision (CV) for 3D scene reconstruction

4 Challenge: Human-robot interaction(hri) limb ownership or embodiment Human hand has 27 degrees of freedom, Human arm has 7 degrees of freedom Current advanced arm designs capture as many as 22 DOF

5 Upper limb prosthetic

6 Paper 2: Study: Research themes and the Brain computer interaction Goal: Define research approaches and applications

7 Paper 2: Applications for patients with severe motor disabilities have been the driving force of most brain-computer interface research. Brain-Computer Interfaces Thought as Input Evaluating Interface Adaptive Interface

8 Detection Algorithm Brain Interface Challenge Approximately 100bn neurons neuron interaction create electric pulse folding in brain... source of pulse

9 Device: 1. emotiva 2. neurosky 3. xwave

10 Brain Interaction for Mobile Games Paul Coulton Lancaster University Infolab21, Lancaster University, Lancaster, UK, LA1 4WA. +44 (0) Will Bamford Lancaster University Infolab21, Lancaster University, Lancaster, UK, LA1 4WA. +44 (0) Carlos Garcia Wylie Lancaster University Infolab21, Lancaster University, Lancaster, UK, LA1 4WA. +44 (0)

11 NeuroSky Mindset Frequency Bands Brainwave Type Frequency range Mental states and conditions Delta 0.1Hz to 3Hz Deep, dreamless sleep, non-rem sleep, unconscious Theta 4Hz to 7Hz Intuitive, creative, recall, fantasy, imaginary, dream Alpha 8Hz to 12Hz Relaxed, but not drowsy, tranquil, conscious Low Beta 12Hz to 15Hz Formerly SMR, relaxed yet focused, integrated Midrange Beta 16Hz to 20Hz Thinking, aware of self & surroundings High Beta 21Hz to 30Hz Alertness, agitation

12 Game Design

13 Brain Maze Level Design

14 Mind Gate Control

15 esense Meter ATTENTION esense The esense Attention meter indicates the intensity of a user's level of mental focus or attention, MEDITATION esense The esense Meditation meter indicates the level of a user's mental calmness or relaxation.

16 Coclusion: BCI has potential More research required to validate low cost devices.

17 Other Game:

18 Reference: Lalor, E.C., Kelly, S.P., Finucane, C., et al. Steady-StateVEP-Based Brain-Computer Interface Control in an Immersive 3D Gaming Environment. EURASIP Journal on Applied Signal Processing 2005, R.F. Weir and J.W. Sensinger, Design of artificial arms and hands for prosthetic applications, in Standard Handbook of Biomedical Engineering & Design, Vol. 2, M. Kutz, Ed. New York: McGraw-Hill, Daniel Sjölie,Kenneth Bodin, Eva Elgh, Johan Eriksson, Lars-Erik Janlert, Lars Nyberg, Effects of interactivity and 3D-motion on mental rotation brain activity in an immersive virtual environment, Proceeding CHI '10 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems A headset that reads your brainwaves: Tan Le on TED.com Tan Le s astonishing new computer interface reads its user s brainwaves, making it possible to control virtual objects, and even physical electronics, with mere thoughts (and a little concentration). She demos the headset, and talks about its far-reaching applications. (Recorded at TEDGlobal 2010, July 2010 in Oxford, UK. Duration: 10:38)