Controlling Robots with Non-Invasive Brain-Computer Interfaces

Size: px

Start display at page:

Download "Controlling Robots with Non-Invasive Brain-Computer Interfaces"

Maude Randall
6 years ago
Views:

1 1 / 11 Controlling Robots with Non-Invasive Brain-Computer Interfaces Elliott Forney Colorado State University Brain-Computer Interfaces Group February 21, 2013

2 Brain-Computer Interfaces 2 / 11 Brain-Computer Interface (BCI) Direct channel of between brain and machine Voluntary changes in mental state Bypasses innate motor-based means of communication

3 Brain-Computer Interfaces 2 / 11 Brain-Computer Interface (BCI) Direct channel of between brain and machine Voluntary changes in mental state Bypasses innate motor-based means of communication

4 Uses for BCI 3 / 11 BCI have many potential uses Reestablish communication with people who are Locked-in ALS, stroke, traumatic brain injury Assistive technology electric wheelchairs, computers, telephones Everyday devices video games, monitoring emotional states Driving kewl robots around the lab with your mind!

5 Uses for BCI 3 / 11 BCI have many potential uses Reestablish communication with people who are Locked-in ALS, stroke, traumatic brain injury Assistive technology electric wheelchairs, computers, telephones Everyday devices video games, monitoring emotional states Driving kewl robots around the lab with your mind!

6 Uses for BCI 3 / 11 BCI have many potential uses Reestablish communication with people who are Locked-in ALS, stroke, traumatic brain injury Assistive technology electric wheelchairs, computers, telephones Everyday devices video games, monitoring emotional states Driving kewl robots around the lab with your mind!

7 Uses for BCI 3 / 11 BCI have many potential uses Reestablish communication with people who are Locked-in ALS, stroke, traumatic brain injury Assistive technology electric wheelchairs, computers, telephones Everyday devices video games, monitoring emotional states Driving kewl robots around the lab with your mind!

8 Uses for BCI 3 / 11 BCI have many potential uses Reestablish communication with people who are Locked-in ALS, stroke, traumatic brain injury Assistive technology electric wheelchairs, computers, telephones Everyday devices video games, monitoring emotional states Driving kewl robots around the lab with your mind!

9 Electroencephalography 4 / 11 Electroencephalography (EEG) to measure brain activity Non-invasive, portable, relatively inexpensive Superficial & noisy signals PZ FZ CZ T6 T5 T4 T3 F7 O2 O1 P4 P3 C4 C3 F4 F3 FP2 FP Time (s)

10 Electroencephalography 4 / 11 Electroencephalography (EEG) to measure brain activity Non-invasive, portable, relatively inexpensive Superficial & noisy signals PZ FZ CZ T6 T5 T4 T3 F7 O2 O1 P4 P3 C4 C3 F4 F3 FP2 FP Time (s)

11 Electroencephalography 4 / 11 Electroencephalography (EEG) to measure brain activity Non-invasive, portable, relatively inexpensive Superficial & noisy signals PZ FZ CZ T6 T5 T4 T3 F7 O2 O1 P4 P3 C4 C3 F4 F3 FP2 FP Time (s)

12 Machine Learning & Pattern Analysis Machine Learning algorithms identify patterns in EEG This is extremely difficult because different for each person change over time noise & artifacts the brain is complex! Signal EEG RNN Time (s) 5 / 11

13 Machine Learning & Pattern Analysis Machine Learning algorithms identify patterns in EEG This is extremely difficult because different for each person change over time noise & artifacts the brain is complex! Signal EEG RNN Time (s) 5 / 11

14 Synchronous BCI 6 / 11 Some BCI use patterns associated with external stimuli P300 speller is an example User looks for a specific character in a series or grid of flashing characters

15 Synchronous BCI 6 / 11 Some BCI use patterns associated with external stimuli P300 speller is an example User looks for a specific character in a series or grid of flashing characters

16 Asynchronous BCI 7 / 11 Other BCI do not require external stimuli Mental Tasks is an example Imagine left arm moving moves to the left while silently singing a song moves to the right

17 Asynchronous BCI 7 / 11 Other BCI do not require external stimuli Mental Tasks is an example Imagine left arm moving moves to the left while silently singing a song moves to the right

18 Robots 8 / 11 Controlling robots is a first step toward real-world applications Drive a robot before an electric wheelchair Demonstrates that user has control Just plain fun!

19 Robots 8 / 11 Controlling robots is a first step toward real-world applications Drive a robot before an electric wheelchair Demonstrates that user has control Just plain fun!

20 Ideas for Robot Club, I 9 / 11 Current robot module is hacked together but better systems to come. Some ideas for robot club to start on Streaming video through sockets, to python Better user interfaces Fluid control How far to move Control gripper

21 Ideas for Robot Club, I 9 / 11 Current robot module is hacked together but better systems to come. Some ideas for robot club to start on Streaming video through sockets, to python Better user interfaces Fluid control How far to move Control gripper

22 Ideas for Robot Club, I 9 / 11 Current robot module is hacked together but better systems to come. Some ideas for robot club to start on Streaming video through sockets, to python Better user interfaces Fluid control How far to move Control gripper

23 Ideas for Robot Club, I 9 / 11 Current robot module is hacked together but better systems to come. Some ideas for robot club to start on Streaming video through sockets, to python Better user interfaces Fluid control How far to move Control gripper

24 Ideas for Robot Club, II 10 / 11 Safety systems Don t hit a wall Don t go down stairs Hybrid control Turn away from walls Move toward objects

25 Ideas for Robot Club, II 10 / 11 Safety systems Don t hit a wall Don t go down stairs Hybrid control Turn away from walls Move toward objects

26 Thanks! 11 / 11

Non-Invasive Brain-Actuated Control of a Mobile Robot

Non-Invasive Brain-Actuated Control of a Mobile Robot Jose del R. Millan, Frederic Renkens, Josep Mourino, Wulfram Gerstner 5/3/06 Josh Storz CSE 599E BCI Introduction (paper perspective) BCIs BCI = Brain