Virtual and Augmented Reality Brain Games

Transcription

1 Masaryk University Faculty of Informatics Virtual and Augmented Reality Brain Games MasterŠs Thesis Bc. Roman Konečný Brno, Fall 2016

2

3 Declaration Hereby I declare that this paper is my original authorial work, which I have worked out on my own. All sources, references, and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Bc. Roman Konečný Advisor: Doc. Fotis Liarokapis PhD i

4

5 Acknowledgement I want to thank all the wonderful people who have supported me in my studies and writing this thesis. I also want to thank my supervisor doc. Fotios Liarokapis, PhD for his guidance, support and kindness. iii

6 Abstract Brain-computer interfaces (BCIs) are becoming more and more popular in human-machine interaction. Non-invasive BCIs operate by recording the brain activity from the scalp with Electroencephalography (EEG) sensors attached to the head on an electrode cap or headset without being surgically implanted. The aim of this project is threefold. Firstly, to create a virtual reality/augmented reality game. Secondly, to integrate diferent BCI devices to fully control a computer game. The Ąnal task is to record and analyse brainwaves taken from an experiment with real-users. Implementation will be done in Unity3D and signal processing in Matlab. iv

7 Keywords VR, AR, Brain-Computer Interface, BCI, Unity, Game v

8

9 Contents 1 Introduction 1 2 Game 3 3 Conclusion 5 Bibliography 7 vii

10

11 List of Figures 2.1 Screen shot of the game, work in progress 4 ix

12

13 1 Introduction Over last few years, there has been a big step forward in virtual reality and augmented reality technologies. The headsets are more comfortable, easier to use and more afordable. This, combined together with powerful hardware that is very common in personal computers this days, makes opportunities for developing applications for virtual/augmented reality for greater audience, such as games. The term brain game used in this thesis refers about a computer game, that uses data obtained from brain of the player. The data is collected through Brain-Computer interface (BCI) and can be either input data - player is controlling the game by brain, or as output - then the data is analyzed according to what happened in the game and how the player reacted. The original idea of the thesis was to create a simple racing game in Unity game engine that would be controlled by brain, more speciącally, use SSVEP to control acceleration and steering of the racing car. SSVEP (Steady State Visually Evoked Potentials) was chosen as it has the advantage of having higher accuracy and higher information transfer rate (ITR) compared to other brain computer interface approaches, such as P300. In addition, short/no training time and fewer EEG channels are required [1]. 1

14

15 2 Game The idea of the game is pretty simple - to have a racing game in which the player would control the car on a racing circuit. The circuit has a line in the middle of the road that the player should follow - the further is the car from the line, the less points are given. The other measurement is time - how long will it take to Ąnish the lap. The game was created using the Unity engine, C# programming language and Blender for model of the track. Free asset from Unity Asset Store was used for car. For collecting signals from the brain, Enobio device was used and Openvibe was used as a brain computer interface software. The game also supported stereoscopic mode, so it was possible to play it in 3D. The score was calculated by measuring the diference between position of the car and position of the line in the middle of the track. A simple diiculty system was implemented and consisted of limiting the top speed of the car and steering eiciency. Those two settings could have been set independently and were introduced for making the game more playable when using BCI as a source of input. However, the main issue was in the BCI itself, more speciącally, in the SSVEP used for getting the input. This is described in chapter 3. How the game looks like can be seen in Figure 2.1 3

16 2. Game Figure 2.1: Screen shot of the game, work in progress 4

17 3 Conclusion Game prototype was fully playable using keyboard. When we tried to use BCI approach however, we were not able to get meaningful input. This was tested on a premade scenario in OpenVibe. The player was controlling a spaceship and the aim of this little game was to rotate clockwise and anticlockwise and shoot using the SSVEP. During the testing, it was discovered that the reactions were unpredictable and not suitable for the racing game, as it would be impossible to drive the car using the SSVEP For this reason, the topic was slightly changed. Instead of controlling the game using the BCI, the game would not use BCI as an input method, but rather to monitor the player and his reactions to what is happening in the game. Unfortunatelly, I was not able to Ąnish this in time and that is the reason of this thesis being in such an incomplete form. My apologies to those who are reading this thesis, as it is not a thesis at all. 5

18

19 Bibliography 1. FAZEL-REZAI, Reza (ed.). Brain-Computer Interface Systems - Recent Progress and Future Prospects [online]. 1st ed. InTech, 2013 [visited on ]. ISBN Available from: intechopen.com/books/brain- computer- interface- systemsrecent- progress- and- future- prospects/a- review- of- p300- ssvep- and- hybrid- p300- ssvep- brain- computer- interfacesystems. 7