The Virtual Reality Brain-Computer Interface System for Ubiquitous Home Control

Transcription

1 The Virtual Reality Brain-Computer Interface System for Ubiquitous Home Control Hyun-sang Cho, Jayoung Goo, Dongjun Suh, Kyoung Shin Park, and Minsoo Hahn Digital Media Laboratory, Information and Communications University, Dogok-dong, Gangnam-gu, Seoul , S. Korea {haemosu, bucsu, linuxer, park, Abstract. This paper presents a virtual reality brain computer interface (BCI) system which allows the user to interact physical objects in the ubiquitous home. The system is designed for motion disabled people to control real home facilities by a simple motor imagery and minimal physical body movements. While BCI research has mostly focused on improving classification algorithm, our BCI system uses a simple BCI classification method and the additional locking mechanism to compensate the BCI classification error. This paper describes the design, implementation, and user evaluation of our virtual reality BCI system for ubiquitous home control. The user study results showed the locking mechanism helped to improve user controllability which made the system more feasible and reduced user undesired BCI decision error rates. 1 Introduction In the ubiquitous computing environment, the objects are intelligent and can be controlled remotely via a network. With the advent of ubiquitous computing technologies, it is possible to realize an affordable environment that allows motion disabled people to control the surrounding objects. Motion disabled people usually have a lot of psychological disorders such as depression due to their uncontrollable body. They also feel charging a burden on their family at home. Our research aim is to enable motion disabled people to feel as if they enjoy a real life. We believe that virtual reality (VR) can give them psychological compensates since it provides the sense of actually being there in the physical environment where the spatiality of the virtual objects matched with the position of real objects. However, motion disabled people need a specially designed VR system that will not require a lot of physical body movements for the system operation. Recent research efforts show the feasibility of brain computer interface (BCI) used for motion disabled people [1, 2, 3]. They have achieved a mean classification rate of above 80%. However, BCI is not easily used for application domains due to 10~20% error rate. The random BCI classification errors will create a cyber sickness problem in the virtual environment when user navigation is mismatched with their will. Eventually, this problem will result in loosing user control. It is one of the serious obstacles of using virtual reality BCI system. A. Sattar and B.H. Kang (Eds.): AI 2006, LNAI 4304, pp , Springer-Verlag Berlin Heidelberg 2006

2 The Virtual Reality Brain-Computer Interface System for Ubiquitous Home Control 993 Fig. 1. The overview of the virtual reality brain-computer interface system for controlling a ubiquitous home environment In this paper, we present a virtual reality BCI system for ubiquitous home network control. The system allows the user to control the objects in the virtual environment using user s motor-imagery, and this event triggers the control of objects in the ubiquitous computing environment for real executions. It uses motor-imagery Event- Related De-synchronization (ERD) brain signals of the EEG device for moving left or right direction and an auxiliary control device for moving forward in the virtual environment. We also provide the locking mechanism for enhancing system robustness and usability under high BCI classification error rates. The feasibility of this approach is confirmed through a user study. The result showed that the locking mechanism improved user controllability thereby enhancing system usability. 2 BCI-Based VR Interface for Ubiquitous Home Control Fig. 1 shows the overview of our virtual reality BCI system for ubiquitous home control. The middle and the right image of Fig. 1 show a virtual world and a physical miniature mockup model replicated a real home. The system detects human motorimagery brain signals (i.e., one s imagination about left or right hand moving) by measuring μ-rhythm from the EEG device. When a user imagines left or right hand moving, he/she can incrementally change the orientation viewpoint to left or right direction in the virtual environment. In this system, a user can freely navigate and trigger actions in the virtual environment using the additional direction device. In addition, the action events (in the virtual environment) trigger the events in the real ubiquitous home environment. From our experiences with the early prototype of this system, users seemed to feel fatigue or cyber sickness induced by their unwilling wander in the virtual environment. This happened in association with a wrong BCI direction decision. To solve this cyber sickness problem, we provided a direction locking mechanism. That is, a user could lock the direction when he/she decided the correct direction which way he/she wanted to go. Then, the user could unlock (i.e., release the direction locking) so that he/she could change left or right direction (i.e., rotation in the virtual environment) using this BCI. We have developed two direction locking devices: a pedal-like device and a simple eye-blink detector (shown in Fig 3).

3 994 H.-s. Cho et al. Fig. 2. The system architecture, describing the events triggering among modules Fig. 2 shows the system architecture of our virtual reality BCI system for the ubiquitous computing environment controls. The system consists of three main modules: the signal analysis unit, the virtual environment, and the ubiquitous computing environment control unit. The signal analysis unit contains of three functional modules: the physical control signal input module, the EEG measurement equipment, and the brain signal analysis unit. We sampled brain wave with 256 Hz to make the classification every second. The virtual environment was implemented using YGdrasil (YG) [4]. 3 User Evaluations We conducted an experiment of participants using our virtual reality BCI system to evaluate the effect of the direction locking mechanism (i.e., with or without the direction locking for BCI uses in the virtual environment). Eight graduate and undergraduate male students volunteered as participants in this study. The range of the participants age was 20 to 35 years old. They had high level computer experience and their typical computer usage time was more than three hours a day. However, none of them had prior experience with EEG equipments, BCI, and VR system. That is, all participants were novice users to our BCI and the virtual environment. Fig. 3 shows a participant of our BCI system performing the VR navigation task. In this task, each participant was asked to navigate in the virtual environment to visit three object control regions (i.e., bed lamp, a curtain, and a table lamp) and trigger the operation (i.e., turning on/off the light and moving the curtain up/down). We examined participants with or without the direction locking mechanism that was provided as a foot pedal controller.

4 The Virtual Reality Brain-Computer Interface System for Ubiquitous Home Control 995 Fig. 3. A participant performed the VR navigation task visiting three object control regions such as a bed lamp, a curtain, and a table lamp(on the left). The participant performed the VR navigation task with or without the direction locking mechanism given as a foot pedal controller(on the center) or eye blink detector(on the right). Fig. 4. The left image is the navigation path given to the participants in the virtual environment. The middle and the right image show one participant s actual navigation trails with and without the direction locking mechanism respectively. The experiment result showed that our system had low BCI classification rate with novice users. In the past, an informal user evaluation with trained participants showed 70% on average and up to 90% correct classification rate. We suspect that there might have been a problem with the brain signal gathering or an inappropriate EEG electrode placement on the participant s head because the experiment was done by novice helpers to check the robustness of our system. With this low BCI classification rate, we however, could confirm the effectiveness of this direction locking mechanism for VR navigation and interaction. The results showed it helped our participants were easily navigating in the virtual environment. The average completion time with direction locking was 147 seconds (35% faster) while the average completion time without locking was 414 seconds. In particular, two participants lost their controls in the virtual environment without locking which was never happened with locking. Fig. 4 shows the participant 5 s actual navigation traces in the virtual environment with and without the direction locking mechanism. In Fig. 4, the colored areas indicate the object control regions. 4 Conclusions and Future Work This paper presents a virtual reality brain computer interface (BCI) system designed for motion disabled people to control the ubiquitous home environment. In this

5 996 H.-s. Cho et al. system, users can change left or right direction turns by motor imagery (i.e., human imagination of a left or right hand movement) to navigate in the virtual environment and move forward by an auxiliary control device. Such user interactions in the virtual environment trigger the control of ubiquitous objects in the real home environment. In this paper, we also presented the additional direction locking mechanism to compensate the random BCI decision errors. The goal of this research was to develop a cheap and simple practical BCI system. Through the user evaluation, we found that this locking mechanism was imperative to user navigation and interaction in the virtual environment which also helped reduce the cyber sickness problem. The results also showed that user s navigation trails in the virtual environment was much shorter with the locking mechanism than without locking. We believe the locking mechanism can be used for the dynamic complex brain signal processing. We will also continue to develop other types of direction locking and forward input device, such as other brain signals, sound or jaw movements. We will envision expanding this system to support home or town for wider social interaction for motion disabled people. Acknowledgement I sincerely appreciate Ji-Min Park, Ye-Seul Park and You-Min Kim for their help on experimental setting and process and all participants of this experiment. This research was supported by the MIC (Ministry of Information and Communication), Korea, under the Digital Media Lab. support program supervised by the IITA (Institute of Information Technology Assessment). References 1. Gert Pfurtscheller and Christa Newper: Motor Imagery and Direct Brain-Computer Communication. Proceedings of the IEEE, Vol.89 No.7, Robert Leeb, Reinhold Scherer, Felix Lee, Horst Bischof, Gert Pfurtscheller: Navigation in Virtual Environments through Motor Imagery. Proc. of 9th Computer Vision Winter Workshop, pp Baharan Kamousi, Zhongming Liu, and Bin He: Classification of Motor Imagery Tasks for Brain-Computer Interface Applications by Means of Two Equivalent Dipoles Analysis. IEEE Transactions on Neural System and Rehabilitation Engineering, Vol. 13, No. 2, June Dave Pape, Josephine Anstey, Margaret Dolinsky, Edward J. Dambik: Ygdrasil--a framework for composing shared virtual worlds. Future Generation Computer Systems, volume 19, No. 6, p , 2003.