VIP I-Natural Team. Report Submitted for VIP Innovation Competition April 26, Name Major Year Semesters. Justin Devenish EE Senior First

Transcription

1 VIP I-Natural Team Report Submitted for VIP Innovation Competition April 26, 2011 Name Major Year Semesters Justin Devenish EE Senior First Khoadang Ho CS Junior First Tiffany Jernigan EE Senior First Will Johnston EE Junior First Giancarlo Valentin COMPE Senior First Malthus Yau EE Sophomore First Brandon Yee COMPE Sophomore First Hae Won Park ECE Grad student advisor First Advisors Dr. Ayanna Howard Dr. Edward Coyle Department ECE ECE

2 I. Summary of Goals The main motivation for creating this project is to develop an user interface capable of providing children with verbal and physical limitations with the necessary play experience to develop their cognitive, social, and physical skills while not being constrained to a single-purpose high-cost device. Existing applications such as Proloquo2go [1] offer assistance to individuals with speech difficulties by allowing them to create words and phrases by pressing a series of images. Similarly, Popchilla [2] combines an interactive drawing application with a robot that gives feedback to the user's input. Accessible Messaging [3] provides assistive text typing by highlighting every keyboard element at a time. However, these touch-based tools are developed assuming the user is capable of 'touching' a specific small region with appropriate intensity and timing. This assumption does not generally hold true when considering individuals who possess limited upper body motor controls, such as often observed in children with cerebral palsy. As a result, the I-Natural team decided to create an alternative user interface to be mounted on the forearm and used in conjunction with a popular tablet computer such as the ipad. The ipad was chosen due to its increased pervasiveness among children and adults of all ages. Tablet computers in general have been widely regarded as an ideal way to deliver therapies along with entertainment. We believe that an interface such as the one we propose has the potential to deliver education and entertainment to every child who is unable to interact with the newest technology due to his/her limitations. (a) (b) Figure 1. (a) I-Natural interface Prototype (b) Sample game application

3 II. Accomplishments The hardware component of the interface currently consists of a forearm mountable mobile interface (Figure 1-a) for the user to control the ipad application. The current design uses a micro-controller unit coupled with force sensitive resistors to implement the physical aspects of the user interface. The device recognizes press and swipe gestures generated by a combination of sensors with a finite state machine (Figure 2-b). These readings are sent to the ipad via a Wi-Fi module capable of hosting its own server. With the use of an HTTP request function, the ipad is able to read values from the server and respond according to the actions of the user. The software interface consists of a custom-made ios application that serves as a multipage menu by presenting the user with multiple choices that can be controlled with the I-Natural interface. This menu is responsible for providing the user with the applications available for the I-Natural interface. Currently, the first hardware prototype (Figure 1-a) has been created and successfully used along with the multi-page menu. Testing results for the user interface are shown with an application developed to play a simple game (Figure 1-b). (a) (b) Figure 2. (a) Diagram of the parts composing the I-Natural system. (b) Finite state machine diagram for swipe and press interpretation.

4 Due to the substantial difference between the I-Natural hardware and existing commercial alternatives, we believe it represents an innovative step in the design of educational applications. As a result, future efforts should include patenting of the mountable interface and its gesture detection mechanisms. III. Future Plans Future work on the system will incorporate improved versatility to input commands, adjustable forearm mounting, and broader communication methods. It is necessary to develop a simple mechanism to keep the control unit near the interface sensors without obstructing the user s ability to move freely. Amongst potential communication methods between the I-Natural interface and the ipad, Bluetooth technology seems to be the most promising. Its benefits in comparison to the current Wi-Fi module include greater stability and faster response time. In its current form, the BlackWidow Wi-Fi module takes a non-trivial amount of time to update its readings and this can represent a noticeable response delay for the user. At this point, this problem has been addressed successfully but depending on modifications to the existing algorithm it might reoccur in the future. Incorporating important changes to the interface, the goal of next semester is to create applications that to an increasing degree replace existing specialized devices. The most important aspect of these applications would consist of an augmentative communication device allowing the user to express words and phrase with the use of swipe and press commands. Also, another goal is to introduce and educate robotics via our forearm interface to children and increase their exposure to engineering. Robotics has shown great potential in assisting child therapy [4-6]. Robots also can be a good mediator for engaging children in interactive play [7] or physical therapy sessions. Ultimately, the progress of this project will depend on the degree with which it is accepted by the target audience. As a result, further study with child subjects must be conducted in the long term. Observations resulting from these studies will be regularly incorporated into the existing design until project completion.

5 References [1] S. Sennott and A. Bowker, Autism, AAC, and Proloquo2Go, Perspectives on Augmentative and Alternative Communication, vol. 18, no. 4, p. 137, [2] Bill. (2010, Aug. 13) ipad Controlled Robots Help Kids With Autism. [Online]. Available: [3] B. Clayton. (2010, Nov. 16) Michigan Engineering Students Develop Mobile Communications Technology for Cerebral Palsy Patients. [Online]. Available: [4] B. Robins, K. Dautenhahn, and P. Dickerson, From isolation to communication: a case study evaluation of robot assisted play for children with autism with a minimally expressive humanoid robot, Proc. Second Inter. Conf. Advances in CHI, ACHI [5] A. Billard, B. Robins, J. Nadel, and K. Dautenhahn, Building robota, a mini-humanoid robot for the rehabilitation of children with autism, RESNA Assistive Technology Journal, vol. 19, no. 1, pp , [6] B. Scassellati, How social robots will help us to diagnose, treat, and understand autism, in Proc. of the 12th International Symposium of Robotics Research (ISSR05), [7] A. Duquette, F. Michaud, and H. Mercier, Exploring the use of a mobile robot as an imitation agent with children with low-functioning autism, Autonomous Robots, vol. 24, no. 2, pp , 2008.