Integrate the BlindAid system in a traditional orientation and mobility rehabilitation program

Transcription

1 Integrate the BlindAid system in a traditional orientation and mobility rehabilitation program The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation As Published Publisher Lahav, O., D.W. Schloerb, and M.A. Srinivasan. Integrate the BlindAid system in a traditional orientation and mobility rehabilitation program. Virtual Rehabilitation International Conference, Institute of Electrical and Electronics Engineers. Institute of Electrical and Electronics Engineers Version Final published version Accessed Wed Jan 16 18:14:09 EST 2019 Citable Link Terms of Use Detailed Terms Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.

2 Integrate the BlindAid system in a Traditional Orientation and Mobility Rehabilitation Program Orly Lahav 1, 2 1 School of Education, and The Porter School of Environmental Studies Tel Aviv University, Tel Aviv, Israel lahavo@post.tau.ac.il David W. Schloerb 2, Mandayam A. Srinivasan 2 2 The Touch Lab, RLE MIT Cambridge, MA Abstract In the process of becoming blind, newly- blinded people participate in a rehabilitation program, which includes different skills that a newly- blinded person needs to adapt as a result of his or her lost of vision. The virtual system, the BlindAid, involves active collaboration between orientation and mobility instructors from the Carroll Center for the Blind in Newton, Massachusetts and engineers and cognitive scientists at the MIT Touch Lab in Cambridge, Massachusetts. The two teams collaborated in the integration of the BlindAid system in the traditional orientation and mobility rehabilitation program. In this study we will describe the model of the integration. The three main goals of this model were to study: (1) cognitive mapping process of newly-blinded when using the virtual environment; (2) mental support of the BlindAid system to the newly- blinded; and (3) enhancement of the BlindAid system for the orientation and mobility instructors. The findings supply strong evidence that interaction with the BlindAid system by people who are newly-blinded provides a robust foundation for the participants development of comprehensive cognitive maps of actual unknown spaces during their rehabilitation program. Keywords- Blind; Cognitive processing; Rehabilitation; Orientation; Mobility; Virtual simulation; Haptic devices I. INTRODUCTION People in the process of becoming blind face great difficulties in performing basic daily skills. For most of them, walking in an unknown environment can be unpleasant and uncomfortable. The traditional orientation and mobility (O&M) rehabilitation program supports the acquisition of spatial mapping and orientation skills by supplying two main type of information: perceptual and conceptual. The perceptual level is based on the work of Amendola [1], the pioneer in the field of sensory training [2, 3]. Sensory training is based on systematically collecting information through all of the senses from the immediate environment. The three primary senses used in restoring orientation function are audio, smell, and touch. Kinesthesia is also used for restoring orientation function. At the conceptual level, the focus is on supporting the development of appropriate strategies and orientation solving problems for an efficient cognitive mapping of a space and This project was supported by the National Institutes of Health, National Eye Institute. Grant No. 5R21EY applying it during navigation. Over the years, information technologies have been developed to help people who are blind build cognitive maps and explore real physical spaces. There are two types of O&M aids: (1) pre-planning aids that provide the user with information before his or her arrival to the environment and (2) in-situ planning aids that provide the user with information about the environment in-situ. Pre-planning aids include a verbal description of the space, tactile maps, strip maps, physical models, and Talking Tactile Tablets (TTT) that use tactile maps [4-7]. Over the past few years, in-situ planning aids that have been developed include Sonicguide [8], Kaspa [9], Talking Signs, embedded sensors in the environment [10], activated audio beacon by using cell phones technology [11], and personal guidance systems based on satellite communication [12]. Global Positioning System (GPS) based devices are manufactured by StreetTalk from Freedom Scientific, Trekker from HumanWare, and the BrailleNote GPS from Sendero Group. However, there are a number of limitations in the use of these pre-planning and in-situ planning devices. For example, the limited dimensions of tactile maps and models may result in poor resolution of the provided spatial information (e.g., lack of precise topographical features or of accurate dimensions and location for structural objects) and difficulties in producing and acquiring updated spatial information. As a result of these limitations, people who are blind are less likely to use pre-planning devices in everyday life. The major limitations of the in-situ planning devices are that they can be used only in the explored space. Over the past few years, the use of virtual reality in domains such as simulation-based training, gaming, and entertainment has been on the rise. It has also been used for rehabilitation and learning environments for people with disabilities (e.g., physical, mental, and learning) [13, 14]. Advanced technology, particularly haptic interface technology, enables blind individuals to expand their knowledge by using artificially made reality through haptic and audio feedback. Research on the implementation of haptic technologies within virtual navigation environments has yielded reports on the potential of haptic technology for supporting rehabilitation training with sighted people [15], as well as with people who are blind [16, 17]. Related research on the use of haptic devices by people who are blind includes: identification of textures and object shapes [18], mathematical learning environment and /09/$ IEEE 37

3 exploring mathematical graphs [19], exploring geography maps using audio and tactile feedback [20], and construction of cognitive maps [21, 22]. In this paper we will describe the integration of the BlindAid system in a traditional O&M rehabilitation program. The BlindAid system acts as a pre-planning aid to help newlyblinded people develop awareness about the environments in which they are traveling, in a safe and relaxed manner, as well as provide tools for O&M instructors to evaluate the progress of their clients, while in-situ, during their rehabilitation program. The three main goals of this model were to study: Cognitive mapping process of newly-blinded when using the virtual environment. Mental support of the BlindAid system to the newlyblinded. Enhancement of the BlindAid system to the orientation and mobility instructors. II. THE BLINDAID SYSTEM Developed by the Touch Lab at MIT, the BlindAid system consists of a software package that provides a virtual environment (VE) for people who are blind and a haptic device. The system was designed at the MIT Touch Lab, with assistance from an O&M instructor from the Carroll Center for the Blind (CCB). Fig. 1 depicts a schematic of the BlindAid system. Figure 1. Schematic Diagram of the BlindAid System Using this human-machine interaction, the user gets haptic and audio feedback. The haptic interface allows the user to interact with the VE and provides two functions: (1) it moves the avatar (a human representative in VE) through the VE, and (2) it provides force feedback to the user that gives clues about the space similar to those generated by the white cane. In this study we used the Phantom, a haptic interface with high fidelity in force generation and position tracking. Although there are many haptic devices, the Phantom met the present need for functionality and commercial availability because it produces a mechanical force during a maneuver in the VE if the tip of the stylus attached to it penetrates any virtual object in the VE, such as a wall or a table. In addition to haptic feedback through the Phantom, the user was also given 3D audio feedback through stereo headphones. The audio feedback provides the user with descriptive information as well as sound cues for various artifacts he comes into contact with in the VE. To simplify the signal processing, the Head Related Transfer Functions (HRTFs) were used for this virtual 3D audio processing. The BlindAid system comprises two modes of operation a developer mode and a learning mode. A. The Developer Mode A key feature of the developer mode is the program's ability to import AutoCad (dxf) 2D drawing files to aid in the creation of new virtual maps. This makes it possible, for example, to import a floor plan so that the locations of all the walls of a building are accurately replicated in the VE. After creating a new virtual map, the researcher can add structure components (door, windows, rod, etc.), or objects as furniture (tables, chairs, doormat, etc.). Each component in the virtual map can be represented by geometric properties (size, shape, location), haptic feedback (tactile and or stiffness properties), and audio feedback (hearcon, world, long description). B. The Learning Mode The learning mode, within which the user and the researcher works, includes two interfaces: User Interface The users station consisted of a tabletop space with an arm support, a computer, keyboards, a set of stereo headphones, and the Phantom device through which the subject interacted with the VE. The user interface consisted of the VE that simulates real physical rooms and objects to be navigated by the users using the Phantom (Fig. 2). In addition to the Phantom, the user was able to interact with the VE by using the numeric keyboard. The user interface includes four tools. The first is a second audio tool tool that gives the subject additional audio information. The second tool includes the ability to pause the movement of the Phantom and take a break anytime during the exploration. The last two tools were developed mainly to assist the user to navigate by themselves in the VEs. The return to the starting point tool allows the user to return to the starting point in case of confusion about current location in the VE. The fourth tool, active landmarks, allowed the researcher and the user to install a landmark in the VE. Landmarks are basically audio locators that are placed in the VE by the researcher or by the user. Figure 2. User Interface 38

4 Researcher Interface - The researchers station consisted of a flat panel computer monitor, a mouse, and an additional keyboard that was used as an aid for researchers to observe the progress of the subjects in the VE during the experiments. The researcher interface included several features that serve researchers during and after the learning session. On-screen monitors present updated information on the user s navigation performance, such as position or objects already reached. The researcher interface allows researchers to record and review a user s exploration activities in the VE. During an experiment session, the computer records the avatar s position and orientation within the VE in a text file, along with any command actions. Subsequently, the data file may be viewed directly or replayed by the system like a video recording. (Fig. 3). Figure 3. Research Interface III. ORIENTATION AND MOBILITY REHABILITATION PROGRAM The CCB was founded in Today, the CCB offers a vast array of services to individuals aged 14-90, with an average age of 36 years. The CCB practitioners followed Amendola s thoughts and developed a sensory training methodology and curriculum for O&M skills for people who are blind [2-3]. This methodology is based on systematically collecting information through all of the senses from the immediate environment. The three primary senses used in restoring orientation function are: audio, smell, touch and kinesthesia. The CCB O&M rehabilitation program has two main programs: community-based and campus-based. The campus-based program, which is 16 weeks long, is an intensive and comprehensive rehabilitation course designed primarily for newly-blinded adults to make the physical and emotional adjustments to living with blindness. The first two weeks include functional assessment encompassing activities of daily living, travel skills, use of remaining vision, information management, personal health care, and adjustment to vision loss. At the end of these two weeks the CCB O&M specialists evaluate the client s condition and write their evaluation in a report. This report is a broad evaluation about the clients, including issues such as the clients eye condition; present vision; medical conditions (hearing, balance, stamina); previous O&M training; current functioning in mobility (unaided vision, vision plus low vision aids, human guide, long cane, guide dog, public transportation, independent travel); and stated mobility goals. The second part of the rehabilitation program includes a 14-plus-week program to assist clients in achieving or maintaining personal independence. The campus-based program addresses a variety of needs: O&M skills, counseling, communications skills, daily living skills, health management, and low vision skills. The O&M program includes three parts: orientation, mobility, and cane technique. The orientation part has six components: (i) the use of sensorial landmarks and clues, for example: hearing (direct sound to identify objects, distance or direction, and echolocation), kinesthesia (elevation), touch (object texture and ground texture), and smell; (ii) the use of visual scanning and audible signal to cross a street; (iii) the use of landmarks and clues, such as buildings, street names, and door numbers; (iv) the use of cardinal directions for travel; (v) ability to recover when disoriented (problem solving); and (vi) the manufacture of a mental map for a route. The mobility part has four components: (i) basic mobility skills that include a human guide, for example, to negotiate doorways, stairs, narrow passageways and elevator, ability to travel with a human guide when using a white cane; hand trailing; and protective techniques; (ii) travel indoors in the CCB buildings, for example, how to travel independently to and from each location on specific floors and buildings; (iii) campus navigation, for example, travel to and from different parking lots to different buildings; recognition of east, west, north driveways; walking from different locations on CCB campus to Sargent Street or Centre Street and to the bus stops; (iv) community travel, for example, ability to listen to verbal directions, repeat directions, execute simple and complex routes, travel on a bus or subway independently, being able to ask for help when needed, and interacting with public appropriately. The third part of the O&M program focuses on cane technique. During the 14 weeks of the campus-based program each client had three to four O&M sessions each week for 50 minutes each. IV. THE BLINDAID INTEGRATED IN THE TRADITIONAL O&M REHABILITATION MODEL The integration of the BlindAid system into the CCB traditional O&M rehabilitation program happened in two stages. The first stage, which included planning and design, happened before the use of the BlindAid system by the clients, and the second stage, which happened after the clients started using it, involved tailoring the program so it is responsive to the specific needs of the client as well as the program. The first stage was based on three issues: Design and Development VE Models Ten spaces were chosen by the CCB O&M instructors and the researcher to be modeled as learning VEs. The ten models were parts of the Main building, the dormitory, St. Paul building, and the outdoors main campus. The Main building, the dormitory, and the CCB campus are the most used areas for training at the CCB during the O&M rehabilitation program. Nine VE models had structures and object components of the 39

5 four floors of the Main building, of the four floors of the dormitory, and one model of the CCB campus, which included all buildings, paths, parking lots, streets, and objects that surroundings the campus (see Fig. 4). The tenth model, an area that is not part of the O&M rehabilitation program, was of the main floor of St. Paul building. It was used at the end of the rehabilitation program to understand the clients ability to explore unknown spaces by using only the BlindAid system and to study how clients applied their spatial knowledge in the real physical space. Figure 4. A Blue Print of CCB Campus Design and Development of O&M Tasks The O&M tasks were designed by the researcher and later evaluated by the CCB O&M specialists for safety. The O&M tasks were divided into two categories: O&M tasks in the VE and O&M tasks in the real physical space. Table 1 presents examples. The O&M tasks in the VE were: (i) verbal description of the space; (ii) exploration tasks in which the exploration time was limited by the O&M instructor according to their estimation of the time required for exploring the parallel physical space; (iii) object-oriented task, which occurred after the exploration task, in which the clients were asked to find the VE starting point and to find five objects in the VE; (iv) object-oriented task, in which the client was asked to create his or her path from the VE starting point to one of the VE objects and then asked to describe his or her path; (v) perspective task, in which the client was asked to create his or her path from one of the VE landmarks (A) to another VE landmark (B) and after which he or she was asked to describe their path. There were three types of O&M tasks in the physical space: (i) two object-oriented tasks, in which the client was first asked to perform the same object-oriented task as in the VE and then to go back to the starting point, followed by a second task in which the client was asked to go from the starting point to another object in the same space and afterwards to go back to the starting point; (ii) two perspective tasks, similar to the above task, in which the client was asked to perform the same perspective task required in the VE and then asked to go back, followed by the client being asked to go from location A to another object in the space (C) and then asked to go back to location A; and (iii) pointing tasks in which the client was asked to stand at the starting point facing the same direction as in the VE and to point with his or her finger to the location of five different objects. TABLE 1: ORIENTATION TASKS IN THE VE AND IN THE REAL PHYSICAL SPACE In the VE In the Real Physical Space (i) Task Verbal description Example (ii) Exploration (iii) Object-oriented Go to the starting point / bathroom / iron board / Dina s office / refrigerator / elevator (iv) Object-oriented Create your path from the starting point to the stairs Describe your path from the starting point to the stairs (v) Perspective Create your path from the elevator to the bathroom Describe your path from the elevator to the bathroom (i) Object-oriented (similar From the starting point, go to the VE object to the stairs oriented task) (ii) (iii) Object-oriented reverse Object-oriented new Object-oriented reverse Perspective (similar to the VE object-oriented task) Perspective reverse Perspective new Perspective reverse Hand pointing tasks from the VE starting point Go back to the starting point From the corner of the library corner, go to the kitchen sink Go back to the starting point From the elevator, go to the bathroom Go back to the elevator From the elevator, go to the Low Vision Department Go back to the elevator Point on the location of: Administration Office / kitchen / bathroom Clients O&M evaluation Before integrating the BlindAid system in the traditional O&M program, each O&M instructor was asked to evaluate their clients behavior on four criteria. The instructors were first asked to define their client s abilities in orientation, mobility, and spatial memory on a scale of one to five ((1) very good; (2) good; (3) it depends, some times; (4) has some difficulties; (5) not at all). Next, the instructors were asked to give some background on their client s O&M abilities and skills based on their evaluation report and information on their 14-weeks O&M rehabilitation program. The third assessment included an evaluation of how well the client knew the ten research spaces (four floors of the Main building, four floors of the dormitory, the CCB campus, and St. Paul main floor), using the same scale. For the last assessment, the instructors were asked to recommend special O&M methods for the clients to use in the VE training. The second stage of the integration of the BlindAid system into the CCB traditional O&M rehabilitation program took place after the clients started using the BlindAid system. The 40

6 researcher met with clients two to three times each week for 50 minutes. The first three meetings included reviewing the consent form and getting the participant s signature, answering the O&M questionnaire, and training on how to operate the BlindAid system. At the fourth session, the clients started exploring the VE models that represented the CCB buildings and campus. Each session was dedicated to a different building floor, from simple spaces to complex spaces and included O&M tasks in the VE and in the real physical space, as described above in Table 1. Once a week, the researcher met individually with the O&M instructors. By using the researcher interface the instructor could see the client s exploration performance by using the user s log (see Fig. 3), and they could assess which exploration strategy their client used during his or her exploration and whether a special O&M method was needed to be addressed in the VE training. In addition to gathering information about the client exploration in the VE, the instructor learned how his or her client was performing orientation tasks in the physical space. The model described above for integrating the BlindAid system into CCB s O&M rehabilitation program was studied with an experimental group (n=10) and a control group (n=10). At the time of the conference, detailed results from the actual study as well as preliminary conclusions will be presented. REFERENCES [1] R. Amendola, Touch kinesthesis and grasp (haptic perception). Unpublished Manuscript, held by The Carroll Center for The Blind, MA, [2] N. Campbell, Sensory training. In R. Rosenbaum (Ed.). The sound of silence. The Carroll Center for The Blind, Newton, MA, [3] N. Campbell, Mapping. In R. Rosenbaum (Ed.). The sound of silence. The Carroll Center for The Blind, Newton, MA, [4] M. A. Espinosa, E. Ochaita, Using tactile maps to improve the practical spatial knowledge of adults who are blind, Journal of Visual Impairment and Blindness, 92(5), pp , [5] J.F. Herman, T.G. Herman and S.P. Chatman, Constructing cognitive maps from partial information: A demonstration study with congenitally blind subjects, Journal of Visual Impairment and Blindness, 77(5), pp , 1983 [6] [6] J.J. Rieser, Access to knowledge of spatial structure at noval points of observation, Journal of Experimental Psychology: Learning, Memory, and Cognition, 15(6), pp , [7] S. Ungar, M. Blades, and S. Spencer, The construction of cognitive maps by children with visual impairments, in: Portugali, J. (ed.), The construction of cognitive maps. Kluwer Academic Publishers, Netherlands, 1996, pp [8] D.H. Warren and E.R. Strelow, Electronic spatial sensing for the blind, Martinus Nijhoff Publishers, MA, [9] R.D. Easton and B.L. Bentzen, The effect of extended acoustic training on spatial updating in adults who are congenitally blind, Journal of Visual Impairment and Blindness, 93(7), pp , [10] W. Crandall, B. L. Bentzen, L. Myers and P. Mitchell, Transit accessibility improvement through talking signs remote infrared signage, a demonstration and evaluation, San Francisco, CA: The Smith- Kettlewell Eye Research Institute, Rehabilitation Engineering Research Center, [11] S. Landau, W. Wiener, K. Naghshineh and E. Giusti, Creating Accessible Science Museums With User-Activated Environmental Audio Beacons (Ping!), Assistive Technology, 17, pp , [12] R. Golledge, R. Klatzky and J. Loomis, Cognitive mapping and wayfinding by adults without vision, in: Portugali, J. (ed.), The construction of cognitive maps. Kluwer Academic Publishers, Netherlands, 1996, pp [13] M. T. Schultheis and A.A. Rizzo, The application of virtual reality technology for rehabilitation, Rehabilitation Psychology, 46(3), pp , [14] P.J. Standen, D.J. Brown and J.J. Cromby, The effective use of virtual environments in the education and rehabilitation of students with intellectual disabilities, British Journal of Education Technology, 32(3), pp , [15] C. Giess, H. Evers and H.P. Meinzer, Haptic volume rendering in different scenarios of surgical planning, presented at the Third PHANToM Users Group Workshop, M.I.T., Massachusetts, [16] G. Jansson, J. Fanger, H. Konig and K. Billberger, Visually impaired persons use of the Phantom for information about texture and 3D form of virtual objects, presented at the Third Phantom Users Group Workshop, M.I.T., Massachusetts, [17] [17] C. Colwell, H. Petrie, and D. Kornbrot, (1998), Haptic virtual reality for blind computer users, presented at the Assets 98 Conference, Los Angeles, CA, Available: [18] C. Sjotrom and K. Rassmus-Grohn, The sense of touch provides new computer interaction techniques for disabled people, Technology Disability, 10(1), pp , [19] A.I. Karshmer and C. Bledsoe, Access to mathematics by blind students - introduction to the special thematic session, presented at the International Conference on Computers Helping, [20] P. Parente and G. Bishop, BATS: The blind audio tactile mapping system, ACMSE. Savannah, GA, [21] O. Lahav and D. Mioduser, Exploration of Unknown Spaces by People who are Blind, Using a Multisensory Virtual Environment (MVE), Journal of Special Education Technology, 19(3), [22] S.K. Semwal and D.L. Evans-Kamp, Virtual environments for visually impaired, presented at the 2nd International Conference on Virtual worlds, Paris, France,