Controller Design for a Wearable, Near-Field Haptic Display

Transcription

1 Controller Design for a Wearable, Near-Field Haptic Display Robert W. Lindeman Justin R. Cutler Department of Computer Science The George Washington University nd St NW Washington, DC {gogo jrcutler}@gwu.edu Abstract In this paper, we address the problem of providing near-field haptic feedback in a wearable, scalable manner. Our solution, called the TactaBoard, supports the independent control of 16 outputs on a single controller board using a standard serial port. We have tested the system with several types of output devices, including low-cost pager motors and fans. Based on Pulse-Width Modulation, the system can generate an output frequency from 0.3Hz to 316Hz. We provide a detailed description of the characteristics of our system, and present early results from empirical studies we have conducted with one possible configuration of tactors. Future enhancements to the TactaBoard system include the ability to daisy chain multiple boards on one control bus, and support for other classes of output devices such as those requiring an H-Bridge. Finally, we present some possible applications where this type of system might be useful. 1. Introduction The goal of our current work is to develop a scalable system for providing multiple kinds of touch feedback cues for use in simulation environments (e.g., virtual reality simulations). Several key problems need to be addressed in order to produce a solution that is usable, including unit size, weight, power consumption, and ease of donning/doffing. In addition, it is desirable that such a system also be low-cost, easy to integrate into existing systems, easily reconfigurable, and user extensible. With these goals in mind, we have developed the TactaBoard system. This paper introduces the problems we are trying to solve, describes our system in detail, and discusses some empirical studies we have conducted using one configuration of the TactaBoard system. In order to best frame the problem space we are exploring, we present an illustration of one possible application. The main underlying driving application area that has helped us outline and design the requirements of the TactaBoard system is the area of using virtual reality for dismounted infantry training. Setting out to provide a foot-soldier with a simulated environment containing the right type and amount of fidelity to acquire new skills, or improve existing ones, is an ambitious undertaking. Unlike vehicle simulators, this sort of training environment has the user in close proximity to many possible types of objects, such as walls, doors, windows, weapons, and other soldiers. In addition, the user can interact with these objects directly (e.g., with the hand), through an intermediate object (e.g., resting a gun barrel on a window ledge), or might be struck by an object, like weapons fire. Finally, contact with objects might involve a large amount of surface area of the soldier, such as leaning his back against a wall, or going prone. Thus, the nearfield haptics of this system must be flexible enough to support various forms of contact, yet allow the soldier the freedom to locomote and explore the space. 2. Background Any approach to providing useful cues must start by looking at how humans perceive stimuli from the outside world. Much of the empirical work into determining how we sense touch has focused on the hands, and, in particular, on the finger pad of the index finger [8]. Some approaches combine tactile and kinesthetic stimulation into a single system. Howe [9], Wellman and Howe [21], and Kontarinis et al. [11] describe the evolution and refinement of a teleoperation system for supporting a precision pinch grasp, using a two-fingered linkage, voice coils, and arrays of shape memory alloy wire actuators. Some researchers have focused on our ability to discern combinations of sinusoidal waveforms at differing frequencies [19, 7, 3]. Others have looked at our ability to discern patterns in the presence of temporal masking of pattern elements [8]. Some researchers have begun to explore the use of vibrating motors, similar to those used in pagers and cell phones, as a means of providing inexpensive haptic cues [12, 17, 6]. Hughes and Forrest [10] instrumented a standard desktop mouse with vibration elements and discuss its application to multivariate map exploration. We propose combining low-cost vibrotactile (VT) feedback units with feedback from other channels to relay contact or other information to the user. In the absence of

2 actual, physical walls, tactors mounted on the user (e.g., on the arm) could be triggered to simulate physical contact between the arm and a virtual wall. It is hoped that this integration of VT cues into a virtual environment (VE) system would thereby improve a user's sense of contact made with objects in the VE. 3. Design Our group went through several design iterations before finally settling on our current layout. Our first priority was to support at least 16 outputs on a single board, and to provide scalability by supporting multiple boards on a single communications line. The rest of this section motivates the remaining design decisions, and gives a detailed description of our current setup Design Goals The TactaBoard was designed to meet the following constraints as a low-cost, wearable haptic display: Mobility: The TactaBoard should be battery powered and portable. Ideally, the TactaBoard would be either linked to a handheld computer for autonomous operation or linked via wireless communication to a host. Compactness: The TactaBoard should be approximately the size of a handheld computer Simplicity: A minimum number of components should be incorporated into the design to reduce complexity of assembly and troubleshooting. Expense: Low-cost components and tactors should be used to minimize overall costs. Updatability: It should be possible to update any software residing on the TactaBoard in the field to allow additional features to be added as they are needed, or for bug fixes to be applied Design Approach We originally considered controlling tactors via a multiplexed resistor ladder to create a true variable analog output. With this method, however, each tactor required eight controller pins to allow a range of 256 discrete output levels. This was found to be prohibitive in limiting the number of tactors that could be controlled by one lowcost board. Our next approach was to use Pulse-Width Modulation (PWM) for our tactor control scheme (Figure 1). PWM allows us to vary the amount of time voltage is sent to a device instead of the voltage level sent to the device [2]. By varying the duty cycle of a PWM signal, we can deliver varying power to a device using a single microcontroller output pin. This was consistent with our desire to control at least 16 devices with a single microcontroller. We found that PWM also reduced power consumption significantly compared to our analog output approach. V o l t a g e V o l t a g e 100% 50% 0% 100% 50% 0% (a) time (b) time Figure 1. Varying the output: The resulting output produced by halving the input voltage (a) can be achieved by halving the ON time (duty cycle) within a pulse period (b). It is important to note that the system acts as an inexpensive, multiple-output digital-to-analog converter that is independent of the actual voltage being varied. The current prototype supports any output device requiring 6 volts or less. Devices requiring higher voltages can be supported using additional switching hardware. This allows a single, unified interface to be used, even if the output devices vary Current Hardware Implementation The TactaBoard is based on a Microchip PIC 16F microcontroller (Figure 2). This microcontroller operates at 20MHz and has 8192 words of instruction memory (used to store the program and default values), 384 bytes of RAM, and 256 bytes of EEPROM (nonvolatile memory used to store configuration values for the board) [15]. Support circuitry includes a Maxim MAX233CPP RS-232 level converter [14] and one transistor per tactor for switching the tactor supply voltage. Our targeted PWM frequency was 300Hz for 16 devices, with one-byte resolution (256 steps). With our present firmware, we can adjust the PWM frequency

3 between 0.3Hz and 316Hz. By reducing the number of devices, or the resolution, we can increase the maximum PWM frequency. Host Computer RS-232 MAX233CPP Transistors To Tactors Board Power TTL PIC 16F TTL Figure 2. Hardware schematic The TactaBoard currently is a single printed circuit board (PCB) that measures 8.9cm x 11.4cm (3.5" x 4.5") (Figure 3). Tactor Power Figure 4. TactaBoard with connections 3.4. Interface The host computer communicates with the TactaBoard using a standard RS-232 serial interface and a custom control protocol. The protocol allows for setting tactor values by board and output address. Broadcast commands are also included to allow addressing multiple tactors with a single command. Currently, the TactaBoard can communicate with the host computer at speeds ranging from 2,400 bps to 115,200 bps. For wireless communication between the host computer and the TactaBoard, we use the MaxStream 9XStream-DEV 900MHz wireless development kit [13]. The rate at which output values can be changed is currently limited by communication speeds. For example, at 115,200 bps, values can be changed 2,400 times per second Software Implementation Figure 3. TactaBoard PCB in box The PCB is placed in a 19cm x 11cm x 5.8cm (7.5" x 4.3" x 2.3") box (Figure 4). We connect the tactors to the box via 2.5mm dual-conductor headphone jacks, and the box has two power supplies, one for board power and one for output device power. We use a 3.5mm stereo headphone jack as a serial connector. The TactaBoard protocol and a simple application program interface (API) have been released under the GNU Lesser General Public License (LGPL). The TactaBoard firmware allows for setting 16 PWM outputs to 256 discrete levels. In addition to direct setting of output levels, a calibration curve can be stored on each TactaBoard for different output responses Tactors We have used the TactaBoard with multiple types of tactors. All of the tactors we have used thus far have been DC motors that produce vibration by rotating an eccentric mass. The first tactor we used was a cylindrical motor similar to those found in pagers and cellular phones (Figure 5). The tactor measures 6mm (1/4") in diameter and 14mm (9/16") in length and weighs approximately 2 grams. The tactor has a standard speed of 4,000 RPM at 1.5 VDC.

4 For our pilot studies, we have used disk-shaped DC motors with an eccentric mass. Each of these tactors measures 14mm (9/16") in diameter and is 3mm (1/8") thick. Figure 5. Cylindrical tactor The tactors used in our user studies were disk-shaped DC motors (Figure 6). They are manufactured by Tokyo Parts Industrial Co., Ltd., Model No. FM23A, and have an operating voltage range of V at 30mA. They have a standard speed of 5,000 RPM at 1.3V, and have a vibration quantity of 1.0G. Each of these tactors measures 18mm (11/16") in diameter and is 3mm (1/8") thick. application, a package containing a tactor and an accelerometer seems to be an attractive way of providing more-predictable VT cues. In studies we have done, we found that both the frequency and amplitude of the stimulus change drastically depending on the load placed on a tactor. In other words, a tactor mounted a person's back, possibly under a backpack, will give a different vibrational stimulus than one mounted under light load on the chest, given equal voltage. It is therefore nearly impossible to know the precise magnitude of the stimulus, because not all of the forces are known, and they are potentially changing from moment to moment. Virtual environment systems, however, typically provide no haptic feedback whatsoever, so as bleak as this VT stimulus control situation sounds, providing any such feedback is better than none, as long as the feedback is controlled and tested for the given application. This has required us to strike a balance between the psychophysics of VT feedback, typically requiring exact measures of stimulus in order to measure outcomes, and a more pragmatic approach used commonly in the computer graphics field, where if it looks good enough, it is good enough. To give some indication of the stimulus, we have found a roughly linear correlation between PWM output and stimulus intensity Limitations Our design for a compact, low-cost device is not without its limitations. All of the outputs on a given TactaBoard use a single, user-definable PWM frequency; only the duty-cycle is varied per output. Low-frequency PWM modulation can lead to significant noise and harmonics. However, because DC motors, such as the tactors we use, tend to act as RL (low-pass) filters, this has not been a major issue so far. Also, the current amount of memory allocated for calibration information does not provide the ability to correct for multiple devices on a single board; only a single curve can be defined for all the tactors on a given board. Our device does not allow operation of voice coil type tactors and would require significant additional support hardware to do so. Figure 6. Disk tactor used in experiments Soon after we began experimenting with pager motors, it became clear that one of the major drawbacks of this technology is the variability in vibration characteristics from one motor to another, even with motors of the same make and model. In addition, over time, a motor's characteristics will change, depending on the amount of use, humidity present, etc. This forces one to either use higher-quality tactors, or to introduce some sort of closedloop calibration into the system. Depending on the 4. Empirical Studies We have used the TactaBoard system in one configuration to conduct several user studies currently in preparation for publication. A 3-by-3 array of tactors, with a 6mm distance between tactor centers, was attached to the back of an office chair, similar to the work of Tan et al. [20] (Figure 7). In the following descriptions, the stimulus presentation times refer to the amount of time the PWM signal was applied to the motor. The actual stimulus time will differ from this, as the mechanics of the tactors require "spin-up" and "spin-down" times.

5 One study focused on the ability of users to determine the location of a 1 second stimulus of 92Hz. This study showed that subjects successfully identified the correct location 84.3% of the time (637 out of 756 trials). A deeper analysis of the errors showed a statistical difference in error rates based on the stimulus location, as well as a trend in stimuli higher on the back to be mistakenly identified for lower ones. Figure 7. Tactor array mounted on an office chair A second study required subjects to match the intensities of two 92Hz stimuli feed to different tactors in the array. The goal of this study was to find a mapping of stimulus intensity at one location to an equal intensity at another. Though the findings were inconclusive, the study helped underscore the need for closed-loop correction of VT devices, due to the change in stimulus frequency in dynamic environments where the load on the tactor changes. A third set of studies looked at the influence of visual and vibrotactile cues, in isolation and combination, on a visual search task. These studies showed a significant reduction in trial times when VT cues were used for providing directional information for locating a target in a cluttered environment. Visual cues significantly dominated over VT cues, but VT cues still had an affect. We have also run a number of pilot studies in a different TactaBoard configuration, comparing tactors in a cylindrical form factor with disk-shaped tactors. Our early results show the disk-type tactors to exhibit greater consistency in vibration, and to cover a more-uniform area of the skin. Further pilot studies have compared tactor intensities at a single location on the forearm in a forcedchoice task. This pilot study has given us some insight into designing full-blown studies into just-noticeabledifference determination. This description of user studies is included to illustrate the major contribution of our current system, namely its ease of reconfigurability. All of these studies were run using the same control hardware and firmware, with the only major changes being how and where the tactors were deployed. The software for the user studies was quickly produced (usually over a two-day period), debugged, and pilot tested. The tactors could be very quickly plugged into and unplugged from the TactaBoard, allowing us to keep the tactors in place, and just move the board. This allowed us to gather a large amount of data in a short amount of time, from many different configurations and empirical studies. 5. Applications for VT Feedback A system with the characteristics of the TactaBoard could be applied to many different areas. Arrays of VT feedback devices could be placed on parts of the body (for instance, on the forearms), and users could be fed collision information as their arms intersect virtual objects. This "virtual bumping" into the environment might aid users in maneuvering. Physical props could be outfitted with VT devices to provide feedback for when the prop contacts virtual objects. For instance, a rifle prop could be outfitted to give the user a sense of bumping the barrel into something, or resting it on a support. In addition to virtual contact, many other applications suggest themselves Data Perceptualization Hughes and Forrest [10] talk about data perceptualization as the extension of the notion of data visualization to cover all the senses, as well as the associated cognitive processing. They note that a large percentage of the literature on data visualization deals with presenting data from a single sensory channel. They posit that if we could use multiple channels to provide feedback, we might be able to support the understanding of a larger number of variables. As a data perceptualization technique, we are experimenting with the use of a single tactor, mounted on a stylus, for exploring a volume data set. As the user moves the stylus through the data set, the vibration fed back through the stylus is proportional to the value of a particular variable in the data. It will be interesting to compare this inexpensive device with similar techniques which use force-reflecting devices, such as the PHANToM Spatial Awareness Rupert [17] has developed a system using a vest with tactors sewn into it. This system allows pilots to better judge the down-vector when performing aerial maneuvers that alter the pilot's vestibular system in such a way as to cause possibly-fatal errors in judgment. A similar system could be used by scuba divers to orient themselves with respect to the up-vector. Devices that substitute VT feedback for sounds have been used for years by the hearing impaired. These systems are typically limited to a few (usually two) tactors

6 for feedback. With the TactaBoard system, a large number of tactors could be attached to different parts of the body to increase the fidelity of the feedback possible for the hearing impaired, improving their quality of life. The automobile industry could embed tactors in the driver's seat or steering wheel as a feedback system for alerting or notifying drivers of certain situations. For example, a monitoring system could be used to measure how close a car is to the line markers on the road, and alert the driver using vibrotactile feedback when the car nears the line Navigation Aid GPS systems used today in many vehicles could be coupled with a TactaBoard system in a route-following application to alert drivers when it is time to make a turn. If the tactors are spaced at different locations in the driver's seat, spatial information can be used as well. In firefighting scenarios, a firefighter with a GPS transponder could be guided through a smoke-filled building in order to search for victims (e.g., find the bedrooms). This could be done autonomously, or using a human guide. Because these environments are often very loud, verbal communication is not always an option, so VT feedback could provide the same information using a nonverbal channel Nonverbal Communication Some of our research is driven by the application of VT feedback for allowing members of a special forces team to communicate nonverbally. Tactors placed on the team members can be controlled using standard hand signals interpreted using pattern recognition, passed to team members wirelessly, and displayed using VT feedback. Special forces also often communicate with each other through physical contact. One member might kick the back of the shoe of another member manning a position in front of them to move the person along (e.g., off of a door). They touch shoulders when lining up in a stack prior to entering a room. They maintain contact while moving, so as to track the other's position while covering different fields of fire. We could use VT techniques, coupled with location sensing, to feed similar proximity information to members of a team, so that they can use tactile cues to communicate at a distance, or through walls Computer Interface Support A stylus form factor could be used in a virtual modeling system, such as in molding virtual clay. The VT feedback could be varied as a function of how hard the user is pushing on the surface, taking into account surface compliance, and therefore possibly improving the user's overall sense of the surface being molded. Studies into human VT perception using this point-contact approach could be compared with similar studies done using commercial active-haptic feedback devices [22]. Traditional computer interface devices can be augmented to provide additional information about mouse [1] or TrackPoint [5] movement. Snibbe et al. [18] discuss the use of instrumented, special-purpose interface devices for controlling the flow of digital visual and auditory media in editing and searching tasks. The authors draw on their backgrounds as audio/video engineers to apply their insights into the physical feedback that make non-digital interfaces (e.g., editing machines) easy and precise to use, and how these qualities have been lost in the move to mouse-based control interfaces. This innovative paper underscores the need to include domain-specific knowledge into interface design. 6. Future Work The next phase of development will be updating the TactaBoard design to use the Microchip PIC16F microcontroller [15]. This will allow greater device flexibility by providing more general I/O pins and allowing the use of additional specialized pins, such as analog-to-digital input (A/D) pins. This chip supports 8 10-bit A/D input lines which could be used to allow closed loop calibration of tactors via an external accelerometer. We will be adding support for H-Bridge outputs in order to provide bi-directional control for devices such as fans. Our initial expectation is to be able to control 8 bidirectional outputs with a PWM frequency of 300Hz. We also plan future software optimizations to increase the maximum PWM frequency for both the single- and bi-directional versions of the TactaBoard. In addition to reducing signal noise, this will allow us to control devices that need much faster updates. Once we have increased PWM frequency to 1kHz [16], we should be able to control Peltier devices reliably which will allow the TactaBoard to provide temperature feedback. We plan to add Controller Area Network (CAN) bus functionality to the TactaBoard system by utilizing the upcoming CAN-enabled Microchip PIC18XXXX family of microcontrollers [15]. The CAN bus, developed and widely used by the automotive industry to provide multidrop, bi-directional communication for on-board computer systems, will allow for multiple TactaBoards on a single control channel [4]. A new control protocol is being written to allow the TactaBoard to share a bus with other devices as well.

7 7. Conclusions Providing a general solution to the requirements of near-field haptics is a daunting task. In this paper, we have described one attempt. Our TactaBoard system has been designed with an eye towards simplicity, scalability, flexibility, and low cost. We have begun to measure our success by using it in sample configurations in a number of user studies. In addition, the protocol and API are available from the authors under the GNU LGPL, and a detailed schematic of reference hardware is currently in preparation. Being able to rapidly reconfigure a vibrotactile feedback system has proved invaluable. Using the same controller, we have been able to compare tactors with different mechanical properties and form-factors, in a number of different tactor layouts, such as a 3-by-3 array attached to an office chair, a single tactor on a stylus, and a glove-like device. In addition, we have used the same hardware to control devices with larger voltage requirements, such as muffin fans. By providing a general framework such as the TactaBoard, we hope that devices designed specifically for a given application area can be controlled in a unified manner, possibly even using a single communications link. Acknowledgments Funding for this work was provided by the Office of Naval Research under Grant No. N , and by DARPA under Grant No. N x Some of the work reported here was conducted in the MIS3 group of the Advanced Telecommunications Research Institute International, in Kyoto, Japan. We especially want to acknowledge YANAGIDA Yasuyuki of ATR for his insight into the design of the empirical studies. Finally, we would like to acknowledge the contributions of Dr. John Sibert and Dr. Robert Lavine for their help with the pilot studies. 8. References [1] Akamatsu M, MacKenzie I. Movement Characteristics Using a Mouse with Tactile and Force Feedback. Int'l J. of Human-Computer Studies 1996; 45; [2] Barr M, "Pulse Width Modulation", Embedded Systems Programming, Sep. 2001, [3] Bensmaïa S, Hollins M. Complex tactile waveform discrimination. J. of the Acoustical Soc. of America 2000; 108(3); [4] CAN in Automation, [5] Campbell C, Zhai S, May K, Maglio P. What You Feel Must Be What You See: Adding Tactile Feedback to the Trackpoint. In: Proc. of INTERACT'99: 7th IFIP Conference on Human Computer Interaction 1999; [6] Cheng L-T, Kazman R, Robinson J. Vibrotactile feedback in delicate virtual reality operations. In: Proc. of the Fourth ACM Int l. Conf. on Multimedia 1996; [7] Choi S, Tan H. A parameter space for perceptually stable haptic texture rendering. In: Proc. of the Fifth PHANToM Users Groups Workshop, Aspen, CO, October [8] Craig J. Vibrotactile pattern isolation/integration. Perception & Psychophysics 1998; 60(5); [9] Howe R. A force-reflecting teleoperated hand system for the study of tactile sensing in precision manipulation. In: Proc. of the 1992 IEEE Int l. Conf. on Robotics and Autom. 1992; vol. 2; [10] Hughes R, Forrest A. Perceptualisation Using a Tactile Mouse. In: Proc. Visualization ' ; [11] Kontarinis D, Son J, Peine W, Howe R. A tactile shape sensing and display system for teleoperated manipulation. In: Proc. of the Int l. Conf. on Robotics and Autom. 1995; [12] Massimino M, Sheridan T. Sensory substitution for force feedback in teleoperation. Presence: Teleoperators and Virtual Environments 1993; 2(4); [13] MaxStream, Inc., [14] Maxim Integrated Products, Inc., [15] Microchip Technology Inc., [16] Peltier Device Information Directory, [17] Rupert A. An instrumentation solution for reducing spatial disorientation mishaps. IEEE Eng. in Med. and Bio. 2000; March/April; [18] Snibbe S, MacLean K. Haptic Techniques for Media Control. CHI Letters (Proc. UIST 2001) 2001; 3(2); [19] Tan H, Durlach N, Reed C, Rabinowitz W. Information transmission with a multi-finger tactual display. Perception & Psychophysics 1999; 61(6); [20] Tan H, Lu I, Pentland A. The chair as a novel haptic user interface. In: Proc. of the Workshop on Perceptual User Interfaces, Oct , Banff, Alberta, Canada 1997; [21] Wellman P, Howe R. Towards realistic vibrotactile display in virtual environments. In: Proceeding of the ASME Dynamics Sys. and Control Division, Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Sys., Alberts T. ed. 1995; DSC-Vol. 57-2; [22] Yamashita J, Lindeman R, Fukui Y, Morikawa O, Sato S. On determining the haptic smoothness of force-shaded surfaces. In: Conference Abstracts and Applications, SIGGRAPH 2000; 240.