WEARABLE HAPTIC DISPLAY FOR IMMERSIVE VIRTUAL ENVIRONMENT Yutaka TANAKA*, Hisayuki YAMAUCHI* *, Kenichi AMEMIYA*** * Department of Mechanical Engineering, Faculty of Engineering Hosei University Kajinocho, Koganeishi, Tokyo 184-8584, Japan (E-mail: y_tanaka@a k.hosei.ac.jp) **CANON Inc., Japan ***EIZO NANAO Co., Japan ABSTRACT Recently a number of immersive displays that project virtual environment on large-sized screens have been developed. In applications of virtual reality technology for such large immersiveenvironments, it is important to develop some haptic display that gives a real feedback sensation such as forceor tactile to users from the virtual environment. Especially, it is necessary to develop small, light and safety actuators for the wearable haptic display. In this paper, the haptic display system using pneumatics for theimmersive virtual environment is proposed and developed. The system consists of a tactile and a force reflecting type wearable haptic displays using pneumatic pressure control. Experiments for grasping virtual objects in the virtual environment are performed by the wearable haptic displays using pneumatic actuators. It is experimentally verified that the performance of the haptic display system is effective to touch and grasp the virtual objects. KEY WORDS Virtual Reality, Pneumatic Actuator, Haptic Display, Wearable Interface INTRODUCTION Virtual Reality is technical innovation on humanmachine interfaces in telerobotics, mechatronics, computer networks, virtual prototyping, or every engineering field. The virtual reality is superior to other forms of human-computer interaction because it provides a real-time immersive environment integrated several new communication modalities, such as stereo graphics, three-dimensional sound, force or tactile feedback, and even taste and smell. By providing these sensorial interactions, the virtual reality makes the user feel immersed in the simulation or application in the virtual environment. The virtual reality may be defined as an integrated trio I3 of Immesion-Interaction-Immagination by Burdea and Coiffet [2]. In some applications of virtual reality, it is necessary that operators sense the contact force and tactile sensation when a virtual object is touched and grasped. Especially, the force and tactile sensation to operator's hand in dexterous teleoperated manipulations is one of the important human-machine interfaces. Many researchers have recently developed haptic display devices and systems [1][2]. The term "haptic" originates from the Greek haptesthai meaning to touch that is synonymous with force and tactile sensation. In order to operate manipulation tasks in a large scale virtual environment for a multi-screen projection system Fluid Power. Fifth JFPS International Symposium (c)2002 JFPS. ISBN4-931070-05-3
such as the CAVE[9], with feeling presence and with maximum freedom motion, it is necessary for users to feedback haptic sensations from the virtual environment by portable force and/or tactile feedback interfaces[2]. Especially it is necessary to develop small, light and safety actuators for the wearable haptic display. On the other hand, nonportable haptic feedback interfaces that are mechanically grounded to a desk, ceiling, or floor, have the advantage of their ability to off-load the actuator weight from the user. The disadvantage of nonportable is a reduction in the user's freedom motion. There is a growing need for users to interact virtual object in a large-scale virtual environment and a telemanipulation. In the dynamic force display system, it is necessary to provide sufficient force while keeping the feedback hardware light and portable. Burdea, et al. [4] have developed a portable master device (The Rutgers Master) designed to retrofit a sensing glove using four pneumatic micro-cylinders placed in the palm of a glove on a small L-shaped platform. This glove is not allowed the simulation of virtual object weight as no wrist feedback is provided. A portable hand master with more degrees of freedom has been developed by Bouzit, et al. [5]. Kramer [6] has patented a "Force feedback and texture simulation interface device" that is modified a sensing glove and added to force and touch feedback. Force feedback is provided by multiple plastic cables that are routed from the wrist mount to the fingertips. The CyberGgrasp (Virtual Technologies Inc.) using Kramer's patent is commercially available for force feedback by a human hand. In our previous study [31[10] the portable force feedback interface using pneumatic pipe or bellows actuators have been proposed and developed. Pneumatic actuators are simple, cheap, clean, light and inherently safe due to air compliance. We call this interface device the Fluid Power Glove. Tactile sensations as well as force sensations, however, are also required for initial contact detection to virtual objects because the force feedback does not come about prior to any manipulation tasks. Tactile stimulation can be achieved various ways, being used for virtual environment systems include mechanical pins activated by piezoelectric crystals [12], shape memory alloy [7], solenoid, vibrations from voice coils, or temperature from thermoelectric heat pump. One of the author has also developed a portable tactile display using air jet on a trial purpose to give local shapes of virtual objects and have conducted an evaluation of the perceptual characteristics of the air jet stimulator and the two point difference threshold of index finger pads and thumb finger pads [11]. We call this interface device the Fluid Tactile Display. In this paper, a new type of haptic display system using pneumatics for the immersive virtual environment is proposed and developed. The system consists of a tactile and a force reflecting type wearable haptic displays using pneumatic pressure control. We demonstrate the dynamic force and tactile feedback system for touching and grasping virtual solid objects in virtual environment of 3D computer graphics. The computer graphics display motion pictures of the virtual objects and the operator's hand model according to the actual motion of the operator's hand and the object. The operator can interactively communicate the virtual object with the contact force in the computer world. FLUID POWER GLOVE A photo of the developed Fluid Power Glove is illustrated in Figure 1(a) and a line drawing illustrating components is shown in Figure 1(b). The prototype of the Fluid Power Glove consists of four pneumatic bellows actuators in which air pressure is regulated by a pneumatic pressure control valve, and a fitting glove for the operator's hand. The pneumatic bellows actuators are fixed in palm of the hand side of the fitting glove. Each bellows actuator allows both flexion and extension of the fingers when the compressed air is not supplied to these actuators. When the operator's hand in the virtual world grasps objects such as a virtual rubber ball or solid ball, the compressed air is supplied to the pneumatic bellows Figure 1(a) Photo of Fluid Power Glove Figure 1(b) Line drawing illustrating components in Fig.1(a)
Figure 2 Pneumatic bellows actuators and the operator can feel dynamic contact forces on his fingers. The contact force depends on the dimension of the object deformation as well as its modeled compliance. Figure 2 illustrates a structure schematic drawing of the pneumatic bellows actuator used in this study. The forming bellows is made of the beryllium that has a thickness of 0.09 mm. The bellows has an inside diameter of 7.5 mm and an outer diameter of 12 mm. A nominal spring constant of the bellows is 1.96 N/mm, and an effective sectional area is 0.77 cm'. A part of the bellows is closed, and other end is connected with a piping joint for air supply. The Fluid Power Glove is very light (total weight of 210 g) and wearable for operator's hand. As a result, the operator can be easily and unconsciously fitted to the Fluid Power Glove for dynamic and interacted force feedback from the working equipment in real or virtual environments. The Fluid Power Glove has the advantages of a low cost, a compact and lightweight structure, and is a clean and safety device. FLUID TACTILE DISPLAY Figure 3(a) shows a photo of the developed Fluid Tactile Display and Figure 3 (b) illustrates the configuration of the fluid tactile display. The developed tactile display represents the local object shapes by pressurized air jet from the array of eight nozzles for tactile stimulator. Human's perception stimulated by the jets is affected by the nozzle placement, i.e. the distance between centers of the nozzles not to exceed the two-point-difference threshold of finger pads. In order to design the nozzle placement we conducted an evaluation of the threshold of an index finger pad and a thumb finger pad in our preliminary experiments [11]. An air jet nozzle that has a diameter of 0.6 mm with elements spaced 2.6 mm apart in x-direction and 2.8 mm in y-direction for index finger, and 3.1 mm in x-direction and 3.4 mm in y-direction for thumb. The fluid tactile display device has a dimension of 25 x 10 x 7.5 mm for index finger and 26 x 11 x 7.5 mm for thumb. It has a Figure 3(a) Photo of Fluid Tactile Display Figure 3(b) Configuration of Fluid Tactile Display total weight of 22 g. The tactile display device uses a pneumatic system to separate the display device from the driving mechanism enabling a small and lightweight display. The phase of the air jet from each nozzle is controlled by sixteen on-off valves supplied with the compressed air pressure of 0.4 MPa. Air jet stimulation has the advantage of simplicity, light weight, cleanliness, and lower cost than the vibrotactile of the electrotactile approaches [13]. Furthermore, air jets are noninvasive and do not produce pain. For the above advantages, however, the pneumatic actuators and air jet have lower system bandwidth than for electrical actuators because of air compressibility. HAPTIC DISPLAY SYSTEM It is experimentally investigated that the force and tactile sensation are applied to the tips of operator's fingers and hand by the prototype of the Fluid Power Glove and the
Fluid Tactile Display using pneumatic actuators as shown in Figure 4. Figure 5 shows the configuration of the haptic display system using the force and tactile feedback display. Compressed air is supplied and regulated from an air compressor to the Fluid Power Glove by three electro-pneumatic pressure control valves and to the Fluid Tactile Display by sixteen on-off valves. The value of the regulated air pressure for the force feedback display is measured at the inlet of the pneumatic bellows actuator with a semiconductor type pressure transducer and directly fed back to the reference command path. The reference command for the air pressure to the electro-pneumatic pressure control valves are adjusted at a suitable value according to the interacted forces for virtual grasping objects. The microcomputer commands to operate the pressure control valve according to the dimensions and compliance of the grasped object and the bend of each finger. The virtual environment is modeled with the World Tool Kit R8 (Sense8 Co.) on a graphic workstation (Silicon Graphics Indigo2 Impact10000). The graphic workstation has a graphical refresh rate at 32 Hz. Hand translation and rotation are tracked using a tracking position sensor (the Fastrak; Polhemus Co.) communicating the graphics workstation at 120 Hz in data transmission rate. The position and movement of the Figure 4 Photo of wearable haptic displays fi ngers are repeatedly measured using a sensing glove (the Super Glove; Nissho Electronics Co.) at 143 Hz in data transmission rate. Each joint of flexing finger angle is measured indirectly by a change of resistance of the polymer film in the sensing glove. The output data of the sensing glove is calibrated by grasping a standard object. The calibration has been done to be repeated every time the Fluid Power Glove is put on at the start of a grasping session for the operator. Data communications to control the pressure regulated valves and the on-off valves uses the BSD socket based communication on the Internet Figure 5 Schematic diagram of haptic display system
protocol. The shape of a virtual object and a human hand for visual sensation in virtual space has been created in the graphic workstation by the World Tool Kit. The 3D visual sensation for the operator's hand model and the virtual object in the virtual world is presented in Figure 6. The computer graphics display motion pictures of the virtual objects and the operator's hand model according to the actual motion of the operator's hand and the virtual object. Collision detection between the virtual hand and the object is also calculated in the virtual environment. The operator can interactively communicate the virtual object in the computer world. EXPERIMENTAL RESULTS In our experiments we image to touch and/or grasp a solid cubic objects in the virtual environment. In the case of touching and grasping the virtual solid object, firstly adequate pattern of air stimulus corresponding to geometry positions of the virtual object is displayed. Secondly a constant value of the regulated air pressure is supplied to the pneumatic bellows actuators and the dynamic contact force displays to the operator's fingers at a virtual contact point. When the constant value of the compressed pressure is momentarily supplied to the Fluid Power Glove, the operator's finger is restricted to the exion and the bending angle of the finger is nearly kep fl at a constant value according to the dimension of the cubic object. The operator can feel like touching and grasping the solid cubic object. In comparison with the similarity between real and virtual touch and grasp for the cubic object, the perceptual similarity is evaluated by a four-grade system Figure 7 Experimental results of haptic sensations to the sense of touch and grasp. Subjects were asked a question, "How would you rate the perceptual similarity of the haptic sensation?" The eighteen answerers as the subjects have been accepted in our laboratory. The subjects, ranging in age from 21 to 23 years, make a choice of one answer that the rating grade scale is from "1 (lowest similarity)" to "4 (highest similarity)" after the provided several trials. Figure 7 shows the responses from the subjects on the perceptual similarity test without and with the tactile feedback using the Fluid Tactile Display. More than 70% subjects answered "I could feel the similarity (Grade 4)" or "I could mostly feel the similarity (Grade 3)" in the test with the tactile feedback. These results indicate grasping the solid object significantly depends on the tactile feedback from the virtual environment. Therefore, it is experimentally verified that one can feel like touching and grasping the virtual objects through the combination of the force and tactile sensations for the developed wearable haptic display system. CONCLUSIONS Figure 6 Virtual The dynamic force and tactile display device using the pneumatic actuators and the virtual contact force feedback system have been developed for large immersive environment in virtual reality applications. The dynamic touch and force sensation is applied on the operator's hand by the Fluid Tactile Display and the Fluid Power Glove. The performance of the system has been experimentally investigated. The effect of displaying the dynamic touch and force sensation for the virtual solid cube to the operator's hand is experimentally confirmed by the wearable haptic display. The operator can environment 313
interactively communicate the virtual object in the computer world. The wearable haptic display using pneumatics is portable, lightweight and safety device in virtual reality applications. ACKNOWLEDGEMENTS The authors acknowledge with thanks the encouragement received from the staffs of the SAGINOMIYA SEISAKUSHO, INC. for valuable supply of the pneumatic bellows. REFERENCES [1] Kalawsky, R. S., The Science of Virtual Reality and Virtual Environments, Addison-Wesley, 1993. [2] Burdea, G., Coiffet P., Virtual Reality Technology, John Wiley & Sons, Inc., 1994. [3] Tanaka, Y., Kanamori, T., Dynamic Force Display Device by Pnuematic Pressure Control, SICE Proc. FLUCOME Vol.2, pp.719-723, 1997. [4] Burdea, G., Zhuang, J., Roskos, E., Silver, D., Langrana, N., A Portable Dexterous Master with Force Feedback, Presence-Teleoperators and Virtual Environments, Vold, No.1, pp.18-27, 1992. [5] Bouzit, M., Richard, P., Coiffet,P., LRP Dexterous Hand Master Control System, Technical Report, Laboratoire de Robotique de Paris, pp.21, 1993. [6] Kramer, J., Force Feedback and Texture Simulating Interface Device, US Patent 5184319, 1993. [7] Kontarinis, D., Son, J., Peine, W., Howe, R., A Tactile Shape Sensing and Display System for Teleoperated Manipulation, IEEE International Conference on Robotics and Automation, Vold, pp.641-646, 1995. [8] Tanaka,Y., Kikuchi,T., Kaneko,A., Dynamic Force Display System by Fluid Power Glove Using Pneumatic Bellows, Proc. JHPS Autumn Annual Meeting, pp.121-123, 1998 (in Japanese). [9] Calolina Cruz-Neira, Daniel J. Sandin, and Tomas A. DeFanti:" Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE", Computer Graphics Proceeding, Annual Conference Series, pp.135-142, 1993. [10] Tanaka, Y., Kikuchi, T., Kaneko, A., Dynamic Force Display in Virtual World by Fluid Power Glove, 4th JHPS International Symposium on Fluid Power, Tokyo, pp.187-192, 1999. [11] Kenichi AMEMIYA, Yutaka TANAKA, Portable Tactile Feedback Interface Using Air Jet, The 9th International Conference on Artificial Reality and Telexistence (ICAT'99) Proceedings, pp.115-122, 1999. [12] Ikei, Y., K. Wakamatsu, and S. Fukuda:" Vibratory tactile display of image-based texture," IEEE Computer Graphics and Applications, Vol.17, pp.53-61, 1997. [13] "Selectively Asamura, Stimulating N., N. Yokoyama,and Skin Receptors H. for Shinoda: Tactile Display," IEEE Computer Graphics and Applications,Vol.18, pp.32-3'7, 1998.