Haptic Perception System For Robotic Tele-Manipulation

Transcription

1 aptic Perception System For Robotic Tele-Manipulation Emil M. Petriu School of Information Technology and Engineering University of Ottawa Ottawa, O K1 65 petriu@site.uottawa.ca Thom E. Whalen Communications Research Centre 3701 Carling venue, P.O. Box 11490, Stn. Ottawa, O K2 8S2 thom.whalen@crc.ca Voicu Z. Groza School of Information Technology and Engineering University of Ottawa Ottawa, O K1 65 groza@site.uottawa.ca bstract This paper presents an experimental haptic perception system for telemanipulation operations in telerobotics and interactive virtual environments. This haptic system enhances the visual feedback of the human tele-operators. I. ITRODUCTIO Many operations in hazardous environments such as nuclear stations, mines, polluted environments, war zones, or in remote environments such as the space, or underwater are difficult or impossible to be carried out directly by humans or even by fully autonomous robots. These robots must rely on remote human operator expertise to carry out manipulation tasks, which require a higher level of intelligence. Telemanipulation has been developed as an extension of the manipulative capability of the human hand to these difficult to reach environments, [1], [2], and [3]. There are three basic issues recognized in telemanipulation: mechanisms, sensors and control, and man-machine interfaces, [1]. proper control of telemanipulation operations cannot be accomplished without visual, [4], and force and haptic telepresence capability which allows the human teleoperator to experience the feeling that he/she is in the remote environment in which the robot operates, [3], [5], [6], [7] and [8]. The haptic perception is a complex exploratory act integrating two distinct modes: (i) a cutaneous tactile sensor provides information about contact force, contact geometric profile and the temperature of the touched object, and (ii) a kinesthetic sensory system provides information about the positions and velocities of the kinematic structure (e.g. hand) carrying the tactile sensor. There are various cutaneous mechanoreceptors located in the outer layers of the skin. These receptors are specialized nervous cells or neurons. The free nerve endings are the most numerous and play an active role in the perception of pain, cold and warmth. These mechanoreceptors have preferential frequency response characteristics: the highest sensitivity for the Pacinian Corpuscles (PC) units is around z but they respond from 30 z to very high frequencies. The Rapidly dapting (R) units effective frequency range is between 10 and 200 z, with more sensitivity below 100 z. The Slowly dapting (S) units respond at low frequencies, under z. The human kinesthetic function has a much lower frequency band. This paper discusses an experimental haptic perception system for model-based telemanipulation operations in telerobotics and interactive virtual environments. This haptic system enhances the visual feedback of the human tele-operators enhancing their ability to grope, identify geometric object shape, and measure contact forces and slippage. II. SYSTEM RCITECTURE Fig. 1 illustrates the architecture of the experimental robotic telemanipulation system. It is a bilateral architecture, [9], aiming to couple the human operator and the telerobotic manipulator as transparently as possible. Using a ead Mounted Display for augmented visual virtual reality, Fig. 2, and a haptic feedback system, the human operator controls the operation of a remote robot manipulator equipped with video camera and tactile sensors placed in robot s hand. The tactile sensors, [10], provide the cutaneous information at the remote robotic operation site. The joint sensors of the robot arm provide the kinesthetic information. tactile human-computer interface provides the cutaneous feedback allowing the human operator to feel with his/her own sense of touch the same sensation as that acquired by the remote robot hand from its artificial tactile sensor. robot-like kinematic structure provides the kinesthetic component of the haptic human-computer interface.

2 Sensory data gathered from vision, joint encoders, and tactile sensors are integrated in a unique framework which allows one to deal in a common way with all the properties of the manipulated object, These include the object s 3D geometr/shape, its surface-material properties, and the contact forces occurring during object handling by the robot. This framework also solves sensor redundancy problems occurring when more sensors than actually required are used to measure a given object parameter, [11] and [12]. telemanipulator. This will reduce the bandwidth of the telepresence function and it may affect the fidelity and stability of the telecontrol system, [9], [14] and [15]. It will also affect the sensory-motor adaptation of the human operator, which is quite critical for immersive visual reality based telemanipulation. Psychological research has shown that It can be affected by delays as minute as 0.3 s, [16]. ead Mounted Display aptic Feedback Virtual model of the object manipulated in the physical world Video Camera Robot rm Fig. 2. ead Mounted Display Tactile Sensors Manipulated Object Fig. 1. Video and haptic virtual reality interfaces allow a human operator to remotely control a robot manipulator equipped with video camera and tactile sensors. The multi-sensor data fusion system has a hierarchical architecture based on the S/BS standard reference model for telerobot control, [13]. This architecture has an ascending sensory processing path in parallel with a descending task-decomposition control path connected via world models at each level. The processing time at each level is reduced by the use of a priori knowledge from the world model that provides predictions to the sensory system. The use of a world model promotes modularity because the specific information requirements of the sensory and control hierarchies are decoupled. In many applications as, for instance, space or deep ocean telerobotics, [14], there is a considerable communication delay between the human operator and the remote robotic The time clutch concept proposed in [17], is used to disengage synchrony between operator specification time and telerobot operation time during path specifications. In order to avoid fatal errors and reduce the effect of the communication delay, we are using a distributed virtual environment allowing to maintain a shared world model of the physical environment where the telemanipulation operation is conducted, [18]. nother critical requirement is the need to maintain the synchronism between the visual and the haptic feedback. While being two distinct sensing modalities, both haptic perception and vision measure the same type of geometric parameters of the 3D objects that are manipulated. III. PTIC SESORS FOR TE ROBOT MIPULTOR The robotic manipulator, shown in Fig. 3, consists of a five-axis commercial robot, an instrumented passivecompliant wrist and a 16-by-16 tactile probe, [19]. The compliant wrist allows the hand equipped with tactile sensors to accommodate the constraints of the explored object surface and thus to increase the amount of cutaneous tactile information.

3 Position sensors placed in the robot s joints and on the instrumented passive-compliant wrist provide the kinesthetic information. The tactile probe is based on a 16-by-16 matrix of Force Sensing Resistor (FSR) elements spaced 1.58 mm apart on a 6.5 cm 2 (1 square inch) area, [20] and [21]. The FSR elements exhibit exponentially decreasing electrical resistance with applied normal force: the resistance changes by two orders of magnitude over a pressure range of 1 /cm 2 to 100 /cm 2. The elastic overlay has a protective damping effect against impulsive contact forces and its elasticity resets the transducer system when the probe ceases to touch the object. owever, they may cause considerable blurring distortions in the sensing process if they are not properly controlled. We avoided it by replacing the one-piece pad with a custom-designed elastic overlay consisting of a relatively thin membrane with protruding round tabs, shown in Fig. 4, [20]. This construction allows the material to expand without any stress in the x and y directions making possible its compression in the z direction proportionally with the normal stress component. The tabs are arranged in a 16-by-16 array in such a way that there is a tab on top of each node of the FSR matrix. This tab configuration provides a de facto spatial sampling, which reduces the elastic overlay's blurring effect on the high 2D sampling resolution of the FSR transducer. Experimental results illustrate the positive effect of this overlay design. Fig. 5 shows the 16-by-16 median filtered tactile image of the same washer pressed with the same force on a tab-shaped elastic overlay. Fig. 3. Robotic manipulator equipped with tactile and kinaesthetic sensors Fig. 5 Tactile image of a washer. IV. PTIC FEEDBCK TO TE UM OPERTOR Fig. 4. The FSR sensor array with the tab-shaped elastic overlay on top tactile monitor placed on the operator's palm was initially developed, [22], as a cutaneous telepresence interface allowing a human to virtually feel by touch the object profile measured by the tactile sensors placed in the jaws of the robot gripper, as illustrated in Fig. 6. The tactile monitor consists of an 8-by-8 array of electromagnetic vibrotactile stimulators. The active area of this array is 6.5 cm 2 (same as the tactile sensor). Each stimulator corresponds to a 2-by-2 window in the tactile

4 sensor array. The vibrotactile stimulators are used as binary devices that are activated when at least two of the corresponding taxels (tactile elements) in the sensor arary window are "on". Fig. 7 shows a view of this tactile monitor when it generates a curved edge feedback for the human teleoperator. R O B O T - D TS TCTILE IMGE CQUISITIO TS U M - D TM TCTILE SESTIO TM RECOSTRUCTIO Fig. 8. aptic feedback device. TS = Tactile Sensor TM = Tactile Monitor V. COCLUSIO Fig. 6. Cutaneous Tactile Monitor allowing a human to feel by touch the tactile image acquired by the robotic tactile sensor. Fig. 7. "Vibro-display" image on the Tactile Monitor. It is a rather rudimentary virtual-reality tactile interface, but it does the job of letting the human operator feel object edges the same way the robot "feels." Experiments have shown that getting a natural tactile feedback about the position of the object grasped by the robot arm comes in quite handy during telemanipulation operations. We are using a commercial Virtual and Toolkit for CyberGlove/Grasp, shown in Fig. 8, to provide the kinesthetic human-computer interface. new generation of cutaneous feedback is currently developed to work together with the CyberGlove. The developed system is used for an experimental study of the human-machine haptic perception mechanisms occurring in telemanipulation operations for telerobotics and interactive virtual environments. The main research topics are: Object recognition by integrating haptic and visual sensory data into a composite haptic & vision model of 3D object shape, surface and material properties (e.g. texture, elasticity, heat transfer characteristics), and contact forces and slippage. Development of a haptic & vision model-based coding for distributed interactive virtual reality applications. Using a unique composite haptic & vison model of the manipulated objects could be more efficient than using separate vision and haptic data streams. Such an integral approach makes sense as the role of haptics is to enhance the vision. Both haptic perception and vision measure the same type of geometric parameters of the 3D objects that are manipulated. Study of human-computer interaction aspects of the haptic & visual perception for interactive distributed virtual environments. The aim is the development of a more efficient haptic feedback enhancing the visual feedback to human operators during virtual object manipulation. VI. CKOWLEDGMET This work was funded in part by the Communications Research Centre, Canadian Space gency and by the atural Sciences and Engineering Research Council of. The authors gratefully acknowledge the assistance of Francois Malric.

5 VII. REFERECES [1].K. Bejczy, "Sensors, Controls, and Man-Machine Interface for dvanced Teleoperation," Science, vol 208, no. 4450, 1980, pp [2] G. irzinger, B. Brunner, J. Dietrich, and J. eindl, "Sensor- Based Space Robotics - ROTEX and Its Telerobotic Features," IEEE Trans. Robotics utomat., vol. 9, no. 5, 1993, pp [3] M.C. Cavusoglu, F. Tendick, M. Cohn, and S.S. Sastry, Laparoscopic Telesurgical Workstation, IEEE Trans. Robotics utomat, Vol. 15, o. 4, 1999, pp [4]. Liu, G. Tharp, S. Lai, and L. Stark, Some of What One eeds to Know bout Using ead-mounted Displays to Improve Teleoperator Performance, IEEE Trans. Robotics utomat, Vol. 9, o. 5, 1993, pp [5] D.G. agner and J.G. Webster, "Telepresence for Touch and Proprioception in Teleoperator Systems," IEEE Trans. Syst. Man Cyber, vol. 18, no.6, 1988, pp [6]. Bicchi, E.P. Scilingo, and D. De Rossi, aptic Discrimination of Softness in Teleoperation: The Role of Contact rea Spread Rate, IEEE Trans. Robotics utomat, Vol. 16, o. 5, 2000, pp [7] T. Itoh, K. Kosuge, and T. Fukuda, uman-machine Cooperative Telemanipulation with Motion and Force Scaling Using Task-Oriented Virtual Tool Dynamics, IEEE Trans. Robotics utomat, Vol. 16, o. 5, 2000, pp [8]. Zhang and.. Chen, Control of Contact Via Tactile Sensing, IEEE Trans. Robotics utomat, Vol. 16, o. 5, 2000, pp [9] D.. Lawrence, Stability and Transparency in Bilateral Telerobotics, IEEE Trans. Robotics utomat, Vol. 9, o. 5, 1993, pp [10]. icholls and K. Lee, " Survey of Robot Tactile Sensing Technology," The Int. Journal of Robotics Research, Vol. 8, o. 3, 1989, pp [11] J.M. Richardson and K.. Marsh, "Fusion of Multisensor Data," Intl. J. Robot. Res. Systems, vol.7, no. 6, 1988, pp [12] R.C. Luo and M.G. Kay, "Multisensor Integration and Fusion in Intelligent Systems," IEEE Trans. Syst. Man Cybern., vol 19, no. 5, 1989, pp [13] L. Derobertis and. Milano, " Man Machine Interface for SREM-based Telerobotics," IEEE/RSJ Intl. Workshop on Intell. Robots and Systems IROS'91, Osaka, 1991, pp [14] T.B. Sheridan, Space Teleoperation Through Time Delay: Review and Prognosis, IEEE Trans. Robotics utomat, Vol. 9, o. 5, 1993, pp [15] B. annaford and J-. Ryu, Time-Domain Passivity Control of aptic Interfaces, IEEE Trans. Robotics utomat, Vol. 18, o. 1, 2002, pp [16] R. eld,. Efstathiou, and M. Greene, daptation to displaced and delayed visual feedback from the hand, J. Exp. Psych., Vol. 72, 1966, pp [17] L. Conway L., R.. Voltz., and M.W. Walker, "Teleautonomous Systems: Projecting and Coordinating Intelligent ction at a Distance," IEEE Trans. Robotics utomat., vol. 6, no. 2, 1990, pp [18] F.C.. Groen,. Vlassis, M. Boasson,.J.W. Spoelder, and.e. Bal, framework for maintaining a shared world model in dynamic environments between differentiated embedded systems and allowing interaction with human supervisors, in Proc. 2 nd PROGRESS Workshop on Embedded Systems, Veldhoven, The etherlands, October [19] E. Petriu, W.S. McMath, S.K. Yeung,. Trif, "ctive Tactile Perception of Object Surface Geometric Profiles," IEEE Trans. Instrum. Meas., Vol. 41,o. 1, 1992, pp [20] S.K. Yeung, E. Petriu, W.S. McMath, D.C. Petriu, "igh Sampling Resolution Tactile Sensor for Object Recognition," IEEE Trans. Instrum. Meas., Vol. 43, o. 2, 1994, pp [21] ***, Interlink Electronics Tactile rray Robotics, Interlink Electronics Inc., Santa Barbara, C, [22] E. Petriu and W.S. McMath, "Tactile Operator Interface for Semi-autonomous Robotic pplications," in Proc. Int. Symposium on rtificial Intell. Robotics utomat. in Space, i-sirs'92, Toulouse, France, 1992, pp