The Effect of Haptic Feedback in a Remote Grasping Situation

Transcription

1 The Effect of Haptic Feedback in a Remote Grasping Situation Dominic Rizzo Lisa Messeri Department of Aeronautics and Astronautics Massachusetts Institute of Technology Cambridge, MA March, 00 Abstract In this paper, we describe an innovative, pneumatically-actuated haptic feedback glove. We also examine the effect of haptic feedback from that device on a test subject s ability to properly grasp a virtual remote object. Two parameters relevant to grasping were examined: success and force control. The success parameter is a indication of whether the virtual object was grasped, not grasped or damaged. The force control parameter measures how well a test subject reacts to the grasping sensation provided by haptic grasping feedback. We were able to demonstrate that haptic feedback leads to a statistically significant positive change in force control, and a statistically significant, but mixed, result in grasping success. I. Introduction Man-machine interfaces are a pervasive part of life. They encompass a large array of devices which in turn engage any number of sensory channels. For example, some devices provide visual feedback (computer displays, indicator lights), some provide aural feedback (speakers, buzzers), but very few provide touch, or haptic, feedback. The type of man-machine interface of interest in this paper is telerobotics. In telerobotics, the most important operation involves the use of the remote operator s hands. One of the most important things that an operator can do with his or her hands is remotely grasp an object. This an area where haptic feedback devices can be applied to greatly enhance operator effectiveness by providing touch as well as visual feedback. There are many examples of remotely operated graspers, such as NASA s Robonaut and the University of Maryland s Space Systems Lab s Ranger, where haptic feedback could be added to improve performance of the grasping task. 1,,3 However, the most commonly available haptic feedback devices either occupy a large amount of space, are unsuitable for simulating grasping, or are expensive. It is our goal in this paper to demonstrate an economical, simple and effective pneumatic haptic interface that is able to enhance operators perception of their environment for grasping tasks. We will present our innovative design for a haptic grasping device as well as some experimental results showing how that device positively impacts grasping performance. One of the best known haptic interfaces, the PHANToM from SensAble Technologies, provides excellent touch feedback, but is unusable in grasping applications. Many types of pneumatic displays have been developed as well, but most of them are actuated by expensive, custom pneumatic cylinders or are unable to simulate grasping feedback.,5,6 Student, Dept. of Aeronautics and Astronautics, 77 Massachusetts Ave., Student Member. Student, Dept. of Aeronautics and Astronautics, 77 Massachusetts Ave., Student Member 1

2 There have also been a number of haptic feedback systems actually suited to grasping tasks. The Rutgers Master II glove provides excellent grasping stimulation, and is relatively small. 7, However, it is actuated by a series of pneumatic cylinders; excessive for the simple, but important, task of grasping. Another pneumatic haptic feedback system suitable for grasping feedback was the TELETACT commercial product, formerly offered by Intelligent Systems Solutions, UK. 9 This haptic feedback device utilized Lycra air bladders to provide feedback, but was eventually withdrawn from the market due to problems with air bladder degradation and failure. However, no studies were ever performed to examine the influence of the TELETACT glove on grasping. A number of other haptic feedback devices geared towards grasping have also been designed or implemented, but their effect on grasping performance has not been investigated. 10,11,1 Possibly the most relevant study to our investigation was performed at the University of Salford, UK. An excellent multi-modal feedback glove was constructed, with which the investigators were able to show that air bladders are a appropriate form of actuation for low frequency actions (less than 100 Hz) such as grasping. However, this study focused on the implementation of multi-modal feedback, not the application of feedback to any specific task. 13 In response to the lack of cheap, simple haptic interfaces designed for grasping, we have developed the Pneumatic Haptic Interface for Grasping, or PHIG. Our system is inexpensive to manufacture and not cumbersome to use or transport. It also is able to greatly enhance an operator s perception of a virtual (and by extension remote) grasping environment. II. The PHIG Apparatus The PHIG apparatus serves as the interface between the user and the virtual world. The complete experimental apparatus has two main parts: the PHIG and the virtual world plus haptic control program. The PHIG is made up of the pneumatic apparatus, the drive electronics, and the telemetric glove. The virtual world drives the PHIG over a simple RS- 3 serial control line. Currently the virtual world and haptic control program are implemented in Matlab using Simulink and the Virtual Reality Toolbox. The PHIG is completely decoupled from this implementation and is fully capable of being driven by a standalone program. The pneumatic apparatus is the portion of the system which provides the tactile feedback to the user. A set of black, natural rubber bladders are located inside the telemetric glove, in contact with the bottoms of the user s pointer and thumb. The bladders lie flat when deflated and expand relatively isotropically when pressure is applied. The bladders are connected to pressure sensors as well as valved supply and exhaust pressure lines. The pressure sensors, the binary pneumatic valves and the control software through a RS-3 link are all connected to a Field Programmable Gate Array (FPGA), which directs all of the drive electronics. The FPGA is programmed with a simple proportional pressure control system. The pneumatic valves are directly controlled with a pulse width modulation scheme, which allows them to adequately simulate proportional control for the time scales of interest. Figure 1: Telemetric Glove with Bladder Locations Marked

3 A model Data Glove 5 from 5th Dimension Technologies telemetric glove provided bending information on the user s finger position to the virtual world. Figure shows the telemetric glove with approximate bladder locations marked. The command input to the FPGA is received from the virtual environment. Its value is based on the percentage overshoot of the users actual finger position versus the portion of space occupied by the virtual object. The air bladder-based approach to pneumatic haptic feedback presents a few practical advantages. It is physically smaller and less bulky than even systems which utilize miniature pneumatic cylinders to provide feedback. It is also very simple and economical to manufacture. However, in its current configuration, with only one bladder cell per finger, the PHIG is strongly optimized to provide feedback for grasping actions only. III. Experiment Description The purpose of this experiment was to collect position and timing data on a user attempting to grasp a virtual cylinder, both with and without haptic feedback. Unfortunately due an oversight in the experimental protocol design, the timing data was rendered useless. Data was collected from a total of seven subjects, who each performed six trials, for a total of sets of data. Another two test subjects were used to gather an additional 1 sets of data for a pilot study to verify correct operation of the experiment. Haptic feedback was enabled for three trials and disabled for the other three trials for each subject. A. Experimental Protocol Task: The task in this experiment consisted of grasping a virtual cylinder with two block-like, linear motion grippers. The virtual world is displayed in Figure. The cylinder is meant to simulate an object that is hard, brittle and breakable. The situation is intended to simulate, for instance, a robotic arm remotely grasping something with material properties like a thin-walled ceramic cylinder. In order to mitigate learning effects, the starting distance of the gripper-blocks was randomly varied for every trial. Three different distances were used, each of which appeared twice. In this same vein, the order of the haptic vs. non-haptic trials was also randomly varied. Procedure: For every trial, subjects were seated in front of a laptop running the virtual world simulation and fitted with the PHIG apparatus. A basic introduction to the system and how it operated was then provided to each subject. Subjects were then told the scenario: they are controlling a remote robotic arm attempting to grasp an object. It was explained that the cylinder about to be grasped should be regarded as brittle and breakable. Figure : The Virtual World Subjects were informed of the success criteria, grasp the cylinder tightly enough that they would feel comfortable lifting it, but not so tight as to damage it. The trial was halted by the subject when he or she felt that the success criteria had been met. Subjects were told to move somewhat deliberately in order to compensate for deficiencies in the pressure controller response. However, it became clear that this was not a significant detriment to the experiment, 3

4 as participants tended to not reach the update rate limits of the controller in the normal course of grasping. This was especially true after they were familiarized with the system in the training phase. Subjects were not told to acheive the task as quickly as possible or under any sort of time limit because of our oversight. Because of this all of the timing data that was collected was invalid, since subjects were under no time constraints. If more time had been available to run more trials, the experimental protocol would have been modified to yield valid timing data. The subjects were also familiarized with the equipment in a brief training phase. For the first half of training, haptic feedback was disabled. Subjects then practiced with the telemetric glove in order to familiarize themselves with the correlation between finger motion and the motion of the boxes in the virtual environment. The haptic feedback was turned on for the second half of the training to familiarize the subject with the feel of grasping the virtual object. At the beginning of each trial the subject had to make a fist three or more times in order to complete the automatic calibration and scaling procedure of the telemetric glove. Once that was completed, the trial was run and all relevant data collected. IV. Experimental Results Data was collected in order to evaluate the effect of haptic feedback on force control and grasping success. The variables recorded were finger position from the telemetric glove and time to complete the task. Age and gender data collected from all subjects was analysed with a t-test to confirm that neither of those factors influenced the results. A second t-test was performed to demonstrate that changing the position of the grippers did not significantly impact the time to complete the task. A. Grasping Success The success variable has three distinct states: not grasped, grasped and damaged. Success is determined by the position of the user s fingers relative to the cylinder. If their finger positions are from 0 to 5 percent past the surface of the cylinder, it is not grasped; 5 to 0 percent is considered grasped; and anything past 0 percent is 0 considered damaged. 1 There were two kinds of success measured for each 16 trial. For cumulative success, the cylinder was considered damaged if the damaged criterion was ever met dur- 1 1 No Haptic 1 Haptic 11 ing the trial. Figure 3 shows the raw counts for cumulative 13 success. 10 Final success was determined by the state of the object at the end of the trial. Figure shows the raw 6 counts for final success. Two different metrics for success were used in order to case where a subject reacts to haptic feedback releasing his or her grip, which causes the final position to differ from the maximum 0 position. Not Grasped Grasped Damaged The success counts were created using the average positions of the thumb and pointer in order to compensate Cylinder State for the fact that it was possible for the thumb and pointer to have different success states. This was justified Figure 3: Cumulative Success Counts by the equalizing nature of the forces applied in a real grasping situation. Statistical Analysis: A chi-squared test for independence was used to analyse both types of grasping success data. Any result under 5% probable was considered statistically independent. The position values from trials without haptic feedback were used as the expected values of the chi-squared test.the test re- Number of Trials

5 turned a result of.1% probable for cumulative success and 1.% probable for final success, demonstrating the statistical significance of the results. B. Force Control Force control is defined as (position final position maximum ). It is a measure of the subjects ability to realize that they are grasping too firmly and correct for that. All force data was trimmed in order to remove the initial pre-trial calibration data. An example plot of force control data from one trial with haptic feedback enabled and calibration data removed is shown in figure 5. The dashed horizontal lines indicate the range in which the cylinder is considered grasped. The force control data is the actual position of the thumb and pointer fingers. Statistical Analysis: A two-tailed, paired t-test with a threshold of 0.05 was used to analyse the force control data. The test showed a result of 0.037, which is statistically significant. In order to determine whether more force control was demonstrated with or without haptic feedback, the averages of the data points from the sets were examined. These showed that subjects ability to control force increased, on average, by 05%. Number of Trials No Haptic 1 Haptic 9 5 Not Grasped Grasped Damaged Cylinder State Figure : Final Success Counts V. Impact of Haptic Feedback on Grasping The previous section demonstrated that the presence of haptic feedback causes a statistically significant change in a user s ability to grasp a virtual object. In this section, we will attempt to explain our results as well as discuss work that they suggest for the future. A. Grasping Success The counts of success show that subjects respond to haptic feedback by becoming more successful at grasping, but also that they have a greater tendency to damage the object. Conversely, without haptic feedback subjects were more likely to fail to grasp the object. It is our belief that this phenomenon is explained in two ways. First, haptic feedback actually does increase the user s perception of a remote or virtual grasping Position Ungrasped Grasped Damaged Damaged Grasped Ungrasped Time Pointer Thumb Figure 5: Thumb and Pointer Position task. However, in this case there was a sensory conflict between the type of haptic feedback presented to the users and what they had been told to expect. The soft, pliant nature of the rubber air bladders versus the hard, brittle object that users thought they were grasping may have led to users overgrasping. In reality, users would grasp until the haptic feedback system gave a very strong result, which was unfortunately past the point at which the object was considered grasped. It may be possible to compensate for this effect either 5

6 by calibrating the output from the telemetric glove differently, by moving to a nonlinear controller for the airbladderpressure. B. Force Control The force control analysis demonstrates that haptic feedback increases a user s ability to control force in remote or virtual grasping action. This shows that the PHIG system is extremely useful for enhancing the perception of an operator performing a grasping task. C. Future Work A number of improvements are suggested for the next iteration of the PHIG system, based on lessons learned from this initial design. The pressure control system used was not ideal. The current system lacked somewhat in response time, though it was sufficient to successfully complete the experiment we performed. The largest question remaining is whether or not haptic feedback affects a user s speed in grasping? We believe that it does, but the experimental protocol needs to be changed in order to emphasize speed to the test subjects, so that support can be built for this hypothesis. Two major improvements would greatly improve the current PHIG apparatus. First, a faster controller and pressure response system would be useful to lend a more natural feel to the grasping action. The other major improvement is in the bladder design. The current natural rubber bladders are adequate but difficult to manufacture. We believe that a bladder cast in silicone rubber would be much easier to manufacture and generally be superior to the current solution. VI. Contributions In this paper, we have made three major contributions: We have presented a design for a innovative, economical and portable haptic feedback system integrated into a telemetric glove. We have demonstrated a statistically significant change in grasping success in the presence of haptic feedback (as well as illustrating a possible problem in simulating material properties in a haptic feedback system). Finally, we have shown a statistically significant increase in the presence of haptic feedback in the user s ability to control grasping force. In order to successfully complete any task, it is desireable to engage as many of the senses as possible. Our tactile feedback system is able to provide another channel of information specifically for grasping tasks. It is our firm belief that a small, portable and cheap haptic feedback display of similar design to PHIG can be easily integrated into most remote grasping devices to the benefit of the user s performance at grasping. References 1 Rochlis, J. L., Clarke, J., and Goza, S. M., Space Station Telerobotics: Designing a Human-Robot Interface, Conference and Exhibit on International Space Station Utilization, AIAA, Washington, DC, 001. Ambrose, R., Culbert, C., and Rehnmark, F., An Experimental Investigation of Dexterous Robots using EVA Tools and Interfaces, AIAA Space 001 Conference and Exposition, AIAA, Washington, DC, Parrish, J. C., Sullivan, B. R., and Roberts, B. J., Planning for the Ranger Telerobotic Shuttle Experiment On-orbit Operations, AIAA Space 000 Conference and Exposition, AIAA, Washington, DC, 000. Takaiwa, M. and Noritsugu, T., Development of Pneumatic Human Interface and Its Application for Compliance Display, Industrial Electronics Society, 000. IECON th Annual Conference of the IEEE, Faculty of Engineering, Okayama University, Japan,

7 5 Bouzit, M., Burdea, G., Popescu, G., and Boian, R., The Rutgers Master II-New Design Force-Feedback Glove, Proceedings IEEE/ASME Transactions of Mechatronics, Vol. 7, No., Rutgers University, New Jersey, Moy, G., Wagner, C., and Fearing, R. S., A Compliant Tactile Display for Teletaction, Proceedings of the 000 IEEE International Conference on Robotics & Automation, Department of EE&CS, University of California, Berkeley, CA, Richard, P., and Coiffet, Ph., Dextrous Haptic Interaction in Virtual Environments: Human Performance Evaluations, Proceedings of the 1999 IEEE International Workshop on Robot and Human Interaction, Laboratoire de Robotique de Paris (CRIIF-LRP), France, Gomez, D., Vurdea, G., and Langrana, N., Integration of the Rutgers Master II in a Virtual Reality Simulation, Proceedings of IEEE Vrais 95 International Symposium, Research Triangle Park, North Carolina, 1995, pp Biggs, J., Srinivasan, M.A., and Kay, M.S.(ed.), Haptic Interaction, Handbook of Virtual Environment Technology, Lawrence Erlbaum Associates, Inc., 001, Chapter Kawai, M., and Yoshikawa, T., Stable Haptic Display of 1-DOF Grasping with Coupling Impedance for Internal and External Forces, Proceedings of the 000 IEEE/RSJ International Conference on Intelligent Robots and Systems, Department of Mechanical Engineering, Kyoto University, Japan, Maekawa, H., and Hollerbach, J. M., Haptic Display for Object Grasping and Manipulating in Virtual Environment, Proceedings of the 199 IEEE International Conference on Robotics & Automation, Department of Computer Science, University of Utah, Utah, Wannasuphoprasit, W. and Chanphat, S., A Novel Fluid Haptic Interface, 00 IEEE International Conference on Industrial Technology, Department of Mechanical Engineering, Chulalongkorn University, Thailand, Caldwell, D.G., Lawther, S., and Wardle, A., Multi-Modal Cutaneous Tactile Feedback, Proceedings of IROS, Department of Electronic Engineering, University of Salford, UK,