the almost identical time measured in the real and the virtual execution, and the fact that the real execution with indirect vision to be slower than the manipulation on the simulated environment. The particular master used in this experiment is characterized by extremely low values of inertia and friction 1. The mechanical characteristics of the master manipulator and the fact that it may not be necessary a very high bandwidth of force reflection for the execution of such easy tasks may explain why we did not observe any degradation in the performance when the operator had to interact with the simulated environment using the master device instead of directly touching the objects. The graphic representation of the virtual environment is an isometric projection where the shadow of the end-effector is projected onto the virtual plane to provide the subjects with a 3D illusion [Kim et al., 1987]. Looking at the 2D display used to provide visual feedback during the direct manipulation, it was very difficult to perceive the vertical position of the tip of the pen relative to the horizontal plane (see Figure 7 and Figure 9). Even if the master and slave used in this experiment were direct-drive, low friction manipulator, and therefore they could, alone, provide a very fast response, when coupled together the noise introduced by the encoder, the delay and quantization of the transmission channel, did not allow us to choose a sufficiently high gain for the position-position controller (the theoretical bandwidth was about 5-10 Hz). Moreover, we did not exactly solve the inverse kinematics of the slave manipulator. Conclusion The experimental procedure described in this paper can be a useful tool for the characterization of a master-slave and virtual environment perfromance. The possibility of testing single part of a remote manipulation system permits to identify which area of the design needs more improvement. The experimental data here reported are just a preliminary test run on the physical setup, and much more remain to be done. In particular: the modeling of the virtual environments needs to be improved to simulate a more realistic behaviour. For a truly realistic replica of a real environment it would be necessary to model objects that are deformed when manipulated and to take in account rotations and moments; the graphic display in the virtual and remote manip- ulation should provide the some 3D cue to the subjects; it would be interesting to determine how much of performance degradation when the remote manipulation is performed is due quantization phenomena and delay in the communication channel, to the bilateral control scheme used, or to the particular slave manipulator. Bibliography P.Buttolo, B.Hannaford, Pen-Based Force Display for Precision Manipulation in Virtual, Proc. IEEE Virtual Reality Annual International Symposium, North Carolina, March 1995a P.Buttolo, B.Hannaford, Advantages of Actuation Redundancy for the design of Haptic Displays, Annual symposium on Haptic Interfaces for Virtual Environemnts and Teleoperator Systems, October 1995b IMECE P. Buttolo, Modelli e Strutture di Controllo per Telemanipolazione con Riflessione di Forza, Doctoral Dissertation, University of Padua, February 1995 B. Hannaford, Kinesthetic Feedback Techniques in Teleoperated Systems, In Advances in Control and Dynamic Systems, pp 1-32, C. Leondes, Ed., Academic Press, San Diego, 1991. D.Y.Hwang, B.Hannaford, Modeling and Stability analysis of a Scaled Telemanipulation," Proceedings RO-MAN 94, Nagoya, July, 1994. W.S.Kim, S.R.Ellis, M.E.Tyler, B.Hannaford, L.Stark, "Qu antitative Evaluation of Perspective and Stereoscopic Displays in Three-Axis Man ual Tracking Tasks," IEEE Systems, Man and Cybernetics, vol. 16, pp. 61-72, 1987. K. Kosuge, K. Takeo, T. Fukuda, T. Sugiura, A. Sakai, Y. Yamada, Unified Approach for Teleoperation of Virtual and Real for Skill-Based Teleoperation, Proc. Intelligent Robots and Systems (IROS 94), Vol 2, pp.1242-1247, Munich, Sept. 1994. P.H. Marbot, B. Hannaford, "Mini Direct Drive Arm for Biomedical Applications," Proceedings of ICAR 91, pp. 859-864, Pisa Italy, June 1991. M.R. Moreyra, Design of a Five Degree of Freedom Direct Drive Mini-Robot Using Disk Drive Actuators, Masters Thesis, University of Washington, Department of Mechanical Engineering, 1994. L.B. Rosenberg, Virtual haptic overlays enhance performance in telerpresence tasks, Proc. SPIE Telemanipulator and Telepresence Technologies Symposium, pp 99-108, Boston, October 31, 1994 H. Tokashiki, P. Akella, K. Tanie, Scaled bilateral telemanipulation: An Experimental investigation of scaling laws, Proc. SPIE Telemanipulator and Telepresence Technologies Symposium, pp 214-224, Boston, October 31, 1994 1. the static friction is less than 0.01N in the worst case, the inertia in the cartesian frame is less than 10 gr in the worst case 6
35 average time needed for the completion of the task time [s] 30 25 20 15 10 5 0 1 2 3 4 5 6 7 8 9 10 remote + 2D display remote direct + 2D display virtual manipulation direct manipulation 7 6 standard deviation [s] 5 4 3 2 1 remote + 2D display remote direct + 2D display virtual manipulation direct manipulation 0 1 2 3 4 5 6 7 8 9 10 task Number Figure 8 Average time recorded for 5 different manipulation modalities. We decided for this graphic representation instead of an histogram because the lines passing through the experimental points give a qualitative ideas of the similarities between the shape of the results in the different configurations. The first two continuous line in the top figure, starting from the bottom, almost coincident, represent the time measured in the direct manipulation and in the virtual manipulation. The third line is the time measured in the direct manipulation and vision through a 2D display. The fourth and fifth line are relative to the remote manipulation, without and with a 2D display. Experimental Results A first run of experiments showed some interesting preliminary results (Figure 8). The task number 10 in figure is the average of the 9 tasks measured time. The time needed for the completion of the tasks in the direct and virtual manipulation were almost the same. The time range from a minimum of 3 sec (task 7) to a maximum of 10 sec (task 6). The differences between the recorded time range from less than 5% for tasks 1,2,3,5,8, to about 30% for task 4 and 7. The execution of the tasks in the virtual environment was faster than the direct execution of the tasks looking through a 2D display. The presence of the 2D display increased the average time by about 30%. The average time increased from 5 sec to 6.5 sec. The standard deviations do not differ significantly from the one calculated in the previous two cases. The remote execution of the tasks was significantly slower than the direct and virtual execution. The time measured were about 100% and 130% of the time needed for direct execution, without and with a 2D display respectively. The average time are 10 and 12.5 sec, respectively. Discussion Some of the results were unexcpected. The particular hardware and software implementation may account for Figure 9 Visual feedback to the operator from the 2D display. It is not as easy as in the graphic representation in to perceive the position of the tip of the pen along the vertical axis 5
PBFD position determination (encoders) PBFD dynamics force display (actuators) VR graphic representation collision detection object positions are determined using the second order non-linear model of eq(1), whose parameter are the object inertia and the coefficients of static, coulomb and dynamic friction. In Figure 6, a schematic model of the implementation of the virtual simulation is represented. Description of the experiment virtual objects position update virtual objects dynamics Figure 6 Block diagram of the implementation of the virtual simulation. Each object can interact with the end-effector of the pen-based-force-display or any other object. In our experimental procedure a group of 12 subjects was asked to repeat a sequence of nine simple tasks, probing small parts and fixtures with pen-like tools, with a virtual reality simulation and remotely with a master and slave system. The nine tasks were the following: tip the four corners of a 5mm square drawn on a plate; slide the pen along the four sides of the square without losing contact; tap a certain sequence of points on the plane and on an L-shaped object; follow the contour of the L-shaped object; move a nut from the bottom-left corner to the topright corner of the environment graphically represent in ; follow the contours of three different fixed objects trying not to lose contact; kick a nut in between two poles; touch and press against the middle point of the four sides of a square made out of a rubber band; follow the contour of the elastic square trying not to lose contact; We did not include tasks that would require operation of pick and place because when the experiment was performed the slave robot did not have a gripper. The tasks specified, even if they are not representative of complex manipulations, allow us to characterize the following modes of operation: free movement; shape recognition; application of modulated forces; application of impulsive forces; sliding of objects on a plane; We asked each subject to execute the task at the maximum speed he was feeling comfortable, without committing errors. The time required for each operation was recorded. Remote manipulation was tested allowing the operator both direct vision of the mini setup and indirect-vision using a monitor. As mentioned before this experiment can be effective in decomposing the effects of the single components of the telemanipulation system on the overall performance: the time necessary for the direct execution can be thought of as a natural performance rate of the experiment. the time required to complete the virtual manipulation gives us a measure of how much the performance degrades when the human interacts through the monitor and the force display. This is true if the real time implementation of the model equations of the virtual objects is a good approximation of the real behavior. the time difference between the virtual and the remote manipulation is a reasonable measure of the effects of the slave robot and bilateral controller on the overall performances. end-effector shadow projected on the virtual plane Figure 7 Graphic representation of one of the 9 tasks in the virtual environment. The subject was required to move a nut from the bottom-left corner to the top-right corner of the environment. The end-effector shadow is projected on the virtual plane to give the subject a 3D cue. 4
Software Implementation of the Virtual The quality of the virtual simulation depends both on the particular mechanical interface and on the equations used to model the dynamics of the virtual objects. If the equations are a sufficiently good replica of the real dynamics, the only difference in the feeling perceived in the real and the virtual manipulation is due to the fact that in the second case the operator is forced to touch the environment using a mechanical interface. In our system we didn t implement the full objects dynamics. Some of the properties we didn t model are: it is not possible to apply moments, therefore objects cannot rotate and there is no torque feedback to the operator; objects cannot be deformed, even if it is possible to simulate spongy objects. Besides that, only a limited number of shapes has been implemented, such as cylinder, cubes, nuts and regular polygons. The object i dynamics is described by the following equation: F a Model of the friction along one dimension. We took in account three different component: the static friction (black point), the coulomb friction (dashed line) and the dynamic friction (dash-dot line). The resultant friction is represented by the continuous line. Figure 4 elastic component ẋ anelastic component F oper, i F ji, + F plane, i j + = m i ẋ i + m i a i eq(1) where: F oper, i = force applied by the operator to the object i through the end-effector; F ji, = interaction force between the objects i and j; F plane, i = interaction force between the virtual plane and the object i; m i = object mass; ẋ i = object acceleration; m i a i = virtual gravity force acting on the object; To obtain a faithful replica of the real behavior, we modeled the friction present in the virtual environment including both linear and non-linear phenomena (see Figure 4). The static friction and the coefficient of coulomb and dynamic friction were experimentally measured in the real physical environment. The friction were introduced both to simulate resistance against relative sliding motion between different surfaces and to simulate inelastic collisions. In Figure 5 the springs model the elastic component of the collision, the dampers model the inelastic component. The interaction force between objects depends on the physical properties of the object, such as stiffness and friction, and on the relative position and velocity. A collision between the objects i and j is represented by object i object j Figure 5 Model of collision between different objects. Each object is represented as a combination of a spring, that models the elastic part of the collision, and a dumper, that models the inelastic component. the following equation: K i K j, = ---------------- ( x + i x j ) + F ji K i K j F aji,, F aij,, where x i x i is the relative distance between the two objects surfaces, the vector F aji,, F aij,, is the friction model described in Figure 4, and K i, K j are the stiffness of the two objects i and j. A similar equation has been used to model the collision between the end-effector and the object i F oper i, = K i ( x i x em ) + F ai, The end-effector of the master manipulator is modeled in the virtual environment as a rigid body having zero intertia, smooth surfaces and infinite stiffness. The end-effector position is determined applying the direct kinematics equation to the encoder readings. The 3
Because some of the conditions (see software implementation and experimental results) required to justify the validity of this experimental procedure were not fully satisfied during the execution of this preliminary series of trials, we will put less stress on a rigorous analysis of the data obtained. Hardware Setup The telemanipulation system used in this experiment consists of a 3 Dof, direct drive, parallel, master manipulator [Buttolo & Hannaford, 1995a], [Buttolo & Hannaford, 1995b], and a 5 Dof, direct drive, serial slave manipulator [Marbot & Hannaford, 1991], [Moreyra et al., 1994]. The first two joints on the slave manipulator were not used, so that the kinematic joint solution could be uniquely determined. The controller was implemented on a PC486DX2 on the master side and on a custom TMS320C30 board on the slave side. The PC was also used to simulate the virtual environment. The operators were allowed either direct view or remote view of the real environment. For this purpose a camera and a 19 black&white TV were used. The communication between the two sides was obtained through a parallel link. A photo of the two manipulators is in Figure 3. The algorithm used to control the master and slave robot in the remote manipulation of the environemnt was a real manipulation real manipulation 2D display virtual manipulation 2D display remote manipulation 2D display subject natural performance influence of the display on the performance influence of the PBFD on the performance influence of the slave and bilateral controller Figure 2 Schematic representation of the experiment. During each phase a new element is introduced in the system. The difference between the times needed for completion of two adjacent phases can be considered as a measure of the influence of the single element on the overall performance. simple position-position controller scheme [Hannaford, 1991], [Hwang & Hannaford, 1994]. Figure 3 The experimental system is composed of a direct-drive serial manipulator used as a slave (right), and a direct-drive, 3 Dof parallel manipulator (left). The 3 Dof device is used as a master in the remote manipulation and as a force display during the interaction with the virtual environment. 2
Proceedings IEEE conference on System, Man and Cybernetics, vol 5, pp. 4656-61, Vancouver, BC, October 1995 Manipulation in Real, Virtual and Remote s Pietro Buttolo, Darwei Kung, Blake Hannaford University of Washington Seattle, WA 98195 email blake@ee.washington.edu phone (206) 543-2197 Abstract In this paper we describe a novel experimental procedure for the evaluation of a telemanipulator performance. A group of subjects performed the same set of tasks directly on a physical setup, on a virtual implementation capable of providing visual and force feedback through an haptic display, and remotely on the real setup using a telemanipulation system. Using this experimental procedure we were able to decouple the effects on the overall telemanipulator performance introduced by the single components of the system, master manipulator, display, slave manipulator and bilateral controller. Introduction In the last years numerous research projects merged technologies coming from the telemanipulation and virtual reality disciplines. At the MITI a master and slave system has been implemented where the master is a real, one degree of freedom manipulator, whereas the slave and the environment are simulations run by the computer [Tokashiki et al., 1994]. At Stanford University, experiments have been done to determine how much virtual haptic overlays can enhance performance in telepresence tasks [Rosenberg, 1994]. At Nagoya University an operator can try a task on a virtual setup before executing the remote manipulation [Kosuge et al., 1994]. Virtual reality and telemanipulation are closely related. In the first case the operator interacts with a virtual simulation running on a computer. The operator receives visual feedback from a display and sometimes haptic feedback provided by a manipulator (force display or master). In the second case the operators interacts with a remote environment. Besides the interface with the operator, that as in the virtual simulation consists of a display and a master manipulator, the presence of a slave manipulator and a bilateral controller allows the physical manipulation of the real, remote environment (). We took advantage of these similarities to characterize the performance of a telemanipulation system. Considering the time required for the completion of a specific Operator Operator Operator Force Display Real manipulation Monitor Virtual manipulation Monitor Master bilateral controller Slave Remote manipulation Figure 1 Virtual Remote Real, virtual and remote manipulation. The operator can perceive feelings by directly touching the real environment (top), interacting with a computer simulation through a force display (middle), or by a remote interaction with a master and slave system (bottom) task as a measure of performance, we asked a group of subjects to perform the same set of tasks using direct, virtual and remote manipulation, and we compared the results. In each phase some additional elements are introduced that could be responsible for changes in the measured time (Figure 2). In this paper we will briefly describe the experimental setup used for the experiment, emphasizing the description of the experimental procedure and the analytical implementation of the virtual environment. 1