Operating in Conguration Space Signicantly. Abstract. and control in teleoperation of robot arm manipulators. The motivation is

Transcription

1 Operating in Conguration Space Signicantly Improves Human Performance in Teleoperation I. Ivanisevic and V. Lumelsky Robotics Lab, University of Wisconsin-Madison Madison, Wisconsin 53706, USA Abstract This paper discusses the use of conguration space (C-space) as a means of visualization and control in teleoperation of robot arm manipulators. The motivation is to improve operator performance in tasks involving manipulator motion in complex three-dimensional (3D) environments with obstacles. Unlike other motion planning tasks, operators are known to make expensive mistakes in arm control, due to de- ciencies of human spatial reasoning. The advantage of C-space is that in it the arm becomes a point, a case which humans are much better equipped to handle. To make such operation possible, a tool is proposed that reduces motion in 3D C-space to that in 2D C-space. It is then shown on results from testing 18 human subjects that translating the problem of a three-link 3D arm manipulator motion into C-space improves the operator performance remarkably, by a factor of 2 to 4, compared to usual work space control. 1 Introduction This paper is concerned with the developmentof a human-computer interface which would simplify motion planning tasks encountered in teleoperation of robot arm manipulators. It is a well known fact (this topic has been a subject of several experimental studies [1, 2]) that human operators have a hard time controlling complex jointed devices such as robot arm manipulators. In the past, examples of applications of teleoperation have been reported (e.g. with the NASA shuttle arm) where human operator error resulted in costly equipment damages and failure to complete the task. This is not to say that humans should be eliminated from such tasks: the well-known (if not well-understood) ability of humans to see \the big picture" and to quickly assess changes This work was supported by the Oce of Naval Research Grant N

2 in the environment and modify the task as necessary still makes the human operator an invaluable part of the control loop in most teleoperation tasks. What is needed is machine intelligence tools that would compensate for the shortcomings of human reasoning. In approaching this development task, one needs to clarify rst what specic intelligent skills the machine should be endowed with and how the two intelligences - human and machine's - can be combined in a synergistic fashion. Studies [1, 2] connect operator diculties with peculiarities of human spatial reasoning: humans have diculty handling simultaneous interaction with objects at multiple points of the device's body, or motion that involves mechanical joints (such as in arm manipulators), or dynamic tasks where the eects of inertia and accelerations are essential (such as in assembly or underwater applications). With this in mind, our approach has been to use machine intelligence to reduce the problem to one which humans are known to be good at: experience indicates that one such problem is that of moving a point in a maze. Another task that humans have diculty with and which can be assigned to machine intelligence is avoiding potential collisions. This can be accomplished by continuously checking whether the operator-dictated motion of the arm will result in collision with obstacles, and disregarding or, better, intelligently modifying such commands. Such systems, based on sensitive skin covering the arm, have already been demonstrated in hardware [3]. The focus of this work is to produce a visual interface which would allow the human intelligence to become a contributor to the motion control. We show, in particular, that while human operators nd it nearly impossible not to bump the arm into surrounding objects, they easily achieve real-time collision-free motion using the proposed tools. Our visual human-computer interface transforms the task of controlling a jointed arm manipulator (the work space (W-space) control) to one of control of a point in the corresponding conguration space (C-space). It has been shown [4]that even in a simpler two-dimensional (2D) case the control in C-space improves operator performance rather remarkably, even outperforming the best known algorithms running on modern workstations. Results in 2D, however, are not easily translated to 3D tasks of real-world applications [5, 6, 7]. Such applications can appear in physical or virtual setting, or be presented on a (at) screen. In order to extend known 2D results to a more realistic 3D case, an interface tool is proposed here which reduces the 3D task in hand to a multiplicity of simple 2D tasks. We assume that the C-space representation of the work space is available to the operator. Below, details of the arm model and its C-space are discussed in Section 2, the proposed interface is described in Section 3, and the experimental setup, results and discussion are given in Section 4. 2

3 z θ 1 l 2 θ 2 l 1 l 3 θ 3 P x y Figure 1: The 3D RRR arm manipulator. P 2 The arm Arm Geometry. We consider a three-link three-dimensional arm manipulator with three revolute joints (RRR arm), Figure 1. The arm's rst link, l 1, rotates about a vertical axis, producing joint values 1. Each of the other two links, l 2 and l 3, rotates in a vertical plane; their positions are dened by the values 2 and 3, respectively. For computational eciency links are modeled as generalized cylinders. Note that link l 2 is attached to the side of base l 1, and link l 3 is attached to the side of link l 2, allowing for bigger (or even unlimited) range of joints 2 ; 3. For a more realistic representation, joint values are assumed to be limited by mechanical stops, j i j < 2; i =1; 2; 3. The virtual arm used in the experiments (Section 4) is shown in Figure 2. The arm operates in an environment with stationary obstacles. Conguration Space (C-space). The arm's conguration space (C-space) is formed by all possible combinations of the values of the triple ( 1 ; 2 ; 3 ) each of which denes a position of the arm in the work space. Those positions of the arm that are not permissible due to obstacles or due to the range limits form the C-space obstacles (see e.g. [8],[9]). Since in our scheme the operator will control the robot in C-space, the latter has to be 3

4 Figure 2: A sample task in W-space (Task 1 in Section 4). computed rst. A position in W-space in which the arm touches some obstacle maps in C-space uniquely into a point on the surface of the corresponding obstacle image. The set of all positions in W-space in which the arm touches one or more obstacles will form the surface(s) of C-space obstacles. The said mapping is not linear and so C-space obstacles look very dierent from W-space obstacles. This means the operator will in general see a rather unfamiliar picture, which in principle is a disadvantage. One useful property of C-space obstacles is that, assuming 3 axis corresponds to the \up" direction in C-space, they tend to have the general shape of pillars (see Figure 3; here axes q i correspond to our i ). This fact is exploited below in the design of our interface. 4

5 Figure 3: Three-dimensional C-space of the sample task in Figure 2. 3 Visual Feedback and Control We can now compare two types of visual feedback that that one can realize { in W-space or in C-space. Their use in tests with human subjects is discussed in Section W-space Feedback and Control This scheme corresponds to the commonly used work space control. The user is presented with an image of the three-dimensional work space (that is, its projection on the at screen), complete with the arm and any physical obstacles in the scene. See an example in Figure 2: the darker object is the arm, the lighter object { a relatively complex obstacle. The starting position of the arm is shown in solid, and its target position is shown as a skeleton arm. The user can manipulate the W-space image interactively, including zoom in and zoom out and changing the position of the \camera" (i.e. the viewer's position relative to the 5

6 Figure 4: The master arm used as an input device in W-space. scene). (Note that such exibility of control is feasible in virtual scenes but is not likely to be reproducible in a physical system because it would require either a very large number of cameras or a possibility of an arbitrary positioning for the viewing camera(s)). In an attempt to provide a good interface, an input device has been built, Figure 4, which presents an arm resembling the one on the screen. The operator can use this master arm to guide the virtual slave arm on the screen while receiving continuous visual feedback from the \camera" 1. Given the diculty the operators have with arm control in the vicinity of obstacles, additional visual feedback means have been provided, which inaway simulate a haptic interface. A small bright sphere appears on the point(s) of the virtual arm body that comes into contact with an obstacle. Also, if after the contact the operator continues moving the arm \into the obstacle", a skeleton arm gure will follow the actual motion while the solid arm will stay at the point of contact. The operators then realize they should move the arm back until arriving at the point of contact and then modifying the motion. 1 We considered other types of input devices { mouse and keyboard in particular { and found them to be less eective when dealing with a 3D environment. One might also consider a haptic interface that would provide tactile feedback from collisions between the virtual arm and virtual obstacles. Accomplishing this would not be easy: today there are no haptic interfaces { and principles on which they could be built are unclear { capable of \sensing" contacts with objects at any point of the virtual arm body, and transmitting this information to the operator in a meaningful way. 6

7 Figure 5: Slicing of the 3D C-space in Figure 3. This also corresponds to the W-space in Figure C-space Feedback and Control As explained above, transforming the problem of moving a complex jointed arm (in W- space) to that of moving a simple point (in C-space) makes the task much easier for the humans. But, another problem, which did not appear in the two-dimensional case [4], appears in the 3D case. Note that for non-point objects various means (e.g. perspective projections, shading etc.) can be used to help estimate the object's location. These are not useful for a point: when moving a point on the screen, it is very hard to see where it is located in 3D space. A no less serious problem is that the operator cannot see most of the interior of C-space obstacles, which tend to be cave-like structures, with multiple 7

8 entrances, branches and internal dead-ends (see Figure 3). Attempts to \go inside" those caves would likely lead to an exhaustive search of the insides of the C-space obstacles. In fact, it may be easier to assess in C-space (rather than in W-space) which paths lead to dead-ends and which lead to the target.) making C-space less eective in such cases. This suggests another mapping, which would transform the 3D C-space to some twodimensional space, while retaining information about C-space obstacles. The mapping introduced here is based on the concept of C-space slicing, which is not unlike the slicing used in earlier work on motion planning [8]. One may notice an analogy with the eld of medicine, where it has been common practice to present doctors with a set of 2D pictures representing dierent slices of a 3D object (e.g. a human brain). The fact that doctors are able to mentally reconstruct a 3D space from this set of 2D slices provides a hope that the same principle could be applied to motion planning. Assume each side of the C-space cube is 2 long; if a joint {say, 1 { has range limits, ( 1min ; 1max ), then the parallelepiped in C-space between 1 = 1min ; 1 = 1max is a range obstacle which, from the motion planning standpoint, is as any other obstacle. [C-obstacles shown in Figure 3 represent only physical obstacles; range obstacles are omitted]. We slice C-space bottom up, along the 3 axis; the result is a number of squares each representing a 3 slice. Each side of each square represents the 2 range of 1 ; 2, respectively. It may include slices of regular C-space obstacles and range obstacles if any. If the range of 3 is 3min ; 3max and the slice thickness is 3, then the number of slice squares is m =( 3min, 3max )= 3. Smaller 3 will results in better resolution and bigger m. The maximum m is determined by the screen size and operator's convenience. Figure 5 shows the slice mapping of C-space of Figure 2. Rectangular shaded stripes in the squares and fully shaded squares in the middle of the gure correspond to the range obstacles, more \irregular" shapes { to normal C-obstacles. The 100 squares shown result in a resolution of 3.6 degrees along the 3 axis. In the gure, the bottom left corner corresponds to 3 = 0 and the upper right corner to 3 =2, 3 { which means these slices are actually neighbors. In each row the increase in 3 corresponds to the slices going from left to right; that is, the leftmost slice in row n follows the rightmost slice in the row n, 1 and vice versa. S and T mark the start and target positions corresponding to those in Figure 2. When working in C-space, the operator moves the point representing the arm's current position within a slice or between neighboring slices. One can also try to jump over a number of slices { the computer will allow such moves if the (C-space) straight line between the current and intended positions does not cross any obstacles. To increase visibility of details (e.g., to look at narrow openings), one can zoom-in at a given slice by clicking the mouse (this makes the slice occupy the whole screen) and zoom-out back when needed. As mentioned in Section 2, C-space obstacles tend to have the general shape of pillars. This explains the choice of slicing along the 3 axis (as opposed to 1 or 2 or some arbitrary axis). with slicing along 3, neighboring slices tend to be more similar in appearance - i.e. when slicing higher or lower across a pillar you tend to see the same general shape in the cross-section. This property of similarity is important in that it allows the operator to 8

9 Figure 6: W-space experiment, Task 2 (Section 4). formulate a plan of motion and mentally carry it from slice to slice along an intended path, to see if it will accomplish the task. In other words, similar to a bird-sight view of a maze, the operator will not actually perform the motion until they're certain it is likely to solve the problem. If each slice were vastly dierent from its neighbors, the discontinuity would make it hard to keep track of the change from slice to slice, and thus it would be impossible to do this kind of advance planning without performing any motion. It is clear that in more or less complex tasks such planning is not possible in W-space { the objects being moved are far too complex and their imaginary motion is nearly impossible to visualize. Experimental studies [1, 2] conrm that in practice human operators usually try to solve the bigger problem in smaller stages that are easier to plan. This often results in backtracking when it becomes obvious that previous stages led to an unfavorable outcome. 9

10 Figure 7: C-space experiment, Task 2. This also corresponds to the W-space in Figure 6. On the other hand, an operator working in C-space can carry out the imaginary motion from start to target in a succession of smaller tasks (e.g. nd a way out of or into a hole), never moving the actual arm until it is certain that the contemplated path will lead to the target. To realize this type of advanced planning, the operator needs some training to get familiar with the use of C-space; in our experiments some attempt was made to assess the eect of more or less training on the operator performance. 4 Experimental Results and Discussion In the experiments, 18 human subjects have been tested in the same motion planning tasks with a three-link RRR arm manipulator. The objective was to compare the subjects' 10

11 performance when working in the work space with that in the conguration space. For W-space, the interface described in Section 3.1 was used, and for C-space { the interface presented in Section 3.2. Each subject was tested on two tasks { an easier Task 1, Figure 2, and also Task 2, Figure 6, which was meant to be more of a challenge to human spatial reasoning 2. Table 1: C-space, Task 1 statistics for 18 subjects Mean Min. Max. St.Dev. path len time, sec Table 2: C-space, Task 2 statistics for 18 subjects Mean Min. Max. St.Dev. path len time, sec The experiments consisted of two parts completed on separate days, with Part 1 (planning in C-space) followed by Part 2 (planning in W-space). The reasoning behind this order of experiments is as follows. If, as expected, the subjects' performance in C-space is indeed better than in W-space, those good results would be more convincing if they were obtained in the very rst set of experiments. That is, if those same results were obtained in the Part 2 of the experiment, one could argue that better performance in C-space could have been the result of experience and memorization of the environment acquired in the (Part 1) W-space experiment. With our C-space-then-W-space-experiment order, better performance in C- space would suggest that operation in C-space is indeed more eective than in W-space. Table 3: W-space Task 1 statistics for 18 subjects Mean Min. Max. St.Dev. path len time, sec Part 1. The task is to plan arm motion from the starting to the target position in C- space, using the C-space interface. Each subject completed Task 1 shown in Figure 5, and then a more complex Task 2 shown in Figure 7. Prior to the test each subject went through a training period of about 15 min during which they would familiarize themselves with the interface and solve a simple training example. Of the whole group of 18 subjects, 3 subjects received more extensive training: this was to be used for assessment of the eect of training. 2 One may see resemblance between the shapes of obstacles in the experiments (Figure 2) and the support trusses in the NASA space station project. 11

12 Figure 8: A typical W-space path produced using the C-space interface in Task 1. Table 4: W-space, Task 2 statistics for 18 subjects Mean Min. Max. St.Dev. path len time, sec Subjects' performance was measured by the length of the path produced, determined as the integral of changes in the three joint values ( 1 ; 2 ; 3 ), and the time taken to complete the task. Tables 1 and 2 summarize the subjects' performance in C-space, in Task 1 and Task 2, respectively. Figures 8, 9 show typical performances in Task 1 and Task 2, respectively, when translated back to the W-space. Dark lines show the path of the arm endpoint; also shown is the projection of the path on the workspace oor (xy plane). Part 2: Here the subjects were asked to solve the problem in W-space, using the W-space interface. Once again, the subjects received 15 min of training during which they would solve a simple training example (again, the same three subjects received more extensive training, see above). Each subject would then complete Task 1 shown in Figure 2, followed bytask 2, Figure 6. As before, subjects' performance was measured by the path length and time to completion. Tables 3 and 4 show the subjects' performance in W-space for Task 1 12

13 Figure 9: A typical W-space path produced using the C-space interface in Task 2. and Task 2, respectively. Figure 10 shows a typical example of performance in W-space in Task 1, and Figure 11 shows one for Task 2. Discussion: Note a remarkable improvement in subjects' performance in C-space compared to W-space. As seen in Tables 1-4, for Task 1 the mean length of the paths produced in C-space is roughly 2.5 times better (shorter) than in W-space; for Task 2 it is more than 3.5 times better than in W-space. Furthermore, both the best and the worst performances of the subjects are better in C-space than in W-space (see the min and max columns of Tables 1-4). Even more interestingly, in the more dicult Task 2 the best (shortest) path in W-space is still worse that the worst (longest) path in C-space. For the same subject, only in one instance, in Task 1, one subject performed (marginally) better in W-space than in C-space (producing a path of length 5.21 as compared to 5.59, respectively). In Task 2 the same subject did signicantly better in C-space than in W-space. 13

14 Figure 10: The typical W-space path produced using the W-space interface in Task 1. One parameter that was not measured consistently which could be of great interest in applications, was the number of times the arm hit an obstacle during the task execution. Based on our observations, this number was much higher when working in W-space than in C-space. Overall, the subjects found it much easier to work, and they performed admirably, in C-space compared to the W-space. This dierence was even more evident with the three subjects who received longer training with both interfaces. For these subjects, in C-space Task 1, the mean path length was 1.52 and the mean time 86.3 sec. For the same task in W-space these numbers were 4.36 and sec, respectively. When considering those numbers, one should take into the account that the length of the optimal path for Task 1 is about In C-space the three well-trained subjects produced near optimal paths. On the other hand, when working in W-space, their performance was signicantly worse than that of an average little-trained 14

15 Figure 11: The typical W-space path produced using the W-space interface in Task 2. subject (with 15 min training) in C-space, albeit signicantly better than that of littletrained subjects in W-space. In C-space Task 2, the well-trained subjects' mean path length was 1.63 and the mean time was 92.7 sec. For the same task these numbers in W-space were 7.77 and sec, respectively. Note that the path length 7.77 is only marginally better than the mean of all subjects taken together { training did not help the well-trained subjects in this case. In this more complicated task, the optimal path length is about Here the improvement in performance in C-space (as compared to W-space) is even more dramatic. This conrms other results [1, 2] which suggest that training in general helps little when working in W- space. On the other hand, since the performance of well-trained subjects in C-space was consistently better than of the little-trained subjects, one can conclude that training does in fact help when operating in C-space. Given the overhead of eort a subject needs to master the more complex rules of operation 15

16 in C-space and to get used to the unfamiliar shapes of C-space obstacles, it is reasonable to suggest that training will, in general, signicantly improve the subjects' performance in C-space. 5 Conclusion This paper proposes an approach to human-guided teleoperation of a robot arm manipulator based on operation in the task's conguration space rather than in the commonly used W-space. As described, the approach is applicable only if complete information about the arm manipulator and its environment is available and the corresponding c-space can be computed beforehand. To improve the human interface, a tool is proposed for reducing the motion in 3D C-space to that in a 2D C-space. With it, a two-dimensional, sliced version of the three dimensional C-space is oered to the operator. Instead of directly confronting the problem of collision analysis, which is known to be extremely challenging for the human spatial reasoning, the operator faces the task in C-space where one can concentrate on global navigation, leaving collision analysis to the computer. Thus reduced task becomes a variant of the maze-searching problem. Our experiments with human subjects show that in spite of the strange appearance of this new interface, subjects quickly learn it and perform in it remarkably better than in the familiar commonly used work space. The authors express their gratitude to Jon Lawrence for the help in designing and especially building the master arm used in this work. References [1] V. Lumelsky, S. Rogers, J. Watson, F. Liu, An Experimental Study of Human Performance in Planning Object Motion. Final Report. University of Wisconsin Robotics Lab. July [2] F. Liu, Multivariate Analysis of Human Performance in Motion Planning. M.S. Thesis, University of Wisconsin-Madison, Mechanical Engineering, May [3] V. Lumelsky, E. Cheung, Real-Time Collision Avoidance in Teleoperated Whole- Sensitive Robot Arm Manipulators, IEEE Transactions on Systems, Man, and Cybernetics, Vol. 23, No. 5, pp , [4] I. Ivanisevic, V. Lumelsky, A Human-Machine Interface for Teleoperation of Arm Manipulators in a Complex Environment, Proceedings of the 1998 IEEE International Conference on Intelligent Robots and Systems, October [5] Q. Lin, C. Kuo, Virtual Teleoperation of Underwater Robots, Proceedings of the 1997 IEEE International Conference on Robotics and Automation, April

17 [6] P. Schenker et al., Development of a telemanipulator for dexterity enhanced microsurgery, Proceedings of the 2nd International Symposium on Medical Robotics and Computer Assisted Surgery, pp , [7] I. Hunter et al., A Teleoperated Microsurgical Robot and Associated Virtual Environment for Eye Surgery, Presence, Vol. 2, pp , Fall [8] T. Lozano-Perez, Spatial planning: a conguration space approach, IEEE Transactions on Computers, Vol. 32, No. 3, February [9] J. Schwartz, M. Sharir, On the Piano Movers problem. Part II. General Techniques for Computing Topological Properties of Real Algebraic Manifolds, Advances in Applied Mathematics, Vol. 4, pp ,