Tele-manipulation of a satellite mounted robot by an on-ground astronaut

Proceedings of the 2001 IEEE International Conference on Robotics & Automation Seoul, Korea May 21-26, 2001 Tele-manipulation of a satellite mounted robot by an on-ground astronaut M. Oda, T. Doi, K. Wakata National Space Development Agency of Japan E-mail: oda.mitsushige@nasda.go.jp Abstract An experiment to tele-operate a robot arm on a satellite from an onground control station was conducted by an astronaut who has extensive on-orbit experience in operating the Space Shuttle s robotic arm. This experiment aimed to evaluate / compare a teleoperation system, which we developed, with the teleoperation system of the Space Shuttle Remote Manipulator System (SRMS). There was a time delay of about 6 seconds in the teleoperation control loop of the satellite-mounted robot. However, the teleoperation experiments were satisfactory conducted with helps of various operator assistance functions. 1. Introduction The international space station is currently under construction aiming to finish by the year 2006. Assembly of the International Space Station (ISS) is currently being conducted by astronauts using the Space Shuttle Remote Manipulator System (SRMS). Another large robot arms that are under construction in Japan, Canada, and Europe will be added to the ISS. These robot arms are operated by astronauts on the ISS. The ISS has many experiment modules built by US, Japan, Europe and Russia. However, number of onboard astronauts will be limited to seven persons even after the assembly of the ISS is completed. This limited number of astronauts on the ISS is due mainly to the limited living space on the ISS and the high cost of sending astronauts to the ISS. Therefore, to supplement the limited crew resources on the ISS, a teleoperational robot system would be an attractive tool for the ISS. However, there has been limited experience in the teleoperation of a robot system in space. Germany conducted a space robot technology experiments (1) in 1993 using the Space Shuttle. A small robot arm whose length is about.5m was mounted on an experimental rack of the Space Shuttle s laboratory module named Spacelab. An onboard astronaut mainly operated the robot arm. However, an onground operator conducted some experimental operations. National Space Development Agency of Japan (NASDA) so far conducted two robot technology experiment missions. The first one is the Manipulator Flight Demonstration (MFD) (2) mission that was conducted on the Space Shuttle in August 1997. A 1.5 m-long robot arm, which is a replica of a manipulator system to be used on the ISS was mounted on the Space Shuttle cargo bay and was operated by the astronauts on the Space Shuttle. NASDA is now developing a remote manipulator system for the Japanese Experiment Module (JEM) of the ISS. The purpose of the MFD mission was to verify design of the JEM s manipulator system and its teleoperation system. In order to keep commonality with the SRMS, the MFD robot arm and the JEM s manipulator system use similar control system with the SRMS. Both use a set of 3 degree-of-freedom (DOF) joysticks for manual teleoperation of the robot arm. The onboard astronauts conducted most of the MFD robot operations. However, some experiments were carried out from the ground. The second experiment is the engineering test satellite #7 (ETS-VII) (3) which was launched in November 1997. The mission of ETS-VII was to conduct automated rendezvous docking and space robot technology experiments. ETS-VII consists with two satellites, a larger chaser satellite and a smaller target satellite. The ETS-VII chaser satellite has a 2 m-long robot arm, and the robot arm was used for various robot experiments. The robot experiments included coordinated satellite attitude and a robot arm control, teleoperation of the onboard robot arm, autonomous target satellite capture (5), etc. Operation of the satellite and the onboard robot arm was conducted from NASDA Tsukuba Space Center located in Japan. The communication between the satellite and the onground control system was established using a data relay satellite in the geo-stationary orbit. There is a time delay of about 6 seconds in the robot control loop. Since pure tele-manipulation under such time delay could cause difficulty in the robot arm operation and safety concerns, a supervised robot arm control was used in most of the ETS-VII robot experiments. Some experiments were conducted in the tele-manipulation mode. These experimental operations of the onboard robot were conducted by 0-7803-6475-9/01/$10.00 2001 IEEE 1891

engineers/scientists of NASDA and many other institutes. To see the advantages of the ETS-VII robot teleoperation system over the teleoperation system of the SRMS, and to show taxpayers that the space robot is a useful tool for on-orbit tasks, NASDA conducted tele-manipulation experiments by a Japanese astronaut who has extensive operational experience in operating the SRMS. NASDA astronaut, Koichi Wakata operated the SRMS to retrieve a Japanese satellite, to deploy and retrieve a US satellite, and to support the extra-vehicular activities on STS-72 in 1996. He also operated the SRMS to install two components to the ISS and to support the extra-vehicular activities on STS-92, an ISS assembly flight in 2000. This paper introduces results of the tele-manipulation experiments conducted by the astronaut on the ground. Section 2 shows the teleoperation systems of space robot systems. Section 3 shows experiment system and experiment plan. Section 4 shows experiment results and findings from the experiments. 2. Space robot teleoperation system The existing robot manipulators that are used in space are 1) Space Shuttle s Remote Manipulator System (SRMS) that was developed by Canada and is being used since 1980 s, 2) Germany s experimental robot arm system named ROTEX that was conducted in 1993 in the Space Shuttle s laboratory module, 3) NASDA s robot arm system that was mounted on the Space Shuttle s cargo bay and was launched in 1997 (Manipulator Flight Demonstration), 4) NASDA s engineering test satellite, ETS-VII. Three sets of manipulator systems to be used on the International Space Station (ISS) are currently under development by Canada, Japan and Europe. Canada is developing the Space Station Remote Manipulator System (SSRMS). Japan is developing the Japanese Experiment Module Remote Manipulator System (JEMRMS). European Space Agency (ESA) is developing the European Robotic Arm (ERA) that is used on the Russian segment of the ISS. From a standpoint of teleoperation, these robot systems can be grouped as follows. 1) manipulator system controlled by astronauts on-orbit; SRMS, MFD, SSRMS, JEMRMS, ERA belong to this group. 2) manipulator system controlled from a remote site such as an onground control station; ROTEX and ETS-VII belong to this group. 2.1 Space Shuttle s manipulator and similar systems SRMS, MFD, SSRMS, and JEMRMS use a pair of 3 degree-of-freedom (DOF) joysticks for manual teleoperation. One reason that these systems use similar control device is to maintain a commonality with the SRMS, which has more than 19 years of on-orbit operational experience. As an example of this type system, NASDA s MFD (2) (Manipulator Flight Demonstration) mission is examined here. The robot arm is about 1.5m long and has six rotary joints. Robot control equipment was mounted in the after-cockpit of the Space Shuttle. A set of 3 DOF joysticks was used as the robot control device. Tasks conducted by MFD robot arm included handling a hinged door and de-mating/re-installing an orbital replacement unit. An on-board astronaut conducted these tasks. Some other experiments such as visual inspection of equipment using a hand-eye-camera were conducted by teleoperation from the ground. In the teleoperation from the ground, the robot control commands were generated by the on-ground equipment. However execution of the commands is something like an off-line execution, since commands were executed after verifying the command file was transmitted without error. The robot arm s contact operation was inhibited in this teleoperation experiment due to a safety concern that an unexpected robot arm motion might damage the Space Shuttle or other equipment. 2.2 ROTEX (1) ROTEX (Robot Technology experiment) was conducted by Germany in April 1993 using the Space Shuttle. A small robot arm was mounted in an experimental rack in a laboratory module of the Space Shuttle.The ROTEX robot system was operated by an onboard astronaut and by an onground operator in different telerobotic ground control modes. A SPACEMOUSE was used in this experiment as a robot control device for an astronaut and also for an onground operator. Teleoperation from ground was supported by predictive computer graphics and sensor-based off-line programming. The onboard teleoperation system didn t have such capabilities because of limited computer performance for space application at that time. 2.3 ETS-VII (3) NASDA launched a robot satellite named ETS-VII(Engineering Test Satellite #7) in 1997 to conduct rendezvous docking and space robot technology experiments. ETS-VII is a pair of satellite, a larger chaser satellite and a smaller target satellite. The masses of both satellites were about 2500kg and 400kg, respectively. Both satellites were launched together by NASDA s H-II rocket in November 28, 1997. Orbit of ETS-VII is 550km altitude and 35deg inclination. The target satellite was released from the chaser satellite during the rendezvous docking experiments. Fig.1 shows a picture of ETS-VII. 1892

Fig.1. ETS-VII satellite during checkout The ETS-VII chaser satellite has a 2 m-long robot arm. This robot arm was used in various experiments such as a coordinated robot arm and satellite attitude control experiment, teleoperation of the onboard robot arm and autonomous target satellite capturing experiment. Since ETS-VII flies a low earth orbit on which a satellite flies around the Earth in about 100 minutes, direct communication between the satellite and the onground control station is possible for only 5 to 10 minutes in each 100 minutes. Therefore, continuous communication between the satellite and the onground control station is realized using a data relay satellite in a geostationary orbit. As a data relay satellite, NASA s tracking and data relay satellite (TDRS) that is located over the Pacific Ocean was borrowed. The overall time delay in the robot control loop was about 6 seconds. It is neither easy nor safe to telemanipulate the onboard robot arm from ground under such a large time delay. Therefore, NASDA adopted the supervisory control as a primary robot control mode. In this control mode, instruction to the onboard robot system is generated in advance based on a planned task sequence. A set of instruction is verified using a built-in onboard robot simulator that receives teleoperation commands and responds as if a real onboard robot system. Operator s task is to send this instruction set and monitor its execution. The ETS-VII on-ground robot operation facility was designed considering following requirements. [4] To be easy in learning and operating the system even under the time-delayed and limited communications environments. To be safe and reliable in conducting tasks. From these design requirements, ETS-VII robot teleoperation was conducted mainly in the supervised control mode using the electronic operation procedure that describe the tasks and commanding procedures (4). Tasks are sub-divided into sub-tasks. Sub-tasks that can be automatically executed are basically executed automatically under the control of the electronic operation procedure. Tasks/sub-tasks that cannot be pre-defined or difficult to automatically execute are conducted manually using hand-controller. As the hand-controller, we adopted a set of 3dof joysticks that is similar with joysticks of the Space Shuttle manipulator. One reason that we do not adopt 6dof joysticks / master arm was that the robot arm motion on a satellite is basically slow, and this makes manipulation of position based commanding device to be tiring. Fig.2 shows NASDA s ETS-VII robot teleoperation facility. Fig.3 shows joysticks that are used in this facility. Fig.4 shows execution of the electronic operation procedure. Central sub-window shows execution status of individual command (pre-send-diagnostics / send status / receive status / execution-result). Flowchart shows execution of individual task and sub-tasks. Video images from two CCD cameras, a hand-eye-camera and a camera on a first joint, are sent from satellite, and are displayed on the operation console (see Fig.3). Computer graphics (CG) images that show the robot arm position based on the telemetry data from the satellite and the estimated robot arm position after executing the individual commands are also displayed. (see Fig.5) Fig.2. ETS-VII robot teleoperation facility Fig.3. A pair of 3dof joysticks 1893

Fig.4. Electronic operation procedure Fig.5. CG image of onboard robot (left: estimated pose after execution of commands, right: current position) 3. Experiment The astronaut s telemanipulation experiments were conducted as follows. 3.1 Training In order to assure safe and satisfactory mission, training of the mission is important. However, if the overall system is not well designed, many hours/days are required to receive required training. Because of the limited availability of the astronaut, who was in training for the International Space Station (ISS) assembly flight, only three days were allocated for this experiment including the training and the actual experiment operations. Therefore the first two days were used for trainings, and the actual experiment was conducted on the third day. The astronaut had a general understanding of the satellite and its robot system to be used, but not in details about its design or operations before this training. The training for this experiments were conducted as follows; (a) a lecture about the system 1894 to be used, (b) training to become familiar with the teleoperation system operation, (c) training to learn the required tasks in the actual experiments, (d) training to check whether the required tasks can be performed within the planned time period (rehearsal of the actual experiment). The lectures included the purpose of the experiments, the satellite system, the onboard robot system, the onground teleoperation system, the teleoperation system operation, and the malfunction procedures. These lectures were given in 3 hours. Subsequently, operation training was conducted in 3 hours in the afternoon of the first day to familiarize with the teleoperation system of NASDA and other two institutions. The astronaut operated the system at the console with the joysticks. Simulated telemetry data and video images were automatically generated by the built-in onboard robot simulator. The ETS-VII robot teleoperation system uses a pair of 3 DOF joysticks as the robot control device that is similar but not identical with the Space Shuttle s Remote Manipulator System(SRMS). Therefore, it seemed relatively easy for the astronaut to learn the teleoperation of the robot arm on ETS-VII. The training to learn the required tasks in the actual experiments was conducted in the morning of the second day. The time necessary to conduct the tasks was also checked and the actual experiment plan was slightly modified based on this time measurement. The rehearsal of the experiment was conducted in the afternoon of the second day, and the all planned tasks were checked whether those could be performed in the planned time period. This rehearsal was conducted for the NASDA s experiment as well as for the experiments of NAL and CRL. Each rehearsal took place in an hour. 3.2 Experiment The actual experiments were conducted on March 16, 1999. The actual time line of tasks was as follows.(xx:xx-xx:xx) shows time window in JST(Japan Standard Time). The experiments start early in the morning because of location of the data relay satellite in the geo-stationary orbit. ETS-VII robot experiments were conducted while the satellite could communicate with the data relay satellite and while the satellite was receiving sunlight. Gather. (03:30) Daily task briefing (check of preparation status and others). Besides this briefing, short briefings to check the final preparation are conducted before each session. First session: (04:34-05:16), Start-up the onboard robot system and prepare the experiment. These operations were conducted by NASDA operators in the supervised mode. The astronaut monitored these operations. Then the control of the onboard robot arm was transferred

to the astronaut. The astronaut operated the robot arm using the joysticks. The purpose of this task was to become familiar with the actual robot arm operation. Approximately 10 minutes were used for this operation. Second session: (06:17-06:59) NASDA s tele-manipulation experiment by the astronaut in the tele-manipulation mode. The required task was to trace the surface of the taskboard using a peg attached to the robot arm. (See Fig.5) Third session: (07:58-08:40) NASDA s experiment to conduct the surface tracking in the shared control mode by the astronaut. Forth session: (09:39-10:21) NAL s experiment Fifth session: (11:20-12:02) CRL s experiment Debriefing after experiments (13:00--) Fig.5. Telemanipulation experiment by an astronaut 4. Experiment results In this section, the results of the surface tracking experiments are given. Experiments were conducted in the following cases. (case-1) the astronaut s teleoperation in the compliance control mode (case-2) the astronaut s teleoperation in the force accommodation control mode (case-3) the other operator s teleoperation in the compliance control mode (conducted in other day) (case-4) the other operator s teleoperation in the force accommodation control mode (conducted in other day) The astronaut and the operator were asked to conduct the surface tracking on a sinusoidal-shape portion of the taskboard (see Fig.5) maintaining the contact force to 25 N. In the compliance control mode, robot arm s contact force is determined by the commanded impedance parameters ( Mc, Kc, Dc ) and the instructed robot arm s tip position. In the force accommodation control mode, robot arm s contacting force is automatically controlled by the onboard robot controller to the instructed force level. Therefore, the robot operator does not have to control vertical position of the robot 1895 arm to adjust the contacting force in the force accommodation control mode. Figs.6, 7, 8 and 9 show robot arm s tip position and contacting force. Figs.6,7,8 and 9 are case-1, case-2, case-3, and case-4, respectively. Fig.6. Astronaut in the Compliance mode Fig.7. Astronaut in the Force Accommodation mode Fig.8. Operator-1 in Compliance cont. mode Fig.9. Operator-1 in Force Accommodation mode The spike-like deviations of the arm tip force in each case are due to periodical reset of an integrator within the onboard robot arm control circuit. The evaluation of the experiments was conducted from the following viewpoints. 1) Difference between the operators in the teleoperation performance 2) Effect of the automatic task execution in the force accommodation and the compliance control mode 3) Post-experiment debriefing with the astronaut to gather his impression on the experiments 4.1 Evaluation of the experiment data Against the requested arm tip force of 25N

during the surface-tracing task, results of each test are as follows. Table-1 Result of experiments Operation mode Case Test Subject (*) Average tip force(n) Mean deviation (N) 1 MS Comp. -25.91 2.12 2 MS Force -24.41 1.84 3 OP1 Comp -22.74 3.00 4 OP1 Force. -24.50 1.51 (note) MS: OP1: Force: Comp. Mission Specialist (astronaut) operator-1 Force accommodation mode Compliance control mode From these experiment data, following findings were derived. (1) The experiment data clearly show that the force accommodation control is quite useful in this experiment in senses that the contact force is better managed than in the case of the compliance control mode, and that the difference of control performance between the astronaut and the other operator is small in this control mode. The onboard robot controller manages control of the contact force in the force accommodation control. Therefore onground operator can concentrate in the horizontal control of the robot arm tip position in this case. This is a kind of shared control between the onboard robot controller and the operator. From the debriefing after the experiments, the astronaut and the operator said that operation in the force accommodation control were easier than the compliance control concerning this surface-tracing task. However, it is also shown in other experiment that the compliance control mode is preferable in other tasks such as grasping a payload. The Space Shuttle s Remote Manipulator System (SRMS) does not have these control modes and the contact operation such as berthing a payload to a structure with a small contact clearance and a limited allowable contact force can be a difficult task for the SRMS. (2) Effect of partial automation of subtask In the case of ETS-VII s robot operations, given task is divided into several subtasks, and subtasks are automatically controlled by the electronic operation procedure (see Fig.4). The compliance control and the force accommodation control are also kind of partial automation of the robot arm control. Debriefing after the experiments showed that these partial automations contributed to the easiness of the teleoperation and the training. The fact that the teleoperation of the real space robot by the astronaut was satisfactorily conducted after only the two-day-training shows the advantage of this partially automated teleoperation system. (3) Debriefing from the astronaut Debriefing from the astronaut was conducted immediately following the experiments. Major findings are as follows. Various kind of information is displayed on the operation console such as telemetry data from the satellite, computer graphics (CG) images that show the robot arm s predicted and current positions, and the real images from onboard cameras. The astronaut used mainly the real TV camera images while he conducted the surface tracing task mainly because of his SRMS operational experience where the TV camera images plays a key role in monitoring the robot arm motion. However, he reported that the CG images were very useful for maneuvering the robot arm under the large time delay in the control loop. Training: The astronaut spent only two days for the training. The training using the actual operation facility that has built-in onboard robot simulator was essential for the training efficiency. Preparation for anomalies: The ETS-VII robot operation facility has many built-in electronic operation procedures to cope with anomalies that may happen during experimental operations. These procedures helped establish efficiency in the training and the satellite operations. Conclusions Telemanipulation of a space robot on an orbiting satellite was satisfactorily conducted by an astronaut who has extensive operational experience in the Space Shuttle s Remote Manipulator System. The experiment operations were conducted after two-day-training. The efficient training and satisfactory experiment executions owe greatly to the built-in onboard robot simulator and the partially automated teleoperation system. The experiment results will be applied to the future space missions such as to improve the space station manipulator system and robot satellites. Reference (1) G.Hirzinger, et al., "The first Remotely Controlled Robot in Space, Proc. of 1994 IEEE Int. Conf. On Robotics and Automation, May 8-13, 1994, San Diego, California (2) Y.Horikawa, et.al., On the Results of the Manipulator Flight Demonstration for JEM, 49th IAF Congress, Sepr.28-Oct.2,1998, Melbourne, Australia, IAF98-T-203 (3) M.Oda, Experience and lessons learned from the ETS-VII robot satellite, Proc.of 2000 IEEE Int. Conf. On Robotics and Automation, San Francisco, CA, April 2000, pp.914-919 (4) M.Oda and T.Doi, Teleoperation system of ETS-VII robot experiment system, Proc. of IEEE/RSJ International Conf. on Intelligent Robotics and Systems (IROS 97), Sept.7-11, 1997, Grenoble, France, pp.1644-1650 1896