REMOTE OPERATION WITH SUPERVISED AUTONOMY (ROSA)

REMOTE OPERATION WITH SUPERVISED AUTONOMY (ROSA) Erick Dupuis (1), Ross Gillett (2) (1) Canadian Space Agency, 6767 route de l'aéroport, St-Hubert QC, Canada, J3Y 8Y9 E-mail: erick.dupuis@space.gc.ca (2) MD Robotics Limited, 9445 Airport Road, Brampton ON, Canada, L6S 4J3 E-mail: rgillett@mdrobotics.ca ABSTRACT Remote Operation with Supervised Autonomy (ROSA) was a collaborative research and development project with the goal of developing a variable-autonomy architecture to support ground-based control of space-deployed robotic systems in dynamic workspace environments. The ROSA architecture was successfully demonstrated using two sample applications: a satellite-servicing scenario using a laboratory test-bed and MSS ground control using a high fidelity simulator. The project had two main thrusts: artificial intelligence techniques to enhance the local decision-making capability, and artificial vision to enrich the robot s perception of the environment. This paper describes the ROSA concept of operation and architecture concentrating on the autonomy aspects and it describes the experimental results. INTRODUCTION Most space robotic operations in the foreseeable future will require ground control. Depending on the application, the level of autonomy will wildly vary from case to case. On the International Space Station, operations of the Mobile Servicing System will eventually be conducted partially from the ground to free-up astronauts to conduct more science experiments. In this case, the autonomy requirements are relatively low since the environment is known and static, and the communication delays are relatively short. In contrast, for a rover or manipulator on a Mars exploration mission, the operator will be located on Earth and will interact with the robot only once or twice every day. The communication delays in this case will be on the order of 20-40 minutes. Such an application will therefore require a very high level of autonomy: the robot being sent command scripts for up to 12 hours of operations at a time. In the case of Reusable Space Vehicles performing servicing of satellites in low Earth Orbit, the operator will still be located on Earth. Typical tasks of such a system would involve capture, refueling or maintenance of a satellite. Although the delays are only on the order of a few seconds, the level of autonomy will still be relatively high since the environment in which the robot operates changes dynamically. The objective of the Remote Operation with Supervised Autonomy (ROSA) project was to set-up an infrastructure enabling ground control and supporting varying levels of autonomy. The project had two main thrusts: local autonomy software for space robotics applications and vision algorithms to enrich the knowledge of the environment. This paper will concentrate on the autonomy aspects of the ROSA project. Over the last decade, many approaches have been implemented to circumvent these problems and allow controlling space-based robots from ground stations. In 1993 DLR demonstrated ground control capabilities with the ROTEX [1] flight experiment utilising various modes of control. In 1998, NASDA's ETS-VII satellite was successfully controlled from a ground station at NASDA's Tsukuba Space Center in Japan [2]. ETS-VII was also used to demonstrate ground control using DLR's MARCO architecture [3] and ESA's FAMOUS architecture [4]. Using a similar approach, the Canadian Space Agency (CSA) and MacDonald Dettwiler Space and Advanced Robotics (MD Robotics) developed an architecture for remote operation of space-based assets [5] and tested it in an operational environment by remotely controlling a robotic excavator in northern Canada. Recently, the CSA, DLR and MD Robotics have started to tailor their ground segment software specifically for MSS operations [6][7]. 1

ARCHITECTURE The ROSA architecture is based on the concept of pre-planned operations scenarios composed of sequences of commands with a varying level of autonomy. A ground station provides the capability for generation and verification of the Mission Script, execution of this script including uplinking of the scripts, and monitoring and/or control of transitions between script elements. At execution, scripts can be uploaded to the space segment in entirety or one element at a time depending on the required level of autonomy. All logical branching decisions related to the execution of the script are controlled by a script executor and are typically event-driven. Again, depending on the required level of autonomy, the script executor can be located in the space segment or the ground segment. In the ROSA architecture, the script executor can relay lowlevel commands directly to the equipment being controlled or high-level commands to an intelligent agent in charge of performing the necessary decomposition and monitoring to ensure success of the operation. To support many levels of autonomy, the ROSA architecture implements a tiered approach to task specification. Two technologies were tested for the implementation of the intelligent agent. The first approach used a Hierarchical Task Network (HTN) to provide reactive control capability. The HTN is a set of finite state machines that contain the expert knowledge required to decompose the high-level commands of a script into a sequence of low-level primitive commands and take corrective measures based on discrete events such as sensor, time or operator-entered data. The reactive inference engine that implemented the HTN functionality is dubbed the Amorphous Architecture for Variable Autonomy Robot Control (AArVARC). This HTN-based inference engine is event-driven and provides a capability for immediate reaction based on external information such as sensor data, time events and operator-entered commands. One of AArVARC s most interesting features is its capability to support, by itself, a broad variety of levels of autonomy: from MSS-type operations where autonomy is very low to planetary exploration robots. To ease implementation by non-expert users, AArVARC uses a graphical programming environment: Matlab/Simulink s Stateflow toolbox. This allows operators to use a high level of abstraction when implementing behaviours in the HTN and maximizes the reusability of the software between applications. The software is implemented in such a manner as to retain the same core functions changing only the pool of behaviours and the low-level interfaces to the robot. A deliberative goal-oriented layer (called the Cognitive Controller or CoCo) was added on top of the HTN to provide the capability to generate or modify a plan locally that will subsequently be fed to the HTN or to the primitive executor. CoCo plans a mission using a Goal Achievement approach by considering the current state of the system, the desired state of the system and by laying out a sequence of behaviours to achieve that goal. In principle, the human operator need only provide a single high-level instruction (representing a mission goal). CoCo then plans a mission script to achieve the objectives. The plan invokes lower-level commands passing them along to the HTN or directly to the robot. This provides the capability to accommodate for non-static worksites and mission anomalies. Unlike AArVARC, which is purely reactive, CoCo has a deliberative capability. When encountering anomalies, it has the possibility to generate a new plan. This process is longer than a purely reactive system. However, combining the two capabilities allows the ROSA architecture to take advantage of the quicker reaction of the HTN while having the robustness of an automatic planner when encountering situations where the HTN cannot achieve mission success using the pre-determined set of rules. To take advantage of the broadest range of autonomy, ROSA provides the capability to invoke commands from any level of autonomy seamlessly within a single script. For a satellite servicing scenario, it would therefore be possible to invoke the deliberative layer to plan the capture of a satellite, use the HTN to perform the capture and use scripted primitives for the satellite servicing after capture. RESULTS The ROSA ground control architecture was tested using two sample applications to demonstrate the adaptability of the system to different levels of autonomy. The first application was a satellite-servicing scenario. The ROSA software was implemented on MD Robotics Reusable Space Vehicle Payload Handling Simulator (RPHS), a dual arm robotic testbed used to validate various technologies for satellite servicing operations, shown on Fig. 1. The RPHS facility is located in a dark room environment to faithfully emulate the lighting environment of Low-Earth Orbit to validate various artificial vision technologies. 2

Fig. 1 - Reusable Space Vehicle Payload Handling Simulator (RPHS) The nominal scenario involved locating, tracking and capturing a moving satellite mock-up using a vision system. Transitions between phases of the operation were triggered by sensory events. Anomalies were injected in the scenario to verify the ability of AARVARC and CoCo to deal with off-nominal situations. In one case, the lights were turned off to emulate blinding of the vision system and in another case, the telemetry server was turned off thus emulating the loss of the link between the low-level controller and the intelligent agent. In all cases, both approaches performed admirably. When encountering anomalous conditions, the AARVARC engine executed a pre-determined set of rules to complete the operation successfully whereas the cognitive controller (CoCo) replanned a new operation sequence to accomplish its objectives taking into account the new starting conditions. As a testimony to its robustness, the set of behaviours implemented using the Hierarchical Task Network approach encountered a set of untested conditions during the final demonstration of the project and, nevertheless, succeeded in completing the capture of the satellite mock-up. The second application selected to demonstrate the power of ROSA was a simulation of MSS ground control using the MSS Operations and Training Simulator (MOTS), a high-fidelity dynamic and graphical simulator of the MSS used for astronaut training. See Fig. 2. Fig. 2 - Emulation of MSS on MOTS Simulator 3

Since MSS ground operations will likely be conducted with intensive operator intervention, the cognitive controller approach was not tested. The architecture was set-up such that the intelligent agent, in this case AARVARC, was located in the ground station and the operator monitored and approved all transitions between script elements. The AARVARC engine successfully completed the step-off and stow procedure that was executed during Canadarm-2 s maiden flight. This procedure involved many of the most complex operations to be expected of the Canadarm-2 such as the capture of a grapple fixture and a change of base from one grapple fixture to another. CONCLUSION Remote Operation with Supervised Autonomy (ROSA) is a collaborative research and development project with the goal of developing an architecture to support ground-based control of space-deployed robotic systems in dynamic workspace environments. The ROSA architecture is based on the concept of pre-planned operations scenarios composed of sequences of commands with a varying level of autonomy. It uses a tiered approach to implement different levels of autonomy that includes a Hierarchical Task Network capable of executing a high-level mission script and a deliberative goal-oriented layer capable of re-planning the mission script. ROSA demonstrated satellite docking at various levels of autonomy using MD Robotics Reusable Space Vehicle Payload Handling Simulator. In this facility, two industrial manipulators under controlled lighting environment are used to emulate satellite-docking operations. One manipulator maneuvers a mock-satellite while the other emulates a chaser satellite attempting to dock. Tests conducted using varying autonomy levels demonstrated successful docking in nominal situations as well as in off-nominal situations, such as when the lighting was turned off or when the vision system lost sight of the target. ROSA also successfully demonstrated ground control of the MSS, using a high-fidelity simulator, where safety constraints would preclude the deliberative layer and require that the operations scripts be known ahead of time. Today, ROSA has reached a sufficient level of maturity to become the baseline in many R&D projects both at CSA and MD Robotics. ROSA is also the baseline ground control software for MD Robotics robotic manipulator on the Orbital Express program. REFERENCES [1] Hirzinger, G., Brunner, B., Dietrich, J. and Heindl, J., Sensor-Based Space Robotics - ROTEX and Its Telerobotic Features, IEEE Trans. on Robotics and Automation, Vol.9, No.5, pp.649-663, October 1993. [2] Oda, M., Space Robot Experiments on NASDA's ETS-VII Satellite - Preliminary Overview of the Experiment Results, Proc. 1999 IEEE Conf. on Robotics and Automation (ICRA 99), Detroit, USA, pp.1390-1395, 1999. [3] Brunner, B., Landzettel, K., Schreiber, G., Steinmetz, B.M. and Hirzinger, G., A Universal Task-Level Ground Control and Programming System for Space Robot Applications - the MARCO Concept and its Applications to the ETS-VII Project, Proc. Fifth International Symposium on Artificial Intelligence, Robotics and Automation in Space (isairas 99), ESTEC, Noordwijk, The Netherlands, pp.217-224, June 1-3 1999. [4] Galardini, G.M., Kapellos, K., Maesen, E., Visentin, G. and Didot, F., em Vision and Interactive Autonomy Bi- Lateral Experiments on the Japanese Satellite ETS-VII, Proc. Fifth International Symposium on Artificial Intelligence, Robotics and Automation in Space (isairas 99), ESTEC, Noordwijk, The Netherlands, pp.217-224, June 1-3 1999. [5] Dupuis, E., Gillett, R., Boulanger, P., Edwards, E., and Lipsett, M., Interactive, Intelligent Remote Operations: Application to Space Robotics, SPIE Telemanipulator and Telepresence Technologies VI, Boston, Massachusetts, USA, September 1999. [6] Dupuis, E., and Gillett, R., Validation of Ground Control Technology for International Space Station Robot Systems, Proc. Sixth International Symposium on Artificial Intelligence, Robotics and Automation in Space (isairas 2001), St-Hubert, Canada, June 18-21 2001. 4

[7] Landzettel, K., Brunner, B., Schreiber, G., Steinmetz, B.M., and Dupuis, E., MSS Ground Control Demo with MARCO, Proc. Sixth International Symposium on Artificial Intelligence, Robotics and Automation in Space (isairas 2001), St-Hubert, Canada, June 18-21, 2001. 5