1 o SiPGEM 1 o Simpósio do Programa de Pós-Graduação em Engenharia Mecânica Escola de Engenharia de São Carlos Universidade de São Paulo 12 e 13 de setembro de 2016, São Carlos - SP DEVELOPMENT OF A MOBILE ROBOTS SUPERVISORY SYSTEM Marco Antonio Nunes da Cunha de Arruda, Machine and System Dynamics, marco.nc.arruda@gmail.com Marcelo Becker, Machine and System Dynamics, becker@sc.usp.br Abstract. Robots applications has been increasing last years, from civilian search and rescue, agriculture and medicine to military missions, they are useful to keep humans away from unhealthy and risk tasks. A problem that comes up when it is necessary to work with team of robots is having an adequate supervisory system. Each robot has its own ground control station software, but there is no agreement between those solutions. It is revised and compared solutions in the literature to develop tools for a customized interface for a broadly multi-robot supervisory system. So far, it is shown most important aspects of existing solutions and how they could be used to propose a flexible solution to serves big variety of heterogeneous robotics systems. It is expected with this work to offer an easier way to develop single or multi robot supervisory system. Keywords: Supervisory System, Mobile Robots, Ground Control Station, ROS 1. INTRODUCTION The Robotics Research is an increasing area that evolved in the last 50 years, coming from Industrial applications to civilian solutions, as shown by Garcia et al. (2007). At the beginning, robots were able to execute repetitive tasks and had low perception of the environment. As this area shown great potential, researches gave robots more power to interact with humans, bringing them to more challenging problems. From a known environment, like factories, to urban spaces, robots needed mobility and maneuvering capabilities. Once robots can move around and make decisions about next tasks, there is the necessity of human supervision. In (Chen et al., 2014) it is addressed the Human-Agent Teaming trend, which is about human and agents (a computer program or a robot) interfacing. The authors shows how embracing is the supervision problem in different areas of robotics and relevant concerns for developing a supervision system for a specific application. There are various robotics solutions from different companies. Those solutions could be an embedded software or hardware, supervision software, robot, ground control station, or an entire system. For this reason a new problem rises, the integration of those solutions. There are efforts to solve the software problem. As shown in (Quigley et al., 2009), ROS is a middleware to integrate solutions on robotics software. As a contribution to this effort, this work reviews relevant aspects of existing supervisory systems and ground control station of mobile robots to propose, in the future, a mobile-robot-oriented ROS integrated supervisory system. 2. LITERATURE REVIEW As the main goal of this work is to develop a supervisory system for ROS integrated mobile robots, it is reviewed supervision concepts and ground control station works. 2.1 Supervisory System Supervisory systems are projected to offer visibility and control of agents to a human supervisor. This one must be able to has any relevant information about a system or an agent inside an environment. Also, a supervisor must be able to intervene if an agent doesn t act in a undesirable way of doing a specific task. Sheridan (2002) describes the main role of human during supervision task on Unmanned Aerial Vehicles missions, it is listed below: 1. Planning 2. Teaching 3. Monitoring 4. Intervening 5. Learning
Marco Antonio N. C. Arruda, Marcelo Becker Development of a Mobile Robots Supervisory System Table 1: Level of Automation (source: Parasuraman et al. (2000) TEN LEVELS OF AUTOMATION (LOA) High 10 Full Autonomy: The computer makes all decisions, acts autonomously, and ignores the human operator. 9 The computer informs the human operator, only if it "decides" to. 8 The computer informs the human operator, only if asked to. 7 The computer executes automatically, then informs the human operator as necessary. 6 Allows the human a limited amount of time to veto an action before it is automatically executed. 5 The computer executes the suggestion with approval from the human operator. 4 The computer suggests one alternative. 3 The computer narrows the decision-making to a few selections. 2 The computer offers a complete set of decision / action alternatives. Low 1 Manual Operation: The computer offers to assistance to the human operator, who must make all decisions and take all actions. In Parasuraman et al. (2000) the author classifies supervisory system by the level of automation (LOA). It is identified by the interaction between system and human. The more complex is a system, more automated it should be, releasing human workload. To classify a system, it is used the following criteria. LOA are shown in Table 1. 1. Information acquisition 2. Information analysis 3. Decision and action selection 4. Action implementation Other concern on designing supervisory systems is related to issues on performance of operator. Some points shown in Manzey et al. (2008) are: Automation reliability and complexity of tasks. According to the author, those points affects directly the operator performance. Endsley (1995) describes other import aspect on supervision tasks, Situation Awareness of operators. It is described as three levels: 1. Perception of data and environmental elements 2. Comprehension of current situation 3. Project of future states and events Other important aspects like about operator performance and user interface design can be found on Chen et al. (2011). 2.2 Ground Control Station (GCS) In this Section it is reviewed some works that aim on the development of multi-robots systems with a Ground Control Station for supervision. In Kang and Yuan (2009), it is proposed a design for Unmanned Aerial Vehicles (UAV) supervision. According to the authors, the software must consider hardware limitations, like communications speed, and take advantage of computers screen to show information with "low costs". It means using virtual instruments to make it more intuitive.
1 o SiPGEM-EESC/USP 1 o Simpósio do Programa de Pós-Graduação em Engenharia Mecânica (a) In flight data visualization (b) Flight data analysis Figure 1: Data visualization and analysis (a) Flight data analysis (b) GCS for supervision of multi ground robots Figure 2: Data visualization and analysis Authors also classifies the kind of data that are shown on the software. They are two types: Flight Data and Payload Data. Flight Data are information about the aircraft, like battery level, altitude, speed, etc. Payload Data are interest data for the mission. It could be photos, videos or some kind of measurement. Another important role of GCS software is Data Analysis. For this reason, it is proposed a second screen to be used after mission, just to analyze the data collected. It is called Diagnosis Mode of visualization. The design proposed is shown on Figure 1. Reid et al. (2013) presents a complete Cooperative Ground Robots System for search & rescue and surveillance applications. The GCS proposed is based on Real Time Strategy (RTS) games concepts, once the concern is to provide an interface of agents supervision. The idea is to show on the main screen only vital information about the system. More complex data or detailed information can be accessed, but only if the operator wants to view it. The interface is shown on Figure 2b. In Perez et al. (2013), it is presented a GCS software for multi aerial robots supervision under surveillance task. The software was developed using C++ and Qt IDE framework, it is also open source, which allows the collaboration of third developers. The main screen, as the previous work, shows a map with the position of robots and interest points. There is also the concern to keep only vital information on the screen, but the operator can select a specific robot to see detailed information. The main screen of the software is shown in Figure 2a.
Marco Antonio N. C. Arruda, Marcelo Becker Development of a Mobile Robots Supervisory System (a) Aerial robot built on LabRoM (a) Mapping visualization tool (b) Ground Robot built on LabRoM (c) Robot from Ascending Technologies (b) 3D visualization tool Figure 4: Robot web tools (source: www.robotwebtools.org) 3. DEVELOPMENT For the development of the flexible tool for the design of mobile robots supervisory systems, here proposed, it will be used robots available on LabRoM (Laboratório de Robótica Móvel), the laboratory that is being used on EESC USP (Escola de Engenharia de São Carlos, Universidade de São Paulo). LabRoM has ground and aerial robots, built from scratch and others bought from research companies. Some available robots are shown on Figure 3.. The main idea is to provide an customizable interface, visualization tools that can be added, moved and resized around a computer screen to give the best experience to the operator. It would be like some SCADA software, where it is possible to use only necessary visualization tools. It was chosen web environment for this work. First, it is possible to access to tool from any device that has a web browser. Second, there are already web tools compatible with ROS (Robot Operating System), that are being used in robots at LabRoM. In Toris et al. (2015), it is proposed a efficient way to send robots data through web messages, for cloud robotics purpose. It is used a ROS package to propagate robots meta-data through web sockets, described in (Crick et al., 2011). For the client, developed using JavaScript language, there is a library Rosapi (Osentoski et al., 2011). The current work will be developed using both tools. This way, it is possible to develop the interface. In Figure 4 it is shown some robot tools developed for web applications 1. 4. PARTIAL RESULTS Until this paper, it was created some visualization tools for common robots data: Attitude and GPS-based position. These tools were developed independently and then joined at the same web application using Rosapi. It is possible to connect to a ROS machine that runs the Rosbridge package and select the desired message, since it is the same type of the visualization tool. Figure 4.shows those tools running. It is being used HTML5 and CSS3 standards, JavaScript and NodeJS package manager. All the source code is available on GitHub via the URL (http://www.github.com/eesclabrom/gs_web). 1 http://www.robotwebtools.org
1 o SiPGEM-EESC/USP 1 o Simpósio do Programa de Pós-Graduação em Engenharia Mecânica Figure 5: Proof of concept: Developed web tools 5. REFERENCES Chen, J. Y., Barnes, M. J., Harper-Sciarini, M., 2011 Supervisory control of multiple robots: Human-performance issues and user-interface design, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 41(4), 435-454. Chen, J. Y., Barnes, M. J., 2014, Human agent teaming for multirobot control: a review of human factors issues, IEEE Transactions on Human-Machine Systems, 44(1), 13-29. Crick, C., Jay, G., Osentoski, S., Pitzer, B., Jenkins, O. C., 2011, Rosbridge: Ros for non-ros users, In Proceedings of the 15th International Symposium on Robotics Research. Endsley, M. R., Toward a theory of situation awareness in dynamic systems Human Factors, vol. 37, no. 1, pp. 32 64, 1995. Garcia, E., Jimenez, M. A., De Santos, P. G., Armada, M., 2007, The evolution of robotics research, IEEE Robotics & Automation Magazine, 14(1), 90-103. Kang, Y., Yuan, M., 2009, Software design for mini-type ground control station of UAV, In Electronic Measurement & Instruments, 2009, ICEMI 09, 9th International Conference on (pp. 4-737), IEEE. Manzey, D., Reichenbach, J., Onnasch, L., 2008, Performance consequences of automated aids in supervisory control: The impact of function allocation, In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 52, No. 4, pp. 297-301), SAGE Publications. Osentoski, S., Jay, G., Crick, C., Pitzer, B., DuHadway, C., Jenkins, O. C., 2011, Robots as web services: Reproducible experimentation and application development using rosjs, In Robotics and Automation (ICRA), 2011 IEEE International Conference on (pp. 6078-6083), IEEE. Parasuraman, R., Sheridan, T. B., Wickens, C. D., 2000, A model for types and levels of human interaction with automation, IEEE Transactions on systems, man, and cybernetics-part A: Systems and Humans, 30(3), 286-297. Perez-Rodriguez, D., Maza, I., Caballero, F., Scarlatti, D., Casado, E., Ollero, A., 2013, A Ground Control Station for a Multi-UAV Surveillance System: Design and Validation in Field Experiments, Journal of Intelligent & Robotic Systems, 69(1-4), 119-130. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J.,..., Ng, A. Y., 2009, May, ROS: an open-source Robot Operating System, In ICRA workshop on open source software (Vol. 3, No. 3.2, p. 5). Reid, R., Cann, A., Meiklejohn, C., Poli, L., Boeing, A., Braunl, T., 2013, Cooperative multi-robot navigation, exploration, mapping and object detection with ROS, In Intelligent Vehicles Symposium (IV), 2013 IEEE (pp. 1083-1088). IEEE. Sheridan, T. B., 2006, Supervisory control, Handbook of Human Factors and Ergonomics, Third Edition, 1025-1052. Toris, R., Kammerl, J., Lu, D. V., Lee, J., Jenkins, O. C., Osentoski, S.,..., Chernova, S., 2015, Robot Web Tools: Efficient messaging for cloud robotics, In Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on (pp. 4530-4537). IEEE.
Marco Antonio N. C. Arruda, Marcelo Becker Development of a Mobile Robots Supervisory System 6. ACKNOWLEDGMENTS Authors thanks to USP and CAPES for the support. 7. RESPONSABILITY The authors are the only responsible for this work information.