The Real-Time Development and Deployment of a Cooperative Multi-UAV System

Transcription

1 The Real-Time Development and Deployment of a Cooperative Multi-UAV System Ali Haydar Göktoǧan 1, Salah Sukkarieh, Gürçe Işikyildiz, Eric Nettleton, Matthew Ridley, Jong-Hyuk Kim, Jeremy Randle, and Stuart Wishart 2 1 Australian Centre for Field Robotics School of Aerospace, Mechanical and Mechatronic Engineering The Rose Street Building J04 The University of Sydney, NSW 2006, Australia agoktogan@acfr.usyd.edu.au, 2 BAE Systems, Australia Abstract. This paper presents a systems view of the Autonomous Navigation and Sensing Experimental Research (ANSER) project. ANSER is one of the most advanced UAV projects of its kind, which is aimed at demonstrating Decentralised Data Fusion (DDF) and Simultaneous Localisation and Map (SLAM) building on multiple cooperative UAVs. The demonstration of DDF and SLAM require both navigation and terrain sensors to be carried onboard by the UAVs. These include an INS/GPS navigation system, millimeter wave (MMW) radar, and both a single vision node and a vision node augmented with a laser system. This paper presents the implementation of the algorithms, sensors and platforms, along with their interaction to address this demonstration. Furthermore, the paper will present the high fidelity real time simulator which is implemented to thoroughly test all algorithms and sensors before actual implementation in the environment. 1 Introduction The aim of the Autonomous Navigation and Sensing Experimental Research (ANSER) project is to demonstrate novel algorithms in the area of Decentralised Data Fusion (DDF) and Simultaneous Localisation and Map (SLAM) building on multiple, cooperative UAVs. DDF is an architecture which allows for the ability to decentralise estimation algorithms across multiple sensing nodes so that there is no common failure point. The architecture also provides the ability for scalable and modular data fusion systems thus providing solutions to many network-centric problems which are currently being faced. The paradigm uses a Kalman filter (or more commonly its information form equivalent) in its decentralised form so that local sensor nodes have global knowledge. One such data estimation problem is distributed target tracking. In this scenario multiple sensor nodes are tracking multiple targets. In most forms a centralised architecture is implemented. However using the DDF paradigm allows A. Yazici and C. Şener (Eds.): ISCIS 2003, LNCS 2869, pp , c Springer-Verlag Berlin Heidelberg 2003

2 The Real-Time Development and Deployment 577 for not only a distributed network but also a decentralised one, thus allowing nodes to come in and out of the network in run time. Another example is the implementation of SLAM. In the single vehicle case, SLAM is an algorithm which allows for the simultaneous building up of an environment map, and the use of this map to then localise the vehicle within it. This is true autonomous navigation since no a priori information about the map or vehicle location is required. In the distributed multi-vehicle case, the map between vehicles is shared, better improving the its quality and the quality of the vehicle locations. Applying DDF to this navigation structure then further allows for decentralised paradigm to be applied. In the literature there are numerous innovative research approaches to DDF and SLAM. However the algorithms have been demonstrated on static sensor networks [6] or robotic platforms with the slow and/or simple dynamics [1] [3]. Performing these algorithms and architectures on multiple UAVs is a very challenging exercise [13]. UAVs pose a difficult platform to work on. They have both complex and very high dynamics and flight tests are both expensive and are high risk operations. Any hardware or software system failure may have catastrophic results in any UAV experiment. In this paper we present a system s view of the ANSER project. Section 2 will first introduce the fundamental concepts of both DDF and SLAM. In section 3 the UAVs are presented. Both the navigation and terrain sensors will be presented in section 4. The communication framework CommLibX, and realtime multi UAV simulator, mission control and ground station systems will be presented in section 5. This is an important part of the system since it is used to test all electronic hardware, software modules, their interoperatability and algorithms implemented in the system [5]. Finally the conclusion and further work is provided in section 6. 2 Decentralised Data Fusion A Decentralised Data Fusion (DDF) system comprises of a network of sensor nodes each incorporating a sensor together with local processing and communication capabilities. Each node runs its own local data fusion algorithm and communicates information to other nodes in its neighbourhood. Nodes fuse incoming information with local estimates to produce a global picture of the state of the world. This strategy results in a network of sensor nodes in which the fusion process is decentralised. A general decentralised data fusion system can be characterised by three basic constraints [6]: 1. There is no single central fusion centre and no node should be central to the operation of the network. 2. There is no common communications facility - communications must be kept on a strictly node-to-node basis. 3. Each node has knowledge only of its immediate neighbours - there is no global knowledge of the network topology.

3 578 A.H. Göktoǧan et al. These constraints combine to ensure that there is no single element that is critical to the operation of the network. If any node or communication link should fail, the result is a gradual degradation in network performance rather than catastrophic failure. The conventional structure for multi-sensor systems requires that all sensors send observation information back to a single central point for fusion. This has some advantages in that all information is known at a single point and so, in principle, fusion is based on the best possible information. However centralised structures also have a number of undesirable characteristics: Using a centralised resource for fusion creates both computational and communication bottlenecks as the system increases in size. This limits the ability of a system to scale to large numbers of sensors. A second key problem is that a central fusion site makes the whole system vulnerable to catastrophic failure of the fusion processor. The decentralised system architecture implemented in this research [8] offers a number of advantages over conventional centralised or hierarchical architectures: Scalability: Eliminating the central fusion centre and any common communication facility ensures that the system is scalable as there are no limits imposed by centralised computational bottlenecks or lack of communication bandwidth. Survivability: Ensuring that no node is central and that no global knowledge of the network topology is required for fusion means that the system can be made survivable to the on-line loss (or addition) of sensing nodes and to dynamic changes in the network structure. Modularity: As all fusion processes must take place locally at each sensor site and no global knowledge of the network is required a priori, nodes can be constructed and programmed in a modular fashion. 2.1 Decentralised Applications The DDF architecture is initially applied to the problem of tracking multiple ground targets using a network of airborne sensing platforms [11]. The objective of this problem is for the sensing nodes on each platform to build a composite global picture of targets in an environment using both local sensor observations and information communicated from other platforms in a decentralised manner. This is accomplished by integrating the terrain sensing payload on each aircraft with local processing and communication facilities to form a sensing node. A decentralised network of these nodes can then be formed, where the sensing nodes communicate directly in terms of target information to have a local estimate using global information. This communication has a number of positive effects: The extra information available to nodes through the communicated information improves the quality of the estimates. The robustness of a DDF architecture to node failure is equivalent in the airborne tracking problem to aircraft failure. If an aircraft is damaged or

4 The Real-Time Development and Deployment 579 destroyed, the information it gathered is not lost in a DDF network as it has been communicated to all other vehicles. The picture of all targets in a region is available to all aircraft in the DDF network as soon as any single platform visits that area. Aircraft can therefore have a picture of targets in an area that they have not yet visited. Simultaneous Localisation and Map building (SLAM) for a robotic platform is a process of building a map of the surrounding environment while simultaneously using this map to localise [2]. The DDF architecture is used for decentralising SLAM. The UAV uses a terrain sensor to make relative observations to stationary features or landmarks in the environment. As these observations are relative to the vehicle, an error in the platform pose results in a corresponding error in the landmark location. Over time, the entire map becomes correlated through the motion of the vehicle. For a single platform using the SLAM algorithm to decrease its location uncertainty, it must revisit features it has seen previously. However, with multiple platforms it is possible to share information and so generate a more accurate map and location estimate. Indeed, if certain platforms are directed to revisit shared features, then there will be significant location estimate improvements for all platforms. 3 The UAV The Brumby MkIII UAVs are the robotic platforms used in the ANSER project. The Brumby UAV has been developed over the past six years at The University of Sydney, Australia. In order to meet an increasing payload weight requirement for the research projects, the Brumby UAVs have incorporated a number of changes. Currently we have three different models as shown in table 1. They are designed as test beds for various navigation algorithms, payload sensors and real time distributed data fusion. They offer a large payload volume, a speed range of Knots, a payload mass of over 13kg and over 1 hour of flight capability. The Brumby UAVs are delta winged, pusher aircrafts. The pusher design leaves a clear space in the front for sensors which provides unobstructed view. The Brumby airframe is modular in construction to enable the replacement or upgrade of each component. Each payload position is provided with 24V power and an Ethernet and/or CAN (Controller Area Network) connection to the other onboard systems. The airframe is also capable of carrying a specially designed gimballed scanner that enable radar or laser sensors to be scanned across the ground. 4 Sensors The sensor are grouped as either flight or payload sensors. Flight sensors provide data to be used to monitor and control the UAV. Flight sensors include an Inertial Measurement Module (IMU), Global Positioning Sensor (GPS). Payload

5 580 A.H. Göktoǧan et al. Table 1. Brumby Mk I-II-III UAV specifications MkI MkII MkIII MTOW 25kg 30kg 45kg Wing Span 2.3m 2.8m 2.9m Engine 74cc 80cc 150cc Power 5Hp 5.5Hp 16Hp Max Speed 100 Knots 100 Knots 100 Knots sensors are those used for target detection from the terrain for DDF target tracking and SLAM. Each payload sensor has its own PC104+ based processing and ethernet and/or CAN based communication module. MMW Radar Payload Laser/Vision Payload Vision Camera Payload Fig. 1. Optionally either MMW radar or Laser/Vision payloads can be mounted in the nose cone of the Brumby Mark-III UAV. The vision camera is located under the fuselage Vision Payload: Each UAV is equiped with a vision payload which is capable of detecting artificial targets on the ground. The vision payload consists of a 266MHz PC104+ computer with frame grabber. An image stream of size 50Hz is obtained by extracting the odd and even fields of an interlaced stream of size 25Hz. For DDF picture compilation flights, the vision

6 The Real-Time Development and Deployment 581 node must assimilate both the navigation solution of the platform and raw sensor data into the information matrix and vectors, for use with the information filter as defined in [7]. Laser/Vision Payload: Laser/Vision payload which is also called a Secondary Imaging System (SIS) consists of a laser range finder and a complete vision system. Although the vision payload would provide accurate bearing measurement, its range measurement is very poor if the target size is known. In this payload, the vision system is suplemented with the laser range finder for accurate range measurement. Radar Payload: It is important for DDF that at least one of the platform carries a sensor which returns Range, Bearing and Elevation (RBE) of the ground targets [12][9]. We have developed a 77GHz millimeter wave radar operating on a frequency modulated continuous-wave (FMCW) principle. The whole MMW radar unit is mounted on an attitude-stabilizing gimbal to keep the grazing angle of the radar beam fixed. The gimbal has two axes of freedom; one along the roll and the other on the pitch axis. The gimbal is controlled in real time to counter the motion of the UAV on these axes [4]. The scanning mirror on top of the gimbal reflects the transmitted and received signals to and from the horn antenna of the MMW radar. Received radar signals are processed by dual ADSP SHARC Digital Signal Processor (DSP) for real-time feature extraction [4] [10]. 5 System Software Components CommLibX: The high dynamics nature of the UAV requires both real-time monitoring and control software systems. Each UAV carries a PC104+ based Flight Control Computer (FCC) performing real-time navigation, guidance and flight control. The FCC uses the QNX real-time kernel and communicates with the ground control station via wireless radio modem and with other on board computer nodes via ethernet and/or CAN based local network. UAVs communicate between each other via radio ethernet. The sensor nodes run different OS; Vision and Laser/Vision nodes use Linux while radar uses WinNT. This variety of OS is due to availability of driver support that was available. As communication appears to be a challenging problem, it is solved with a novel communication framework, CommLibX. CommLibX is developed to address OS and transport layer independent multi channel real-time communication [5]. RMUS: Every software and hardware unit running in the UAV experiments must be tested thoroughly. Individual of-line testing of these units may not be sufficient. Their real-time inter and intra-operateability with real hardware must be tested. A Real-time Multi-UAV Simulator (RMUS) has been developed to address these problems. RMUS can be used both in real-time hardware-inthe-loop (HIL) and off-line simulations. It is a flexible distributed simulation architecture designed to address multi-uav experiments. Details of the RMUS architecture, CommLibX framework and their applications in various multi-uav experiments can be found in [5].

7 582 A.H. Göktoǧan et al. Fig. 2. Mission Control System (left) has been developed on the RMUS architecture. Ground Station System (right) allows real-time display and log all telemetered UAV data. MCX: RMUS architecture allows us to develop real-time on-line applications. The Mission Control System(MCX) is an RMUS based application which is used to monitor multiple UAV locations, attitude, paths, sensor frustums and target location estimates in real-time. The logging and replay function along with the interactive 3D display on MCX makes it possible to look at the scene from any angle before, during and after the flight tests. Ground Station: Each UAV has its own Ground Station (GS) system in the ground control center. The purpose of the GS is to display and log all telemetered data of its associated UAV in real-time. GS with its utility programs can replay the logged flight for further off-line analysis of the flight mission and UAV performance. 6 Conclusion and Future Work This paper has presented a summary of the DDF architecture, picture compilation and SLAM algorithms and some of the major system components of the ANSER project. In the current phase of the project, five Brumby Mk III UAVs have been built and tested. So far more than 40 successful flight missions have been performed for data collection. We also successfully demonstrated real-time DDF picture compilation on multi-uavs. Multi-UAV SLAM is demonstrated on real-time simulation with real flight data. Demonstration of real-time multi-uav SLAM is scheduled for late Four UAVs will be flown simultaneously for this demonstration. The Brumby UAV airframe development continues for new models for larger and heavier payload capacity. In the RMUS environment we are currently testing the application of the DDF architecture for decentralised multi-uav coordinated control.

8 The Real-Time Development and Deployment 583 Acknowledgements. The authors wish to thank BAE Systems for their support in providing funding and systems engineering towards this project. Also, thanks to Alan Trinder from ACFR for his assistance with the UAV construction. References 1. Gamini Dissanayake, Hugh Durrant-Whyte, and Tim Bailey. A computationally efficient solution to the simultaneous localisation and map building (slam) problem. In IEEE International Conference on Robotics and Automation 2000, pages , San Francisco CA, M. Dissanayake, P. Newman, H. Durrant-Whyte, S. Clark, and M. Csorba. An experimental and theoretical investigation into simultaneous localisation and map building. In 6th International Symposium on Experimental Robotics, pages , Sydney, Hugh Durrant-Whyte, Somajyoti Majumder, Marc de Battista, and Steve Scheding. A bayesian algorithm for simultaneous localisation and map building. In 10th International Symposium of Robotics Research - ISRR 2001, Lorne, Victoria, Australia, Ali Haydar Göktoǧan, Graham Brooker,, and Salah Sukkarieh. A compact millimeter wave radar sensor for unmanned air vehicles. In Field and Service Robotics, FSR 03, Lake Yamanaka, Japan, Ali Haydar Göktoǧan, Eric Nettleton, Matthew Ridley, and Salah Sukkarieh. Real time multi-uav simulator. In IEEE International Conference on Robotics and Automation, Taipei, Taiwan, S. Grime and H. Durrant-Whyte. Data fusion in decentralized sensor networks. Control Engineering Practice, 2, No 5: , E. Nettleton and H. Durrant-Whyte. Delayed and asequent data in decentralised sensing networks. In G.T. McKee and P.S. Schenker, editors, Sensor Fusion and Decentralised Control in Robotic Systems IV, volume Proc. SPIE, pages 1 9, E. Nettleton, H. Durrant-Whyte, and S. Sukkarieh. A robust architecture for decentralised data fusion. In International Conference on Advanced Robotics, Eric W. Nettleton, Hugh F. Durrant-Whyte, Peter W. Gibbens, and Ali Haydar Göktoǧan. Multiple platform localisation and map building. In G.T. McKee Schenker and P.S., editors, Sensor Fusion and Decentralised Control in Robotic Stystems III, volume 4196, pages , Boston,USA, Matthew Ridley, Ali Haydar Göktoǧan, Graham Brooker, and Salah Sukkarieh. Sensor nodes for decentralised data fusion with multiple uninhabited air vehicles. In The 2nd International Workshop on Information Processing in Sensor Networks (IPSN 03), Palo Alto, California, USA, S. Sukkarieh, E. Nettleton, J. Kim, M. Ridley, A. Goktoǧan, and H. Durrant- Whyte. The ANSER project - multi-uav data fusion. IJRR (to appear), Salah Sukkarieh, Ali Haydar Göktoǧan, Jong-Hyuk Kim, Eric Nettleton, Jeremy Randle, Matthew Ridley, Stuart Wishart, and Hugh Durrant-Whyte. Cooperative data fusion and control amongst multiple uninhabited air vehicles. In ISER 2002 Seventh International Symposium on Experimental Robotics, Salah Sukkarieh, Eric Nettleton, Jong-Hyuk Kim, Matthew Ridley, Ali Haydar Göktoǧan, and Hugh Durrant-Whyte. The anser project-multi uav data fusion. Internatioal Journal of Robotics Research, 2002.