QUADCLOUD: A Rapid Response Force with Quadrotor Teams

Transcription

1 QUADCLOUD: A Rapid Response Force with Quadrotor Teams Kartik Mohta, Matthew Turpin, Alex Kushleyev, Daniel Mellinger, Nathan Michael and Vijay Kumar Abstract We describe the component technologies, the architecture and system design, and experimentation with a team of flying robots that can respond to emergencies or security threats where there is urgent need for situational awareness. We envision the team being launched either by high level commands from a dispatcher or automatically triggered by a threat detection system (for example, an alarm). Our first response team consists of autonomous quadrotors with downward-facing cameras that can navigate to a designated location in an urban environment and develop a integrated picture of areas around a building or a city block. We specifically address the design of the platform capable of autonomous navigation at speeds of over 3 mph, the control and estimation software, the algorithms for trajectory planning and allocation of robots to specific tasks, and a user interface that allows the specification of tasks with a situational awareness display. Keywords Aerial robotics Multi-robot systems Field robotics 1 Introduction Over the last decade, aerial robotics has received a lot of attention and there is extensive literature on both indoor [16, 18] and outdoor platforms. Indeed by some estimates, 1 the UAV market is estimated to exceed $2 B in the next 3 years, and 1 K. Mohta (B) M. Turpin V. Kumar GRASP Laboratory, University of Pennsylvania, Philadelphia, PA 1914, USA kmohta@seas.upenn.edu A. Kushleyev D. Mellinger KMel Robotics, Philadelphia, PA 19146, USA N. Michael Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA Springer International Publishing Switzerland 216 M.A. Hsieh et al. (eds.), Experimental Robotics, Springer Tracts in Advanced Robotics 19, DOI 1.17/ _38 577

2 578 K. Mohta et al. this forecast is conservative since it does not account for the thousands of micro- UAVs that are likely to be fielded in the near future. At the smaller (and lighter) end of the spectrum, quadrotors and hexrotors have become a standard platform for robotics research worldwide with the potential to support many indoor and outdoor applications [14]. In spite of the limitations of battery technologies and their inherent inefficiency, there are a number of short duration missions that make for very interesting applications. In this paper, we describe a rapid response force consisting of team of quadrotors that can quickly respond to disasters and emergencies by providing situational awareness before human responders can get to the scene. Our main goal in this paper is to describe the component technologies, our approach to the architecture and system design, and experimental data in support of an integrated system with applications to first response. Our paper builds on previous work on designing outdoor quadrotor platforms. A tutorial on quadrotors is provided in [14]. In particular, the work by Huang et al. [7] is notable in its study of lift and drag in flight conditions. Commercial platforms for applications such as aerial photography are available from companies like DJI, 2 Ascending Technologies, 3 and Parrot. Multi vehicle demonstrations have been shown at conferences like ICRA 213 and shows like the 5 robot outdoor event [2]. However, our main focus is on building a system of platforms that can function as a cohesive unit to perform a range of tasks within the broad scope of emergency response. There is extensive literature on the control for quadrotors. Backstepping approaches to control system design based on linear control laws are discussed in [3]. A nonlinear controller that incorporates the curvature of the SE(3) is described in [11, 15], while a closely related approach used in control showed significantly better performance over linear controllers [15]. Similarly the use of linear filters or extended Kalman Filters for state estimation in is quite standard [1]. However a UKF estimator yields significantly improved performance [17]. Our paper builds on these approaches as described later. Finally, the study of quadrotor teams is also relevant to this work. Our own previous work is described in [9, 13]. Representative papers from other groups who have done similar work include [6]. The work in this paper primarily addresses a framework for coordination of aerial robots without assigning labels to the individual vehicles and without specificity of the number of robots in the team. Another related body of work is the design of user interfaces for multi robot control [4]. However, there is relatively less literature on the design of user interfaces for large teams of aerial robots where the three-dimensional environments and the short term duration of the missions impose new constraints

3 QuadCloud: A Rapid Response Force System Design 2.1 Architecture The goal of QuadCloud is to monitor an area that is the size of a few city blocks, approximately a total area of 4 m 4 m, while being responsive to the operator s commands and dynamic information collected by one or more robots. We want the response times to these requests of surveillance to be less than 3 s. This requires the robots to have speeds of around 1 ms 1, and acceleration of 3 ms 2. We want the size of the robots to be as small as possible for ease of handling and deployment (imagine deploying a team from the back of a pickup truck), and have enough payload capacity to carry all the components needed for autonomy and surveillance. Since the system runs outdoors, the robots also need to have enough thrust to be able to cope with moderate winds. A key requirement for QuadCloud is that a single operator must be able to deploy, control and monitor the team of robots easily. This requires sufficient autonomy on each robot to able to handle the navigation task from trajectory generation to position stabilization on-board. Further, on each robot, we need downward-facing cameras to provide imagery, and the computational resources to allow processing of images at around 5 Hz from the on-board camera. Each robot must have the requisite on-board intelligence to look for salient information. In this paper, we do assume that the targets of interest are relatively simple and can be easily identified using regular cameras at heights of around 1 2 m. However, the algorithms must allow for false positives and a probabilistic representation of the belief state of the environment. Finally, in order to send data back from all parts of the monitored area and to respond to commands issued by the operator, each robot must be able to communicate with the base station within a 4 m distance. Thus the architecture must incorporate some elements of decentralized planning, control, estimation and target detection and localization while allowing for a centralized, cloud computing model for command and control by the user and task planning. Further, we want a framework in which the user is agnostic to the number of robots, their identities and what exactly their individual states are. This attribute of anonymity increases the robustness of the system to failures and decreases the overhead on the human user. The simplest architecture with these attributes is shown in Fig. 1. Robots are able to localize and control their motions to destinations, freeing the operator to work with high level task specifications such as target destinations or areas for surveillance. A centralized goal assignment and planning module decides which robot responds to what request and when. Individual robots independently decide how to follow these requests. Each robot periodically sends back its position estimates and images from the on-board camera to the base station. This information is presented to the operator on a simple user interface and through the user interface, the operator is also able to command goal positions which are sent to the planner.

4 58 K. Mohta et al. Base-station Input User Interface Positions Goals Positions Images Odroid-U2 Robot Quadrotor Goal Assignment and Planning Waypoints Fig. 1 Decentralized robot motion planning and control along with a centralized model for human interaction and task assignment Fig. 2 The KMel kquad5 quadrotor equipped with a u-blox NEO-6T GPS module, a Matrix Vision mvbluefox camera, a Ubiquity Networks Bullet M2 and a Odroid-U2 quad-core ARM single board computer 2.2 The Robots The robots used for this project, shown in Fig. 2, are quadrotors designed and developed by KMel Robotics. These quadrotors have a tip-to-tip diameter of about.54 m and weigh around.95 kg in the configuration used for this project. They are equipped with an ARM Cortex-M3 processor, 3-axis accelerometer, gyroscope, magnetometer and a pressure sensor while a u-blox NEO-6T GPS module was added in order to get position estimates. A control loop running at 5 Hz on the ARM processor stabilizes the attitude of the quadrotor. The communication with the on-board processor in order to receive sensor outputs and send thrust and attitude commands is done via a UART port. All the high-level computations on the robot are performed on an Odroid-U2 quad-core ARM single board computer. This compact board has four Cortex-A9 cores each running at up to 1.7 GHz allowing us to have a powerful processor in a small form-factor. The Odroid-U2 comes with a big passive heat-sink which we replaced with a small active heat-sink cutting its weight from 13 g to around 5 g. Each quadrotor is also equipped with a Matrix Vision mvbluefox camera in order to capture images for target detection and surveillance. We wanted each robot to send images to the base station for surveillance purposes, thus we had a requirement of

5 QuadCloud: A Rapid Response Force long-range communications with sufficient bandwidth. Since the Odroid-U2 does not have any built-in wifi, but has an ethernet port, we decided to use a Ubiquity Networks Bullet M2 which provides the required bandwidth with a range of more than 35 m. This allows us to stream compressed images at more than 4 fps from a single robot or around 5 fps from each robot when we have a team of 8 quadrotors sending images back. The Bullet M2 comes with a large and heavy antenna connector which we replaced with a smaller one, and we also removed the plastic shell for weight saving, reducing its weight from 18 to 5 g. The robots use a 3-cell 2.2 Ah LiPo battery which gives a flight time of around 8 1 min with the current configuration of the robot. The software system on the robot consists of three main components: estimation, control and image processing. The image processing is simple and is composed of detecting specified features (that characterize the targets) in the image and image compression for transmitting the images from the camera to the base station. The estimation and control subsystems are described in more detail in the next section. We use ROS as the framework for all the software running on the Odroid-U2, because it provides a good inter-process communication framework allowing transparent relocation of the processes across the machines allowing us to run a particular set of nodes on the robot for testing and running others on a separate computer speeding up the development phase. 2.3 Estimation and Control An overview of the estimation and control systems running on each robot is shown in Fig. 3. We use the GPS, IMU, magnetometer and pressure sensor for state estimation. First, the GPS latitude, longitude and height are transformed to a local cartesian frame using GeographicLib [8]. We ignore the height from the GPS measurement since it has a large drift and instead rely on the pressure sensor for getting the height. This processed output of the GPS along with the other sensor outputs, are then fed to a Trajectory Generator Odroid-U2 Pos Vel Acc Yaw Position Controller Pos Vel R UKF Thrust R des Attitude Controller R Onboard filter Quadrotor Motor Speeds IMU Mag Baro GPS Actuators Rigid body dynamics Sensors Fig. 3 The estimation and control systems running on each robot

6 582 K. Mohta et al. Unscented Kalman Filter (UKF) in order to generate full 6-DoF pose estimates. The UKF state that we use is, x = [ p T ṗ T ψθφa T b ] T where p is the world-frame position of the robot, ṗ is the world-frame velocity, ψ, θ and φ are the yaw, pitch and roll respectively and a b is the accelerometer bias along the three axes. Our control architecture has the common cascade structure of backstepping controllers [3, 16], with the attitude controller as the inner loop and the position controller as the outer loop around it. The controller is based on the non-linear controller developed in [11]. The attitude controller runs at a very high rate (6 Hz) on the on-board processor stabilizing the orientation of the quadrotor, allowing us to run the position controller at a much slower rate (5 Hz) on the Odroid-U2. This position controller takes position commands sent by a trajectory generator and, using the position estimates, converts them into thrust and attitude commands which are sent to the attitude controller running on the on-board processor. Finally, the attitude controller running on the quadrotor takes the thrust and attitude commands and converts them to commanded motor speeds. 2.4 Experimental Benchmarking We have done extensive testing in order to test the performance of our estimation and control algorithms. The estimates of the UKF during a representative hover test are shown in Fig. 4. The plots also show the output from an OptiTrack system which was set up outdoors in order to provide ground truth reference. From the plots we can see that the errors have a standard deviation of around 16 cm in the horizontal plane and 39 cm in the vertical direction. 3 Communication and Supervision Communication The base station communicates with each of the robots via wifi through the Bullet M2 high-power long-range wifi modules. We want each robot to send back position estimates at 5 Hz and image data at 5 Hz. The bandwidth requirement is dominated by the transmission of images from the multiple robots to the base station. The camera on each robot has a resolution of which leads to a raw gray-scale image size of approximately 1.2 MB, so for raw image transmission at 5 Hz we require a bandwidth of about 48 Mbps. The Bullet M2 claims a maximum bandwidth of around 65 Mbps, but in real-world testing, we got a data rate of about 5 Mbps. Thus

7 QuadCloud: A Rapid Response Force (a) x[m] OptiTrack UKF (b) Roll [deg] OptiTrack UKF y[m] z[m] Pitch [deg] Yaw [deg] Fig. 4 UKF estimates during a representative hovering experiment in an open area. Ground truth from an OptiTrack motion capture system, which was set up specifically for this experiment, are shown for reference. The position tracking errors had a standard deviation of.158 m in the horizontal direction and.386 m in the vertical direction. a Position, b orientation it is not possible to send raw images back from each of the robots at the desired rate. To reduce the bandwidth requirement, we decided to jpeg compress the images before sending. This brings down the size of the images from 1.2 MB to about 13 kb allowing us to stream images at 5 Hz from up to 1 robots. If we want to add more robots, we would need to decrease the frame rate of the image data being sent back from the robots. A frame rate of 2 Hz is sufficient for surveillance purposes and would allow us to scale to around 2 robots. User Interface Since all the computations for autonomy are being done on the robots themselves, the operator does not need a very powerful base station to control the team; the base station can just be a small laptop. As mentioned earlier, the robots send their position estimates to the base station. This information is presented to the operator in the form of markers on an overhead schematic map of the area. Figure 8 shows some screenshots of the user interface during an experiment. In addition to monitoring the system, the user is able to send goal positions to the system without needing to specify which robot is assigned which goal. Using the algorithm described in the next section, the system assigns the goals to the robots in order to minimize the maximum travel time and plans trajectories for each of them. This reduces the cognitive burden on the operator by allowing the operator to focus on the high-level tasks.

8 584 K. Mohta et al. 4 Combining Assignment and Trajectory Planning To safely navigate the team of robots to goal locations, a motion planning algorithm is required that computes collision-free trajectories and respects the dynamics of the robots. It is well known that extending single robot motion planners to plan trajectories for a team of robots implies exponential computational complexity [5]. One attempt to solve this problem of computational intractability is to use a two step algorithm that decouples the path from the time parameterization of the trajectories [1]. These decoupled approaches first plan motions for each individual robot while disregarding collisions with other robots. The second step is to specify the time parameterization the robot follows its path. Unfortunately, these approaches are not complete and cannot guarantee they will find a solution if one exists. Fortunately, our team of robots are identical and we can exploit this interchangeability to generate collision-free time parameterized trajectories in a computationally tractable manner using the Gap algorithm. Goal Assignment and Planning (Gap) In this paper, we leverage the authors previous work Goal Assignment and Planning, or Gap [19, 2], to plan complete, collision-free trajectories with a computational complexity of O(N 3 ). This approach assumes full knowledge of obstacles present in the environment. The robots are modeled as second order systems with spherical extent. The radius of the robots is taken at a conservative 2 m to ensure that errors in localization will not cause a catastrophic collision. The Gap algorithm is a decoupled motion planner that maintains completeness by leveraging the interchangeability of the robots. This algorithm begins by finding the cost associated with planning trajectories from the initial state of each robot to every goal location. We use Dijkstra s algorithm to quickly find these N 2 motion plans where robot i has cost C ij to travel to goal j. The next step is the assignment of goals to robots where each robot is assigned to one goal. This assignment can be represented by a permutation matrix φ, where φ ij = 1 if and only if robot i is assigned to goal j. The Hungarian Algorithm is used to find the assignment which minimizes the maximum cost assignment: minimize (φ ij C ij ) p φ i=1,2,...,n j=1,2,...,n where p is a very large constant. In practice, p = 5 is used. Then, robots are prioritized using simple geometric considerations that are fully detailed in [2]. Finally, robots are assigned their full time parameterization to construct trajectories that guarantee collision avoidance. For a team of 6 robots, these plans are generated in under.1 seconds. Additional details of this algorithm including boundary condition requirements for completeness are presented in [2].

9 QuadCloud: A Rapid Response Force Experimental Results 5.1 High Speed Tests Since we are flying outdoors, we can fly along long trajectories, which provide sufficient distance to accelerate to high speeds. We ran some high speed tests with one of the quadcloud robots where we commanded the robot to fly approximately 8 1 m at speeds up to 15 ms 1 and looked at the effect of drag. For a quadrotor, if we ignore drag, since the thrust is only along the body Z-axis, we expect the accelerometer on the quadrotor to measure zero acceleration on the X and Y axes while the Z axis will measure the effect of thrust [12]. But, as shown in [12], the accelerometer on a quadrotor measures some acceleration in X and Y axes when moving due to the drag force acting on the robot. From Fig. 5, we get, D cos φ a x = m a z = T D sin φ m where D is the drag force and T is the thrust. Thus, when the robot is flying fast and there is considerable drag, we expect it to have a significant measurement on the accelerometer X-axis. In our data (Fig. 6), we can see that the accelerometer measures approximately 4 ms 2 acceleration along the X-axis when the quadrotor is flying at speeds of around 15 ms 1. Using the above equations, from the accelerometer measurement and orientation estimates, we can estimate the drag force on the robot. The computation gives the magnitude of the drag force to be approximately 4.2 N which is significant considering that it is half of the gravitational force acting on the robot. This is a large external force that is not modeled in our estimator and can lead to errors in our position and orientation estimates. We are still looking into the effects of drag on the orientation estimates and also ways in which we can use the drag measurements for velocity estimates. Fig. 5 Forces acting on a quadrotor moving towards the right with velocity v Drag φ Thrust v Gravity

10 586 K. Mohta et al. (a) Velocity Accelerometer (b) Roll [deg] Pitch [deg] IMU UKF Fig. 6 Measurements during a high speed test. a Velocity and acceleration, b orientation Fig. 7 An outdoor experiment with six robots 5.2 Experimental Results with Multiple Robots in an Open Field Here, we describe an experiment we conducted with six robots in an open field (Fig. 7). Instead of flying around real-world obstacles, we provide virtual obstacles to the planner so that we can perform the experiment in a much safer manner. Figure 8

11 QuadCloud: A Rapid Response Force (a) (b) (c) (d) Fig. 8 A series of snapshots of the user interface while running an experiment with six robots. a Initial positions, b user species goal positions, c planner assigns the goals and generates trajectories, d actual trajectories followed by the robots shows the various steps involved in the experiment. We start the robots from the ground with a separation of about 4 m between each other so that we can take-off without worrying about collisions between the robots. Once they take-off and reach a specified height we switch to the trajectory tracker which takes in inputs from the central planner and sends position commands to the position controller. Once this stage is reached (Fig. 8a), the operator can command the robots from the user interface and send the robots to the desired goal positions (Fig. 8b). Upon receiving the goal positions, the planner assigns the goals to the robots and plans trajectories for each of them (Fig. 8c) which are then followed by the robots (Fig. 8d). As mentioned in the previous section, the planner models the robots as circles with a radius of 2 m, even though the actual robot radius is around.3 m, in order

12 588 K. Mohta et al. Table 1 Mean error between the desired position and estimated position in the horizontal plane for each robot during the six robot experiment Robot XY error (m) to allow some localization and control errors. Table 1 provides estimates of the controller errors during the six-robot experiment which shows that most of the robots have errors between.2.7 m. Adding the localization error of approximately.2.5 m gives us a total error of around m thus justifying the choice in the planner. Aggregation of Visual Imagery for Situational Awareness The base station receives the images from the quadrotors as well as their pose estimates. Using the pose information, we can correct the perspective distortion of the image and project them onto the ground plane. This allows us to create an overhead map of the environment using the team of quadrotors. An example of this is shown in Fig. 9 where images from three quadrotors are being used. Fig. 9 A mosaic being created using the images from three robots. The robot positions can be seen by the red circles in the image

13 QuadCloud: A Rapid Response Force Conclusion and Future Work We described the component technologies, the architecture and the software for QuadCloud, a prototype of a rapid response team for first response and disaster recovery. The key contributions are (a) the design of robust outdoor platforms with speeds that exceed 3 mph with an absolute positioning accuracy of well under 5 cm; (b) the architecture design that enables the rapid deployment of a small team of quadrotors for surveillance without specifying the roles of individuals; and (c) the design of algorithms for estimation, control and planning for multiple quadrotors. We showed experimental results with a team of six quadrotors. Our analysis suggests that the team size can be scaled up to 2 units without compromising system performance. Our future work would address vision-based stabilization to increase robustness to GPS drop out and experimentation with larger teams. Acknowledgments We are grateful for the support of ARL grant W911NF-8-2-4, ONR grants N , N and N , NSF grants PFI and IIS , and TerraSwarm, one of six centers of STARnet, a Semiconductor Research Corporation program sponsored by MARCO and DARPA. References 1. Achtelik, M., Achtelik, M., Weiss, S., Siegwart, R.: Onboard IMU and monocular vision based control for MAVs in unknown in- and outdoor environments. In: IEEE International Conference on Robotics and Automation (ICRA) (211) 2. Ars Electronica: Spaxels Ars Electronica Quadcopter Swarm Bouabdallah, S., Siegwart, R.: Full control of a quadrotor. In: 27 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp (27) 4. Envarli, I.C., Adams, J.A.: Task lists for human-multiple robot interaction. In: IEEE International Workshop on Robot and Human Interactive Communication, 25, pp IEEE (25) 5. Erdmann, M., Lozano-Perez, T.: On multiple moving objects. In: 1986 IEEE International Conference on Robotics and Automation. vol. 3, pp (1986) 6. Hoffmann, G.M., Rajnarayan, D.G., Waslander, S.L., Dostal, D., Jang, J.S., Tomlin, C.J.: The Stanford testbed of autonomous rotorcraft for multi agent control (STARMAC). In: The 23rd Digital Avionics Systems Conference, pp. 12.E.4/1 1. IEEE (24) 7. Huang, H., Hoffmann, G., Waslander, S., Tomlin, C.: Aerodynamics and control of autonomous quadrotor helicopters in aggressive maneuvering. In: 29 IEEE International Conference on Robotics and Automation (ICRA), pp (29) 8. Karney, C.F.F.: GeographicLib library Kushleyev, A., Mellinger, D., Kumar, V.: Towards a swarm of agile micro quadrotors. In: Robotics: Science and Systems (212) 1. Latombe, J.C.: Robot motion planning (1991) 11. Lee, T., Leok, M., McClamroch, N.H.: Geometric tracking control of a quadrotor UAV on SE(3). In: 21 49th IEEE Conference on Decision and Control (CDC), pp (21) 12. Leishman, R.C., Macdonald, J.C., Beard, R.W., McLain, T.W.: Quadrotors and accelerometers: state estimation with an improved dynamic model. IEEE Control Syst. Mag. 34(1), (214)

14 59 K. Mohta et al. 13. Lindsey, Q., Mellinger, D., Kumar, V.: Construction with quadrotor teams. Auton. Robots 33(3), (212) 14. Mahony, R., Kumar, V., Corke, P.: Multirotor aerial vehicles: modeling, estimation, and control of quadrotor. IEEE Robot. Autom. Mag. 19(3), 2 32 (212) 15. Mellinger, D., Kumar, V.: Minimum snap trajectory generation and control for quadrotors. In: 211 IEEE International Conference on Robotics and Automation (ICRA), pp Shanghai (211) 16. Michael, N., Mellinger, D., Lindsey, Q., Kumar, V.: The GRASP multiple micro-uav testbed. IEEE Robot. Autom. Mag. 17(3), (21) 17. Shen, S., Mulgaonkar, Y., Michael, N., Kumar, V.: Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft MAV. In: IEEE International Conference on Robotics and Automation, Hong Kong (214) 18. Shen, S., Michael, N., Kumar, V.: Autonomous multi-floor indoor navigation with a computationally constrained mav. In: 211 IEEE International Conference on Robotics and Automation (ICRA), pp (211) 19. Turpin, M., Michael, N., Kumar, V.: Concurrent assignment and planning of trajectories for large teams of interchangeable robots. In: 213 IEEE International Conference on Robotics and Automation (ICRA), pp Karlsruhe (213a) 2. Turpin, M., Mohta, K., Michael, N., Kumar, V.: Goal assignment and trajectory planning for large teams of aerial robots. In: Proceedings of Robotics: Science and Systems. Berlin (213b)