The Path to Real World Autonomy for Autonomous Surface Vehicles

Transcription

1 Authors: Howard Tripp, PhD, MSc, MA (Cantab), Autonomous Systems R&D Lead, ASV Global, Portchester, United Kingdom, Richard Daltry, CEng, MRINA, Technical Director, ASV Global, Portchester, United Kingdom, Abstract This paper presents a new autonomy architecture, developed and tested over the past two years which has taken a dumb remote-controlled platform and built it into a state-of-the-art Autonomous Surface Vehicle, able to navigate safely at high speeds in a COLREG aware manner, across some of the busiest waterways in the world. The architecture is made up of a hierarchy of decision making components sharing a common world model. These components consist of: a route planner, path planner and last response engine. Each element of the hierarchy is working to reduce risk within decreasing time and space horizons, ultimately providing a minimum risk navigational solution. MAST and the collision avoidance system is now at the advanced prototype stage hence. Dstl and ASV welcome interest from other organisations who would like to explore and experiment with running their systems either onboard or otherwise in command. 1. Context ASV Global has built a reputation as the leading manufacturer of (ASVs). In parallel to developing a wide product range of platforms, ASV Global has developed its own autonomous control system, ASView. ASView has been in development for 9 years and is actively used on over 80 platforms. It supports chart based operator situational awareness with pre-planned vehicle actions. The system currently relies on human input with limited on-board decision making. To fuel growth of the industry, the goal moving forward is to develop a commercially useful autonomy system including and exceeding collision avoidance. To ASV Global, real world autonomy means increasing the capability of the ASV to undertake decision making and thus enabling the remote operator to focus on safety critical issues and utilise natural human intuition. To provide the most efficient use of resources throughout the system as a whole, including the human element. In pursuit of this goal, ASV Global has developed an architecture which provides a robust foundation for the development of autonomy over the next decade. Throughout the design of the architecture a key element is the capability for expandability and future improvement, ensuring the system can meet current and future industry goals. Initial tests and demonstrations of the system utilised data fused from radar, cameras and AIS sensors. This has been used in proof of concept demonstrations and showcased at the recent Unmanned Warrior naval exercise in the UK, deployed on multiple ASVs. Collision avoidance behaviours, taking into account important aspects of the collision regulations at sea, have been a critical part of this early development around which operational behaviours have been built.

2 These developments are accelerating the adoption of ASVs in the maritime industry by enabling capabilities such as over the horizon operations and the control of fleets of ASVs with minimal human input. Examples of applications spread across both military and commercial domains and include ISTAR, hydrographic survey, support of underwater systems and in the future, commercial shipping. The programme was directly funded by UK MoD through Dstl and led by ASV Global in partnership with Roke Manor Research and Cambridge Pixel. 2. Project Introduction The developed capability centred around the guiding vision of a high speed (>30knots) Unmanned Surface Vehicle (USV), notionally based outside Portsmouth Harbour, that could be tasked to autonomously intercept and investigate a target many nautical miles out at sea. This required development of high speed collision avoidance algorithms and on the fly route, path and trajectory replanning all running directly on-board the USV for total self-sufficiency. The USV used was the Dstl Maritime Autonomy Surface Testbed (MAST) vehicle based on a bladerunner racing hull. Fig 1 - The MAST platform is a 34ft Bladerunner racing hull retro-fitted with ASV control system and extensive additional testbed hardware and software The programme focused on researching what could be achieved using only current COTS sensor systems including AIS, radar, cameras and accelerometers which was successful and demonstrated that a great deal can be achieved using this approach. However, sensing remains a crucial performance driver and that there are many system/ platform level trade-offs to be considered and no obvious one-size-fits-all sensor suite. The concept and architecture was developed from scratch and is based on directly manipulating risk and uncertainty. This gives two key benefits: firstly a direct way for users to dynamically adjust risk appetites as mission goals change; and secondly a continuous realtime visualisation of what I am thinking over the next few seconds, minutes and hours for intuitive operator supervision and trust building. This contrasts with many collision avoidance systems that are purely reactive, and hence give no advanced warning about their intentions. The system was incrementally developed with an agile customer demonstration drum beat and numerous R&D testing activities including 54 days of MAST on the water R&D and testing time. This practical real-world approach is significant as handling the real world noise was consistently more challenging than expected in all areas, and rapidly raised the TRL of the subsystems. As a result the programme has delivered hands-on capability and created the foundations of a testbed that is available for further exploitation.

3 A competitor analysis carried out late in the programme showed that this is potentially the fastest full autonomy USV capability in the world; that the MAST platform creates more sensor processing challenges than most competitors; that other collision avoidance systems are more reactive and tend to plan over much shorter timeframes; and has a sensor fit that is broadly similar/comparable to other approaches. In short, this programme is one of the world leaders in this field to the best of our knowledge. Equally this is evidenced by the fact ASV Global have licenced the technology developed back from Dstl via the Ploughshares organisation, for a significant upfront cost and potentially recurring license fees as it is commercialised. A further indirect benefit is that this programme has been instrumental in developing better working relationships with local and national authorities including Southampton Vessel Traffic Service, Portsmouth Queens Harbour Master and the Maritime Coastguard Agency. These authorities approved autonomous operations for this programme in their busy controlled waters, something that would have been highly unlikely just 2 years ago. 3. Technology Overview Much of the development work undertaken in this programme is commercially sensitive and hence much of the system detail is deliberately restricted to high level discussion. 3.1 System Overview Fig 1 - Schematic overview of the Advanced USV system

4 The Advanced USV System runs directly onboard the USV and consists of a hierarchy of decision making components sharing a common world model. Each element of the hierarchy works to reduce risk with decreasing time and space horizons, ultimately providing a minimum risk navigation solution: The Route Planner generates routes over long distances taking account of water depth (and potentially shipping lanes). The Path Planner considers the next few minutes, choosing paths that minimises collision risk in a COLREG compliant fashion with other vessels. The Trajectory Planner manages the next few seconds to smoothly control and predict USV motion based on a dynamics model. The Last Response monitors the immediate vicinity and provides a safety critical engine cut out, if an unexpected obstacle is detected. The system includes a sensor processing pipeline, with a 360 panoramic camera, broadband radar, AIS sensor, compass, GPS and an IMU, all of which is combined into tracks and vessel position information in a world model. Further processing then calculates track statistics and generates future trajectory predictions for targets. These predictions are processed based on COLREG rules within a 3D risk landscape to calculate risks of being in the wrong place at the wrong time. Chart features such as buoyage and shore data is also added to the risk landscape for use by the path planner. The system is at the point where it would be possible to command MAST to autonomously navigate around the Isle of Wight under suitable weather conditions. 3.2 Programme Achievements The Advanced USV programme was predicated on a series of demonstrations to showcase the development of advanced autonomous navigation. Demo 1 (November 2015) demonstrated a basic risk-based path planner undertaking simple collision avoidance in an open water environment at low speed (~15 knots) using only AIS sensor input from a single target. Achieved capability for one-on-one crossing situations. Demo 2 (March 2016) added collision avoidance with multiple vessels using a fused AIS and radar sensor input and sea state monitoring to limit speed. Achieved capability for reasoning about compound multi party crossing situations. Unmanned Warrior 2016 (October 2016) The MAST and collision avoidance system was used extensively in military exercises over a three week period around the Isle of Skye. Achieved long endurance testing in more hostile environments with additional behavioural modes of operation. Demo 3 (November 2016) demonstrated multiple vessel collision avoidance at higher speed with automated last response functionality and basic route planning. Achieved the ability to operate fully unmanned at 30knots with autonomous emergency stop.

5 Demo 4 (February 2017) demonstrated the addition of an initial vision processing system, a trajectory planner, AIS metadata processing to support detection/classification of objects and live extraction of chart data (both depth and objects). Achieved the integration of vision processing detections, basic vessel classification and dynamics aware planning for more advanced COLREG compliance. Demo 5 (March 2017) Added improved vision processing and showcased an integrated multi-phase representative mission on a long transit through shipping lanes and controlled waters. Achieved an end-toend mission with unscripted collision avoidance. (Note: severe weather on the day unfortunately cut short the demo on safety grounds, but the system operated entirely successfully as expected). 3.3 Specific Technologies Developed Core System One of the key insights from this programme was the representation of rules within the International Collision Regulations at Sea (COLREGs) as dynamic mathematical functions to represent risk. The system uses information about the relative positions & velocities of obstacles and the USV to classify it, and then generate risk distributions based on the predicted position, appropriate avoiding action and the error/uncertainty. With a heavy emphasis on real world trials, the programme demonstrated that there are large numbers of challenges in handling the real world noise. Systems and algorithm that seem promising and successful in simulation in the laboratory had difficulty on the water. This level or ruggedisation challenge should not be underestimated. To give one example consider a basic COLREG overtaking vessel must keep clear. In human terms this is easily understandable, but for a machine we need to define when an overtake begins and ends, precise angles, relative velocities and thresholds. In theory this manageable but now what if you find yourself on the threshold one of these how much hysteresis or fuzzy logic is needed? What if there is error and jitter in both detection and control? What if you loose and reacquire (an apparently different) vessel track midway through but need to remember that you are overtaking? What if you or it alters course and how is that differentiable from noise? What are the processing latencies in the system and how do you account for environmental effects and vessel dynamics? How do you ensure safety at all times? Some of these issues can be explored in simulation, but it is difficult to fully simulate the true level of real world uncertainty and plethora of special cases without extensive on the water experimentation.

6 3.3.2 Sensing Sub System Fig 2 - An example of AIS, radar & vision sensor outputs all being fused into tracks MAST was fitted with a SIMRAD 4G broadband radar which is the most critical sensor of the collision avoidance system. Configuring an controlling the radar, plot and track extractor on just a small mobile platform was an ongoing an persistent challenge. A significant amount of effort was spent working with the supplier of the radar track extraction system, in order to optimise the performance in a variety of conditions. The 360 panoramic Pt Grey LadyBug 5 camera was used for visual object detection. Work was undertaken to explore various detection algorithms including: Image masking, Horizon detection, Colour detector, Edge detector, Local Variance detector and particle probability hypothesis density (PPHD) filter. Other sensor related work undertaken included the numerous enabling sensors such as compasses & GPS which again although seem conceptually simple had many challenges. Monitoring the vertical acceleration of the boat using IMUs with FFT & physics based algorithms to infer maximum safe speed was also developed. Database algorithms were also developed to read and process electronic chart information for extraction of information relevant to collision avoidance for example water depth information, tide data and chart features such as buoys and marks Final Outcomes and Deliverables An internal comparison with other research activities in the field was undertaken in December This re-affirmed our belief that the work undertaken in this programme is at the forefront of the ongoing research activities in the industry and highlighted: The capability developed under this project is potentially the fastest full autonomy capability in the world. The light displacement, high speed platform creates more sensor processing challenges than most competitors. Other collision avoidance systems are more reactive, operating over much shorter timeframes (more like trajectory planners than path planners), whereas this system provides not just short term planners but more longer range planning. The sensor fit on MAST is broadly similar to other products/projects.

7 Demo 5 (as noted in section 3.2 above) was the final deliverable of the project; a showcase of all the technological developments made to date. Each leg of the scenario demonstrated a particular element of the functionality that has been developed as part of the project. The MAST USV was demonstrated operating autonomously (with full collision avoidance) from just outside the entrance to Portsmouth Harbour out to the outer approaches of the Solent, this included crossing the main shipping channel twice and operating at high speed. The Solent is one of the busiest shipping areas in the world and this was a significant achievement for the project. Key functionality demonstrated during this run included various potential collision scenarios (including emergency evasive maneuvering), the last response engine, route planning, automatic sensing of the sea state and corresponding adjustment of the USVs speed. The demonstration also included a demonstration of the work done to date using machine learning techniques to automatically classify different types of vessels and buoyage. Fig 3 - Overview of demo 5 operational area

8 4. Conclusion MAST and the collision avoidance system is now at the advanced prototype stage. It is clear that the system now has a level of performance and assurance that provides the opportunity for two crucial exploitation testbed activities: 1. The modular architecture approach allows additional or alternative subsystems to be easily compared against the current baseline, to evaluate performance improvement. Given that it is now established it is fundamentally possible, research can be focused towards pushing the limits (e.g. operating closer in shore or in more challenging environments). 2. Integrating into the wider context. There are numerous higher layer mission management and situational awareness systems at various TRL. MAST now provides the opportunity for those systems to experiment with real-world uncertainty by integrating with an actual representative platform. Dstl and ASV therefore welcome interest from other organisations who would like to explore and experiment with running their systems either onboard MAST or otherwise controlling or commanding it.