AIAA 3rd "Unmanned Unlimited" Technical Conference, Workshop and Exhibit 2-23 September 24, Chicago, Illinois AIAA 24-6424 Test and Integration of a Detect and Avoid System Mr. James Utt * Defense Research Associates, Inc., Beavercreek, OH 45431 Dr. John McCalmont s Directorate, Air Force Research Laboratory (AFRL/SNJT), Wright Patterson AFB, OH 45433 Mr. Mike Deschenes Defense Research Associates, Inc., Beavercreek, OH 45431 Remotely Operated Aircraft (ROAs) currently do not have convenient access to civil airspace due to their inability to provide an onboard capability to see and avoid air traffic. Defense Research Associates, Inc. and AFRL/SNJT have developed affordable technology based on silicon charge couple device sensors and passive moving target detection algorithms. Previously, an all-software implementation of the algorithms demonstrated concept feasibility by using video data recorded during flight testing. This paper documents subsequent implementation and field testing of a real-time version of the system with field programmable gate arrays handling the detection processing and multiple sensors to demonstrate the wide field of regard required. The implementation met demonstration goals by functioning reliably in real-time and providing detection and false detection performance comparable to that of the previous, non-real-time version. I. Introduction Federal Aviation Administration (FAA) Regulation 761.4 states remotely operated aircraft must provide an equivalent level of safety, comparable to see-and-avoid requirements for manned aircraft in order to operate like manned aircraft in the National Air Space (NAS). The capability must be effective against all air traffic, with or without active, transponder-based collision avoidance systems. Currently, no ROA see and avoid capability exists. ROAs operating in the NAS must obtain Certificates of Authorization, a cumbersome process, and/or use either chase planes or ground-based observers. The Air Force Research Laboratories s Directorate (AFRL/SN), and Defense Research Associates, Inc. (DRA) have developed technology called Detect and Avoid (DAA) that has the potential to meet the FAA s see and avoid requirement. Air Force Research Laboratory, s Directorate (AFRL/SN) has been investigating methods of reducing the number of false alarms in missile approach warning systems. One method showing promising results is the addition of a discrimination algorithm which discerns the motion of the approaching missile relative to the motion of the background. AFRL/SN and the Predator/Global Hawk Program Offices sponsored DRA to adapt this missile detection technology to the See and Avoid application. DRA used a validated AFRL/SN human vision model called OPEC and custom simulation software to numerically quantify the detection ranges required for an equivalent level of safety. DRA performed developmental flight demonstrations on a surrogate ROA aircraft. These demonstrations verified predictions that DAA technology will meet Global Hawk and Predator requirements, as shown by McCalmont, Utt, and Deschenes. The methodology for these demonstrations was to fly sensors in a surrogate ROA in near-collision scenarios while recording sensor data. The data was then processed on the ground using general-purpose computers running the detection algorithm software much slower than real time. These demonstrations proved the concept of using this affordable sensor technology and detection algorithm to meet the needs of Global Hawk and Predator. * Vice President, Systems Development, 3915 Germany Lane, Suite 12, Beavercreek, OH 45431. Threat Warning Team Leader, AFRL/SNJT, 35 C Street, Hangar 4B, Wright Patterson AFB, OH 45433. Engineering Team Leader, Systems Development, 3915 Germany Lane, Suite 12, Beavercreek, OH 45431. 1 Copyright 24 by the, Inc. All rights reserved.
TCAS See and Avoid TCAS Procedural II. Scope of Demonstration Operating manned aircraft in civil airspace is a relatively safe proposition due to the arsenal of tools and technologies built up since the invention of flight. Figure 1 illustrates the layered nature of these tools and technologies. Pilots are uniformly trained to follow the same procedures and interact in the same manner with air traffic control and each other. Radars monitor aircraft positions in areas of dense traffic. Transponders automatically announce aircraft positions. More sophisticated technologies, such as Traffic Collision Alert System (TCAS), use transponders to autonomously exchange deconfliction information. Finally, pilots use their eyes to see and avoid air traffic. Except for the latter, all of these tools and technologies can be made available for unmanned aircraft. The purpose of the work described in this and related paper is to develop technologies for ROAs that performs this function. More specifically, the purpose is not to develop a system that replaces any, much less all, of these other tools and technologies. Air Traffic Management Figure 1. Reasons Why Manned Aviation Is Safe The demonstration documented here focused on building a practical real-time implementation of the DAA system capable of delivering performance comparable to that observed for the non-real-time software implementation previously demonstrated. Additionally, real-time multi-target tracking software was written and integrated with the detection sub-system. The result was a complete wide field of view, multi-sensor, real-time onboard brassboard system.. The demonstration did not address integration of the system into an unmanned platform. III. Objectives The effort focused on meeting two objectives: 1. Demonstrate real-time operation of integrated detection and tracking system. 2. Demonstrate real-time tracking of targets across multiples sensor fields of view. IV. Implementation The Detect and Avoid (DAA) concept uses several key technologies: CCD sensors, new discrimination algorithms, and field programmable gate arrays (FPGAs). The DAA concept is to use three sensors to provide adequate non-cooperative target collision avoidance protection. The brassboard implementation is shown, along with it s field of regard, in Figure 2. The architecture implemented is shown in Figure 3, along with instrumentation for the demonstration. Note that this brassboard implementation provides approximately one half of the intended final azimuth coverage. The additional coverage can be obtained by using shorter focal length lenses and larger detector arrays. Both of these can be obtained commercially, but were not available for this demonstration. The sensors are high resolution (megapixel), low cost, digital video cameras available in the commercial market. The selected sensor provides high spatial resolution (~.5 milliradian) while maintaining a large field of view (~3 degrees horizontal by ~3 degrees vertical). The key to this capability is the 124 by 124 pixel silicon focal plane array. In addition to the high resolution, the current selection of digital video sensors are smaller, lighter weight, lower power, and lower cost. The sensor used in the earliest demonstrations was a Dalsa CA-D7-124T. This was subsequently replaced by Basler megapixel sensors, which offer comparable sensitivity and higher frame rates. The principal discrimination algorithm processes digital video from the sensor. The algorithm assumes that energy in the scene is constant over the interval between N frames of digital video data. It then estimates, for each pixel, the displacement of the energy during the N frame interval. The estimates are in the form of two-dimensional displacements called flow vectors. Together, these vectors characterize global scene motion. (In the case of an aircraft, the scene is really stationary while the aircraft moves). The set of vectors can be used to detect moving targets. This is possible because moving targets have motion differing from that of the background. 2
1 15-15 Sense 35 4 Covers 3 degrees elevation 85 Detect Track/Declare Covers 1 degrees azimuth (~1/2 of final design) Figure 2. Brassboard Coverage and Equipment 19 chassis Processor (FPGAs) Processor (FPGAs) Processor (FPGAs) Buffer (DRAM) FSB Tracking and Declaration (Pentium 4) PCI Diagnostic Tools High Bandwidth Camera Link Video Low Bandwidth Binary Data Digital Video Recorder Digital Video Recorder Inertial Measurement Unit Digital Video Recorder IRIG Timer Data Logger GPS Figure 3. Brasboard Real-Time Architecture FPGAs are moving rapidly into the mainstream of embedded high performance computing applications and as such, provided an important enabling technology for DAA. A commercial PCI board capable of expansion by adding daughter cards was selected to facilitate rapid development of a real-time detection processor. Figure 4 3
shows the architecture of both the PCI card and the daughter cards, provided by Nallatech Corporation. This hardware was used to implement the detection processor. Figure 4. Commercial Real-Time Processing Hardware A Kalmam tracker was implemented in software, illustrated in Figure 5, running on a Pentium 4 microprocessor. A modified version of the Linux open source operating system was also used. The tracker is the principal application running under Linux, but some ancillary functions responsible for logging low-bandwidth data and managing peripherals. Figure 5. Real-Time Tracking Software Design 4
V. Methodology The demonstration utilized the six-seat, twin-engine Aero Commander aircraft shown in Figure 6 as a surrogate ROA. The Beech Bonanza shown in the same figure was the intruder aircraft. Three sensors were mounted in a specially designed aircraft nose cone as shown in Figure 7 to provide coverage shown in Figure 2. The processing and data recording equipment were installed in a custom-designed rack, which replaced the middle row of seats in the Aero Commander. The aircraft were flown toward each other in a series of near-collision engagements. A 5 altitude separation was maintained for safety purposes. Engagements concentrated on nose-on geometries since this was the most challenging (smallest profile of the approaching aircraft and longest detection range requirement). Engagements were also contrived to close in the overlap region of two sensor fields of view. Specific engagements flown are shown in Table 1. During each engagement, the operator could view video data from any one sensor at a time. Output from the detection processor (i.e. the FPGA boards) was indicated on the screen in real time. Output from the track processor was logged to a hard disk for later, off-line analysis. Raw video data from all three sensors was recorded with a time-stamp on each frame. GPS and inertial measurement data were also timestamped and recorded. Intruder Aircraft Installation Controls Surrogate UAV Test Aircraft Figure 6. Demonstration Assets Figure 7. Plan, Side, and Three-Dimensional Drawings of s Installed on Surrogate UAV 5
Table 1. Demonstration Scenarios Engagement Number Altitude Diff (ft) Approach Angle (deg) Offset (NM) Heading 1 5 2 5 3-5 4-5 5.5 6.5 7 1.5 8 1.5 9 5 1 5 11 5 1. 12 5 1. 13-5 1. 14-5 1. VI. Results and Conclusions The system functioned reliably during the demonstration. Specifically, the three detection processor sub-systems kept pace with data from the sensors and did not drop any frames of raw data. Likewise, the tracking software kept pace with the detection processors without dropping frames of detection data. and false detection probabilities were consistent with results from the previous, non-real-time demonstration. Specifically, initial detection ranges were consistently greater than four nautical miles with confidences near 1% and false detection probabilities around.5%. The tracking software tracked targets across the seam between two sensor fields of view reliably, but briefly showed two tracks in the overlap region in some of the engagements. Based on these observations, the brassboard system met the demonstration goals, albeit with the caveat that the tracking software for handling the overlap regions needs improvement. VII. Acknowledgements The authors wish to thank the Unmanned Air Vehicle Battlelab for supporting the flight demonstration of the technology discussed. VIII. References 1. Global Hawk ORD CAF 353-92-I/II-C Basic Systems for the Global Hawk Unmanned Aerial Vehicle (ROA) System, Air Force Command and Control & Intelligence, Surveillance, and Reconnaissance Center, (September 2) 2. RQ-1 Predator ORD # CAF 3-9-I-A, 24 June 1997 3. MQ-9 Capability Development Document (Draft), 3 Sep 23 4. FAA Order 761.4J: Special Military Operations, Change 2 (Effective July 12, 21) 5. McCalmont, J., Utt, J. and Deschenes, M., Detect And Avoid Technology Demonstration, Air Force Research Laboratory (AFRL/SNJT), Wright Patterson AFB, OH, and Defense Research Associates, Inc., Beavercreek, OH (22) 6. Bryner, M., McCalmont, J., Utt J., Global Hawk Definition of Equivalent Level of Safety Requirements See and Avoid (SAA), Proceedings of the 23 UAV Technical Analysis and Applications Conference 7. Air Force Instruction 11-22, Vol. 3, General Flight Rules, (June 98) 6