North Carolina State University Aerial Robotics Club

Transcription

1 North Carolina State University Aerial Robotics Club MAE Box Oval Drive Raleigh, NC May 30, 2013 AUVSI Seafarer Chapter Re: NC State Aerial Robotics Club Journal Paper Submission Contest Judges: The North Carolina State University Aerial Robotics Club has prepared an entirely new system to meet the requirements of the 2013 SUAS competition. Fenrir, the team's new aircraft, carries a payload including a Piccolo LT autopilot, IDS machine vision camera, and an x86 Flight PC. The system uses a 2.4GHz link for manual flight control, a 5.8GHz link for imagery downlink, a 900MHz link for autopilot control and telemetry, and an additional 2.4GHz link for SRIC communication. The payload is powered by V, 3300mAh lithium polymer batteries. Fenrir has a 10-foot wingspan, is about 10 feet long, and weighs in at 34 pounds gross weight. It is powered by a gasoline engine and carries 50oz of fuel allowing a flight time upwards of 1 hour. The system has logged over 8.5 hours of flight time and has proven to be both reliable and capable of excelling at the mission requirements of the competition. Enclosed please find our Journal Paper, as required by the competition rules. Sincerely, NCSU ARC

2 NCSU ARC 2013 AUVSI SUAS Journal Paper North Carolina State University Aerial Robotics Club Department of Mechanical and Aerospace Engineering North Carolina State University, Raleigh, NC The academic year was very productive for the NCSU Aerial Robotics Club. The developments made can be measured in leaps and bounds compared to previous years, and were all innocently precipitated by the desire to upgrade the system s flight computer. Failing to find space in the legacy air vehicle, ARCWulf, the team designed and built an entirely new system. Imagery and aircraft teams worked concurrently, and fed off of each other s enthusiasm and ideas to create the newly designed system, Fenrir. Fenrir outperforms ARCWulf on all accounts - It has proven capable of fully autonomous takeoff and tight search pattern tracking for efficient search area coverage. Additionally, the new imagery system provides higher quality images than the club has seen before, and downlinks the images to the ground where the newly developed target recognition software rapidly completes automated target detection and provides the user an interface to enter target identification data. Extensive flight and ground testing has given the team confidence that the system will excel in completing nearly all the primary and optional performance parameters for the 2013 competition. Figure 1: Fenrir system in flight. A. Mission Requirements Analysis I. Systems Engineering Approach Drawing on the success of and knowledge gained from our previous systems, the Aerial Robotics Club (ARC) at North Carolina State University (NCSU) has worked hard to develop an entirely new system to meet the mission requirements of the 2013 AUVSI Student Unmanned Aerial Systems (SUAS) competition. Six Key Performance Parameters (KPP s) were set by the Request for Proposal (RFP): Autonomy, Imagery, Target Location, Mission Time, Operational Availability, and In-flight Re-tasking. Each KPP includes Threshold and Objective goals. Thresholds indicate minimum performance levels that qualify as attainment of their respective KPP s. Objectives indicate higher level performance, exceeding the Threshold requirements. Performing up to the Objective standard will result in a higher score. Where safety and reliability would allow, the team chose to set the Objectives as its minimum design goals for the system s performance. This meant planning for a system capable of autonomous target identification, locating targets within 50 feet, completing the entire mission in a single attempt within 20 minutes, the ability to shift the search area during flight, and autonomous takeoff and landing. Additional optional goals were specified by Correspondence: ncsu.arc@gmail.com

3 the RFP, including providing real-time actionable intelligence data, connecting to the Simulated Remote Information Center (SRIC), and imaging an off flight path target. All of these options were adopted by the team as secondary goals. In addition to meeting the goals specified for the 2013 competition, the team aspired to develop a system that allowed for expansion and improvement to meet future mission requirements, while being more user-friendly to operate. On the air vehicle side, this meant making allowances for extra payload space and computing power to allow flight testing of various other university projects and prototype equipment. On the ground station, this meant improvement of the system to be more quickly and readily deployable without the need for excessive setup and teardown work to support a single mission. B. Design Rationale 1. Aircraft During the second half of 2012, the NCSU Aerial Robotics Club designed, developed and built a new airframe to meet the current mission requirements, allow for future expansion of capabilities, and improve mission performance compared to the club s previous airframe, ARCWulf. Over five years of flying and competing with ARCWulf, various systems were added to the aircraft. All these additions were aimed at improving the overall mission performance, but as capabilities were added, unavoidably so was weight. ARCWulf, being a modified version of the popular Telemaster RC model, was not engineered specifically to manage the weight or bulk of these payloads. After an update to the onboard payload computer, it became very clear to the club that mission performance was suffering due to the air vehicle. ARCWulf was operating above its design gross weight, resulting in poor takeoff and climb performance, especially in the tight confines of our practice field. This also limited the aircraft to shallow bank angles (wide turn radius) to maintain altitude resulting in search pattern performance that compromised efficient coverage of the search area. Plus, the airframe was simply out of space, excluding the possibility of further systems integration. A new design was deemed necessary to replace ARCWulf. This new aircraft would be Fenrir, which was designed to exceed the performance and load-carrying capabilities of ARCWulf and better meet the current mission requirements. Fenrir s design details will be discussed later under the Air Vehicle UAS Design section. 2. Payload By far, the most important payload component installed in the system is the camera, which helps the team meet the Imagery and Target Location KPP s as well as the additional goal of imaging the off flight path target. This year, the club decided to switch from our old camera, a Nikon D60 DSLR, to a machine vision camera as the primary flight camera. The decision to switch was made based on the factors of weight, image quality, and ease of programmatic control. The full camera, battery, and lens weight of the Nikon D60 is 26.7oz, heavily contributing to ARCWulf s gross weight and performance problems. Knowing that most of the imagery system testing would be conducted on ARCWulf before the new airframe was ready, a smaller, lighter camera was desired. While weight was a driving factor in the decision to switch to a smaller camera, the club felt strongly that it must not compromise on image quality and, if possible, improve it, as quality is clearly one of the most important factors in quickly and easily identifying targets. In the past, the club has found that machine vision cameras of sufficient quality were outside of the club s budget. However, as high resolution cameras become more widespread in industrial applications, high resolution machine vision cameras have come down significantly in price. In addition, machine vision cameras can forgo human interface features, such as a screen and buttons, for reduced weight and size. This means that they can include many of the same high resolution and quality sensors found in DSLR s, at a fraction of the weight. Another important feature in the club s search for a new camera was programmatic control. The Nikon D60 is a very powerful camera, but it is intended to be used by a human, not a computer. The Picture Transfer Protocol (PTP) provided a rudimentary featureset for capturing images; however, settings still had to be manually set and verified on the camera itself, and, more painfully, the implementation was unreliable. Machine vision cameras, as the name suggests, are designed from the ground up to be controlled by a computer. These cameras provide APIs allowing access to all camera features, allowing fully automated camera control, and thus providing in-flight access to modify the camera configuration. 2 of 20

4 Early in the search for machine vision cameras, it became clear that the APIs for our best camera options only support processors based on the x86 architecture and would not be compatible with our existing ARM-based Pandaboard flight computer. With this knowledge and a desire to move toward the ability to run some image processing onboard the aircraft, research began into building a lightweight yet capable x86 flight computer. Software is an essential part of the payload system, and high quality, reliable, and maintainable software was a focus this year. Each component of the system needed to be resilient to failure in other parts of the system, such that one failing system does not bring others down with it. This is particularly important for image capture, where a loss of communications must not prevent further capture of images, which could be downloaded and processed upon landing. For maintainability, a central library usable in many different programs prevents code duplication, and makes creating a new program simple. 3. Autopilot The Piccolo LT is a high-grade, off-the-shelf autopilot system made by Cloud Cap Technology. The system provides the capability to meet all of the autopilot-driven KPP s including the Autonomy and In-flight Re-tasking Thresholds and Objectives. It also supports attainment of the Imagery and Target Location KPP s by providing high-fidelity GPS position and attitude data to the imagery system through its serial connection to the flight computer. The Piccolo Command Center ground station software meets the safety requirements by displaying the airspace boundaries, airspeed, altitude, and current vehicle position on the autopilot ground control screen. The autopilot also provides the required failsafe and aerodynamic termination capabilities. Drawing on the club s 7 years of positive experience with this autopilot and the fact that it meets all of the requirements, the club elected to continue using the Piccolo LT autopilot on the Fenrir platform. C. Expected Performance The club has currently logged 22 total flights on the Fenrir platform and 8.5 hours of total flight time, 76% of which was fully autonomous. Much of the imagery and SRIC systems have undergone testing in ARCWulf all year and have over 15 hours of flight testing. Results of this testing will be discussed later under the Test and Evaluation Results section. Based on flight test experience this year, the system is well prepared to meet the mission requirements of the 2013 SUAS competition. The team expects to meet at least 5 Objective KPP s and at least the Threshold for the remaining KPP. While testing has demonstrated reliable autonomous navigation and takeoff performance, autonomous landing performance has not reached a satisfactory level as of the time of this paper s submission. More auto-landing tuning will be conducted and the team hopes to be ready to perform this at competition. In addition to the primary KPP s, we expect to meet many of the stretch objectives outlined in the rules. Tables 1 and 2 show the team s performance expectations. KPP Threshold Requirement Objective Requirement Autonomy Will Meet May Meet (Auto-landing) Imagery Will Meet Will Meet Target Location Will Meet Will Meet Mission Time Will Meet Will Meet Operational Availability Will Meet Will Meet Dynamic Retasking Will Meet Will Meet Table 1: Fenrir expected performance for KPP items. In testing, the autonomous target recognition software reliably detects targets with a low false positive rate. Three of the five required target characteristics are currently attempted by the system. These characteristics have close to a 50% false positive rate, so the characterization objective may not be met. Since the system currently has no mechanism for differentiating between classification and identification, and not all characteristics are attempted, the system will not meet the identification objective. 3 of 20

5 Secondary Requirement Off-Axis Target Pop-up Target Auto Target Detection/Queuing Auto Target Classification Auto Target Identification Electronic Data Submission Actionable Intelligence SRIC Secret Message Expectation Will Meet Will Meet Will Meet May Meet Will Not Meet Will Meet Will Meet Will Meet Will Meet Table 2: Fenrir expected performance for secondary requirements. A. Air Vehicle II. UAS Design As previously discussed under the Aircraft Design Rationale, the club decided to design and build a brand new aircraft. Fenrir was designed specifically to meet the 2013 mission requirements and allow for future expansion of capabilities as mission requirements mature. It is larger than ARCWulf, featuring a 10-foot wingspan and an overall length of over 10 feet (including its air-data boom). Empty weight is 19 pounds, and the aircraft was designed to have a maximum gross weight of 45 pounds. Ballasted test flights have been made with satisfactory performance up to the design gross weight. The up to 26 pound allowable payload capacity represents a drastic improvement over ARCWulf s 8 pound design capacity. The new aircraft was intentionally designed with a voluminous payload bay and the ability to carry dramatically heavier payloads than its predecessor. The current payload only weighs 15 pounds, which allows plenty of room for expansion/improvement of systems and adaptation to changing mission requirements that are expected in the future. The payload bay was designed to be modular, featuring a set of mounting rails and a regular, defined pattern of fasteners to which payload modules may mount. Payload modules may be built independent of the aircraft following a payload module design guide and installed to the rails without modification to the aircraft. A separate module is used for the flight computer, networking and radios, the camera gimbal, the autopilot system, and the payload power system. All may be easily removed for maintenance and system improvements. The club chose a pusher-propeller, twin-tailboom configuration for Fenrir. This allowed for a large, highly accessible payload bay forward of the wing. The aft-mounted engine with camera in the forward payload prevents exhaust gases and fluids flowing across the camera, keeping the lens clear of residue, reducing obscuration and assisting in meeting the Imagery KPP. The air-data boom, including pitot tube and static ports, is mounted in front of the nose in the cleanest possible air for best air-data accuracy. The configuration lends itself well to redundant flight controls as it has two rudders. The aircraft uses separate servos for the left and right ailerons and flaps, left and right rudders, and left and right half of the elevator. The airplane was designed with adequate control power to be flown safely following the failure of a single side of any and all of these flight controls, contributing greatly to the safety of operating the aircraft. Finally, this configuration leaves the engine and propeller guarded by the tail section, drastically reducing the possibility of a team member accidentally coming into contact with the rotating propeller. This makes the aircraft much safer to work around on the flight line. As an extra line of redundancy, power to Fenrir s flight control servos is provided by two separate lithium polymer batteries. A circuit between the batteries and servo power bus handles parallel loading of the batteries. When the voltage of the two batteries matches, both are drawn from in parallel, splitting the load and depletion evenly. In the event of a battery voltage mismatch, only the higher-voltage battery is used. In the event of a totally dead or shortcircuited battery, that battery is isolated from the system and safe control can continue from the remaining battery. This ensures that a single battery failure can not result in a loss of safe control of the aircraft. It was also desired to achieve a high flight endurance. ARCWulf was limited to 20 minutes of flight time, which, while adequate to meet the mission time Objective KPP, was a limiting factor during flight testing and did not allow much margin for mission changes. Fenrir was designed with enough fuel and battery capacity for approximately 1 hour of loiter time. This would allow for more goals to be met during each flight test and the possibility of longer mission times, should an emergent target require a longer period of surveillance than anticipated. An additional goal was to develop an aircraft with a high maneuverability envelope to allow for tight waypoint 4 of 20

6 navigation tracking and dense search pattern pathing. ARCWulf s maximum safe bank angle was set at 25 degrees, allowing a minimum theoretical turn radius of 246 ft. Fenrir was designed with adequate structures and flight controls to safely execute turns at 75+ degrees of bank, yielding a theoretical turn radius of just 31 feet. This would allow for more dense coverage of the search area, assisting in better attainment of the Imagery and Target Location KPP s. The aircraft was also designed with an adequately high power-to-weight ratio to allow for steep (45 degree) climbs, and with flaps to allow steep yet controlled descents, contributing to better attainment of the Autonomy Threshold KPP. B. Data Link The system uses a variety of radio frequencies for air-ground communications between subsystems. For manual aircraft control, a 2.4GHz Frequency-Hopping Spread Spectrum (FHSS) system is used, sending control inputs from the external pilot s controller to a receiver onboard the aircraft. This system is capable of operating in the same environment as other common 2.4GHz radio-control systems, wireless networks, video and other devices without being negatively impacted by interference. The autopilot uses a two-way 900MHz link to pass commands and telemetry between the air vehicle and the ground station. This link is handled by the Piccolo s internal radio, based on the Xbee Xtend 1-watt radio. Via this link, autopilot operators are able to task and dynamically re-task the aircraft at any time during flight as well as monitor real-time telemetry from the aircraft. Manual flight controls can also be passed over this link via the autopilot ground station s manual flight control console. The payload uses a two-way 5.8GHz link primarily to pass images from the aircraft to the imagery interface. This is accomplished using an Ubiquiti M5HP bullet on both the aircraft and the ground station. This provides Mbps of throughput for imagery downlink. An additional 2.4GHz Ubiquiti M2HP bullet is contained in the aircraft payload which operates as the SRIC link. This link is capable of connecting with the SRIC router and has been tested under a variety of conditions, discussed later in the SRIC Design Section. For communication between personnel, a 462MHz FRS (Family Radio Service) VHF radio system is used. This is a band used by readily available 2-way radios and offers several miles of line-of-sight range. This link allows the ground control trailer personnel to communicate to external personnel such as the external pilot, flight test director and observers. To enhance system safety, the data link architecture has been designed such that, in the event of a comms emergency, manual aircraft commands, autopilot commands, and telemetry data can all be passed over multiple channels. The 2.4GHz manual aircraft control link, the 900MHz link, and the 5.8GHz link are all capable of passing commands from the safety pilot to fly the aircraft manually. The 900MHz system serves as the primary link for autopilot command, control and telemetry; however, it is also possible to continue control of autonomous flight while receiving telemetry and passing commands via the 5.8GHz link. This arrangement makes the links for manual flight control doubly redundant and autonomous flight control redundant, greatly enhancing system safety. C. Ground Control Station To provide a sterile work environment for the UAS operators and a means of transportation of systems to the flightline, the team uses a dedicated Ground Control Trailer (GCT). This 14-foot enclosed trailer provides workstations for 2 autopilot operators and 2 imagery operators, and contains the infrastructure necessary to support operation of the aircraft. The GCT contains permanently installed central AC and DC power systems, a dedicated Local Area Network (LAN), and all ground-based antennas. The GCT is equipped with an auto-tracking antenna mount that keeps the patch antenna for the 5.8GHz imagery link directed towards the aircraft at all times. The tracker controller and the 5.8GHz Ubiquiti M5HP Bullet both interface directly to the GCT LAN. The tracking and other antennas are weatherproof and permanently mounted to the roof of the trailer to both allow a clear view of the aircraft and reduce the setup time of the ground station. The GCT has a permanently mounted system of intercoms and FRS VHF radios to facilitate communication between personnel. A 2-place intercom and accompanying headsets at both the Autopilot and Imagery control stations allow 2 operators at each station to converse privately. Each operator is then able to transmit to the rest of the team over the VHF radio via push-to-talk switches at each station. Operators outside the GCT carry personal VHF radios. This system allows operators to focus on their primary tasks while also providing for more effective communication between personnel. It also facilitates communication if the mission requires the flight line crew and external pilot to operate in a remote location from the GCT for any reason. The need for such a system was made apparent during 5 of 20

7 demonstration of the ARCWulf system at the 2012 SUAS competition, when a communication breakdown due to remote operations cost the team significant time and mission success. While not permanently installed, the GCT allows operators to set up all necessary ground station computers and additional hardware in the time before an impending mission. These systems can remain in place during transit to the flightline, further reducing the required setup time immediately prior to the mission. External systems such as the GCT s AC generator and a Ground Power Unit (GPU) for the aircraft are staged for immediate deployment upon arrival at the flightline. Thanks to this high level of system readiness, setup times from arrival at flightline to commson readiness average under 5 minutes, contributing to reliable attainment of the Operational Availability and Mission Time KPP s. 1. Autopilot Interface The autopilot ground control station is contained within the GCT. It consists of two operator stations, the autopilot ground station radio console and manual control console, and one or two laptop computers running the Piccolo Command Center (PCC) software. PCC provides an intuitive autopilot control and telemetry interface. A Primary Flight Display (PFD), shown in Figure 2, shows the operator the aircraft s attitude, airspeed, altitude, heading, and waypoint information. A moving map display, shown in Figure 3, shows the aircraft s current position, all flight plans, airspace boundaries, and a satellite image of the ground. Flight plans and boundaries can be drawn and modified directly on this display. Multiple other windows can be configured to monitor all telemetry parameters received from the aircraft, as well as adjust the autopilot s configuration including the flight control gains. Primary and secondary autopilot operators are responsible for separate tasks during flight. The primary operator is responsible for operation of the autopilot while the aircraft is under autonomous control. Such operations include launching, waypoint and search area navigation, and landing. For missions not requiring dynamic retasking, the secondary operator monitors aircraft systems. For missions where dynamic retasking is a requirement, the secondary operator is also responsible for updating flight plans and airspace boundaries. This division of duties allows for more efficient modifications to the flight plan while maintaining a high level of situational awareness by the operators, a critical factor in safe UAS operations. Figure 2: The Piccolo Command Center s Primary Flight Display Figure 3: PCC s moving map display, showing airspace boundaries, flight plans and the aircraft s position. 6 of 20

8 2. Imagery Interface The primary imagery interface is provided through the imagery ground computer. This computer serves as the final storage point of all images and performs all image analysis. The majority of the imagery system is controlled through a series of command line programs that are monitored in a terminal. Flight PC programs for image capture and transfer are monitored remotely on the imagery ground computer via an SSH (Secure SHell) session. A heavily modified version of the Mirage image viewer is used for image viewing, manual targeting, and monitoring of automatic targeting. The standard view can be seen in Figure 4. Mirage receives images for display from a central database, which is continually updated as new images are taken. Figure 4: The modified Mirage imagery viewer, showing automatically- and manually-detected targets. Additionally, all marked targets for that flight will be displayed on the image, regardless of whether or not they were originally marked from that image. Manual targets are marked in green and automatic targets in red. This works by querying the database for all targets within the coordinates the currently-displayed image covers. The advantages of displaying all targets are two-fold. For manual targeting, the computed target location from multiple images can be averaged together to help reduce GPS error that may be present in a single image. For automatic targeting, it allows for easy inspection of target candidates, without the need to find the original image in which the target was detected. All target markers can be clicked on to get more information, including location and target characteristics, as shown in Figure 5. For manual targeting, clicking an empty area of the image allows the addition of a new target at that point, or averaging of that point into an existing target. Automatic target recognition, characterization, and identification is only monitored through Mirage, but runs as a separate program, which is described later in the Data Processing section. Figure 5: Target information displayed via the Mirage interface. 3. External Pilot Interface Outside the trailer, the external safety pilot uses a Futaba 8FG transmitter for manual aircraft control. This transmitter utilizes a robust 2.4GHz FHSS protocol to communicate with the aircraft. A toggle switch on the transmitter switches control of the aircraft between autopilot and manual control via a multiplexer onboard the aircraft. The external pilot also has a VHF radio with an earbud-type headset, allowing for communication with other personnel while still permitting him to hear the aircraft for aural monitoring of engine performance. A push-to-talk switch on the transmitter allows the pilot to transmit over the radio without hampering his ability to manually control the airplane. In the unlikely event of a failure of the external pilot s primary transmitter, a failsafe system automatically switches 7 of 20

9 control back to the autopilot system. The external pilot can then utilize the autopilot ground station s manual control console to manually pilot the aircraft via either the primary 900MHz autopilot link or the 5.8GHz payload data link. These three systems for manual flight control provide double redundancy and a high level of safety. During flight testing, the primary 2.4GHz manual control link has never been lost, but the primary backup systems are tested at each flight test. D. Payload 1. Autopilot System The autopilot system onboard the aircraft consists of a Piccolo LT and a variety of peripherals aimed at improving overall system effectiveness. The basics include a GPS antenna and air-data boom, however a high-accuracy magnetometer and laser altimeter have also been added to the system to further increase performance. A servo signal multiplexer also serves as a safety switch between manual and autonomous control. The Piccolo LT autopilot is mounted on the autopilot payload module, central to the fuselage, on a vibrationdamping foam mount designed to reduce engine vibration reaching the unit and allow cleaner gyro sensor data. A 3.5in Antcom L1 Active GPS antenna is installed on top of the autopilot module, just under the top skin of the fuselage. This is used by the autopilot for 3D position (latitude, longitude and altitude) and time data. This position data is also transmitted to the imagery system to determine the location of each captured image. An air-data boom, including a pitot tube and static ports, extends from the nose of the aircraft. This provides basic static and total pressure measurements to the autopilot, allowing for barometric altitude and airspeed measurement. In the nose of the aircraft, away from electromagnetic interference, a Honeywell HMR2300 magnetometer is installed, giving the autopilot three-dimensional magnetic field data. This is used to determine the aircraft s magnetic heading, aiding in wind estimation and improving navigation performance. This heading is also used in the event of a loss of GPS lock for dead-reckoning navigation until a GPS lock can be reacquired. Further, magnetic heading is used by the imagery system to determine the orientation of each captured image. Last but not least, a LaserTech TruSense S-200 laser rangefinder is installed in the aft fuselage, looking downward through the lower fuselage skin. Coupled with a Moster Aerospace PTD-A011 Peripheral Translator connecting to the autopilot s CAN bus, this rangefinder is used as a laser altimeter, yielding highly accurate AGL (Above Ground Level) altitude data. AGL data is very important to both autonomous landing performance and the imagery system, in which the AGL altitude is used to determine what area of the ground covered by each image. In the aft section of the fuselage lies an Acroname RxMux. This multiplexer circuit allows switching between two sets of Pulse Width Modulation (PWM) servo signal inputs to a single set of outputs. An R/C receiver, coupled to the external pilot s ground station controller, provides the primary input to the multiplexer. The secondary input takes in the autopilot s servo command PWM outputs. The aircraft s control system servos are connected to the multiplexer s output. When the external pilot clicks a switch to hand control of the aircraft to the autopilot, the multiplexer switches from the R/C receiver input to the autopilot input, allowing autonomous control. In the event of any autopilot problems, up to and including a full autopilot hardware failure, the multiplexer can switch control back to the R/C receiver and the airplane can remain under safe manual control. This system contributes greatly to the overall operational safety of the system. As Fenrir was a brand new design, there was no pre-existing Piccolo software configuration that provided an acceptable starting point. An Athena Vortex Lattice (AVL) model, used in the stability and control analysis of the aircraft, was used to generate stability derivatives to build a Flightgear simulation model. This was used for Softwarein-the-Loop simulations to set the initial flight control gains, and later Hardware-in-the-Loop simulations to further refine those gains. After achieving satisfactory simulation results, the configuration was flight tested on the actual aircraft. Some minor tuning was still required, but the final control gains were extremely close to those determined through simulation. Results of that flight testing will be discussed later. From the Piccolo Command Center, mission limits can be set within the Piccolo autopilot that help meet several of the safety requirements. The system is configured to automatically activate aerodynamic termination following a 3 minute loss of autopilot communications. Coupled with failsafe settings in the external pilot s controller that pass control to the autopilot in the event of failure of that link, the start of the 3-minute countdown to aerodynamic termination is assured in the event that both primary aircraft control links are lost. In the event that the autopilot appears to lose control or proceeds to fly the aircraft beyond the airspace boundary limits, control will be passed back to the external pilot, who may manually assert flight termination should the need arise. Per the requirements, aerodynamic termination is enacted through a specified set of control inputs: closed throttle, full up elevator, full right 8 of 20

10 rudder, full right aileron, and full flaps down. During initial flight testing of Fenrir, this set of control inputs was tested and confirmed to result in a stable spin mode, resulting in a minimum-energy condition and the safest possible termination of the aircraft. During flight testing, it was determined that the time required to attain a GPS lock after system startup was inconsistent, ranging from a matter of seconds to tens of minutes. This inconsistency would hinder our ability to meet the Threshold Mission Time KPP of mission completion within 30 minutes, much less the Objective of 20 minutes, and even endanger attainment of either Operational Availability KPP. In order to better assure mission readiness, a procedure has been put in place to power on the autopilot with its 900MHz radio disabled. Ground power and a hard-wired ethernet connection to the plane allow the system to be started and a GPS lock acquired during initial setup time, saving precious time after the official mission start. Following comms-on clearance, the hardwired connection is used to re-enable the autopilot s 900MHz radio and the normal wireless link is then established. Circuitry onboard the aircraft allows for seamless transition from GPU to battery power shortly before engine start, preventing a system reboot and likely loss of GPS lock. 2. Imagery System The imagery payload onboard Fenrir consists primarily of a camera, flight computer, camera gimbal, 5.8GHz radio, ethernet switch, and data contribution from the autopilot. The flight computer is used for control of the camera, image capture, onboard image storage, and imagery downlinking to the ground. The flight computer interfaces to the camera and 5.8GHz Ubiquiti Bullet through a Gigabit ethernet switch. The flight computer also uses an RS-232 link to the autopilot to receive telemetry data. This data is packaged with the images at the time of capture for storage and downlinking to the ground. Autopilot attitude data is also used to control the camera gimbal. An overview of the system is shown in Figure 6. Imagery System 5.8GHz Bullet Gigabit Switch Flight Computer Autopilot Camera Gimbal Gimbal Controller Ethernet USB Servo PWM RS-232 Figure 6: An overview of the major components of the imagery system. As previously discussed, the club desired to move away from our old DSLR camera to a machine vision camera. After much research, an IDS UI-549SE Machine Vision camera was chosen as the primary flight camera. The full weight of the IDS UI-549SE+lens is only 7.8oz, a dramatic improvement from the over 2-pound Nikon D60. This reduction in weight was very helpful for testing in ARCWulf, which was overweight with the old camera. Though the same resolution as the Nikon D60, the IDS UI-549SE provides superior image quality. The lens chosen to go with the camera, a Kowa LM5JC10M, has a significantly higher resolving power of 200 line pairs/mm vs 50 line pairs/mm of the lens used with the Nikon D60. Additionally, the lens selected only degrades to 160 line pairs/mm 9 of 20

11 at the corners. Test results confirming image quality increases are discussed in the Test and Evaluation Results section. As desired, programmatic control saw a drastic improvement with the new camera. The IDS camera provides a full API with access to all camera features. The club has written a Python module that exposes these features to the rest of the imagery system, providing a simple and, more importantly, reliable interface to the camera. All aspects of the camera s configuration are set through the controlling computer, meaning that settings can be changed in flight, should the need arise. As early research indicated, the API for the IDS camera we chose does not support our previous ARM-based flight computer. A new miniitx form-factor x86 flight computer was built. This system provides several benefits in addition to simply being compatible with our camera. It includes multiple RS-232 COM ports which can be used to interface to multiple peripherals, like the autopilot, without the need for additional USB-to-RS-232 adapters. It also provides considerably more computing power; it utilizes a dual-core 2.6GHz Intel Core i3 processor, compared to the dual-core 1GHz Pandaboard. It is also equipped with a 256GB Solid State Drive (SSD), a major upgrade over the 64GB SSD on the previous flight computer. With the IDS camera able to capture up to 6 images/second, additional storage space onboard the aircraft was a necessary improvement. Onboard Fenrir, the IDS camera is mounted in a custom-designed 2-axis gimbal. In nadir mode, used while capturing imagery during normal search operations, the gimbal receives pitch and roll data from the autopilot via the flight computer and maintains the camera pointed toward the nadir. The gimbal has a range of motion of 40 degrees in roll and 35 degrees in pitch, allowing nadir-centered pictures at any aircraft attitudes within this range. Telemetry fed to the imagery system from the gimbal controller indicates whether or not the gimbal was properly aligned to the nadir when each image was captured, allowing rejection of misaligned images from target positioning. Coupled with the camera s Field Of View (FOV), the gimbal s range of motion also allows the camera to be positioned to capture targets up to 72 degrees off nadir below the aircraft to capture imagery of targets not captured in the nadir FOV. This enables the system to meet the stretch goal of off flight path target identification. The gimbal also provides for an autonomous self-calibration routine through use of an onboard accelerometer. Following initial setup or maintenance on the gimbal assembly, a self-calibration routine is run on the ground during which the gimbal slowly sweeps, recording true gimbal pitch/roll data vs servo command and maps its control table accordingly. This allows for a high level of gimbal accuracy without the need for time-consuming and error-prone manual calibration. High gimbal accuracy plays a key role in meeting the Objective goal for the Target Location KPP; just 1 degree of gimbal misalignment in pitch and roll can result in a 7.4-foot target location error from an altitude of 300ft AGL. The software design is split into several discrete parts in order to achieve the design goals of maintainability and reliability. PostGIS enabled PostgreSQL database The database is the heart of all information in the system. This database stores information about each flight, each image taken, and each target found. Since it is PostGIS enabled, location information is a first class type. Each image is stored along with a polygon geometry describing the area it covers, and each target is stored with a point geometry describing its location. By storing location information as a primary type in the database, very powerful queries on the data can be performed. For example, the image viewer is able to query for all targets contained within the current image, which returns a list of targets that requires no further processing. Furthermore, this database acts as a primary source of image information. That is, client programs need only query the database to find new images. They need no connection to the image capture or downlink systems. Core libraries These libraries provide a single API for commonly accessed features. These include a flight class for performing database actions on a flight level, such as getting a list of all images, or inserting a target. An image class wraps images captured by the system, providing easy access to metadata embedded within them, as well as convenience functions, such as converting X,Y coordinates in an image to Lat,Lon coordinates on Earth. Autopilot and camera classes provide a common interface to those devices. With these core libraries, the remainder of the system components can focus on their task without needing to implement common features. Telemetry daemon This is a simple wrapper around the autopilot class. It buffers autopilot telemetry, allowing other programs to request telemetry from a specific time. This ensures images get tagged with precise telemetry, even if they arrive from the camera several seconds late. Gimbal control Receives telemetry from the telemetry daemon, and commands the gimbal to remain nadired. The gimbal will report nadiring status, which is buffered by this program, allowing nadir status at a specific time to be queried. The nadir status information will be used to establish trust that the image was taken level. 10 of 20

12 Image capture Due to the modular design of the system, this program simply stitches together multiple components; using the camera class to capture images, embedding received telemetry and gimbal nadir information in images, and inserting into the database if specified. This program provides a configurable interface to the operator, allowing several camera settings to be specified, as well as several options for saving, such as simply saving to disk, inserting into the database, or passing to the downlinker. Image downlink This simple program separates the downloading process from image capture, ensuring that image capture is not dependent on the network. Image viewer The human interface to the images captured. The core libraries are used to provide a visual interface to the information available about a given flight, as discussed previously in the Imagery Interface section. Autonomous target recognition Uses the core libraries to acquire images to analyze, and detects and characterizes targets within them. The methods used are further discussed in the Data Processing section. 3. Simulated Remote Information Center (SRIC) The club was pleased with its performance successfully gathering the SRIC information in 2012, but desired to improve its integration with the remainder of the system. Building on last year s system, a 2.4GHz Bullet M2HP is used as the primary SRIC communication router. The Bullet is configured to connect to two different networks. On the wireless side, it connects to the SRIC ground router as a client, setting the provided static IP address on the SRIC subnet. On the ethernet side, the Bullet is connected to the aircraft network, with a known IP address. The Bullet will act as a gateway to the SRIC subnet by forwarding on all traffic bound for the SRIC network, but received on the ethernet network. The flight PC is configured to use the Bullet as its gateway to the SRIC network, so all requests are forwarded to the Bullet, and in turn the SRIC network. In flyby testing, where the aircraft was flown at 300ft AGL about 150ft east of the SRIC system at its closest point, the system was able to connect and download about 4MB of data before losing its connection. While this is a small amount of data, it is more than sufficient for downloading small text files. By only doing a flyby of the SRIC location, more time can be spent covering the search area. In the event that a flyby is not sufficient, the aircraft can perform a 300ft orbit around the SRIC location at 300ft AGL, which is sufficient to maintain a constant connection with the SRIC router. E. Mission Planning The goal of mission planning is to set up the system to accomplish as many of the KPPs as possible in the allotted mission time. Most mission planning is accomplished before a mission begins, although it is possible to dynamically re-task the system at any time to meet the KPP for re-tasking. Prior to mission set-up, flight plans are created for autonomous takeoff, for search area coverage, and for autonomous landing. Autonomous takeoff flight plans are based on which takeoff direction is favorable for the expected wind conditions; however, this can be changed prior to the mission as necessary. A flight plan is generated to take the aircraft from takeoff to the start of the waypoint navigation path, through the waypoint path and to the start of the search area. The search area pattern typically consists of flight paths in parallel rows that are connected with 180 degree turns. The turns are all performed in the same direction so that the aircraft progresses from one side of the search area to the other. After the aircraft traverses from one side of the search area to the other, it enters another series of parallel row search patterns that are perpendicular to the previous set. The combination of search pattern directions provides efficient and complete coverage of the search area. After coverage of the search area is complete, the autopilot operator instructs the aircraft to return to the search area entry/exit point and then to enter into the landing pattern. Autonomous landing patterns are defined in the mission planning and selected at mission time depending on existing wind conditions. In the event that dynamic restasking is necessary, the secondary autopilot operator manages adjustments to the airspace boundaries and flight plan while the primary operator focuses on the aircraft, monitoring its navigation performance and guiding it between mission stages. Once the secondary operator configures and sends the updated boundaries and flight plans to the aircraft, the primary operator may activate them as necessary. F. Data Processing There are two different paths for data processing once images have been downloaded, manual and automatic target detection, classification, and identification. 11 of 20

13 Manual targeting is performed using the custom Mirage image viewer, as described in the Imagery Interface section. Targets are marked in the viewer, which automatically computes their location based on the image telemetry, and are inserted into the database with the characteristics provided by the operator. One operator is dedicated to performing manual targeting, as well as finding the off-axis and pop-up targets. Once all targets have been identified, they can be quickly exported to the format specified by the RFP, using a single script. Automatic target recognition, characterization, and identification run as a separate program on the ground. During a mission, it is configured to connect to the database, and will process each new image inserted, placing the results back into the database. An overview of the process used to detect and characterize targets can be found in the following section, starting at step four. 1. Method of Autonomy 1. Capture image 2. Queued for download 3. Downloaded to ground payload PC, inserted into database 4. Target recognition software begins analyzing new image in database 5. Image searched for contrasting blobs in the hue channel of the Hue, Saturation, and Luminance (HSL) color space 6. Blob area and dimensions checked against RFP requirements Target candidates 7. Inside of target candidate checked for another contrasting blob Letter candidates 8. If found, many rotations of letter candidate run through Optical Character Recognition (OCR) 9. If OCR finds letters, highest confidence rotation used Letter 10. Target orientation derived from letter rotation and image heading Orientation 11. If letter valid, consider valid target Target 12. Target distance from centroid to edge measured at each angle Shape signature 13. Signature aligned with reference shapes and total square error between reference and signature taken. Least error Shape 14. (Currently incomplete) Target color 15. (Currently incomplete) Letter color The automatic target detection and classification system fulfills the automatic detection and automatic characterization objectives specified in the RFP. The system is capable of detecting targets with a low false positive rate, as well as classifying letter, orientation, and shape. At the time of this paper s submission, target and letter color are incomplete. As not all characteristics are characterized, and our testing shows that classification has a higher false positive rate than detection, the system will not meet the autonomous identification objective. Each characteristic requires a different method of analysis to determine, which are summarized below. Target detection Targets are detected by searching for contiguous, contrasting blobs in the hue channel of the original image. These blobs are compared against the RFP requirements for target size and total area, using the image altitude metadata. Only targets that meet the requirements are kept as target candidates. Only target candidates later found to contain a letter are considered valid targets. Target location The target location is simply taken at the centroid of the target blob. The target X,Y location in the image is converted into a global coordinate by projecting the image WGS84 coordinate into the Universe Mercator Projection (UTM), computing an offset in meters based on the image capture altitude, and projecting back into WGS of 20

14 Letter detection Letter detection in the target is performed similarly to target detection. Contrasting blobs are found within the bounds of the target blob. Letter recognition and orientation The Tesseract Optical Character Recognition (OCR) library is used for letter recognition. Since OCR is not rotation invariant, the letter blob is fed in to OCR in an array of different rotations. The letter returned with the highest confidence is taken as the target letter. The orientation is derived from the image heading and rotation used for the letter selected. Shape From the centroid of the target, the outline of the shape is traced, and the distance from the centroid is recorded with respect to rotation, resulting in a 1D signal of distance from center with respect to rotation angle. This signal is then normalized and cross-correlation is performed against a large database of template shape signals. The maximum of the cross-correlation between the shape and the template gives the angular offset needed to align the shape to the template, giving our algorithm rotation invariance. Once aligned to each template as well as possible, the sum of squared error is computed between the aligned signal and the template signal, and these are ranked smallest to largest. The template that results in the smallest squared error is the best match, and most likely the correct shape. A. Aircraft III. Test and Evaluation Results Before payload integration, Fenrir was subjected to a rigorous series of flight tests. These included initial stability checks; stall tests in power on, power off and all flap configurations; airspeed envelope expansion; flutter testing, and maneuverability testing. After passing all of these with satisfactory performance, the airplane was ballasted up to its design maximum gross weight and the tests repeated. All tests were passed successfully so the full payload was installed and mission performance testing began. At the current gross weight of 34 pounds and on a standard day at 350 ft MSL, Fenrir s minimum takeoff distance over a 50-foot obstacle is 250 feet and its maximum rate-of-climb is approximately 2000 feet/minute. As shown by Figure 7, this is a major improvement over ARCWulf s 650 foot takeoff distance and 200 feet/minute climbrate. Figure 7: Takeoff/initial climb profiles for Fenrir and ARCWulf. Distances shown are from start of ground roll to 50ft AGL. These improvements help make flight operations at our relatively small flight test facility safer and help us better achieve large altitude changes between waypoints, aiding in our meeting of both the Threshold and Objective Autonomy KPP s. Currently, Fenrir s autopilot bank limits are set at 45 degrees, allowing a turn radius of 115 feet. Using simple 180-degree turnarounds, this allows us to set our search-pattern straight path segments at 230 feet apart, less than half of ARCWulf s minimum path spacing, as shown in Figure of 20

15 Figure 8: Minimum path spacing for ARCWulf and Fenrir platforms in a typical search pattern. This provides for excellent image overlap while flying at low altitudes, improving our performance at meeting the Imagery and Target Location KPP s. Although the airplane is capable of safe 75+ degree banked turns, it was determined during testing that the Piccolo controller does not perform reliably well at bank angles exceeding 45 degrees, as altitude tracking suffers beyond this point. With future plans for autopilot upgrades, we hope to be able to increase the attitude limits and take better advantage of Fenrir s expanded maneuverability. Testing also validated Fenrir s 1-hour loiter endurance. Accounting for time for takeoff and landing, this allows for over 3 times the effective mission time, letting us complete goals more efficiently during flight testing without the disruption of stopping to refuel. The initial autopilot tuning was accomplished within just the first 4 flights on the airplane. B. Autopilot As discussed previously, autopilot integration into Fenrir, an entirely new design, required determination of a full set of controller gains for accurate and reliable path tracking in autonomous flight. Gains determined through Softwareand Hardware-in-the-Loop simulations provided a safe starting point, but flight testing was necessary to refine them for improved tracking performance. Lateral controller gains adjust how aggressively the autopilot attempts to track waypoints as well as response to flight path disturbances caused by wind. Accurate path tracking is crucial not only to the Autonomy KPP, but also to the Imagery KPP to ensure sufficient coverage of the target search area. Below, the left side of Figure 9 shows an initial tuning flight using gains determined through SiL and HiL simulations. Fenrir s track is indicated by a series of dashed blue lines while programmed waypoint path is shown in green. At the start of the recorded track, tracking oscillations can be seen on the leg from waypoint 45 to 46. The turn from waypoints 46 to 47 was overly aggressive, followed by an over-correction resulting in the airplane entirely missing waypoint 47. After stabilizing on the path to waypoint 48, the autopilot began the pre-turn too early, leading to further oscillatory corrections near the waypoint path. In-flight tuning to the lateral controller gains was performed to improve waypoint path tracking and the Piccolo s recovery from wind-induced path deviations. 14 of 20

16 Figure 9: Tracking performance before (left) and after (right) autopilot lateral gain tuning. Once an effective set of controller gains was reached, Fenrir s tracking performance was considerably improved, as evidenced by the right side of Figure 9. The recorded track shown indicates the autopilot s ability to tightly track the programmed path. This guarantees full image coverage of the search area, reliable navigation within confined airspace boundaries, and safe autonomous operation of the aircraft. During flight testing, AGL altitude readings from the laser altimeter were compared to normalized barometric altitude. Data shows that laser AGL readings are negatively impacted when the aircraft flies over a dense grouping of trees, like those surrounding our practice field. The laser receives the highest return from the canopy of trees instead the ground, yielding variances in AGL accuracy as shown in Figure 10. However, over flat ground, the laser altimeter data correlates closely with barometric data, providing extremely accurate AGL altitude information. This validates the laser altimeter s usefulness in the search area and near the runway. With the laser altimeter s ability to accurately estimate altitude within 10 cm, it remains a valuable asset to autonomous landing performance and imagery operations Normalized Barometric Altitude Laser AGL Altitude 400 Altitude (ft) Time (min.) Figure 10: Laser and Barometric AGL altitudes plotted vs time. 15 of 20

17 C. Imagery During the Fenrir build process, the IDS UI-549SE machine vision camera was integrated into the old ARCWulf platform, allowing extensive testing to be done well before the new aircraft was complete. Over the course of the year, over images have been captured during flight. Image resolution and quality were verified through a series of subjective and objective tests. All test flights with the camera onboard were flown with targets placed in the field. The images captured were subjectively compared for sharpness and clarity on the targets. Comparisons were made with previous flights and image settings, as well as center versus edge locations in the image and various altitudes, where significant improvements were noted. This subjective comparison offered a quick glance at image quality, and allowed rough tuning to be done on the fly. Diagnosing and fixing problems such as focus and exposure issues could be done quickly via this method. Figures 11 and 12 show an equal altitude comparison of the IDS image resolution at the center versus at the corners. The resolution of both cameras is comparable at the center, yet vastly different at the edges of the image. While the Nikon D60 shows a significant reduction in image quality near the edges, the IDS camera loses very little quality. This improvement in image quality makes more of the image usable for finding targets, which is especially critical for targets that may be near the edge of the search area. Figure 11: Target images from the Nikon D60 DSLR camera, taken from the center (left) and edge (right) of images taken from 250ft AGL. 16 of 20

18 Figure 12: Target images from the IDS UI-549SE camera, taken from the center (left) and edge (right) of images taken from 250ft AGL. In addition to subjective tests, a resolution target was used to objectively measure image resolution and quality. The club built a resolution target based on the 1951 US Air Force Resolution Target variant used at Webster Field. This target is composed of sets of three parallel lines, equal in width and spacing, where each set is of decreasing size. The smallest set where each line can still be distinguished in a given image can be used as a measure of the image resolution. This target is used on the ground before each flight test to ensure that the camera is optimally focused at infinity. The target is placed in the field to provide an objective measure of image resolution as the camera is tuned. In-flight images of the resolution target proved the consistent resolving power between the center and edges of the IDS camera s FOV seen in earlier subjective tests. An example can be seen below in Figure 13. Figure 13: Resolution target images from the IDS UI-549SE camera, taken from the center (left) and extreme edge (right) of images taken from 150ft AGL. 17 of 20