Helo UAS Helicopter Unmanned Aircraft System Authors: Robert A.Gleich III, Robert C. DeMott II, James W. Homan Lucas Libraro, Skylar Roebuck, Phillip Diana Gus Mancone, David Supola, David West Advisor: Dr. Robert H. Klenke Submitted: 25 May 2010 Abstract This paper discusses s design and implementation of an unmanned aircraft system (UAS) to compete in the 2010 Association for Unmanned Vehicle Systems International (AUVSI) UAV competition. The system is able to autonomously fly a series of waypoints to reach a search area. Once in the search area, the vehicle is able to locate ground targets and identify target characteristics, communicating this information to operators in near real-time. The vehicle platform is an electrically powered, radio controlled helicopter. The complete solution includes a custom developed autopilot, or flight control system (FCS), an ISR payload featuring a gimbal-stabilized, high-definition (720P HD) video camera, and a ground control system (GCS) that provides mission planning, control, and analysis capabilities. The system has been designed with utmost importance placed on operator, observer, and vehicle safety. Testing, operating, and failure procedures have been implemented to ensure safety throughout development and competition. Page 1 of 25
Table of Contents ABSTRACT... 1 TABLE OF CONTENTS... 2 DESIGN OVERVIEW... 3 Custom Flight Control System... 3 COTS Component Selection... 4 VEHICLE... 4 FLIGHT CONTROL SYSTEM... 6 SOFTWARE... 6 HARDWARE... 7 Main Control Unit... 7 Sensors... 7 Communication... 8 UAV Safety Switch... 8 Overall FCS Architecture... 9 GROUND CONTROL SYSTEM... 10 GCS SERVER... 10 GCS CONTROL CLIENT... 10 Map Display... 11 Telemetry Gauges... 11 Parameter Control Forms... 11 TARGET GEOREFERENCING AND IMAGE CLIENT SYSTEM... 12 GIMBAL... 14 Function of Subsystem 1: Network communication with Ground Station... 15 Function of Subsystem 2: Controlling the Servo... 15 Function of Subsystem 3: Interlacing the FCS data... 16 Function of Subsystem 4: Tracking the Servo Position... 17 IMAGE CLIENT SOFTWARE... 18 Video Viewer Client... 18 Image Analysis Client... 19 OPERATIONAL SETUP... 22 FLIGHT CONTROL OPERATOR... 22 VIDEO SYSTEM OPERATOR... 23 INTELLIGENCE ANALYST... 23 SAFETY... 23 FAIL-SAFE FLIGHT MODES... 23 Stability Augmentation Mode... 23 BATTERIES... 24 TRANSMISSION LOSS... 24 FAILURE MODES ANALYSIS... 24 CONCLUSION... 25 ACKNOWLEDGEMENTS... 25 Page 2 of 25
Design Overview At the competition banquet in 2005, RADM Jack Chenevey USN Retired stated that "a helicopter is the right vehicle for the competition." Analyzing mission objectives, current technology, and the operation of unmanned aircraft in the Global War on Terror, the team agreed with RADM Chenevey. The ability to maneuver in confined spaces, loiter in a location for extended periods of time and perform vertical take-off and landings are features that fixed wing aircraft do not possess. Several different design approaches were utilized to develop and integrate the various subsystems. Based upon previous fixed-wing unmanned aircraft systems (UAS) research, the team developed a custom flight control system (FCS). High-resolution video and 802.11 Wifi communications systems have also been implemented. Commercial-off-the-shelf (COTS) components have been utilized for digital video and wireless communications hardware. These systems have been designed, integrated, and tested under a thorough review process focused on mission functionality and overall safety. The Helo Unmanned Aircraft System (Helo UAS) is designed to meet all requirements to compete in the 2010 AUVSI UAV Competition. System Architecture Overview Custom Flight Control System In general, systems utilizing COTS products realize a savings in development time and cost. However, a custom FCS was developed in favor of using a commercially available unit for Page 3 of 25
several reasons. Previous experience and research of unmanned systems at VCU has been utilized in the creation of custom FCS hardware and software for the Helo UAS. When considering the black-box nature of many commercial systems and issues such as expandability and debugging, a custom FCS was the logical design choice. Additionally, many commercial systems do not easily adapt to various helicopter airframes. The Helo FCS has been tailored to the unique flight characteristics of the helicopter, but can be adjusted to autonomously control other airframes without difficulty. Lastly, the Helo FCS is produced at a fraction of the cost of comparable commercial systems. COTS Component Selection Many components of the UAS are COTS products. A COTS Analysis Process was created to ensure that the product selected would best meet the mission requirements. Baseline specifications were established for each component. Using this baseline, the team compiled a list of several candidate products. A point-scale system, awarding to products based on their mission pertinent features, was then used to determine the winning product. The winning product then underwent a thorough review to verify that it met all system requirements. A simplified example of the point-scale system used to select an omni-directional antenna is shown below. This antenna is mounted on the helicopter, making product weight the most important factor in the design decision. COTS Analysis Example Features Gain Input Pwr. Weight Dims Price 2.4 GHz compact, vertical omni 8 dbi 100 W 198 g 16" $89.95 2.4 GHz compact, vertical omni 11.8 dbi 150 W 976 g 36" $99.95 High Gain Omni Wireless Antenna 15 dbi 50 W 100 g 16 $169.95 Wireless Network High Gain Omni 12 dbi 50 W 800 g 16 $119.95 802.11 Magnetic Mount Mobile WiFi 7.8 dbi 50 W 680 g 12" $49.95 Medium sized Magnetic Mount 5.5 dbi 50 W 544 g 6" $44.95 802.11 Mobile WiFi Antenna 5.5 dbi 50 W 284 g 4" $19.95 Total Scoring 20% 5% 40% 20% 15% 100% 2.4 GHz compact, vertical omni 5 7 9 5 7 7.00 2.4 GHz compact, vertical omni 8 10 0 0 6 3.00 High Gain Omni Wireless Antenna 10 5 10 5 2 7.55 Wireless Network High Gain Omni 8 5 1 5 4 3.85 802.11 Magnetic Mount Mobile WiFi 5 5 4 8 8 5.65 Medium sized Magnetic Mount 2 5 6 9 8 6.05 802.11 Mobile WiFi Antenna 2 5 8 10 10 7.35 Vehicle The vehicle chosen is a Miniature Aircraft X-Cell Ion-X electrically powered helicopter. To maximize flight time and payload capacity, the helicopter is equipped with longer rotor blades (800mm vs. stock 690mm) and a longer tail boom. The helicopter is also upgraded with a Hacker motor using single stage gear reduction, improving performance and reducing vibration. Page 4 of 25
With two 8 AH lithium polymer motor batteries, the helicopter, equipped with a ten pound payload, still achieves a fifteen minute flight time. If the helicopter is configured with four 8AH batteries, the maximum flight time increases to 25 minutes. This capability coincides with the competition s twenty minute mission objective. Although a gas powered helicopter of similar size provides a longer flight time and increased payload weight, electric power is used for several reasons. Electric motors produce less vibration than their gas equivalents. High-frequency vibrations have proven to be detrimental to sensors used in the flight control and imaging systems. Electric motors also do not produce exhaust, which can be problematic for optical sensors, such as cameras. X-Cell Ion-X Helicopter (Shown Without Video Gimbal) Page 5 of 25
Flight Control System The AUVSI mission rules require the flight control system to be capable of the following abilities: 1) Navigate a predetermined flight path consisting of a series of GPS coordinates and altitudes. 2) Allow for ground station operators to dynamically change the vehicle s target altitude and airspeed in real time. 3) Allow for ground station operators to specify new waypoints and search areas in real time. In order to meet these requirements, a custom rotor-wing based autopilot system was developed by VCU students. This system is based on previous work conducted at VCU in the area of fixedwing UAVs and adapted to handle the intricacies of a rotor-wing aircraft. Software The FCS software consists of a main control program that runs at 50Hz and is responsible for reading from all sensors, running the control algorithms and handling radio communication with the ground station. This 50Hz operation allows for smooth control of the helicopter and rapid response to both sensor readings and ground station commands. The flight control algorithms are based on standard PID (Proportional, Integral, and Derivative) control loops using feedback from both onboard and external sensors. Autonomous flight of the helicopter is divided into the following aspects, each being controlled by one or more PID loops: Heading, Altitude, Lateral Movement, and Forward Flight. Start Get Sensor Reading Error = SetPoint Measurement; If ( abs(error) < Deadband ) Error = 0; If ( Error > pband ) perror = 1.0F; Else if ( Error < -pband ) perror = -1.0F; Else perror = Error/pBand; Calculate Error Calculate Proportional Error Calculate Derivative Error Calculate PID Output derror = (perror - lasterror) / dt; output = pgain * perror + igain * ierror + dgain * derror; if ( output > 1.0F ) output = 1.0F; else if ( output < -1.0F ) output = -1.0F; ierror += perror * dt; if ( ierror > imax ) ierror = imax; else if ( ierror < imin ) ierror = imin; Calculate Integral Error Calculate PWM Values End Standard PID Control Algorithm Because of this division of control, each algorithm could be thoroughly tested and tuned independent of the other algorithms. This was achieved by selectively toggling autonomous control of each algorithm, with the RC pilot controlling the remaining aspects of flight. Since Page 6 of 25
each algorithm was tuned both individually and in combination with the others, more optimum flight performance was achieved. Hardware Main Control Unit The FCS software runs onboard an embedded computer known as the Suzaku. The Suzaku features a 48MHz softcore processor, 16MB of RAM and a 1 million gate FPGA. Since the Suzaku features an FPGA, it can easily be reprogrammed to communicate with multiple sensors and I/O devices through its onboard peripheral bus (OPB). In addition to interfacing with sensors and communication equipment, the Suzaku is also capable of monitoring and generating the PWM signals necessary for controlling the helicopter servos. This unit is mounted on a custom designed PCB motherboard along with the barometric sensors and the radio modem. FCS Main Control Unit Sensors For attitude, position and velocity information, the MIDG II - INS/GPS unit from Microbotics was used. The MIDG II is a compact, lightweight unit that combines a 5Hz GPS, 3-Axis Rate Gyro, 3-Axis Accelerometer and 3-Axis Magnetometer with an advanced Kalman filter capable of operating at 50Hz. With proper mounting and vibration dampening, this unit has proven extremely reliable, providing the FCS with accurate and precise sensor readings. Attitude accuracy is within 0.4 degrees and position accuracy is within 2 meters. Because the MIDG II can run at 50 Hz (same as FCS), it will always provide up-to-date information to the FCS algorithms, improving system response. Page 7 of 25
Microbotics MIDG II INS/GPS Unit Although the MIDG II also provides altitude information, a temperature compensated barometric pressure sensor was chosen as the source for this information. This choice was necessary due to the fact that the MIDG II relies on its GPS to determine altitude. In the event that GPS communication is lost, this data would no longer be valid. Communication The Communication System provides a link to transfer commands, parameters, and sensor data between the FCS and the GCS. A MaxStream 900Mhz OEM RF module was selected for the FCS side of the link and communicates at 9600 baud. This OEM module was chosen for its ability to mount directly on the FCS motherboard, significantly reducing the total size and weight of the FCS when compared to the use of an external modem. MaxStream OEM RF Module UAV Safety Switch The AUVSI mission rules state that the vehicle must be capable of manual override by the safety pilot at any time. To achieve this objective, a UAV safety switch from Electro Dynamics is utilized. This device is essentially a 5 channel, optically isolated switch for PWM signals. One set of inputs is from the RC receiver (safety pilot) and the other set is from the FCS. The set of signals that are forwarded to the servos is controlled by the auto/manual channel from the RC receiver. This allows control of the helicopter to be selected independently of the FCS. Even if the FCS fails completely, the safety pilot can regain control. This device operates on a separate power supply from the FCS to further minimize risk of failure. Page 8 of 25
UAV Safety Switch Overall FCS Architecture The overall configuration of the FCS components is shown below. FCS Architecture Page 9 of 25
Ground Control System The AUVSI rules state that the GCS must be capable of the following features: 1) The system will display all no fly zones and search areas to the operators and judges. 2) The system will display vehicle position with respect to no fly zones and search areas. 3) The system will display current vehicle altitude above mean sea level. 4) The system must be capable of dynamically adding waypoints and modifying the vehicle s flight-path. The ground control system software has been designed to meet these requirements in a user friendly manner. GCS Server The Ground Control System consists of a server component and a client component. The server connects to a radio modem on the ground to communicate with the helicopter FCS. The server component is responsible for sending all commands and parameters to the FCS and receiving and processing all sensor readings/data. The server allows a potentially unlimited number of specialized Ground Control clients to connect to the system and coordinates connections between the clients and the helicopter FCS. GCS Control Client The control client consists of the following components: Map Display Telemetry Panel Telemetry Gauges Parameter Adjustment Forms GCS Control Client Page 10 of 25
Map Display The map display features a satellite image of the flight area overlaid with the vehicle s current position, heading and flight path, the competition search areas, no fly zones and identified targets. Flight control operators can add new waypoints and modify the flight path using a simple point and click interface. Search areas and no fly zones can be modified in a similar manner. When entering waypoints, the interface will take no fly zones into account to prevent the operator from planning routes that would lead the vehicle out of the competition area. Waypoint changes will automatically be uploaded to the helicopter and confirmed on the map display. Telemetry Panel The left hand side of the control client displays the following telemetry data: Mode of vehicle operation (Manual/Autonomous) Vehicle Altitude (above sea level and above ground level) Vehicle Position (Latitude, Longitude) Vehicle Orientation (Pitch, Roll, Yaw) Vehicle Velocity Info (Air Speed, Ground Speed, Ground Track) Vehicle Battery Levels and approximate flight time remaining Status of wireless communications link Diagnostic information Units of measure can also be toggled between English, Metric and Nautical where appropriate. Telemetry Gauges Selected telemetry data is also shown in the form of graphical instrument gauges for easier reading by the operators and judges. The following gauges are available: Altimeter: Displays both current altitude and target altitude Speedometer: Displays both current air speed and target air speed. Compass: Displays vehicle s current heading and ground track. Attitude: Displays vehicle s current pitch and roll. Gauge position, size and units of measure can also be adjusted by the user. Parameter Control Forms The control client also features collapsible parameter entry forms along the right hand side of the display. This allows for quick access to flight parameters without cluttering the display area. In addition to allowing adjustments to target air speed and altitude, the flight operator can make adjustments to PID control parameters and servo trim values if necessary. Page 11 of 25
Target Georeferencing and Image Client System The AUVSI rules state that the Imagery System must be capable of the following features: The system will locate and display images of targets to the judges. The system will be able to identify target parameters from various altitudes. The system will be able to capture images and identify target parameters up to sixty degrees from directly below the vehicle. The system will be able to provide actionable intelligence: imagery, location, and parameters to the judges. System Overview The Flight Control System (FCS) on the UAV is capable of relaying information about its location and current orientation in 3D space to the Ground Control Station (GCS) where image analysis is performed. The information that the FCS sends includes the altitude, latitude, longitude, roll, pitch and yaw of the UAV. This information, while useful, is meaningless in its raw state when trying to determine the location of a target object as seen from the camera in the UAV s payload. The new Image Client, a program that is part of the GCS, will have access to Page 12 of 25
the data from the UAV s flight system along with data from the payload describing the camera s current pan and tilt angles. Once the operator captures an image from the UAV s camera, the new Image Client program then calculates the actual longitude and latitude of the area that the camera was viewing at the time of the capture. In the case where the camera is looking directly down at the ground, this calculation will be of limited use as the longitude and latitude calculated will be near the same as that of the UAV s current longitude and latitude. This is not always so, however, because the camera could be looking at a location in front of, behind or to the side of the UAV s actual location. This situation is where the utility of the calculation performed by the new program is evident. The program will also provide the operator on the ground with a few other very important abilities. The program will display the video received from the HD camera that is part of the UAV s payload. The operator then uses this video stream to look for targets in the search zone. When the operator determines that the UAV has found a target, he/she can issue a command to the program to capture the current frame of the video being displayed. The operator can also control the pan and tilt of the camera gimbal through this interface, as well as command the video camera to zoom in and zoom out. The second component to this program is an image analysis client. This separate program allows another operator to review the images captured from the high definition video camera and determine if a target has been found. If it is determined that a target has been found, the image analysis client then has the ability to store information about this target (such as color, shape and alphanumeric data). This stored information can then be used to output a formatted report containing information on every discovered target along with a thumbnail of the image where the target was discovered. The image analysis client is also where the georeferencing calculation will be performed. The operator of the analysis client can click anywhere in the image that has been flagged as suspicious and will be given a corresponding latitude and longitude. This is done using an integrated version of the Google Earth rendering system that is running in sync with the video being received from the helicopter. With every image, meta-data from the helicopter is stored in a database. When the image is loaded, this meta-data is sent to the Google Earth rendering engine and is used to locate the virtual camera exactly where the physical camera was when the image was retrieved. With the Google Earth image synched to the physical helicopter, it is then a trivial matter to send a pixel location in the image to the Google Earth rendering engine and retrieve a corresponding latitude and longitude. This second program will make the analysis of images from the UAV far simpler and more efficient than having one operator do both the target recognition and the analysis of the image with the target. Camera The VCU AUVSI competition team elected to use a Sanyo High Definition (HD 4000) video camera. This camera had a few special features that made it attractive. First and foremost, it is capable of outputting H.264 video over an Ethernet link with no specialized internal hardware. This makes it very easy to interface with the video camera and receive high definition video using a compression algorithm that allows for both quality and speed. This camera is also has a separate input that allows an external system to control its zoom level. Through this, the camera gimbal will not only be able to control the pan and tilt of the camera, but also its zoom. While the camera is capable of producing video at resolutions greater than 1080p, for the purpose of this competition and to keep bandwidth usage to a minimum, we will only be operating it at 720p. Page 13 of 25
Gimbal Gimbal Control Unit The RoBoard serves as the center of the system. It processes all data for the project s scope. A PC running a test program is used to emulate the FCS box sending serial data. A real FCS box could be easily dropped into the system at this point. The PC program was written to produce packets that look exactly like FCS packets by reading data logs that were provided by FCS box developers. The RoBoard was chosen for many reasons in addition to its fast processor and large ram capacity. The RoBoard has an Ethernet input that allows it to communicate directly with the router. It was assumed that data would be passed between the gimbal system and the ground station in VACS format. The code can be rewritten to output KLV packets if necessary. The servos are brought through a small initialization routine when the client connects to center the servos. This wasn t strictly necessary after servo feedback was finished, but does provide for a more uniform starting position for the camera. The servo feedback is used to verify Page 14 of 25
that the servos are at the positions that correspond to what the ground station thinks is 0 degrees pan and tilt. Function of Subsystem 1: Network communication with Ground Station The function of this subsystem is to open a socket through which communication with a ground station can take place. The socket opens over any specified port and the client then connects, give instructions and receive data. The packet format is a modified VACS packet, removing unnecessary components for TCP/IP communication, such as the sync bits, the destination and the source. The pan and tilt are stitched into the FCS meta-data (in another subsystem) and the length and checksum are updated appropriately. The modified packet is send to the connected client as frequently as the FCS box sends metadata in. Ground Station to RoBoard Packet Byte Field 0 Message ID 1 Length (N)... Payload (N Bytes) N+2 Checksum A N+3 Checksum B Message types: Message ID Message Type Payload 0 Initiate - 1 Move Camera 2 Meta data packet First two bytes will be pan of camera, second two bytes will be tilt, last two bytes will be zoom Instrument Report Packet plus camera position data (See XML below for details) Function of Subsystem 2: Controlling the Servo The first function of the project is controlling a servo based gimbal using instructions sent wirelessly from a ground station. The RoBoard will be running a server which the ground station will connect to in order to interact with it. The packet will be sent in a modified VACS format Page 15 of 25
through the wireless router where the server will receive it and check it for validity. Upon receiving a valid packet, the server will check the checksum of the packet and the message ID to determine what to do with the packet. If the received packet is valid and a movement packet, the server will begin moving the servos to the appropriate positions, reading the location of the servos as they move. The validity of the packet is determined by Message ID and Checksum. Once those are verified the data payload of the specified length is pulled from the packet and can be used. The pan and tilt is be extracted from this data and the servos can be moved. The correct pulse width is calculated and generated to move the servos. This is done gradually using feedback to determine the current servo position and incrementing that in order to achieve more controlled movement speeds. If the data indicating the actual positions of the servos shows they are incorrectly oriented a new pulse width will be generated to correct it. The RoBoard's built in PWM controller will generate the waveforms needed to keep the servos in the proper positions. The server software opens a socket and listens for connections from a client over a specified port. Once a client is connected that client is listened to in its own thread. Once the client is connected to the server it can begin sending packets; the server is immediately ready to respond. Listening to the client in a thread allowed for reading from the socket to be blocking, which took out the need to constantly poll the socket for new information to read, making a simpler program. Function of Subsystem 3: Interlacing the FCS data The data from the FCS box will conform to the VCU Aerial Communications Standard serial packet format: Byte Field 0 Sync1_Byte 1 Sync2_Byte 2 Destination address 3 Source Address 4 Message ID (Low Byte) 5 Message ID (High Byte) 6 Length N (Low Byte) 7 Length N (High Byte)... Data Payload (N Bytes) N+8 Checksum A N+9 Checksum B The "Instrument Report Packet," which is the packet containing the aircraft orientation data, has a message ID of 150, and is the packet containing the relevant data. Receiving Packets from the FCS Box A PC program was written to produce FCS Data packets. This PC program would read in Page 16 of 25
a.data file which contained sample FCS data logs from past helicopter flights. Once communication was open between the program (Windows PC) and the RoBoard then FCS packets could be sent. The program reads the log until a full packet has been found and sends it over serial. Later on it was found that this wasn t strictly necessary; the program could have simply sent the data over serial byte by byte and achieved the same result. During design and testing of this subsystem we verified that in fact we were receiving each packet in its entirety and that all verification bits looked correct. Some tasks completed during the phase were to verify the sync1, sync2 and check-sum on each packet from the FCS were correct. When receiving a packet on the RoBoard, the serial port listener thread will forward the raw data to a parsing algorithm. The parsing algorithm will verify Sync1, Sync2, Message ID, Length, Data Payload and it will calculate its own checksum and verify that against the one sent in the packet. Once the packet is deemed valid the function will store the packet into a data storage array for stitching. Receiving Data from the Servo Receiving data from the servo was also part of Subsystem 3 in our project implementation. The RoBoard has a built in A/D converter that is able to read the analog voltage drop over the servos potentiometers. This is done by reading the voltage on one side of the potentiometer and subtracting that from the read voltage on the other side of the potentiometer. The relationship between this reading and the servo s position was found to be linear. The RoBoard had a function available in the library that returned the digital value from the A/D converter. Once the digital data was available it could be processed and converted into useful, degree based measurements. The servo is constantly polled for the voltage drop over the potentiometer and an average over 30 samples is used to determine the most accurate position. This is because there is a very slight amount of variance from one reading to the next due to noise and this helps to remove that; though a hardware filter might have been more appropriate. Interlacing the FCS data and the data from the Servo In our system there is a function dedicated to stitching the pan and tilt into the FCS Packet. This function will modify a char[] that is passed into it. The function requires that the array passed in is an FCS packet in VACS format. The function will stitch pan and tilt to the data payload as two new bytes and then will update the Length and Checksum bytes while dropping the sync, source and destination bytes. Sample FCS packet in: 76 63 00 01 A6 00 24 00 00 0E 93 B9 00 03 82 E5 00 00 51 0C 00 00 C6 FB 00 00 09 8C 00 00 95 66 00 00 01 96 00 00 0B E5 00 00 B3 D6 4D CE Modified packet with pan 90 tilt 90 03 00 26 00 00 0E 93 B9 00 03 82 E5 00 00 51 0C 00 00 C6 FB 00 00 09 8C 00 00 95 66 00 00 01 96 00 00 0B E5 00 00 B3 D6 5A 5A 00 73 Function of Subsystem 4: Tracking the Servo Position As described in Subsystem 2 the servos will provide feedback as they move. The Page 17 of 25
RoBoard has a thread dedicated to listening for this feedback. The RoBoard contains a built in Analog to Digital converter and a library of functions to work with this hardware. The value of the A/D converter can be directly read into an integer value using the readchannel command from the RoBoard C++ library. Once that integer value was obtained it was important to find a strong algorithm to associate a degree amount with this digital servo reading. After testing and some calculations it was determined that the relationship was linear or extremely close to being linear. An algorithm that is able to calculate the servo reading to the accuracy of 1 degree, as determined by the project requirements, was developed. It was determined that the bounds of the servo given its position on the gimbal, the range of readings over the potentiometer and the integer value associated with the maximum boundaries of servo movement needed to be known in order to actually calculate the position accurately. Each servo needed to be tested for these numbers and have the numbers manually entered into the algorithm for computing position. Knowing these numbers also allowed for an added fail safe that would not let the servos move beyond their bounds. Image Client Software Video Viewer Client The video viewer client allows the operator to perform two major functions: connect to the video camera to receive streaming video and connect to the gimbal control system to issue commands and retrieve meta data from the helicopter. Once connected to the camera, the video viewer client will begin receiving the H.264 encoded video streaming from the camera attached to the gimbal, decoding and displaying it to the operator for further use. While the operator is viewing the streaming video, the video viewer client is taking screenshots from the video and storing them in a database along with their associated meta data at a rate of 10 images per second. These images will allow the image analyst to go backward and forward in the video stream from a flagged target, in case the video viewer client operator missed the target by a couple of frames and there is a better image of it further back. Once the operator sees a target, he or she need only hit the suspicion button on the client to flag the current image as suspicious. Once an image has been flagged, it will then appear in the image analysis client for further analysis. Page 18 of 25
Shows current metadata received from the GCS Suspicion button Connect to Gimbal Control Server Connect to video stream received from Roboard Image Analysis Client The image analysis client complements the video viewer client and allows for better workflow during the competition. This program will be running on a separate computer from the video viewer client (though it is possible to run both programs on the same computer) and have a separate operator. This operator can concentrate solely on analyzing flagged images from the streaming video, instead of have to analyze images while still looking for new targets. The image analysis client connects to the database that the video viewer client is storing its images and meta data in. Every 5 seconds the image analysis client poles the database looking for new targets. If a new one is found, the client updates the viewing pane to have a thumbnail of this target image. The operator then clicks on this thumbnail to view a full size version of the same image and perform further analysis on it. If the image contains a target, the operator has the capability of storing information about this image to a database for future use. This information includes the latitude and longitude of the target as well as the alphanumeric it contains, its shape and its color. This information in the database can then be used to produce a formatted report on the found targets that includes thumbnail images of the target itself. The image analysis client uses the Google Earth rendering system in the background to get an accurate reading on the latitude and longitude of the target that has been found. When the operator clicks on an image to view it full size, the meta data associated with that image is used to sync the Google Earth engine with the physical location of the helicopter in the virtual map. Whenever the user clicks on a location in the image, the pixel information of that click is sent to Page 19 of 25
the Google Earth engine. The Google Earth engine then returns the exact latitude and longitude of the location that the user clicked in the image. All of this is performed in the background, as the Google Earth engine is not actually displaying anything. To the operator, this process is completely seamless. User can click on viewer to pull up the target data form User can submit info to acquired image database and it will save the georeferenced local Suspicious images appear in the filmstrip Filmstrip is updated 1once/sec Clicking an image will cause it to appear in viewer Power System In order to power the Helo UAS, several different batteries are required. The most critical of these are the 18.5 volt lithium polymer batteries used to power the helicopter s motor and the 5 volt nickel metal hydride receiver battery. The 5V NiMH battery also powers the helicopter servos and the UAV safety switch. The FCS, including the radio modem and sensors, are run off of an 11.1V LiPo battery. The IRIS system is powered from a 14.8V LiPo and uses two battery elimination circuits (BECs) to step down the voltage to the required levels. The overall configuration of the power system is shown below. Page 20 of 25
18.5V LiPo 18.5V LiPo 5V NiMH Helo Motor UAV Switch RC Receiver Servos 11.1V LiPo 14.8V LiPo 5V FCS Imaging Control Unit MIDG II IMU 12v BEC Video TX Wifi Amplifier 7.2v BEC Canon S2 Helo UAS Power System Page 21 of 25
Operational Setup In order to increase efficiency, team members will be given specific tasks and work in parallel to complete the mission. In addition to the RC Safety Pilot, the mission team will consist of the following members: Flight Control Operator Video System Operator Two Intelligence Analysts During the competition they will work in and around the team s trailer as shown below. Trailer 72'-8" Image Analysis Image Analysis 30'-0" Intelligence Analyst Intelligence Analyst Video Control Operator Video Station 35'-0" 15'-9" 900 MHz Antenna 2.4 GHz Antenna Status Monitor Control Client Flight Control Operator RC Safety Pilot Flight Control Operator The flight control operator will be in charge of running the GCS Control Client and communicating with the RC safety pilot. He will be responsible for entering waypoints, adjusting target altitude and airspeed and plotting flight paths through the search area. Additionally, he will make any changes to the no-fly zones and search areas as required by the judges during the mission. The flight control operator will be located closest to the RC safety Page 22 of 25
pilot to allow for efficient communication about the status of the helicopter. Should a problem be detected with the FCS or wireless link, he can quickly notify the RC safety pilot to take manual control. A second monitor will be connected to the flight control operator s computer in order to allow the judges to easily view the current status of the mission at any time. Video System Operator The video system operator will be located inside the trailer and be in charge of running the video system. He will be able to control the level of autonomy of both the camera aiming process and the image capture process. By analyzing the live video feed, he will be able to flag certain locations as high or low priority based on the presence of potential targets. By flagging these locations, the flight control operator and the intelligence analysts will be automatically notified through their respective GCS clients. The flight control operator can then modify the flight path to get a closer look at the targets if necessary. Intelligence Analyst Two intelligence analysts will also be located inside the trailer and be in charge of analyzing the high resolution images captured by the IRIS. When they locate a target, they will record all the relevant parameters (size, shape, location, etc) and prepare actionable intelligence for the judges. Like the video system operator, they will also have the ability to flag certain areas for further analysis. Safety Fail-safe flight modes The flight control system incorporates additional flight modes that ensure maximum safety in the event of a system failure. These flight modes can also be activated by the safety pilot or the Flight Control Operator. Return Home The return home system allows the aircraft to safely fly to a set of GPS coordinates designated as the home location. Home coordinates are set automatically during autonomous takeoff unless manually entered by the Flight Control Operator before flight. The coordinates can also be modified after takeoff if necessary. Stability Augmentation Mode Stability augmentation is an additional mode of operation intended to assist the RC safety pilot in returning the helicopter safely, should it be necessary. In this mode of operation, the FCS will hold the last set altitude, heading and orientation set by the RC pilot. The RC pilot will be able to command changes in the FCS parameters using the RC transmitter. For instance, increasing the throttle/collective stick will command the FCS to increase the target altitude. Returning the stick to center will lock in the helicopter s current altitude. Similar adjustments can be made to the helicopter s heading and orientation. This is similar to the stability augmentation mode of the Yamaha RMAX Helicopter. Page 23 of 25
Flight Terminate The exact definition of flight termination depends on status of available sensors and batteries. In any flight termination event, the aircraft will conduct a minimum energy landing within the specified 500 ft radius. Batteries Battery voltages are continuously monitored by the FCS (and transmitted to the GCS) to ensure they remain in safe operating regions. Safe operating thresholds are calculated based upon the current aircraft distance from the home location, with an additional safety margin. If any battery voltage drops below this safety threshold, the FCS enters Return Home mode. In the event that the flight critical motor batteries become depleted before the aircraft is able to return home, the aircraft will perform a minimum energy landing. Transmission Loss Transmission loss events have been divided into two categories depending on the type of signals lost. Loss of signal from the Safety Pilot RC transmitter results in the UAV switch entering FCS mode. This event also triggers the FCS to enter Return Home mode. Transmission loss from the GCS can be detected by both the FCS and GCS. In this event, the FCS enters Return Home mode and the Flight Control Operator is notified. Failure Modes Analysis Failure Response Loss of RC Ground operators are notified and FCS enters return home mode. Communication (Initial) Loss of RC Aircraft will execute flight termination procedure. Communication (30 sec) Loss of GCS Ground operators are notified and FCS enters return home mode. Communication (30 sec) Loss of GPS Link (Initial) Aircraft will hold zero degrees pitch and roll and attempt to maintain a hover. If helicopter begins to drift into a no fly zone, RC safety pilot will take manual or stability augmented control. Loss of GPS Link (1 min) RC safety pilot will take manual or stability augmented control. Loss of IRIS Link Image system will continue to capture images automatically. Images will be analyzed after landing. FCS, Receiver or Motor Ground operators are notified and FCS enters return home mode. Battery approaches RC safety pilot may choose to take manual or stability augmented minimum safety level control. Motor or Receiver Aircraft will execute flight termination procedure. Battery voltage falls below critical level FCS battery dies RC safety pilot will take manual control. Page 24 of 25
Conclusion Throughout the process, design decisions were made to based upon the rules and mission objectives of the AUVSI competition. A helicopter was chosen as the ideal vehicle for the mission, primarily for its ability to loiter in a given location and efficiently gather actionable intelligence. The flight control system is capable of autonomously flying the aircraft, while the imagery system provides actionable intelligence to ground control operators and judges. Furthermore, an iterative testing process with emphasis on safety has served to minimize the risk of failure. The team has recognized the educational value of the project and is proud to be participating in the competition. In conclusion, the VCU believes they have been successful in creating a complete Unmanned Aircraft System capable of meeting the objectives of the AUVSI competition. Acknowledgements The would like to acknowledge the following individuals for their support. Dr. Robert H. Klenke, Advisor Mr. Mark Sternheimer Mr. Mark Schon Page 25 of 25