Flight Testing Of Fused Reality Visual Simulation System Justin Gray, Systems Technology, Inc. 13th Annual AIAA Southern California Aerospace Systems and Technology (ASAT) Conference April 30 th 2016, Santa Ana, CA 1
Overview General Objective Fused Reality Inflight System Description Test Methods and Conditions Test Results Outcomes and Observations Lessons Learned Conclusions 2
General Objective To assess handling qualities, there is no substitute for flight test. ~ Simulators have improved fidelity, but the human physical response to the dynamic response of the test aircraft can still not be adequately replicated in a simulator. ~ Flight testing comes at a much higher cost than simulation, especially when additional flight assets, such as aerial refueling tankers, are required for the testing. What if important handling qualities evaluations tasks that include aerial refueling, formation flying, etc. could be accomplished without the support aircraft? ~ This will not only save valuable flight test dollars, but it will also provide enhanced safety, since additional flight test assets will not be placed in close proximity to the test aircraft. ~ A flight test evaluation system based on a novel head mounted display visual system that combines virtual and real world elements was developed by Systems Technology, Inc. This presentation describes a flight test evaluation of such a system known as Fused Reality. 3
Fused Reality: Helicopter Cabin Gun Trainer Fused Reality combines real time video via a camera on a head mounted display and virtual elements to provide an overall immersive environment. The original concept used a real-time blue screen technique as the keying technology that distinguishes real from virtual: ~ Wherever the keying color is seen in the video, the color is replaced by the virtual scene.. ~ IR markers are used to track the position of real world objects that allows for a seamless interaction between the real and the virtual world. 4
Fused Reality: Initial Inflight System Initially, a cockpit window border keying technology using two color bands was employed: ~ Under consistent lighting conditions, the approach worked well and led to an effective demonstration. ~ As the program advanced, limitations of the border technology were exposed as integration with an actual aircraft was attempted. ~ Sensed electromagnetic properties in the out-the-window scene were used to successfully key out the cockpit window area. This allowed for an inflight technology demonstration and evaluation using the Calspan Learjet Inflight Simulator. 5
Fused Reality Inflight System Description: System Architecture 6
Fused Reality Inflight System Description: System Components IR tracking cameras 7
Test Methods and Conditions: Evaluation Aircraft GA8 Airvan The GippsAero GA-8 Airvan (N215AV) is a strut braced, high wing, eight seat, Federal Aviation Administration (FAA) type certified aircraft. The GA-8 has fully reversible flight controls driven by conventional mechanical pulleys and pushrods. Because of modifications, the aircraft has a special airworthiness certificate in the experimental category for research and development. 8
Test Methods and Conditions: Safety & Evaluation Pilots Approximately ten hours of flight time in the NTPS Airvan were used in support of this program: ~ Roughly half of those hours dedicated to system integration and checkout ~ The remaining hours to formal evaluation sorties Four NASA Armstrong test pilots participated in the formal evaluation flights over several days in mid-january 2015. 9
Test Methods and Conditions: Evaluation Tasks Precision Landing Formation Flying Virtual Mode Drogue Tracking Formation Flying Stencil Mode 10
Test Methods and Conditions: Evaluation Procedures Precision Landing ~ 500 AGL Virtual Altitude ~ Centerline approach and landing ~ Offset approach and landing (200 offset, correction at 200 AGL) ~ Desired/Adequate Landing Boxes: ±25 /±50 lat, ±250 /±500 lon ~ Desired/Adequate Airspeed: ±5/±10 KIAS Drogue Tracking ~ 5 minutes of practice followed by 30 second evaluation ~ Desired/Adequate (Lateral/Vertical): ±6 /±12 Formation Flying Virtual & Stencil Mode ~ 5 minutes of practice followed by 30 second evaluation ~ Desired/Adequate (Station Keeping Guides): ±5 /±10 NVG Mode Demonstration, time permitting 11
Test Results: Fused Reality Video 12
Lateral and Vertical Path to Runway (ft) Airspeed (KIAS) Test Results: Precision Offset Landing Example Pilot 4 offset landing run shown here received HQR 3 Pilot 4: Lag is not a factor at all what I am seeing is the airplane through the system and the system is not in the way. Successful landings achieved by all four evaluation pilots Indicates that the FR system can be used to conduct repeatable precision approach and landing evaluations at altitude. 500 400 Lateral Offset Virtual Altitude 100 90 80 300 70 60 200 50 100 40 30 0 20 10-100 0 10 20 30 40 50 60 70 80 Time (sec) 0 0 10 20 30 40 50 60 70 80 Time (sec) 13
Tracking Error (ft) Fore/Aft Longitudinal Error (ft) Test Results: Drogue Tracking Example Pilot 4 drogue tracking run shown here received HQR 8 Pilot 4: Very loose laterally. This is really hard. A little more fieldof-view would help the fore/aft cues. The Airvan was never intended to perform such a demanding task and control limitations, particularly in the lateral axis, were challenging for the pilots. 10 8 6 Lateral Tracking Error Vertical Tracking Error 0-10 4 2-20 0-30 -2-4 -40-6 -8-50 -10 140 145 150 155 160 165 170 Time (sec) -60 140 145 150 155 160 165 170 Time (sec) 14
Tracking Error (ft) Fore/Aft Longitudinal Error (ft) Test Results: Formation Flying Example virtual mode formation flying task shown here received HQR 4 Pilot 4: The fore/aft was the hardest thing for me to learn, but once I got a better sense for that, it made it a little easier. The evaluation pilots were able to successfully formation fly with a virtual target aircraft using the FR system in full virtual and stencil modes. 5 4 3 10 8 6 2 4 1 2 0 0-1 -2-2 -4-3 -4 Lateral Tracking Error Vertical Tracking Error -6-8 -5 150 155 160 165 170 175 180 Time (sec) -10 150 155 160 165 170 175 180 Time (sec) 15
Lessons Learned Checkout flights were vital to insure that the Fused Reality system was properly integrated with the test aircraft. For flight test projects with limited flight hours, inflight trouble shooting should only be done when absolutely necessary. ~ Better to land and fix the problem on the ground than burn flight hours trying to solve the issue in the air. Different flight test organizations have different rules, processes, procedures and philosophies, but the common goal is safety first. ~ Communication, respect, an open mind, and concentration on that common goal enable successful collaborations. When at a standstill, the FR navigation can drift because it keys off of ground track, but as soon as the aircraft is moving, the system locks on to the correct position. 16
Lessons Learned (continued) FR system needs to initialize from a stabilized flight condition at the start of each evaluation run. Sun angle was at times a factor for the IR cameras used for the evaluation pilot head tracking and for the video camera attached to the HMD. ~ Best results were achieved when the cameras were not pointed directly at the sun. Regarding the sun angle, when the sun angle was to the evaluation pilot s right, the right eye display could wash out. ~ NVG-like blinders attached to the HMD could alleviate this issue. Regarding the evaluation tasks: ~ Markings that provided next to the virtual runway that provided an aim point for the lateral offset would aid the precision offset landing. ~ Field-of-view of the HMD can limit longitudinal closure cues used in the drogue tracking and formation flying tasks. 17
Conclusions Using a NTPS Gippsland GA-8 Airvan research aircraft, a new standalone system for closed-loop handling qualities assessments was successfully demonstrated that allows virtual out-of-the-window elements to be combined with the real-world cockpit visual environment. As a result of these flight tests, all of the issues uncovered in the initial flight test campaign on the Calspan Learjet Inflight Simulator and during the initial GA-8 checkout flights have been corrected and tested in flight. Many enhancements for Fused Reality as well as other potential training uses were identified by the team which will aid in improving and commercializing this technology. 18
Questions? 19