APPLICATIONS TO SYNTHETIC AND PERIPHERAL VISION DISPLAY SYSTEMS FOR MANNED AND UNMANNED AIR VEHICLES. A thesis presented to.

Transcription

1 APPLICATIONS TO SYNTHETIC AND PERIPHERAL VISION DISPLAY SYSTEMS FOR MANNED AND UNMANNED AIR VEHICLES A thesis presented to the faculty of the Russ College of Engineering and Technology of Ohio University In partial fulfillment of the requirements for the degree Master of Science Behlul J. Poonawalla November 2007

2 2 This thesis titled APPLICATIONS TO SYNTHETIC AND PERIPHERAL VISION DISPLAY SYSTEMS FOR MANNED AND UNMANNED AIR VEHICLES by BEHLUL J. POONAWALLA has been approved for the School of Electrical Engineering and Computer Science and the Russ College of Engineering and Technology by Michael S. Braasch Professor of Electrical Engineering and Computer Science Dennis Irwin Dean, Russ College of Engineering and Technology

3 3 ABSTRACT POONAWALLA, BEHLUL J, M.S., November 2007, Electrical Engineering APPLICATIONS TO SYNTHETIC AND PERIPHERAL VISION DISPLAY SYSTEMS FOR MANNED AND UNMANNED AIR VEHICLES (159 pp.) Director ofthesis: Michael S. Braasch Spatial disorientation plays a large role in fatal accidents especially in General Aviation (GA). In the event of low visibility, maintenance of spatial awareness is a crucial factor, to the pilot, in keeping the aircraft at a level attitude. That said, current GA cockpit instrumentation provides no significant solution to alleviate the spatial disorientation problem encountered by pilots. Latest advancements in navigational technology and the development of Mircoelectromechanical System (MEMS) for precise attitude determination have led to the research and development of a prototype Synthetic and Peripheral Vision Display (SPVD) system at Ohio University. This thesis discusses the architecture and flight tests conducted that document the performance and viability of a prototype SPVD. Additionally, it also discusses the study of a series of Human Factors flight trials designed to test the efficacy of the system. Furthermore the thesis provides an insight into the utilization of SPVD s for remote piloting of Unmanned Air Vehicles (UAV s). Approved: Michael S. Braasch Professor of Electrical Engineering and Computer Science

4 4 ACKNOWLEDGMENTS Three years since I first started my masters program in Electrical Engineering, here at Ohio University and I finally am writing my acknowledgements page for my Masters Thesis. It s been an exciting ride especially for someone with international origins. Words can merely not express my gratitude and appreciation to all those who have supported me throughout my course of study here at Ohio University. I cherish all my moments experienced in the wonderful county of Athens, Ohio. I express my sincere appreciation and gratitude to my thesis advisor Dr Michael Braasch, Professor, Department of Electrical Engineering & Computer Science and Director, of the Avionics Engineering Center. I am indebted to his kindness, friendship and for being my teacher. A quote on the side of the drawer cabinet in his office reads The mediocre teacher tells. The good teacher explains. The superior teacher demonstrates. The great teacher inspires. ~William Arthur Ward Dr Braasch is an inspiration to me in ways unimaginable. I owe him my lifetime gratification. I thank Dr Angie Bukley, my Controls and Guidance teacher. A very good friend and always took time off from her busy schedule to talk to me. I wish her all the very best in her future endeavors with the University of Tennessee Space Institute. I d like to thank Dr Frank van Graas, my committee member and teacher too. I learned a lot from his Navigation classes. Each time you stop by to talk to him you definitely will come off learning something new. Thanks for all the helpful suggestions.

5 5 Thanks to Dr Maarten Uijt de Haag, for being on my thesis committee too and for some helpful remarks and guidance initially during the start of my research with the Avionics Engineering Center. Thanks to Dr William (Gene) Kaufman, Assistant Professor of the Department of Mathematics for being on my committee. I appreciate your time in reviewing my thesis. I wish to thank Dr Michael Mumper, erstwhile Associate Provost for Graduate Studies for his kind words, faith and trust that he saw in me. Thanks for giving me the opportunity to work with the office of Graduate Studies, where I met the most wonderful people. I thank Vic, Usha and Ishan Matta for being there for me always. They are now part of my family and I am indebted to their love and affection. I wish them all the happiness. I am grateful to Mr Michael Tedesco, my friend and supervisor at the office of Graduate Studies. He s the coolest boss one can come across. Learnt a lot of web development tricks from him. Thanks for all the good times in the GSS Attic. I am grateful to Sai Kalyanaraman, for being my friend and colleague and all the fun times we had grilling and going out. His remarks, suggestions and guidance have played and important role in this research. I thank Ananth Vadlamani for all his advice and help during my graduate course work, especially during the GPS receiver design course. I learnt a lot from him and also for being a very good friend and senior student. I am thankful to Tabassum (Ruhi) Khan for her invaluable friendship and support.

6 6 I wish to thank Dr David Diggle for his enthusiasm and support. Mr. Kadi Merbough for helping making things work. Mr. Tom Arthur and Mr. Kevin Johnson for all the fun times we had debugging Unmanned Air Vehicle issues and for all their inputs during this research. Mr. Jamie Edwards for being my safety pilot during the Saratoga flight tests and for all the RC model flying, you are the best. I thank the entire staff and my friends at the Office of Graduate Studies. Thanks to the entire faculty and staff of the Avionics Engineering Center. My friends and colleagues from 234A, Sunny Pandya, Chris Engel and Matt Smearcheck I wish to express my thanks to my uncle and aunt Mr. and Mrs. Mushtaq and Tasneem Lokhandwala for all their help, support and thoughtfulness. I thank Mr. Salim Lokhandwala for all his worldly insights and support always. I d like to thank Qusai Poonawala my cousin and friend, for all his advice on being a student in the United States. Thanks to Abhijeet Singh, Nirav Parikh and Ashita Rastogi for being friends. We had some memorable times here in Athens and a lot of fun on adventurous trips. Thanks to Saket Khajuria, Tehsin Aurangabadwala and Ali Baldiwala for their support and friendship. I thank the FAA/NASA Joint University Program for Air Transportation and Safety for funding this research.

7 7 Most of all I d like to thank my parents Jeevan and Maryam Poonawalla for all their love, constant support and encouragement. They have molded me to be the person I am today. Thanks to my brother Aamir with who I have shared my deepest joys and sorrows. Lastly I d like to thank the Lord above for all his guidance and help always.

8 8 TABLE OF CONTENTS Page ABSTRACT... 3 ACKNOWLEDGEMENTS... 4 LIST OF TABLES LIST OF FIGURES NOMENCLATURE INTRODUCTION DISORIENTATION IN FLIGHT Types of Spatial Disorientation Visual Disorientation Vestibular Disorientation Proprioceptive Disorientation NEED FOR SYNTHETIC AND PERIPHERAL VISION DISPLAYS Modes of Vision and the Importance of Peripheral Vision Focal Vision Mode Ambient Vision Mode Peripheral Vision Horizon Displays Research Over the Years The Malcolm Horizon and Other Developments HUMAN FACTORS STUDIES Human Factors Measurement in Aviation... 30

9 Flight Performance Non Flight Performance Physiological Conditions Subjective Conditions Why Human Factors Research? SUMMARY OF PRIOR RESEARCH AT OHIO UNIVERSITY Background on Work Done by Douglas Burch Background on Work Done by Jahnavi Chakrabarty SYNTHETIC AND PERIPHERAL VISION DISPLAY FOR MANNED GA AIRCRAFT System Architecture and Setup Equipment Installation Software Modifications and Additional Equipment Installation FLIGHT TESTING THE SYNTHETIC AND PERIPHERAL VISION DISPLAY SYSTEM Test Flight Briefing and Pre-Flight Procedures Cooper-Harper Pilot Rating for Aircraft Handling Qualities Flight Tests Flight Test Flight Test Flight Test RESULTS... 58

10 Flight Test Flight Test Subject Pilot Questionnaire and Comments from Flight Tests 1 and Flight Test Interpretation of Results UNMANNED AIRCRAFT RESEARCH Remotely Piloting Unmanned Air Vehicles using Peripheral Vision Video Displays Initial UAV Test Setup and Results Proof of Concept Using Microsoft Flight Simulator Updated UAV Flight Testing and Results CONCLUSIONS FUTURE WORK REFERENCES APPENDIX A: FLIGHT TEST 1 RESULT PLOTS APPENDIX B: SCANNED SUBJECT PILOT QUESTIONNAIRE FOR FLIGHT TEST 1 AND APPENDIX C: FLIGHT TEST 2 RESULT PLOTS APPENDIX D: SCANNED COOPER-HARPER MODEL RATED BY SUBJECT PILOTS FOR FLIGHT TEST APPENDIX E: SYNTHETIC AND PERIPHERAL VISION CODE E.1 Visual Basic Code for the Forward Looking Synthetic Vision Display E.2 Visual Basic Code for the Peripheral Vision LED Display

11 11 LIST OF TABLES Page Table 1: Test Matrix for the 3 Flight Tests 53 Table 1: Mean and Standard Deviation of Recovery Rates and Times for All Data Sets Table 2: Cooper-Harper Pilot Ratings for Different Equipment Configurations..64 Table 3: Absolute Roll Angle Values When an Unusual Attitude was Detected 68

12 12 LIST OF FIGURES Page Figure 1: Illustration of the Coriolis Illusion...21 Figure 2: Illustration of The Leans.22 Figure 3: Illustration of The Oculogravic Illusion.. 23 Figure 4: Architecture of the Synthetic Vision Display System by Douglas Burch.35 Figure 5: Comparison of Synthetic Vision Performance with the Out of the Window View...36 Figure 6: Architecture of the Synthetic and Peripheral Vision Display..40 Figure 7: Piper Saratoga PA-32, Ohio University Flying Laboratory.43 Figure 8: Synthetic Display Screen and LED Strips Placed Inside the Cockpit of the PA Figure 9: Data Processing, GPS and Inertial Equipment on Industrial Racks.44 Figure 10: Schematic Layout of the Equipment Installed 45 Figure 11: Sony Video Camera Used to Monitor Pilot Eye Movements.47 Figure 12: Modified Cooper-Harper Handling Qualities Rating Scale 51 Figure 13: Recovery Result for Subject Pilot 3.59 Figure 14: Recovery result for Subject Pilot 4.59 Figure 15: Recovery Result for Subject Pilot Figure 16: Recovery Result for Subject Pilot Figure 17: Recovery Result for Subject Pilot 5.62

13 13 Figure 18: Recovery Result for Subject Pilot 2 63 Figure 19: Recovery Result for Subject Pilot Figure 20: Recovery Result for Subject Pilot Figure 21: Military UAV Accidents Occurring Due to Human Related Issues at Various Phases of Flight...72 Figure 22: Gasoline Powered RC Model with a Set of Three Cameras Mounted on the Wing Providing a Peripheral View to the Remote RC Pilot.74 Figure 23: LCD Screen Setups on a Tripod for Remote Piloting.. 75 Figure 24: Snapshot of the Video Images as Observed on the Ground Station 76 Figure 25: GPS Time and Position Stamped Images From the VEO Keychain Camera 76 Figure 26: Microsoft Flight Simulator Test Setup with an Additional Side View.78 Figure 27: Simulated Traffic Pattern Performance by an RC pilot Using MS Flight Simulator WITHOUT THE SIDE VIEW.. 78 Figure 28: Simulated Traffic Pattern Performance by an RC Pilot Using MS Flight Simulator WITH LEFT SIDE VIEW AVAILABLE.79 Figure 29: TV Displays Show Left Side and Forward Looking Views as Transmitted from the UAV. 80 Figure 30: (L) Belly View of the UAV with FM Transmitters Attached. (R) Present Ohio University Test Bed. 81 Figure 31: Set of Three TV Sets Providing the UAV Pilot with a Panoramic View. 81

14 14 NOMENCLATURE AHRS AI AOPA CG FAA GA GPS HF ICAO IMU IMC IRB JUP MEMS NAS NASA NTSB PVHD RC RPV Attitude and Heading Reference System Attitude Indicator Aircraft Owners and Pilots Association Center of Gravity Federal Aviation Administration General Aviation Global Positioning System Human Factors International Civil Aviation Organization Inertial Measurement Unit Instrument Meteorological Conditions Institutional Review Board Joint University Program Microelectromechanical System National Airspace System National Aeronautics and Space Administration National Transportation Safety Board Peripheral Vision Horizon Display Radio Controlled Remotely Piloted Vehicle

15 15 SD UAV VMC Spatial Disorientation Unmanned Air Vehicle Visual Meteorological Conditions

16 16 1 Introduction If you are looking for perfect safety, you will do well to sit on a fence and watch the birds; but if you really wish to learn, you must mount a machine and become acquainted with its tricks by actual trial. Wilbur Wright, 1901 Air traffic has dramatically increased in the last decade and as forecasted this will continue to grow. Military, civil and general aviation (GA) have been major contributors to these growing trends occupying the manned airspace. The International Civil Aviation Organization reported a 5% increase in 2006 over 2005 for the world s airlines as measured in tonne-kilometres performed [1]. Recent developments have also shown a sharp rise in the use of Unmanned Aircraft for a plethora of civilian and military applications. Thus, safety of the aircraft, pilots and passengers is an important cause of concern for the aviation industry. Studies have indicated that a large number of aviation accidents have human error involved and specifically more so in the areas of general aviation. Therefore, human factors studies have utmost importance in the overall development and certification of a system. Similarly, in the area of remotely piloted vehicles (RPVs) human error related accidents are far higher than for manned aircraft. The Aircraft Owners and Pilots Association (AOPA) define GA as all civilian flying except scheduled passenger airlines [2]. Nevertheless, GA comprises the majority of flying that take place in the United States. GA offers the flexibility that at most times

17 17 commercial or other airlines fail to offer. To note just a few: rescue operations, cropdusting and law enforcement flights besides business and recreational aviation. The list can go on and on. The point is GA represents a major user of the National Airspace System (NAS). The number of GA accidents is significantly higher than commercial airlines and aviation safety is an important concern. The 2006 AOPA Air Safety Board Nall Report states that of a total of 1076 pilot related accidents 242 were fatal [3]. Spatial disorientation accounts for 90% of these fatal accidents as reported by the AOPA Air Safety Foundation [4]. Loss of orientation has been a constant contributing factor in most of the fatal accident occurrences in GA. Notable celebrity accidents include the one involving John F Kennedy Jr and his wife while aboard a Piper-32R-301. The National Transportation and Safety Board (NTSB) reported the possible cause of the accident as, the pilot's failure to maintain control of the airplane during a descent over water at night, which was a result of Spatial Disorientation (SD). Factors in the accident were: haze, and the dark night [5]. Visual, vestibular and muscular cues provide orientation information to a pilot. Errors in the pilot s ability to perceive aircraft attitude using these cues results in spatial disorientation [6]. The AOPA air safety board statistics in the period from 1987 to 1999 point out that accident due to SD occurred every eleven days [7]. Efforts are being made to invigorate the GA market to make it more affordable and attractive to the public at large enabling one to get a private pilot license with the only requirements being source of financial funds and basic pilot physical fitness. [8]

18 18 This thesis is an effort to discuss the importance of providing pilots (GA or UAV) with additional channels of information in the form of synthesized displays and peripheral vision enhancements. Early detection of unusual attitude scenarios using advanced attitude monitoring equipment help mitigate instrumentation as a factor when SD sinks in. Human factors studies and comprehensive flight tests were conducted taking into consideration various aspects that formed the basis for this research. It is hypothesized that the Synthetic and Peripheral Vision Display (SPVD) system helps as noted in [9] to: - Overcome SD by providing visual references for unusual attitude recovery - Lower the accident rate in GA as well as in the UAV sector - Provide infrequent and low time GA pilots with secondary channels of information at a fairly inexpensive rate. - Reduce pilot training time and currency time requirements This thesis is organized in the following manner: First, the effects and causes of disorientation will be summarized. Prior research activities at Ohio University and the need of additional displays will be discussed briefly. After this a thorough description of the current system setup and its design and working will be explained. Flight testing procedures and test results alongwith human factors studies results and conclusions will be described alongwith possible future work. UAV research activities and the use of peripheral vision video displays for guidance and navigation of these Remotely Piloted Vehicles (RPVs) are also provided.

19 19 2 Disorientation in Flight The AOPA Air Safety Foundation terms disorientation as one of GA s biggest killers [4]. Research has proven the crucial role which vision plays in a human s ability to maintain balance. Situational Awareness (SA) is a condition that develops in the pilot s brain on the continuous usage of aircraft instruments and flight related information [10]. While in motion and especially during Visual Flight Reference (VFR) conditions, a pilot deprived of visual cues, can easily succumb to a phenomenon known as Spatial Disorientation (SD). Other forms of disorientation also exist that arise due to motion and will be discussed later in this chapter. Disorientation mainly occurs due to conflicting inputs between the vestibular, proprioceptive and visual systems. 2.1 Types of Spatial Disorientation Spatial disorientation, especially in-flight SD can be broadly classified into three major types [11], [9]: 1. Visual Disorientation (Eye Movements) 2. Vestibular Disorientation (Inner Ear Fluid Movements) 3. Proprioceptive Disorientation (Muscle Movements)

20 Visual Disorientation Visual disorientation in aviation refers to the lack of position and motion relative to the earth [12]. 25% of the fatal accidents in GA and the military aviation sector are due to visual disorientation. It if often referred to as vection illusion [11]. 90% of total sensory input to the brain comes from vision and an estimated 90% of these visual inputs are provided by peripheral vision [11]. Traditional instruments do not address this additional channel of sensory information. The peripheral vision is exploited in this thesis. Visual disorientation is a serious source of disorientation especially in IMC Vestibular Disorientation Vestibular disorientation does not have the conscious or commonly occurring flight symptoms of vision, hearing and touch. It is realized only when we experience dizziness, nausea; vomiting etc. Vertigo would more accurately describe this form of disorientation or it feels similar to a bad aftermath from a roller coaster ride [9]. The inner ear makes up the vestibular system which consists of the otolith organs that sense linear acceleration experienced by the body and the semicircular canal that sense angular acceleration [11].

21 21 The Coriolis illusion is a commonly occurring type of vestibular disorientation. Figure 1, shows an illustration of how the coriolis illusion occurs [11]. Physiologically it is one of the hardest disorientations to recover from. Abrupt head movements during prolonged turns usually results in coriolis disorientation [11]. Figure 1: Illustration of the Coriolis Illusion Ref: Reinhart, Richard O., 1993, Fit to Fly: A Pilots guide to Health and Safety.

22 Proprioceptive Disorientation Proprioceptive and Vestibular disorientation symptoms can be closely linked to each other. Constant positional and postural movements by the pilot give rise to proprioceptive disorientation. Improperly executed turns without visual references can lead the pilot to interpret the gravitational force incorrectly. Two common types of proprioceptive disorientations are the Leans and the Occulogravic illusion. Figures 2 and 3 provide detailed illustrations of these occurrences [11]. Sensing of a large bank angle by the pilot when the aircraft is actually in a straight and level condition is when the Leans illusion occurs [11]. Figure 2: Illustration of The Leans Ref: Reinhart, Richard O., 1993, Fit to Fly: A Pilots guide to Health and Safety.

23 23 During aircraft acceleration or deceleration the otolith organs sense a nose high attitude with respect to gravity [11]. Without a cross-check of the instruments this phenomenon causes the pilot to react by maneuvering the aircraft to pitch down resulting in oculogravic disorientation [11]. Figure 3: Illustration of The Oculogravic Illusion Ref: Reinhart, Richard O., 1993, Fit to Fly: A Pilots guide to Health and Safety.

24 24 3 Need for Synthetic and Peripheral Vision Displays In the early days of aviation, navigation was based primarily on visual reference to landmarks, buildings and other points of interest. At night, however, these references were generally not visible. It was very difficult to recognize 3D forms at night without enough light sources. This gave way to lighted tower beacons, flashing coded light signals that guided the airplanes [8]. But these too were less useful in low visibility conditions such as fog, haze and cloudy weather conditions. New means of navigation had to be provided to aviators to be guided to the airstrip. GA cockpit instrumentation has not seen a major transformation over the past few decades. But the advent of newer technologies such as GPS and Microelectromechanical System (MEMS) Attitude and Heading Reference (AHRS) Units have made the cost of GA instrumentation design and installation relatively cheap and affordable [13]. Improvements in the design of the GA cockpit can help eliminate accidents occurring due to visual flights into Instrument Meteorological Conditions (IMC). A virtualized view of the outside world can aid the GA pilots to overcome spatial disorientation quicker and avoid putting the aircraft in an unusual attitude [13].

25 25 New researches on cockpit designs are mainly concentrated towards the forward looking view and very little consideration is given to information from other sources e.g. your peripheral vision. 3.1 Modes of Vision and The importance of Peripheral Vision The visual system is the most important of all sensor system cues that are available to a pilot for maintenance of awareness inside the cockpit. The dominance of biological conflicts occurring inside, between the visual systems itself is important to be understood. Humans do not possess attitude and altitude detectors and thus vision again is crucial in providing precise 3D information [12]. There are multiple systems inside the visual system that participate at various levels when viewing 3D information. But the two most important of these systems are [12], [14]: 1) Focal Vision Mode What 2) Ambient Vision Mode Where Focal Vision Mode Focal mode basically answers the question What is it we are looking at [14]. Focal vision inclines more towards the central visual fields. Thus, well represented or well defined is what is processed by the focal vision mode. For example, the focal

26 26 vision system reads instruments, identifies targets, or points the missile. If the focal vision system has deficiencies or is incorrect it can easily jeopardize a flight mission. Thus concentration towards the fine details i.e. high frequency spatial signals is what best represents the focal mode [12]. Focal vision is more effective when there is enough luminance and less refractive error. Thus, for example during night vision or more so during night flying the pilot if deprived of other sensory cues and only relies on IMC to guide his aircraft then it can be fatal [11]. In such cases ambient visual mode is what counts the most and this concept is further explained in the next section Ambient Vision Mode Ambient vision concentrates more towards the questions Where ; where is the observer in space or what is moving the earth or the observer [14]. This vision most likely involves identifying coarse details (low frequency spatial signals) and thus the peripheral visual fields are stimulated [12]. Low light conditions or degradation of image quality does not affect the ambient vision mode. Ambient vision looks at the overall picture; the finer details are neglected. Surrounding environmental conditions and other inputs are very important in the ambient vision mode. Night flying or night vision again is a very common example in understanding this mode of visualization. As explained in the focal mode during IMC the pilot is

27 27 deprived of surrounding cues. If additional channels of information are provided that address these issues then the pilots can have a more safe and sound flight. Another example that explains the ambient vision mode was performed at the National Consortium for Aviation Mobility in Oshkosh, WI. A person was given a book to read and was asked to walk on an upraised platform and was told to concentrate completely in reading while walking. During the first run, the reader subconsciously stopped at the end of the platform. On the second run, the volunteer was asked to wear a foggle ; an eyewear that restricts ambient visual cues and again told to walk the platform. This time the reader was unaware of his position and stepped off the platform, tripping himself [13]. Various such examples have been demonstrated illustrating the importance of ambient vision mode and more so the peripheral vision input. But it cannot function independently and works in conjunction with the focal mode. Ambient vision is susceptible to spatial disorientation and other sensory degradation. Hence peripheral vision represents an uncharted frontier and plays an important role in human aviation engineering.

28 Peripheral Vision Horizon Displays Research over the Years In an aircraft flying in IMC the only valid visual cue for a pilot under spatial disorientation is the artificial attitude indicator [14]. This chapter gives us an idea of the research that has been done in the past with regards to exploiting the ambient vision mode and providing additional visual references to pilots, especially, GA and low time pilots. Most of this research has been concentrated in developing a system that exercises the peripheral vision. This led to the development of Peripheral Vision Horizon Displays (PVHD) The Malcolm Horizon and other developments A larger display representation of attitude information provides a better perception of one s position while flying [12]. For example, a wider (than normal) horizon display or Attitude Indicator (AI) would definitely help in maintaining spatial awareness. But an aircraft cockpit has very limited space and therefore, the implementation of a novel solution to spatial disorientation problem has been difficult. But not so for Dr Richard Malcolm who was one of the first few to design a crude peripheral vision display system known as the Malcolm Horizon [15]. It consisted of a laser beam that was projected from behind the pilot seat [12]. The laser bean was a red line that stretched across the instrument panel of the aircraft. The underlying principle

29 29 was that as the aircraft banked or pitched the red line moved accordingly on the instrument panel. This would enable the pilots to sense the motion of the aircraft using peripheral vision. This idea was well appreciated initially and was ground tested, as well as implemented in the military SR 71 aircraft [12]. But there were some serious technical difficulties that arose in flight. During severe flying conditions and extreme pitch and bank angles the laser beam became a nuisance since it was reflected off the windshield and canopy of the aircraft [12]. The laser intensity also varied as it was reflected of various instruments. Due to these issues the Malcolm Horizon did not gain widespread acceptance. Nevertheless the Malcolm Horizon concept paved way for further research into utilizing peripheral sensory cues while flying. Suggestions were made for helmet mounted displays. This allowed a wider field of view without much interference from aircraft instrument panels and other physical restrictions. All these techniques definitely reduced pilot work load, prevented spatial disorientation and improved precision in attitude control [14].

30 30 4 Human Factors Studies Human Factors (HF) is a field of study that characterizes the man and machine relationship so that one can design better systems from the perspective of the human user. In aviation one of the most important HF considerations is the interaction of the pilot with the aircraft. The pilot s capabilities, limitations, behaviors and his or her personal opinions are studied. This helps in improving the performance and the overall well being of operators and the system [8]. This chapter gives us some introduction into human factors studies and its importance in the development of a system. For more resources and insights the reader is directed to reference [8]. 4.1 Human Factors Measurement in Aviation Human Factors measurements for aviation are more or less similar to HF measurements for other systems. The only difference may be in environment related contents and some physiological areas. HF measurements are basically very abstract in nature. For example, some of the major HF measurements are pilot workload, stress and fatigue.

31 31 Reference [8] categorizes aviation HF measurements into four broad groups viz: - Flight Performance - Non flight Performance - Physiological Conditions - Subjective Conditions Flight Performance Flight performance is usually measured taking the pilot into considerations since the aircraft and pilot are correlated. Pilot flight performance is measured in terms of how well the aircraft is being controlled by the pilot. The flight performance index can be determined by measuring the deviations of the state of the aircraft from its measured reference state. Flight performance can be easily characterized Non Flight Performance Non flight performance measurements are a bit more subjective to characterize and not as straight forward as the flight performance metrics. It does not deal with the aircraft state, but more with how well do aviation personnel and authorities carry out their duties. For example, communication between air-traffic control and the pilot is a non flight performance characteristic. Flight simulators and other advanced technologies are used to measure non flight performance matrices whilst on the ground as well as in flight.

32 Physiological Conditions In HF, physiological condition measurements are of very little relevance. But nevertheless its measurement cannot be ruled out. Most commonly used physiological measurements are heart rate and blood pressure [8]. Physiological conditions are very difficult to measure since most of the measurement methods are intrusive and are very impractical to be measured during in-flight. Simulators are more suited for these kinds of measurement conditions Subjective Conditions Subjective measures are an integral part of HF measurements in aviation though their validity and reliability are questionable. Subjective conditions usually deal with measuring handling qualities of aircrafts. In such cases the evaluations given by experienced pilots are very important and highly desirable even with the advancements of sophisticated testing systems. Subjective techniques usually include questionnaires, surveys, interviews and ratings. For example, in the later part of this thesis the Cooper- Harper handling qualities scale [16-17] is used to characterize the efficacy of the developed Synthetic and Peripheral Vision Display System.

33 Why Human Factors Research? Advanced developments in glass cockpit display technologies and navigation units have made GA flying easier. But research in the areas of situational awareness, behavioral analysis; workload, stress and fatigue do not receive the due attention needed. Simulators have been proven to be a good way in testing human factors measurements but they still do not reflect true performance conditions. Thus for actual or true flight performance metrics of the pilot and aircraft, HF measurements and research play an integral role. Military standards, FAA guidelines and other such regulatory bodies provide the literature required for HF measurements. There are two general approaches to HF evaluations a) top-down or systems approach b) and a bottom-up or monadical approach [8]. Top-down considers the system as a whole in its evaluation whereas the bottom-up deals with every individual part of the concerned system. Successful HF research in aviation usually correlates to less number of aviation accidents and increased safety.

34 34 5 Summary of Prior Research at Ohio University As mentioned earlier, we have realized the importance of maintaining spatial awareness whilst in flight. We have also discussed on how synthetic and peripheral vision cues can act as additional sources of information to the GA pilot and help minimize the effects of spatial disorientation. The Avionics Engineering Center at Ohio University has been actively involved with research activities in the area of advanced cockpit display technologies for the last five years [10, 13, 18, and 19]. Some of the work done here elaborates previous work done by individuals at the Center. Initial implementation and evaluation of a heads-up synthetic vision display system was carried out by Mr. Douglas Burch. This work was then carried forward by Ms Jahnavi Chakrabarty whose research focused more on developing the LED enabled peripheral vision display system. Both of these research activities were carried out under the guidance of Dr Michael Braasch who is the principal investigator in these research activities. Chapter 5 provides an overview of these prior research implementations and findings and also lays the foundation for this thesis. All the research work was primarily carried out under the auspices of the FAA/NASA Joint University Program (JUP) for Air Transportation Safety and Research. MIT, Princeton University and Ohio University are the primary investigators in this consortium.

35 Background on Work Done by Douglas Burch Work done by Douglas Burch and Dr Michael Braasch was instrumental in the development of an advanced cockpit display system. This section provides a brief outline on the work done by them. For further reading please refer to [18]. The underlying principle in this research work was the use of aircraft state vector information estimated by various navigation grade sensor suites. These measured values are used to drive a computer generated synthetic vision display system [18]. The state vector basically comprises of attitude and position information of the aircraft. The basic architecture of the system setup is a shown in Figure 4 below. Figure 4: Architecture of the Synthetic Vision Display System by Douglas Burch

36 36 The system consists of a Novatel OEM 4 GPS receiver that receives GPS signals from an antenna mounted on top of the aircraft. This provides precise position information. Attitude information is provided by a Crossbow AHRS unit. Inputs from these navigation sensors are then processed by a QNX enabled real time processing computer that outputs aircraft state vector information required by a 3D drive engine to render a synthetic vision display of the outside world. The 3D engine used is a free off the shelf commercial gaming engine that works with visual basic code. The synthetic vision display code was developed in Visual Basic and processed by a windows operating system. To render the synthesized view a terrain database was provided and was loaded on the same computer as the 3D game engine. A view of how the synthetic vision matches up to the outside world is shown in Figure 5 below. Figure 5: Comparison of Synthetic vision performance with the out of the window view

37 37 The figure above shows two views of UNI Runway 25. The left side is the real world view and the one on the right is the synthesized view. There is a slight discrepancy but the results are highly compelling. During the installation of the synthetic vision display various options were considered. Initially a head up mounted projected system was used. The synthesized view was observed as a projected image on a combiner glass screen. This showed only the front view of the system. But it was later suggested that panel mounted LCD screens be used as they have better performance and are not a hindrance to the pilots since they work directly from the inputs provided by a VGA cable of the computer. This reduced installation work. To provide peripheral vision cues, a multi-view synthetic vision display system comprising of side looking displays were considered and mounted on the side of the door. This, however, was rejected by the FAA since they could block the view out of the window. Flight tests results and pilot suggestions were positive and suggested its use as secondary guidance tool. Thus, the synthetic vision display idea did hold merit but there were several issues that needed to be resolved in terms of screen placements, digital readouts and other technicalities.

38 Background on Work Done by Jahnavi Chakrabarty As mentioned above in the earlier section the multi-view synthetic vision display system was definitely a good way for providing secondary information. Hence, to provide a better and more robust way for peripheral vision cues led to the conceptualization and development of the LED-forward looking synthetic vision display system by Ms Jahnavi Chakrabarty and Dr Michael Braasch [19]. We have discussed the importance of peripheral vision to reduce the dependence on instruments. The LED displays system is a novel way to utilize this information source. During quarterly JUP forum discussions it was suggested by Dr John Hansman from MIT to use vertical LED strips on the sides of the windshield to provide roll angle information. This method exercised the peripheral vision that detected various bank angles. The benefits of the LED system include the fact that they provide no hindrance to the pilot and do not obstruct the normal view of the pilot. They can be easily mounted on the sides of the windshield of the aircraft using Velcro. It does not cause unwanted reflections as was observed by the utilization of the Malcolm Horizon concept. The system architecture, equipment installation and comprehensive flight tests will be discussed in the next chapter. Since the LED work provides the basis for my research work the next chapter explains the system in detail. The research work carried out by Ms Chakrabarty received positive responses from pilots during the initial flight tests conducted using the forward looking-led symbiotic display.

39 39 6 Synthetic and Peripheral Vision Display for Manned GA Aircraft The GA synthetic and peripheral vision display setup currently installed and operational inside a Saratoga PA-32. The aircraft cockpit has been configured to provide a forward looking panel synthetic vision display and a pair of LED strips one on each side of the aircraft windshield for peripheral input. The synthetic vision display provides a front looking virtual display of the outside world. It is mounted in front of the pilot. The LED displays provide roll angle information of the aircraft with respect to the horizon. The objective of this system is to help spatially disoriented pilots detect an unusual scenario well in advance and avert fatal accidents. This research concentrates more towards the flight testing and comprehensive human factors studies to evaluate the performance and efficacy of the system. 6.1 System Architecture and Setup The architecture and system setup is similar to the design as described in [19]. Most of the components are similar and the only changes are additional equipment and

40 some software modifications. The schematic diagram of the basic architecture of the system is shown below in Figure Figure 6: Architecture of the Synthetic and Peripheral Vision Display. Ref [19]

41 41 The synthetic vision display consists of a panel mounted 8.4 Backlit TFT LCD display. It operates on a 5V DC supply and is easily mounted. It is connected to a laptop computer via a VGA cable. Precise position and velocity information is provided by a Novatel OEM 4 GPS receiver connected to a GPS antenna mounted on the top of the aircraft. Signals received from the Novatel GPS antenna mounted above the center of gravity (CG) of the aircraft are passed on to the GPS receiver for processing. Measurements are provided out of the receiver at a rate of 20Hz and data is sent out in ASCII format at 115,200 baud. Aircraft attitude information is given by a Crossbow Attitude and Heading Reference (AHRS) unit which is an inertial sensor suite and provides robust roll and pitch information at the rate of 60Hz. The Crossbow AHRS is a high accuracy, inertial navigation unit that works well with low flight dynamics. Data output by the GPS receiver and the AHRS units are input to a QNX platform based data collection computer that processes the data in real time mode. Data is provided to the computer via RS-425 links and attitude information is processed using software written in C. The primary synthetic vision display software is written in Visual Basic and runs via a Windows XP based laptop. The visual basic code utilizes the state vector data received from the QNX computer to manipulate a revolution 3D graphics engine to render the virtual picture of the outside world [10], [18]. Appendix F.1 provides further reference to the visual basic code.

42 42 The peripheral vision display system consists of two LED strips each mounted on the side of the windshield of the aircraft. Each strip consists of a set of 56 LEDS that provide attitude information based on roll information received from the Windows laptop [13], [19]. Visual Basic code that drives the LED strips is mentioned in Appendix F.2. Inputs from the QNX data processing computer is given to a second Windows based laptop that processes the state vector information. 6.2 Equipment Installation The current prototype working system is installed inside the Ohio University 1980 Piper Saratoga PA-32 shown below in Figure 7 [9]. It is single engine, low wing aircraft which can seat up to six occupants and is an excellent research platform. Figure 8 shows the panel mounted synthetic vision display mounted in front of the pilot s field of view. It can be clearly seen that the panel display does not obstruct the pilot s normal view of the outside world and instruments and is also foldable when not in use. LED strips are seen placed on the side of the cockpit windshield. Red arrow markings show their placement inside the cockpit.

43 43 Figure 7: Piper Saratoga PA-32, Ohio University Flying Laboratory. Ref [13] Right LED Strip Left LED Strip Figure 8: Synthetic Display Screen and LED Strips Placed Inside the Cockpit of the PA-32

44 44 All the data processing equipment, laptops and LED display hardware along with the GPS receiver and AHRS unit is mounted on industrial racks. Fasteners, aluminum brackets and Velcro is used to mount all the equipment securely (Figure 9). Figure 9: Data Processing, GPS and Inertial Equipment on Industrial Racks

45 45 A schematic layout of the equipment installation is shown in figure 10. It gives an idea of the wiring and other idiosyncrasies related to the system that can be observed from the figure [9]. Figure 10: Schematic Layout of The Equipment Installed. Ref [13].

46 Software Modifications and Additional Equipment Installation During the initial development of the peripheral vision display system, the LED strips showed roll angles up to 28. So the maximum extremes that the LED strips could go were to angles of about ± 28 on a 56 count LED strip. Colored LEDS were used with green LEDS in the middle of the strip indicating 0 or a steady level flight. Red LEDS were used to depict the extreme angles of 28. The LED strip scale is linear and every LED represents a one degree angle. The color of the LED between 0 and 28 is amber. Our first comprehensive flight tests that were conducted along with human factors research, were tailored to the 28 bank angle limit [20]. But suggestions and recommendations indicated that this limit be raised to measure larger roll angles. To incorporate this scale change, the peripheral vision visual basic code was modified to show roll angles of about ± 60 at both extremes [20]. The scale used for the LED strips this time around was non-linear, though the colors of the LEDS were more or less the same, except they now represented different values at the extremes viz ± 60. This provided the safety pilot some more room for disorienting the subject pilot s while testing the performance of the system. This is further explained in the next chapter on flight testing. As shown in Figure 11 below; a Sony video Camera was installed inside the cockpit of the aircraft. This camera was essentially used to monitor the subject pilot eye movements as per suggestions given by human factors studies experts.

47 47 Large font digital readout is also provided on the synthetic vision display to give the pilots roll and pitch information. Minor visual basic code modification was required to output this data on the screen. Sony Video Camera Figure 11: Sony Video Camera Used to Monitor Pilot Eye Movements Ref [20]

48 48 7 Flight testing the Synthetic and Peripheral Vision Display System Fight tests conducted by Mr Douglas Burch and Ms Jahnavi Chakrabarty in their theses [18] [20], gave enough evidence that the system was in good working condition and complied with all the FAA rules and regulations. Positive comments received from a few subject pilots tested in their research compelled a follow on comprehensive test procedure. Human factors studies testing methods were also conducted to fully validate and test the system. Night time flying was best suited to test the system. 7.1 Test Flight Briefing and Pre-Flight Procedures The tests were conducted in accordance with the Ohio University Institutional Review Board (IRB) guidelines for human subjects in research. Each and every flight test conducted had a safety pilot on board the aircraft. Flight tests were conducted at night under VMC. Majority of the pilots tested were students from Ohio Universities Department of Aviation. Flight times ranged from about 100 hours to a 1000 hours and pilot ratings varied from private pilot license and instrument rating to commercial license as well. This provided the test engineers a broad range of experience to test the system.

49 49 Cooper-Harper qualitative pilot rating scale [16], questionnaires and other tools were used to fully test the efficacy of the system. A confidentiality statement was signed and dated by each and every subject pilot Cooper-Harper Pilot Rating for Aircraft Handling Qualities Handling Qualities is defined as those qualities or characteristics of an aircraft that govern the ease and precision with which the pilot is able to perform the tasks required in support of an aircraft role [17]. The above definition implies that the combined performance of the pilot and the aircraft is used to perform a well defined role for example recovery from unusual attitude conditions. The ease, controllability and other performance characteristics are evaluated and a rating is given [17]. The Cooper Harper pilot rating is a scale that helps a pilot judge an aircraft role, specifically controllability, on a scale of 1 to 10. A rating of 1 is given if the system performance to a particular task is flawless and no modifications or ramifications are required [16]. A rating of 10 suggests that the system performance is not adequate and it consists of major deficiencies. The scale progresses linearly in an ascending order with increments of 1. Professor Robert Stengel of Princeton University suggested the use of this scale during our course of discussions at the JUP forum in April It was hypothesized that a qualitative evaluation of the synthetic and peripheral vision display system will

50 50 provide the necessary results that sometimes quantitative results fail to provide. The Cooper-Harper model with a minor change is shown in Figure 12 below [16], [20]. The Cooper-Harper model attempts to quantify human characteristics and its effects on the task performance matrix of pilots. It also gives a measure of the handling qualities of the aircraft. As observed from the chart, evaluations are based on controllability, workload and performance goals. Accordingly the pilot quantifies his readings on a scale of 1 to 10 as explained earlier. Therefore subjective assessment translates into a quantitative one.

51 51 How well does the system help you to assess the aircraft attitude and how easy is it to use the system to correct aircraft attitude? Figure 12: Modified Cooper-Harper Handling Qualities Rating Scale Ref: NASA TN D-5153 [15-16].

52 Flight Tests Three comprehensive flight tests were carried out to fully characterize the efficacy of the system. All in all, a total of 27 subject pilots tested the synthetic and peripheral vision display system. The tests were carried out during the spring and summer months of 2006 and Though the system performed well during ground testing it was very important that the same results be obtained when tested under real conditions. Before every flight test a brief description of the test setup and testing procedures were explained to the subject pilots followed by a confidentiality statement approval. The next few sections below detail the testing methods and procedures carried out. Table 1 lists a summary of the flight tests that were conducted.

53 53 Table 1: Test Matrix for the 3 Flight Tests Flight Test Flight Test 1 Flight Test 2 Hypothesis Low time infrequent GA pilots will have quicker recovery time from unusual bank conditions than traditional instruments. All equipment configurations tested. The increase in bank angle information displayed on the LED strips would provide the safety pilot additional room to disorient the subject pilots and help create unusual attitude situations. Qualitative analysis might provide the required results than a quantitative one Cooper Harper Rating. All equipment configurations tested. Flight Test 3 Stress was on quicker unusual attitude detection. Two equipment configurations were tested the LED peripheral display and the traditional instruments only Flight Test 1 Flight test 1 was conducted over a period of eight days from May 31 st to July 7 th The results and observations of this flight test were documented in form of a paper at the 25 th DASC Conference in October 2006 [9]. Before each flight test, the subject pilot was given a short description and demo of the test setup to get them acquainted with the system. The safety pilot and subject pilot enter the aircraft along with the test

54 54 engineer to begin the test. After the preliminary checks, the safety pilot switches on the aircraft and gets the engine rolling. The aircraft is kept stationary on the ground for about 10 minutes, for AHRS initialization and calibration and for GPS signal acquisition. Once initialized the safety and subject pilot prepare for take-off [9]. Once airborne the safety pilot performs maneuvers to disorient the subject pilot and put the aircraft in an unusual attitude. While the safety pilot is in control of the aircraft the subject pilot is instructed to keep his eyes closed and head down. On the words of Your Airplane from the safety pilot the subject pilot takes over the airplane and using the tools and instruments available at that time recovers the airplane. On the words of Your Airplane from the subject pilot the test engineer sitting in the back seat of the aircraft puts a marker in the data set to identify later, when the recovery took place and measure recovery time from thereafter. The results and calculation procedure are better explained in the next section. See Figure 13 for an illustrated example of how recovery time was calculated by visual approximation. When the subject pilot feels the aircraft is in a level attitude he utters the words Level. The safety pilot affirms if the aircraft is truly in a straight and level flight and confirms it by saying Level again. This completes one testing scenario using the configuration of equipments available to the pilot. Four equipment configurations were tested: 1) All Instruments Available (LEDS, Synthetic Vision, Traditional Instruments) 2) Synthetic Vision and Traditional Instruments Available 3) LED display and Traditional Instruments Available

55 55 4) Traditional Instruments only For every configuration the subject pilot performed a total of 4 recoveries, thus totaling to a sum of 16 unusual attitude recoveries. The flight tests were designed and configured in such a way so that identical testing conditions can be achieved for all subject pilots. 11 subject pilots were tested in all for this preliminary flight test. Most of the unusual attitude recoveries were performed from angles between ± 24 and ± 28. The safety pilot was specifically instructed to maintain identical conditions so that there is some uniform conformity between results obtained. Ideally, these test scenarios must be run under IMC but due to constraints this was not the case. To reproduce some kind of IMC, night time flying was best suited for the approximation. Post-processing on the data sets collected and analysis was conducted and the results of which are shown in section 8 and Figures 13 to Flight Test 2 Flight test 2 was conducted a bit later during the end of spring Five consecutive days of testing during the last week of May were conducted. Similar to flight test 1, the subject pilots were given a pre flight test briefing. However, compared to the last test procedure in flight test 1, a different approach to test the system was implemented. First of all, a Sony Video camera was used to record eye movements of the subject pilot s during recovery scenarios. The safety pilot switched on the recording

56 56 mode on the camera just before the subject pilot was given control of the airplane for recovery. To mark a completion of one test the video camera was blacked out in between recording by a simple palm wave in front of the camera lens [20]. By monitoring eye movements, human factors experts suggested that we might be able to observe the eye ball movement of the subject pilots and figure out what is it that the subject pilot glances, at the first instance when recovering the aircraft from an unusual attitude [20]. The Cooper-Harper model was used to qualitatively evaluate and rate the system. As explained earlier a qualitative analysis seemed to be a better way to evaluate the efficacy of the system. Before boarding the aircraft, the subject pilots were briefed on how the Cooper-Harper scale works and on what basis are they to rate the synthetic and peripheral system. The subject pilots were also informed that they will have to orally announce the ratings for the system, mid-way through the flight tests. The test engineer then notes these values down on a separate piece of paper. At the end of the flight test, the subject pilot was again given a printed Cooper-Harper scale and was told to reevaluate the system. Refer to Appendix D for further details on how the scale works and also how the pilots rated the system. The same four equipment configurations were tested as earlier with four unusual attitude scenarios for each configuration. Ten subject pilots tested the system. Each subject pilot was a student at the Ohio University Department of Aviation. The LED display system was modified to display bank angles of up to ± 60 as explained earlier. This time around the recoveries were performed from approximately ± 40 of roll angle depth [20]. This helped the safety pilot to get a larger range of bank angles to disorient

57 the subject pilots. ± 28 seemed a bit small for unusual attitudes and did not help in creating anxiety and stressful conditions for the subject pilots Flight Test 3 Flight test 3 initiated in response to the outcome of flight test 2 and was conducted in quick succession after the completion of flight test 2. Tests were conducted just prior to the last week of August The hypothesis was that with the use of the LED and instrument configuration the subject pilot will detect the unusual attitude quicker than when he is given only traditional instruments. Six student pilots and only two recoveries for each test configuration were performed. There was no synthetic vision display available to the pilot s. In-flight testing procedures were different and did not test all four configurations. Only two system setup configurations were tested 1) Traditional Instruments alone and 2) LED peripheral vision display strips and Traditional instruments [20]. In these flight tests the focus was not on measuring recovery time but rather on measuring, at which bank angle the subject pilot was successful in detecting an unusual attitude. The safety pilot disoriented the subject pilot s by engaging him in activities such as reading detailed charts or aviation maps or books. This helped divert the pilot s mental thinking and engross them enough to reach a disoriented state of mind. When the subject pilot feels or detects an unusual attitude he orally announce it and the test engineer notes down the roll angle at which it was detected.

58 58 8 Results Results for flight tests 1 and 2 are illustrated by way of plots (Figures 13 to 20) and Tables 2 and 3. Flight test 3 results are shown in Table 4. Matlab was used as the processing tool for the collected data sets. 8.1 Flight Test 1 Test results indicate roll and pitch recovery with respect to recovery time. Figure 13 indicates the method by which these results were analyzed. As seen in figure 13 the pointer shows when the words your airplane were spoken by the safety pilot and the next pointer shows the point where the recovery actually started. All these markers were visually inspected and an approximate recovery time was calculated when a steady level flight was reached, usually when the roll angle indicates 0 degrees. A summary of all the results is provided in Table 1. Only four example set plots are shown below. All the other plots are provided in Appendix A. The results show the peripheral vision display system provided a level of attitude awareness equivalent to that provided by traditional instruments or combinations with traditional instruments and forward-looking synthetic vision.

59 59 Recovery Rate Figure 13: Recovery Result for Subject Pilot 3. Ref [9] Figure 14: Recovery result for Subject pilot 4. Ref [9]

60 60 Figure 15: Recovery result for Subject pilot 8. Ref [9] Figure 16: Recovery result for Subject pilot 10. Ref [9]

61 61 Table 2: Mean and Standard Deviation of Recovery Rates and Times for All Data Sets. Ref [9]. Configuration Recovery Time (sec) Recovery Rate (deg/sec) Mean Standard Absolute Mean Standard Deviation Deviation SVD-LED-INSTRU LED-INSTRU SVD-INSTRU INSTRU ONLY

62 Flight Test 2 The flight test 2 roll and pitch recovery plots are shown below. The plots are similar to plots observed from flight test 1. Figures 17 to 20 provide a few sample data sets collected and plotted using Matlab. Table 2 indicates the Cooper-Harper ratings given by the subject pilots for every test configuration. Appendix D shows scanned Cooper Harper ratings by subject pilots and provides further information. Figure 17: Recovery result for Subject pilot 5. Ref [20]

63 63 Figure 18: Recovery result for Subject pilot 2. Ref [20] Figure 19: Recovery result for Subject pilot 7. Ref [20]

64 64 Figure 20: Recovery result for Subject pilot 9. Ref [20] Table 3: Cooper-Harper Pilot Ratings for Different Equipment Configurations. Ref [20] Subject Pilot Rating SVD-LED- INSTRU In flight End LED-INSTRU SVD-INSTRU INSTRU In flight End In flight End In flight End Test Pilot Test Pilot Test Pilot Test Pilot Test Pilot Test Pilot Test Pilot Test Pilot Test Pilot Test Pilot

65 65 As with Flight test 1, the flight test 2 roll and pitch recovery results indicate the peripheral vision display system provides attitude awareness equivalent to that of the other systems. The Cooper-Harper results indicate that some pilots are set in their ways and prefer traditional instruments over new systems. However, those pilots who were not particularly favorable towards the traditional instruments either rated the peripheral vision system equal and in some cases better.

66 8.3 Subject Pilot Questionnaire and Comments from flight tests 1 and 2 66 After the conclusion of every flight test, each and every subject pilot was given a post-flight questionnaire that stated system performance related questions. 20 of the 21 pilots tested provided their answers to the questionnaire. They were all asked to give their comments and suggestions. A summarized overview of these comments is given below. Appendix B shows scanned questionnaires of hand written entries to illustrate on how the subject pilot s responded to the set of questions. Some of these comments and suggestions have been documented in [9]. 1. Accurate Depiction (17 of the 20 pilots felt the system performed accurately) - Excellent computer graphical image - Well defined horizon - Pitch indication would help - Peripheral vision LEDS were too accurate resulting in fixation 2. Spatial Awareness (16 of the 20 pilots felt that the system helped in maintaining spatial awareness) - Synthetic Vision was a good visual reference to maintain awareness - Peripheral LED displays subliminally enhanced reaction time - Quicker familiarization with surroundings

67 67 3. Unobstructed View (11 out of the 20 pilots agreed that the synthetic vision display obstructed the normal line of vision of the pilot ) - Synthetic vision display obstructed the altimeter a bit - No obstruction with regards to LED strips - Blocked the natural line of sight - Additional channel of information 4. Best Scenario for Visual Cues (10 out of the 20 pilots felt that the peripheral LED display in conjunction with the traditional instruments provided the best visual cues) - Not complicated and confusing - Helpful during night time flying - Combination of all the configurations provided the best visual cues - Pitch reference line would be helpful 5. Primary or Secondary Aid (5 out of the 20 pilots will rely on the LED strips as primary sources of information) - With training and familiarity, the system in time can definitely be the primary source of usage - Additional tool for precise attitude recovery - Helpful for initial response to recovery 6. Suggestions and Improvements - Integration of the LEDS on the side of the LCD screen - Bright display screen and destroys night vision, need dimmer switch - Brighter luminance from LEDS

68 Flight Test 3 As discussed earlier, the purpose of flight test 3 was to evaluate the efficacy of the peripheral vision system in helping pilots identify unusual attitude situations. Table 1 indicates the bank angles at which the subject pilots verbally confirmed proceeding towards an unusual attitude condition. The test results show that half of the subject pilots identified the entry into an unusual attitude situation significantly sooner with the peripheral vision system than without. For the other half, the performance was roughly equal. Table 4: Absolute Roll Angle Values When an Unusual Attitude was Detected. Ref [20] Subject Pilot LED-INSTRU INSTRU DIFFERENCE (1) (2) (3) = (1) (2) In Degrees In Degrees In Degrees Test Pilot Test Pilot Test Pilot Test Pilot Test Pilot Test Pilot

69 Interpretation of Results Results obtained from flight test 1 do not support the hypothesized improvement in response time as initially predicted. Table 2, however, indicates that the synthetic and peripheral vision system matched up to the traditional instruments [9]. There are a couple of possibilities that can be related to these results obtained. One is that, either the advanced cockpit displays do not help at all or improvement in recovery time is not possible. The other explanation which is far more likely to be the case is that the benefits of these advanced displays lies not in the quick recovery from an unusual attitude condition but in the early detection and recognition of such situations. This second possibility was not tested while performing fight test 1 since it is very difficult to test in a controlled and repeatable manner [9]. A few other possibilities that were suggested by pilots were that they had too many things to look at and since their eyes were trained to monitor only the attitude indicator they subconsciously did just that. Also the engine noise was a possible give away of the aircraft approaching an unusual attitude since the sound of the engine gets louder when the aircraft descends. Based on the above conclusions from flight test 1, further testing was carried out. Results obtained from flight tests 2 were supportive of the synthetic and peripheral vision display system. The results can be termed as a response, but there are enough instances when the LED display system out performs the traditional instruments. As observed from Table 3, Cooper-Harper ratings given for the various test configurations are not universal though there are favorable instances to prove that the synthetic and

70 70 peripheral vision display system performs better than traditional instruments. Eye monitoring was not an easy task and results obtained from the video camera recordings were not accurate enough to completely judge the system performance. This was likely because this was a crude way of observing eye ball movement. Table 4 summarizes results obtained from flight test 3. Flight test 3 did not measure recovery time. Test procedures were designed such that we measure the bank angle at which the subject pilots were able to detect the unusual attitude condition. Results from flight test 3 showed that 3 out of the 6 pilots performed better using the LED peripheral vision display in early unusual attitude detection than with traditional instruments alone. The other half of the pilots performed equivalently with the LED peripheral vision display when compared with traditional instruments alone.

71 71 9 Unmanned Aircraft Research The design, development and implementation of Unmanned Aerial Vehicles (UAVs) over the past decade have revolutionized the aerospace industry. Although most UAVs are currently being used for military purposes, a large number of civilian applications are also envisioned. These include border patrol, communications relays, search-and-rescue, traffic monitoring and various others. The rise in the number of UAV applications has been paralleled with a rising number of UAV incidents and accidents. Currently, the UAV accident rate is 100 times higher than that of manned aircraft [21-22]. Human factors play a significant role in UAV accidents. Growth of UAV popularity and expected future integration into the NAS presents a demand to investigate the safety issues concerning these vehicles. It is said that it is relatively easier to command or pilot a manned aircraft than an unmanned aircraft. Figure 21 shown below, summarizes a recent summary of military UAV accidents related to human factor errors. It can be seen that the Predator UAV has the highest percentage of accident occurrences [21].

72 72 It is hypothesized that the benefit of peripheral vision displays (as demonstrated in manned aircraft) could be achieved in remotely piloted vehicle (RPV) operations as well. A wide field of view could improve UAV/RPV safety by 1) increasing situational awareness around the landing site (e.g., improved base-to-final turns); 2) providing better depth perception and hence improvement in stabilization during approach and landing operations; and 3) enhanced traffic and obstacle awareness leading to improved collision avoidance. Human Factor Issues In Accidents % of accidents Hunter Shadow (HF) Shadow (TALS) Pioneer Predator 0 Military UAVs Figure 21: Military UAV accidents occurring due to human related issues at various phases of flight. Ref: DOT/FAA/AM-04/24, December 2004 [21]

73 Remotely Piloting Unmanned Air Vehicles using Peripheral Vision Video Displays The research activities described in earlier chapters for GA gave the impetus to further test the peripheral vision concept for unmanned aircraft; guidance and navigation. For the past three years Ohio University has been engaged in UAV research and especially in the areas of remote piloting. In October 2006, defense systems giant Raytheon announced their revolutionary technology for remote piloting of unmanned aerial vehicles called The Universal Control System (UCS) [23]. By virtue of providing panoramic / peripheral video displays it is hypothesized that the safety and reliability of these vehicles can definitely be enhanced Initial UAV Test Setup and Results The initial test vehicle that was used as a UAV was a Hobbistar 60 MKIII trainer Radio Controlled (RC) model. Specifications can be found on the Hobbico website [24]. It has a gas engine and the receiver works on a 72 MHz band. Equipment mounted inside the fuselage of the UAV was as under: - Novatel Superstar II GPS receiver for logging in GPS position measurements. - Novatel GPS antenna. - VEO keychain camera to collect pictures of remote locations.

74 74 - PIC Microcontroller Emulator board for processing GPS data and for triggering the key chain camera and time stamping the date and time on a text file at which the picture was clicked. Additional equipment mounted on the wing of the airplane was three Microwireless Micro-Cam 2 cameras to provide a panoramic / peripheral view from the UAV as shown in Figure 22 below. Video images from these three cameras were wirelessly transmitted to the ground on a 2.4 GHz FM transmission link. Video images transmitted from these cameras were received by high gain antennas mounted on 12 LCD screens. Figure 23 gives a better picture of the setup. Figure 22: Gasoline powered RC model with a set of three cameras mounted on the wing providing a peripheral view to the remote RC pilot

75 75 Figure 23: LCD screen setups on a tripod for remote piloting Flight test were performed on a regular basis with this setup during the summer months of 2005 and During this time it was first hypothesized that streaming video data be used for remote RC piloting. A snapshot of what the RC pilot sees with the LCD screens is shown in Figure 24 below. As it can be very clearly observed the image resolution of these low cost cameras are not good enough for a safe UAV flight. There also seems to be quite a lot of aliasing effect when the propellers rotate. Fuel spill over from the gas engine on the camera lenses distorted the video images from. Engine vibrations were also a cause for concern. LCD screens that were used do not perform well in daylight conditions. It s very difficult to view video images on the screen in broad daylight. GPS time and location stamped images are also shown in Figure 25.

76 76 Figure 24: Snapshot of the video images as observed on the ground station Figure 25: GPS time and position stamped images from the VEO keychain camera Proof of Concept Using Microsoft Flight Simulator 2004 The above results were very promising and gave the impetus to further research and development in this area. Instead of investing in high end cameras and sophisticated equipment, an easier solution to test the viability of this concept was the use of a simulator. Therefore the acclaimed Microsoft Flight Simulator 2004 software was used

77 77 as a test engine. A standard RC Real Flight gaming controller was interfaced with the software to simulate a feel for real conditions [20]. Testing scenarios comprised of providing the test subject RC pilot with and without an additional side looking computer display for the same simulated conditions each time. Since Flight Simulator does not have an option for an RPV or UAV, the tests were conducted with the software configured to emulate a single-engine Cessna. These views are out of the window cockpit views and thus provide similar conditions to a RC pilot flying a UAV using displays. Four RC pilots were tested with this equipment setup. Only a single set of results are shown here by way of plots obtained from the simulator. In this test the RC pilot was asked to perform a simple traffic pattern under simulated conditions of weather and airport conditions. Seattle-Tacoma airport and stormy weather conditions were set up for this particular test result shown below. Figure 27 shows the traffic pattern performed by the RC pilot when he was void of a side looking display. Figure 28 shows the traffic pattern flying performance of the RC pilot when he was given an additional left side looking visual of the surroundings. It can be clearly seen that the RC pilot performs better with an additional side view (Figure 26).

78 78 Figure 26: Microsoft Flight Simulator Test Setup with an additional side view. Figure 27: Simulated traffic pattern performance by an RC pilot using MS flight simulator WITHOUT THE SIDE VIEW. Ref [20]

79 79 Figure 28: Simulated traffic pattern performance by an RC pilot using MS flight simulator WITH LEFT SIDE VIEW AVAILABLE. Ref [20] Updated UAV Flight Testing and Results The simulator tests provided initial confirmation of the hypothesis that an RPV operator could perform better with peripheral vision displays than without. As a result, an improved UAV testbed was developed to test the concept in the field. The UAV was updated to an electric powered RC model trainer. New cameras with good resolution were ordered. An important change was the use of a lower frequency FM video transmitter unit. Video images were now being transmitted from the UAV on a 900MHz wireless FM transmission link. There are three such wireless transmitting and receiving unit pairs along with three cameras mounted on the wing of the airplane. Most of the test

80 80 setup looks similar to the one mentioned above in section A few images are provided below to give the reader a better picture of the setup. Refer figures 30 and 31 below. Three standard CRT television sets were used for displaying the transmitted video images. This provided some shielding in sunlight and definitely performed better than the LCD screens. Figure 29 gives a better idea of what the RC pilot sees. Since the picture below is a photograph of the setup there are some discrepancies like the patch of black band in the video. In reality the RC pilot sees an uninterrupted clear video output. Figure 29: TV displays show left side and forward looking views as transmitted from the UAV. Ref: [20]

81 81 Figure 30: (L) Belly view of the UAV with FM transmitters attached. (R) Present Ohio University test bed. Ref: [20] Figure 31: Set of three TV sets providing the UAV pilot with a panoramic view Due to the use of lower frequency FM transmitters the effect of interference was reduced considerably. Vibration is not an issue any more due to the benefits of using an electrically powered engine. The transmission link of the equipment mounted, has a range