Flight Data Handling with Augmented Reality. Doctoral Symposium ICRAT 18, Castelldefels, Barcelona (Catalonia) June 25 th 29th 2018

Transcription

1 DLR.de/fl Chart 1 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Flight Data Handling with Augmented Reality Doctoral Symposium ICRAT 18, Castelldefels, Barcelona (Catalonia) June 25 th 29th 2018 Hejar Gürlük DLR Institute of Flight Guidance Controller Assistance

2 Chart 2 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Background Scope of Thesis Concept Study Summary & Outlook

3 Chart 3 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Safety Accidents per flight phase Accidents Fatalities Runway 6% 62% Highest incidents and accident rates are related to Runway (62%) in contrast to in flight (3%) (ICAO Safety Report 2015) Loss of Control in Flight 3% 60% Tower: Errors relate mainly to the field of visual perception major contributing factor for runway incursions (Hilburn, 2004) Error types related to tower environment Perception Memory Decision Making Response Execution Rule Breaking

4 Chart 4 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Head Down vs. Head Up Problem Head Up Head Down Tower Controller Working Position Head Down (HD): Information systems and devices; frequent usage but scan times not as long as head up (Pinska, 2009) Head Up (HU): Outside view Crucial for maintaining adequate situational awareness (Hilburn, 2004) Remains most important source of information (Ruffner, 2008) Problem: frequent transitions from HU to HD: delayed detection and interpretation of visual information major error source for runway incursions (Hilburn, 2004)

5 Chart 5 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Information Complexity Widely distributed head-down displays Controllers have to mentally merge the different information sources (HD / HU) Increasing information complexity (System Wide Information Management) This can lead to higher workload in terms of prolonged times for information acquisition and analysis Tower Controller Working Position

6 Chart 6 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Low Visibility Conditions decreased capacity (throughput) and increased delay at airports operating under CAT II/IIIA-C

7 Chart 7 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Background Scope of Thesis Concept Study Summary & Outlook

8 Chart 8 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Solution Approach and Aim of the Thesis Projection of relevant flight and topological data into the real outside view by means of augmented reality prolongation of head-up times, improved situational awareness, reduced information search maintain capacity (even under low visibility conditions) safety benefits Context-adaptive information presentation: user and task-dependent presentation of only operationally relevant information What to present? When to present? How to present? reduction of controller workload (information acquisition and -analysis) Motto and Aim: display the right (amount of) information at the right time!

9 Chart 9 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Augmented Reality overclutter Issues: - overclutter - untailored information presentation - missing interaction or feedback - integration of AR into operations unclear - display ergonomics, perceptual issues, registration errors Research gap: operational concept for such an assistance system (e.g. for different operational conditions) -> HOW TO WORK WITH THIS? systematic assessment of operational and human performance benefits

10 Chart 10 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Main Research Question of Phd Thesis What are the human performance and operational benefits of context-adaptive augmented reality for air traffic control towers?

11 Chart 11 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Scope of Phd Thesis (I) Development, Implementation and Validation of Operational Concept Workplace Analyses & Model Development (2015) 1 st Concept study adaptiveaugmented outside view (2016) 2 nd Concept study integrated information management (2017)

12 Chart 12 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Scope of Phd Thesis (II) work in progress work in progress 3 rd Study low visibility and concept transfer on Hololens (2018) AR Attention Guidance (09/2018) Validation of Operational Concept (End 2018)

13 Chart 13 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Background Method Concept Study Summary & Outlook

14 Chart 14 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Concept in a nutshell Adaptive Information Management Context Surveillance Data Flight Plan Data Environmental Data Assistance System Data presentation of operat. relevant information reduction of workload and displayclutter enhanced situational awareness Increase capacity and safety under IMC? Direct interaction indirect interaction Integrated Information Management

15 Chart 15 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Background Method Concept Study Outlook & Summary Ausblick Research Aim Experimental Setup Results Conclusion

16 Chart 16 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 2 nd Concept Study Flight Data Handling with Augmented Reality DLR 360 Apron and Tower Simulator LTU7HJ

17 Chart 17 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Research objectives Evaluation the current state of the operational concept Identify most promising CWP configuration Test the effect of augmented reality based flight data handling on human performance

18 Chart 18 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Research hypotheses H1 An augmented outside view with integrated information management (H1.1) leads to significantly longer head up times (H1.2) yields improved situational awareness (H1.3) yields lower workload compared to a conventional tower controller working position H2 Direct interaction in the augmented outside view yields (H2.1) higher situational awareness (H2.2) lower workload compared to indirect interaction

19 Chart 19 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Experimental Setup [1] TFDPS (tower flight data processing system) [2] Weather Display [3] Air Situation Data Display [4] Ground Radar Display

20 Chart 20 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Configurations Configuration Description Technical Setup C1 C2 C3 C4 conventional CWP indirect interaction direct interaction both interaction methods experimental tower controller working position (no augmented outside view) C1 + augmented outside view + coupling with TFDPS C1 + augmented outside view + coupling with 3D idrive Controller. Data entry via context-menu. No TFDPS available! C3 + augmented outside view + TFDPS: data entry both via TFDPS and 3D controller possible Configuration C4 context-adaptive augmented outside view with both interaction possibilities Configuration C4 context-adaptive augmented outside view with both interaction possibilities

21 Chart 21 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Eyetracking: Areas of Interests Wearable eye tracking glasses SMI Eye Tracking Analyzer (DLR)

22 Chart 22 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Scenarios and Experimental Procedure Test persons (n=4): 3 DFS Aerodrome Controller, 1 Military Aerodrome Controller Hamburg Local Control IFR Traffic (just few VFR flights) Arrivals only on RWY 23, Departures from RWY 33 only VMC Traffic volume (above medium, rather high) R/T with 2 Pseudopilots no telephone coordination with other ATC units Part I: 4 simulation runs á 45 minutes (ca. 48 IFR, 2 VFR flights) Eyetracking recording during simulation runs Post-Run Questionnaires & Debriefings Part II: Workshop

23 Mean Dwell Times (in %) Chart 23 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Results: Eyetracking 100% 90% 80% 70% % 50% 40% 50,8% 58,2% Head-Up Head-Down 30% 20% 34,5% % 0% 10.3 C1 C2 C3 C4 Controller Working Positions Standard Errors for Head-Up: [C1]: ±1.60 [C2]: ± 4.37 [C3]: ± 3.81 [C4]: ± 5.16 Standard Errors for Head-Down: [C1]: ±2.29 [C2]: ± 2.69 [C3]: ± 1.34 [C4]: ± 2.46

24 3D SART Score Chart 24 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Results: Situation Awareness (3D SART) 100,0 Mean Situational Awareness 90,0 80,0 70,0 60, ,0 40, ,8% 58,2% 30, ,5% 20,0 10, ,0 C1 C2 C3 C4 Mean SA 51,3 60,3 22,3 72,0

25 Overall Worklad Score Chart 25 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Results: NASA-TLX Workload Scores 34,5% 20,0 NASA TLX (Weighted) 18,0 16,0 14,0 12,0 10,0 50,8% 58,2% 8, ,0 4,0 2, ,0 C1 C2 C3 C4 Overall Workload (all subjects) 8,0 7,6 10,6 6,6

26 Chart 26 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Background Method Concept Study Conclusion & Outlook

27 Chart 27 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Conclusion (I) In general, configuration C4 was rated as the most beneficial and best-suited CWP for an augmented reality based flight data handling configuration C4 showed mean dwell times for head up increased more than 25% compared to a conventional tower CWP (hypothesis H1.1) configuration C4 promotes the situation awareness at most (hypothesis H1.2) configuration C4 NASA TLX workload ratings were the lowest when compared to the other CWP configurations (hypothesis H1.3) the results support the hypothesis 1 (H1.1, H1.2 and H1.3) almost entirely

28 Chart 28 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Conclusion (II) Is Head Up Only a desirable approach? Despite the high dwell times for head up configuration C3 (89.3%), direct interaction in the augmented outside view did neither automatically yield higher situational awareness (hypothesis H2.1) than C2 (indirect interaction) (actually the opposite was the case! C3 obtained the lowest S.A. ratings) C3 yielded highest workload scores (hypothesis H2.2) possibly due to. absence of TFDPS (important for flight planning). fairly high traffic volume in the scenarios,. insufficient training, unusual working method,. limitations related to implementation and nonergonomic handling Thus, hypothesis H2.1 and H2.2 could not be confirmed!

29 Chart 29 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Summary Preliminary results although of descriptive nature are promising Combined direct/indirect Interaction with AR flight data is the most beneficial and best-suited configuration for an augmented reality based flight data handling Substantial increase of head up time when working with direct/indirect Interaction with AR flight data could be a strong indicator for improved situation awareness But things are not that simple, with a look at configuration C3 There must be some sweet spot for head up migration tolerant system development: at early stages of system development, integration of AR within the operational environment should be done carefully in small steps (thus existing ATC systems such as TFDPS should be incorporated within new introduced assistance systems. At least in the beginning )

30 Chart 30 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Outlook Future work will focus on the enhancement of the interaction concept Evaluate the suitability of the operational concept under all-weather operations (3 rd study) Final Validation Campaign currently under preparation sufficiently large sample size (n=15 ATCOs) will be determined in order to apply inferential statistics and test for significant effects and to have fun and to keep me busy with lots of data collecting and analyzing

31 Chart 31 > Flight Data Handling with Augmented Reality > Hejar Gürlük > ICRAT 2018 > 2018/06/29 Take Home Message Results encourage a paradigm shift for flight data handling with augmented reality: away from a display system to an operating system!

32 CONTACT Folie 32 Hejar Gürlük German Aerospace Center (DLR) Institute of Flight Guidance Controller Assistance That s all! Thank you very much for your attention! Thank you!