Calhoun: The NPS Institutional Archive Faculty and Researcher Publications Faculty and Researcher Publications 2012-07-31 The eyes: Windows into the successful and unsuccessful strategies used during helicopter navigation and target detection Yang, Ji Hyun http://hdl.handle.net/10945/43298
The eyes: Windows into the successful and unsuccessful strategies used during helicopter navigation and target detection. Ji Hyun Yang PhD & Quinn Kennedy, PhD 831-656-7582 http://movesinstitute.org
The Research Team The Lab Ji Hyun Yang, PhD Quinn Kennedy, PhD CDR Joe Sullivan, PhD Jesse Huston Marek Kapolka Erik Johnson Thesis Students LT Chris Neboshynsky, USN LT Brad Cowden, USN CDR select Chris Kirby, USN MAJ Shane Price, USA LCDR Eric McMullen, USN This research is funded by the Navy Modeling and Simulation Office (NMSO) 2
Our focus: Overview 1) Understand the cognitive strategies and visual scan processes utilized by pilots during challenging helicopter situations. 2) Determine what types of errors are made. Our goal: To improve pilot training for these challenging situations and therefore improve mission effectiveness and reduce mishaps. 3
Overview Study 1: Determine if visual scan characteristics predict performance in overland navigation and target detection tasks. (thesis work by LT Chris Neboshynsky) Study 2: Determine if confidence is a good indicator of good performance. (thesis work by LT Bradley Cowden) 4
The Problem Helicopter overland navigation and target detection while navigating are cognitively complex and demanding tasks. Requires continuous monitoring of system and environment parameters Training novice helicopter pilots is challenging. Military instructor usually is the flying pilot Timely advice Safety 5
The Problem Currently, instructors have very few salient cues to rely on to assess trainee s navigation performance during flight. Specifically, instructors have very little insight into trainees cognitive states, e.g, when trainees are lost, do they realize it? http://www.iranian.com/main/2009/aug/becoming-mind-reader 6
The Problem Sullivan (2010). Analysis of scan pattern as an indicator of expertise to guide instruction. 7
Possible Solution: Visual Scan Patterns Consistent with other work (Marshall, 2007; Karsarkis et al, 2001; Van Orden et al, 2001), our previous results indicate that visual scan data can provide valuable training information to instructors (Sullivan et al, 2011; Yang et al, 2012). Expert pilots tend to have more frequent and rapid eye scan patterns than novice pilots. Experts tend to visually look ahead on the map but also revisit areas on the map they just flew over to retain confidence in their orientation. http://www.egr.vcu.edu/page.aspx?id=109 8
Purpose: Study 1 1) Explore whether visual scan characteristics predict performance in navigation and search and target detection tasks. 2) Extend our previous results by creating a more realistic task and setup: 9
Study 1 Predictions Visual scan characteristics predict: 1. Navigation performance (RMS error) 2. Target detection performance (true positives). http://photobucket.com/images/sam%20launcher/?page=1 http://www.militaryaircraft.de/pictures/military/aircraft/a-1/a-1.html 10
Participants 11 male participants: 8 2 1 Mean (sd) Range age (yrs) 35.09 (4.23) 29-41 total flight hrs 1561.82 (774.17) 750-3000 total overland hrs 950 (535.26) 350-2000 low level navigation experience 2.64 (1.12) 1 (none) 4 (considerable) target detection experience 3.18 (1.25) 1(none) - 5 (extensive) 11
Methods Pilots completed eyetracking calibration and a simulated flight. The simulated flight was comprised of three components: 1. Overland navigation task (12 waypoints) 2. Target detection task while navigating (5 mins) counterbalanced 3. Target detection with auto navigation (5 mins) 12
Methods Target detection: 5 friends (crashed plane) 5 foes (SAM launcher) Placement of targets varies in difficulty 13
Results: Visual Scan predicts Navigation Performance *Smaller RMS error associated with: Navigation only: Shorter looks at the map and less time looking OTW. Navigation and TD&I: More saccades, shorter looks at the IP, and less time looking at IP. But not total flight hours; neg. trend with self reported navigation experience. *all p s <.10 14
Results: Visual scan predicts TD&I *More accurate TD&I performance associated with: Navigation and TD&I: lower blink rate, fewer OTW fixations, fewer OTW-IP view changes. TD&I only: fewer fixations, fewer saccades, fewer fixations on IP. Navigation + TD&I TD&I *all p s <.10 Plane true positive (max = 5) SAM true positive (max = 5) Missed targets (max = 10) 3.00 (1.10) range: 1-5 4.00 (1.10) range: 2-5 3.73 (1.79) range: 1-7 3.36 (.81) range: 3-5 3.64 (.81) range: 2-5 2.91 (1.14) range: 1-4 But not total flight hours or self reported TD experience 15
Did we meet our secondary goal? 4 participants also had completed the Sullivan et al (2011) study. Technology used in the original study 65 degree field-of-view out-thewindow display Auto-rotation & fix-translation map display Joystick that required constant pressure to maintain attitude Pilot seat without helo cabin Terrain texture Electronic cockpit Technology used in the study you completed today 180 degree field-of-view of out-thewindow display Touchscreen map display (rotate and translate as you control) Cyclic control stick Pilot seat in helo cabin Higher resolution terrain texture Electronic cockpit 1: greatly detracts from the training potential 2: detracts from the training potential 3: has about the same training potential 4: enhances the training potential 5: greatly enhances the training potential The technology in the study you completed today of this simulation tool compared to the technology used in the original study. 16
Original setup (Sullivan, 2010) New setup Field of view map display joystick pilot seat terrain cockpit mean 4.5 3.25 3.75 3.25 4.25 3 sd 0.58 0.50 0.50 0.50 0.96 0.82 17
Study 1 Summary Preliminary evidence that visual scan characteristics predict task performance. Total flight hours and self reported experience did not predict performance. Results suggest that successful navigation and TD&I require different visual scan patterns: Navigation: quick and more frequent eye movements TD&I: slower and less frequent eye movements 18
Study 2 Investigate correlation between pilot s navigation performance and confidence Are pilots more confident when they navigate well? What are the insights gained by modeling perception/misperception in a Bayesian framework? 19
20
Human-in-the-loop experiment2 (2) Measure pilot perception & confidence Four routes with two routes with autopilot 15 NPS student and faculty participants All with overland navigation experience Ages 27-41 Average 35.5 13 male, 2 female Total flight hours Average 1,402.7 hours UNCLASSIFIED 21
CORRECTNESS On-track Off-track Results: Navigation Performance Navigation Performance Low Struggling. Aware that aircraft is off track CONFIDENCE High Lost and doesn t realize it. 22% 78% On course and lucky. On track and certain. When Off-track, perception was wrong 78% of the time. 27% 7% 8% 58% Only 2 participants believe they are overconfident in their navigation skills (8 Neutral, 5 Negative ) Dangerous Quadrant UNCLASSIFIED 22
ERROR pa = Distance between perceived and actual location Actual flight trajectory Perceived trajectory 2.5 Great Circle Distance = arcos(sin φ a sin φ p + cos φ a cos φ p cos (χ a χ p )) R a = actual aircraft position, p = perceived aircraft location φ =latitude, χ = longitude, R = Earth s radius at Twentynine Palms = 6372.8 km Spearman s Rank Correlation Coefficient (α = 0.05) ManNav1 2.5 ManNav2 2.5 AutoNav1 2.5 AutoNav2 2 2 2 2 1.5 1.5 1.5 1.5 1 1 1 1 0.5 0.5 0.5 0.5 0 0 0.5 1 Confidence 0 0 0.5 1 Confidence No correlation between navigation perception error and confidence UNCLASSIFIED 0 0 0.5 1 Confidence 0 0 0.5 1 Confidence 23
ERROR pa = Distance between perceived and actual location Actual flight trajectory Perceived trajectory Intended trajectory WP 3 ERROR ai = Distance between actual and intended location WP 2 WP 1 ERROR pi = Distance between perceived and intended location 0.5 ERRORai 0.5 ERRORpi 0.4 0.3 0.4 0.3 ρ=-.59 Biased perception towards intended route 0.2 0.2 0.1 0.1 0 0 0.5 1 Confidence 0 0 0.5 1 Confidence
Error distance Error distance Error distance Error distance 1.5 ManNav1 Perceived-Actual Perceived-Intended Actual-Intended 1.5 ManNav2 1 1 0.5 0.5 0 0 0.2 0.4 0.6 0.8 1 Confidence 0 0 0.2 0.4 0.6 0.8 1 Confidence 1.5 AutoNav1 1.5 AutoNav2 1 1 0.5 0.5 0 0 0.2 0.4 0.6 0.8 1 Confidence 0 0 0.2 0.4 0.6 0.8 1 Confidence 25
26
27
Study 2 summary Participants spent second most time in dangerous quadrant No correlation between confidence and perceived location error Bias towards the intended route MORS-TISDALE award finalist Implementation Add findings to current aviation physiological instructions and aviation pipeline training Integration into sim event Adjust Go-No-Go requirements for 28
Future Directions 1. Create a portfolio of scenarios. 2. Create easily interpretable, real-time psychophysiological output of trainee s cognitive state and strategies. 3. Bi-directionality of support between humans and machines 4. Make it portable. 5. Conduct a field study. 29
Q & A 30
Contacts Quinn Kennedy, PhD mqkenned@nps.edu x2618 GE-228 Ji Hyun Yang, PhD jyan1@nps.edu x3004 WA-212B 31
EXTRA SLIDES 32
Results: Visual scan patterns differ across scenarios Scan Time Navigation Navigation +TD&I TD&I OTW Scan % 63.94 (14.92) 75.85 (15.84) 86.11 (9.07) IP Scan % 8.20 (4.52) 5.89 (3.68) 3.17 (3.92) Map Scan % 27.85 (17.19) 18.25 (16.50) 10.72 (9.16) View Changes per Min Navigation Navigation +TD&I TD&I Total View changes 31.43 (9.87) 22.13 (9.32) 15.49 (8.83) OTW to MAP 10.11 (5.35) 7.22 (4.78) 5.10 (4.27) OTW to IP 4.18 (2.43) 3.16 (1.66) 2.25 (.82) MAP to IP 1.54 (1.16).90 (.92).43 (.46) MAP to OTW 9.92 (5.48) 6.84 (4.33) 5.06 (4.18) IP to MAP 1.35 (1.15).56 (.58).46 (.58) IP to OTW 4.34 (2.64) 3.46 (1.80) 2.19 (.74) Bolded # s: significant differences after Bonferroni correction 33
Results: Visual scan patterns differ across scenarios Navigation Navigation + TD&I TD&I Blinks per minute 9.79 (4.59) 6.34 (3.17) 11.09 (4.20) Saccades 113.95 (22.03) 106.28 (21.58) 107.05 (23.45) Median dwell time.21 (.04).23 (.04).24 (.04) Mean Fixations 2.40 (.28) 2.71 (.45) 3.30 (.71) OTW mean 2.92 (.88) 3.78 (1.07) 5.03 (1.51) fixations IP mean fixations 1.22 (.14) 1.17 (.11) 1.08 (.09) Map mean fixations 2.11 (.47) 1.86 (.36) 1.76 (.31) Bolded # s: significant differences after Bonferroni correction at alpha =.05 34
TFH results Trend for more saccades per minute with higher TFH (Spearman s r =.588, p =.057). Decrease in the amount time spent looking at the map (Spearman s r = -.376, p =.031) TFH not associated with dwell duration or fixation frequency. However, TFH was associated with less variability in dwell duration (Spearman s r = -.620, p =.042) and fixation frequency (Spearman s r = -.523, p =.099). 35