Relationship between the technical skills and eye-movement indicators of pilots

Transcription

1 Relationship between the technical skills and eye-movement indicators of pilots Lijing Wang a, Hongpeng Li, Dayong Dong b, Xiuli Shu a School of Aeronautic Science and Engineering of Beihang University, Beijing, CHINA; b Shanghai Aircraft Design and Research Institute, Shanghai, CHINA ` This paper aims to find the relationship between flying technical skills and eye-movement indicators of pilots by researching the flying skills and eye-movement parameters in the initial training phase. The operation performance of pilots was considered as an indicator of the level of the pilots technical skills. The fixation count, the fixation duration, the saccade amplitude, the saccade angle, the saccade time and the average fixation time were all considered as the indicators of eye-movement. The data was recorded during the flight process when pilots operated the simulator according to the given mission. The data showed that there was a very strong correlation between the pilots performance level and the fixation count, the average fixation time and the saccade time. Besides, there must be a specific law of variation among them. The conclusion was similar to that of the existing expert-novice model about eye-movement research. The expert-novice model only described the phenomenon that there may exist some differences on eye-movement indicators among pilots with different levels of technical skills. However, in this paper, the relationship between the level of flying skills and eye-movement indicators was found. Based on this conclusion an evaluation criteria based on eye-movement indicators could be formed to assess the level of pilots technical skills objectively. Keywords: technical skill, performance, eye-movement indicator, pilot 1. Introduction It is generally believed that the flying skills consist of two parts: technical skills and nontechnical skills. Harper thought that comparing with non- technical skills on the level of social and cultural, such as communication, cooperation and decision among crew in modern commercial flights (i.e. human factors skills), the technical skills refer to a series of technical skills pilots must have to accurately operate the aircraft and accomplish the flight task successfully, which include controlling height, heading, speed and flight state and so on (Harper R. P., Cooper G. E., 1986). Pilots can control the flight state effectively and overcome all adverse effects of flight activities caused by the complex flight environment only when they have well technical skills to ensure the flight safety. A pilot s technical skills mainly are formed in the initial training process. An accurate assessment of a pilot's technical skill level can not only assist in finding problems in the existing training, but also be guidance for the next training stage, in turn devote significance to the formation of the technical skills. The widely used subjective evaluation methods have the problem of reliability and consistency, so researchers tried to study objective evaluation

2 methods. This article mainly aims to find the relationship between the technical skills and eye-movement parameters of pilots who are on the initial training stage. A pilot s technical skills was decided by his performance reflected by parameters of the flight. The fixation count, duration of fixations, amplitude of the saccade, angle of the saccade, duration of the saccade and average duration of fixations are used as eye-movement characteristics. There already exist some studies showing that the pilots operating performance is significantly related to the eye-movement and the eye-movement indicators between novices and experiential pilots are significantly different. Kasarskis (2001) found the landing performance of experienced pilots was evidently better than novices based on VFR rules, and the eye-movement characteristics were significantly different. Valerie studied the operation difference between the experienced astronauts (experts) and commercial airline pilots (novice) when maneuvering the spacecraft simulators. The experienced astronauts had few errors and faster reaction time, and there were significant differences on eye-movement characteristics. However, the study didn t show the relationship between the technical skills and the eye-movement. Studies already showed that the pilots technical skills and eye-movement characteristics gradually change with the process of training. Comstock researched the eye-movement characteristics of pilots in the initial training process, and found the average fixation duration of main panel increased with the level of training. According to expert-novice model and Comstock s experiment, we propose the following hypothesis: in the initial training phase, pilot's technical skill levels correlates with the eye-movement characteristics and the eye-movement indicators will change regularly as the technical skill levels increasing. Performance was used as the indicator to measure the pilot s technical skill levels. The indicator could be obtained by calculating the bug of pilots flight parameters and standard parameters. The fixation count, duration of fixations, amplitude of the saccade, angle of the saccade, duration of the saccade and average duration of fixations were selected as eye-movement parameters. The experiment procedures are as follows. Select untrained college students to conduct the simulator flight experiment and record their eye-movement indicators after the initial flight training. Then let them conduct the simulator flight experiment again and record their eye-movement indicators after 4 hours of relevant training. During the experiment we can find that there is no evident correlation between the characteristics of eye-movement and performance in turn stage, while the correlation is significant in the landing stage, in which the fixation count and total time of saccade are both negative related to performance, while the average fixation time has a significant positive relation with performance. 2. Methods 2.1. Experiment design 24 undergraduate students took pat in the experiment. They were all novice cadet without any flying experience. All the subjects were trained based on the same saccade strategies. Then they performed a same flight task, and the data of performance and eye-movement was obtained for further analysis. It could be considered that the same flight mission meant the same mission difficulty, in turn the same workload level for the subjects. And the psychological state of the subjects was also similar when they were in the same training phase. This could remove the effects caused by factors besides technical skills, which included the psychological

3 state when performing the task, the mission difficulty and workload level. The same saccade strategies adopted and the same training process could also remove the effects on eye-movements caused by different education background and flight experience. The flight simulator used was a fixed simulator of J8. It has three visual displays and two touch screen instrument displays. The maneuvering gear consisted of digital touch console, side bar, throttle and pedals. The eye tracker used was Eyelink II produced by SR company of Canada. In the flight process, the flight route of shape P (Fig. 1) which included six stages including taking off, climbing, flying, hovering, approach and landing was used, and each stage had its own specific requirements of flight parameters (Fig. 1): Fig.1 The P flight task 2.2. Experiment process The experiment was divided into two parts, training and assessment. The training part was divided into theory training and simulator flight training. First, a theory training included eight 2-hour lessons was conducted by a theory instructor, in which the basic principles of flying, aircraft maneuvering knowledge and saccade strategies of visual instrument were taught. Then twenty 2-hour training lessons on the simulator were conducted by the simulator instructor, which mainly focused on the basic flight skills of taking off, landing and local flight. Finally the subjects were asked to conduct the P-shape flight training in two 2-hour lessons. In the assessment part, subjects would perform a local flight of the P-shaped that described in the training part in accordance with the accuracy. The data of flight performance and eye-movement was recorded during the flight Data analysis Eye-movement data required to be collected in this experiment includes the fixation count, the fixation duration, the saccade amplitude, the saccade angle, the saccade duration and the average duration of saccade.

4 Performance evaluation is based on the difference between the flight track of the pilot and the standard track. Flight track of the pilot was obtained from the flight parameters recorded by the simulator. And standard track was obtained by the instructor s data of test flight. The difference between the flight track of the pilot and the standard track means the difference of flight parameters between the two tracks. Performance indicator can be calculated from 5 secondary indicators using Formula. 1. D1 represents the score of the speed bug between the flight track and the standard track: 1 point for each 10km/hr bug; if the speed bug was more than 50km/hr, score 5 points. T1 represents the score of the time duration of the speed bug: 1 point for each 10 seconds; if the duration was more than 50 seconds, score 5 points. D2 represents the bug of the bank angle between the flight track and the standard track: 1 point for each 2 degree; if the difference was more than 10 degree, score 5 points. T2 represents the score of the time duration of the bank angle bug: the calculation method was the same as T1. D3 represents the bug of the height between the flight track and the standard track: 1 point for each 100m; if the difference was more than 500m, score 5 points. T3 represents the score of the time duration of the height bug: the calculation method is the same as T1. D4 represents the bug of the course between the flight track and the standard track: 1 point for each 2 degree; if the difference was more than 10 degree, score 5 points. T4 represents the score of the time duration of the course bug: the calculation method is the same as T1. D5 represents the bug of the rate of change of vertical speed between the flight track and the standard track: 1 point for each 10m/s2; if the difference was more than 50m/s2, score 5 points., each difference scores 1 point, more than scores 5 points. T5 represents the score of the time for the formation of the difference of the rate of change of vertical speed,calculation method is the same as T1. Performance=i=1510 Di1+0.1Ti5 (Formular.1) After the experiment, flying performance and eye-movement indicators collected directly by Eyelink during the flight were tested by the test of the fitting level (K-S distribution). Spearman s correlation analysis was adopted to analysis the relevance between each eye-movement indicator and performance. 3. Result 3.1. The test of the fitting level of subjects parameters In order to determine what test method should be used in the data analysis process, each subject s eye movement indicators were tested by the test of the fitting level (K-S distribution).the result demonstrated that all the progressive significance of subjects flying eye-movement indicators are not related------the distribution of eye movement indicators is not relatedly different from normal distribution, statistical results showed that the distribution of each eye movement indicators is a normal distribution Spearman correlation analysis between flight performance and eye movements parameters in various subjects: Spearman s correlation coefficient for ranked data method was adopted to select relevance between each eye movement indicator and performance of fight. The result demonstrated that

5 in the stage of landing, the fixation count and the performance of flight are highly positively related (p<0.01); the average duration of fixation and the performance of flight are highly negatively related (p<0.01); the total time of saccade and the performance of flight are negatively related (p<0.01). Fig.3 Scatter diagram of the relationship between flight performance and number of the fixation spots (Y= flight performance; X= number of the fixation spots) As can be seen from Fig.3, flight performance was positively correlated with the fixation count, P<0.01, namely fewer the fixation count are, higher performance the initial pilot can get. Fig.4 Scatter diagram of the relationship between flight performance and average fixation duration (Y= flight performance; X= average fixation duration) Flight performance is negatively correlated with the average fixation duration, P<0.01,.namely shorter average fixation duration is, higher performance the initial pilot can get.

6 Fig.5 Scatter diagram of the relationship between flight performance and total duration of saccade (Y= flight performance; X= total time of saccade) Flight performance is positively correlated with total time of saccade, P<0.01,namely more total duration of saccade is, higher performance the initial pilot can get. 4. Conclusion In the landing stage the fixation count and total time of saccade are both positive related to performance, while the average fixation time has a significant negative correlation with performance. This conclusion matches the result of expert-novice mode research that the fixation count of experts is clearly larger than that of novices, the average fixation time and total time of saccade are also less than novices. There are some differences between this conclusion and Comstock s about the initial flight training, that is, in the initial stages of training, shorter pilots average fixation duration is and more fixation are, better performance will be got. 5. Discussion This paper found that the characteristics of pilots eye-movement would change regularly while their flying skills levels changed. After a further training for each stage and a further research for each indicator, a set of corresponding relationship of the indicators of eye-movement and flight performance can be formed, as well as the rules for evaluating the technical skills by eyemovement indicators. Its meaning is great that it can help pilots to take specific saccade strategies for pilots according to different training stages. As for the measure of improving their saccade efficiency, speeding up the process of training, we will develop in the next phase of work. References Coulter, A Flying on autopilot. Airline Fleet Management, 80: Comstock, J.R., G.D. Coates and R.H. Kirby Eye-Scan Behavior in a Flight Simulation Task as a Function of Level of Training. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 29(4): Brown, D.L., et al Instrument scan strategies of F-117A pilots. Aviat Space Environ Med, 73(10):

7 Sullivan, J., et al Training simulation for helicopter navigation by characterizing visual scan patterns. Aviat Space Environ Med, 82(9): A., V., et al Characterizing Scan Patterns in a Spacecraft Cockpit Simulator: Expert Vs. Novice Performance. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 49(1): Kasarskis, P., et al COMPARISON OF EXPERT AND NOVICE SCAN BEHAVIORS DURING VFR FLIGHT in 11th International Symposium on Aviation Psychology. The Ohio State University: Columbus. Morrow, D.G., et al Expertise and Age Differences in Pilot Decision Making. Aging, Neuropsychology, and Cognition, 16(1): Schriver, A.T., et al Expertise differences in attentional strategies related to pilot decision making. Hum Factors, 50(6): Yang, J.H., et al Pilot performance: assessing how scan patterns & navigational assessments vary by flight expertise. Aviat Space Environ Med, 84(2): Kennedy, Q., et al Age and expertise effects in aviation decision making and flight control in a flight simulator. Aviat Space Environ Med, 81(5): Taylor, J.L., et al Pilot age and expertise predict flight simulator performance: a 3-year longitudinal study. Neurology, 68(9): Bellenkes, A.H., C.D. Wickens and A.F. Kramer,1997. Visual scanning and pilot expertise: the role of attentional flexibility and mental model development. Aviat Space Environ Med, 68(7): Steelman, K.S., et al Auditory, visual, and bimodal data link displays and how they support pilot performance. Aviat Space Environ Med, 84(6): Russi-Vigoya, M.N. and P. Patterson, A pilot study to evaluate gaze behavior in aircraft simulations. Biomed Sci Instrum, 48: Kirby, C.E., An Analysis of Helicopter Pilot Scan Techniques While Flying at Low Altitudes and High Speed. DTIC Document. :119. Ebbatson, M., et al Combining control input with flight path data to evaluate pilot performance in transport aircraft. Aviat Space Environ Med, 79(11): Croft, J.L., D.J. Pittman and C.T. Scialfa, Gaze behavior of spotters during an air-to-ground search. Hum Factors, 49(4): Sarter, N.B., R.J. Mumaw and C.D. Wickens Pilots' monitoring strategies and performance on automated flight decks: an empirical study combining behavioral and eye-tracking data. Hum Factors, 49(3): Tvaryanas, A.P Visual scan patterns during simulated control of an uninhabited aerial vehicle (UAV). Aviat Space Environ Med, 75(6): Talleur, D.A. and C.D. Wickens, The Effect of Pilot Visual Scanning Strategies on Traffic Detection Accuracy and Aircraft Control, in 12th International Symposium on Aviation Psychology. 2003: Dayton, OH. Ottati, W.L., J.C. Hickox and J. Richter, Eye Scan Patterns of Experienced and Novice Pilots during Visual Flight Rules (VFR) Navigation. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 43(1): Daniel J. Callan, U.M., Eye Movement Relationships to Excessive Performance Error in Aviation, in Human Factors and Ergonomics Society 42nd Annual Meeting.: Chicago

8 Tole, J.R., et al Visual scanning behavior and mental workload in aircraft pilots. Aviat Space Environ Med, 53(1): Papin, J.P., P. Naureils and G. Santucci, Pickup of visual information by the pilot during a ground control approach in a fighter aircraft simulator. Aviat Space Environ Med, 51(5): Thackray, R.I. and R.M. Touchstone, Visual serach performance during simulated radar observation with and without a sweepline. Aviat Space Environ Med, 51(4): Wetzel, P.A., G.M. Anderson and B.A. Barelka, Instructor Use of Eye Position Based Feedback for Pilot Training, in Human Factors and Ergonomics Society 42nd Annual Meeting. 5: Chicago. Jacob R J K,Karn K S Eye tracking in human-computer interaction and usability research: ready to deliver the promises. Computer Vision and Image Understanding, 24: Wen-qi, G., Fu-chun, Z., Li-bo, W., & Hui, L Flight training performance assessment model and implementation. Electronic Design Engineering. Zhongqi, L., Xiugan, Y., & Yubo, F Pilot performance evaluation model based on BPneural networks. Journal of Beijing University of Aeronautics and Astronautics, 36, 4, Wenge, W A subjective and objective method for determining index weights of training evaluation. Ship Electronic Engineering. Burki-Cohen, Go, et al Flight simulator fidelity considerations for total air line pilot training and evaluation