Determining the Impact of Haptic Peripheral Displays for UAV Operators Ryan Kilgore Charles Rivers Analytics, Inc. Birsen Donmez Missy Cummings MIT s Humans & Automation Lab 5 th Annual Human Factors of UAVs Workshop 15 May 2008
Research Motivation There is a transition of the military toward UASs As of October 2006, the Army has used hand-launched UASs to fly over 400,000 hours of support missions for Operations Enduring Freedom and Iraqi Freedom (Tsach, et al., 2007) Since 2001 Air Force reduced fighter inventory by 152 while simultaneously increasing UAS platforms by 113 (Randolph, 2007) By OCT 2007 Air Force shifted 120 pilots from the cockpit to UAV ground control stations (Rise, 2008) COCOMs are calling for force multiplication (Culbertson, 2006) Need autonomous technology for UAV and interface help for single operator who now is in supervisory role. Team of people flying one UAV One person supervising mission of multiple UAVS 2
Background General Problem: Lots of information to present, information overload is a possible problem Multimodal displays Auditory, or Haptic feedback as redundant modalities for presenting information 3
Background Why Multimodal? Resources within a sensory modality are limited There is so much a person can visually attend to May be able to offset resource demand onto alternative sensory modalities Other sensory channels could also provide redundant encoding for important information Auditory and haptic channels can provide means for passive monitoring Usage of these alternate modalities may offer superior performance (e.g., faster response time in the haptic channel) 4
Sonifications Previous phase of the project focused on auditory feedback comparing discrete to continuous audio Engine sound while driving Sonification is artificial continuous audio that is matched to a monitored task (e.g., radar) Sonifications is shown to aid anesthesiologists in monitoring patient status (Watson & Sanderson, 2004) Deviations from normal breathing patterns matched to musical notes allowed doctors to aurally detect a difference in patient status, while visually focusing on other tasks. Graham & Cummings (2007) showed that continuous audio can aid human supervision of both single and multi UAV operations better than discrete audio. But must have proper system integration to prevent masking 5
Haptic Feedback Research Haptic feedback has been shown to support operator performance in different domains, driving (Schumann, Godthelp, Farber, & Wontorra, 1993; Suzuki & Jansson, 2003), aviation (Sklar & Sarter, 1999). Sklar & Sarter (1999) showed that when compared to visual cues, both tactile, and redundant visual and tactile cues result in faster response to and higher detection rates for mode transitions in an automated cockpit system. Burke et al. (2006) compared the effects of visual-auditory and visual-tactic feedback using a meta-analytic approach on 43 different studies. Adding an additional modality was found to enhance performance overall Visual-auditory feedback most effective for moderate workload, single task conditions Visual-tactile feedback more effective for high workload, multiple task conditions. 6
As the command and control tasks shift from many operators controlling one vehicle to one controlling many, high operator workload can become a major issue. Auditory warnings provided to the operator can be masked by environmental noise and other auditory alerts. Haptic feedback may provide better support for command and control, and merits research in this domain. As a follow on to the research of Graham & Cummings (2007), a multi-uav supervisory control experiment was conducted to assess the potential benefits of continuous haptic feedback. 7
Multiple Autonomous Unmanned Vehicle Experimental Test Bed (MAUVE) Right Display Left Display Multi-Modal Workstation 8
Left Display Overview The three major screen elements on the left display are: 3 1 Map Display 2 Mission Execution 3 Mission Time 2 1 9
Right Display Overview The four major screen elements on the right display are: 1 15 Minute Timeline 2 5 1 2 Decision Support 3 UAV Health & Status Updates 4 Chat Box 5 4 3 UAV Status 10
Design of Experiment Pilot study 12 participants MIT students Single factor repeated measures design: Continuous haptic feedback Threshold haptic feedback Mapped to two control tasks Course deviations: continuous in nature Late arrivals: discrete in nature 11
Apparatus Continuous feedback for late target arrivals: The pressure vest inflated with the number of air bladders filled proportional to the target priority Stayed on until the participant responded to the late arrival. Threshold feedback for late target arrivals: The pressure vest inflated for late arrivals and stayed on for 2000 ms. All bladders were filled, independent of target priority. 12
Apparatus Continuous feedback for course deviations: The wristband buzzed continually throughout the experiment The buzzing intensified as the UAV deviated further off course. Threshold feedback for course deviations: When a UAV deviated from its course by 10 degrees, all five motors on the wristband turned on for 600 ms. 13
Reaction Times Continuous: 2.46 sec faster course deviation reaction time Threshold: 1.74 sec faster late arrival reaction time Course Deviation Reaction Time (sec) 13 12 11 10 9 8 7 6 Continuous Threshold Late Arrival Reaction Time (sec) 7.5 7 6.5 6 5.5 5 4.5 4 3.5 3 Continuous Threshold Haptic Feedback Type Haptic Feedback Type 14
A threshold haptic alert is more effective for tasks that require monitoring discrete events (e.g., late arrival to a target) A continual alert is more appropriate for tasks that require monitoring events that are continuous in nature (e.g., monitor how well a UAV keeps its course). Type of device is different possible confound 15
Subjective responses NASA TLX No differences for perceived workload Post-test responses Almost all participants thought that the haptic cues helped them in responding to both late arrivals as well as course deviations. Some even said that they depended on haptic as the primary source of identifying an off-nominal state. There were also concerns about operator annoyance with prolonged use of these haptic feedback devices. Participants in general preferred threshold feedback for course deviations, and found the continual buzzing to be annoying They still performed better with continuous feedback. 16
Discussion It is unclear how operators acceptance of these alerts would change performance with long term use. Acceptance of an alert has an important influence on the use and hence the effectiveness of it Low acceptance observed in this study may lead to disuse of the alerts in the long term, and result in degraded performance Effectiveness may also guide perceived usefulness and therefore acceptance. If operators realize that their performance, in fact, improves with the alert, then they may accept the alerts more. 17
Discussion Pilot study need more representative population Lack of a baseline to assess if and how much haptic feedback helps An explicit comparison between audio and haptic feedback is required 18
Acknowledgements This work was funded by an Army Phase II SBIR We would like to thank the following people for their contributions: Brian Malley, MIT Dimitry Kudryavtsev, CRA Hudson Graham, MIT 19
Questions? Birsen Donmez Post-doctoral Associate Humans & Automation Lab Massachusetts Institute of Technology (617) 253-0993 bdonmez@mit.edu Ryan Kilgore Senior Scientist Charles River Analytics, Inc. Cambridge, MA (617) 491-3474 rkilgore@cra.com