Engaging Innate Human Cognitive Capabilities to Coordinate Human Interruption in Human- Computer Interaction: The HAIL System Operator Console Cognitive System Components Weapon System HAIL Engine Dr. Daniel C. McFarlane Lockheed Martin Advanced Technology Laboratories mcfarlane@acm.org 856-792-9708 June 30, 2003
Table of Contents HAIL Overview Cognitive System Research Cognitive System Development R&D Process Summary 6/30/03-2
HAIL Overview What is HAIL? Technology Overview Operator Console Cognitive System Components Weapon System HAIL Engine 6/30/03-3
What is HAIL? Human Alerting and Interruption Logistics alert mediation technology HAIL has two parts: (1) The HAIL Engine: A cognitive system that delivers cognition-based resources for human alerting activities based on proven empirical human-subjects research (2) The HAIL System: An open architecture decision-support support system (DSS) component built around the HAIL Engine that delivers the capabilities of the HAIL Engine for human decision-makers within their work environments The first transition of the HAIL technology is into the US Navy s Aegis Weapon System The HAIL technology has two parts: (1) (1) A cognitive system as as a core HAIL Engine; and (2) (2) A decision-support system (DSS) that realizes the the capabilities of of the the HAIL Engine for for human users in in real world work environments. 6/30/03-4
What is HAIL? (cont.) Support for increased human capability: (1) Reduce human interruption rate by decoupling the alert rate from human interruption rate The HAIL solution is generic and does not address the platform-specific issue of reducing alert rate overall (2) Increase human control of information environment with negotiation-based coordination of interruption events (3) Increase operator access into past information with context recovery aids HAIL is is a platform-independent software component that actualizes the the accepted NRL NCARAI alerting research findings for for improving operator performance in in the the presence of of alerts. 6/30/03-5
Technology Overview (cont.) Aegis UI Core UI Interface Manager HAIL Engine Alert Managers Information Generation The HAIL Engine resides at at the the Business Tier in in an an N-Tier Architecture. It It encapsulates proven business logic on on how humans succeed at at making effective decisions in in alerting-dense work environments. 6/30/03-6
Technical Overview (cont.) RUP-guided transformation of proven research results into a mature componentc omponent-based cognitive system component Founded on Vision of future human workload for interrupt-driven responsibilities Interdisciplinary NRL research Documented requirements from multiple domains Process RUP-guided (Rational Unified Process) Iterative spiral software development User-centric UI prototyping and usability testing HAIL Alert Engine A Reusable Component-Based Open System Approach OO, Open Architecture, User-Centric, Modular, Extensible, Reusable, Well Documented, Robust, Scalable The HAIL software development approach is is using the the same tools and process as as Aegis Open Architecture to to minimize transition effort. 6/30/03-7
Cognitive Systems Research Vision Problem Description Operator Console Cognitive System Components Weapon System Approaches Examples Research Foundation HAIL Engine Human-Subjects Experiment 6/30/03-8
Vision of future human workload for interrupt-driven responsibilities Quote: Future human-computer interface will be rooted in delegation, not the vernacular of direct manipulation. (Nicholas Negroponte, Being Digital, 1995, p. 101) Increased alerting requirement identified through experience in applied research on intelligent command and control systems at NRL ISART (Interactive Situation Assessment and Rollup Tool) 6/30/03-9
Problem Description Success of tactical operations depends on high quality decision-making. However, this is not possible without better alerting technology The problem has four parts: 1) People have cognitive limitations 2) No decision-support support for management of context switching in current alerting mechanisms 3) Alerting mechanism delivers a large quantity of critical information but its current human interruption rate outpaces operators ability to keep up 4) Information overload continues to worsen There is a threshold for people s ability to handle interruptions without support, and when this threshold is exceeded, situational awareness fails and decision quality drops People have limited ability to to handle interruptions without support. 6/30/03-10
Approaches Remove person Training Incentives Selection of personnel Construction of a cognitive system for a DSS 6/30/03-11
Example Domain Commercial Air National Transportation Safety Board, 1988 Northwest Airlines Detroit Metropolitan Airport McDonnell-Douglas Douglas DC-9-61 Preflight checklist interrupted by Air Traffic Control Change in taxi-way instructions and weather alert for possible windshear Never resumed checklist task; Flaps in wrong position for take-off Task resumption aids could have prevented this accident A. Too many interruptions B. No operator control C. No trend information D. No recovery support Many platforms could be be improved with Interruption Support technology. 6/30/03-12
Example Domain Aegis Alerting 6/30/03-13
Example Domain Aegis Alerting Plasma Touch Panel 3. Operator decides to press REVIEW ALERTS button. Menu Bar 4. These displays are Primary Secondary refocused for the alert task. Hooked Hooked Track Track Window Window Tactical Situation Tactical Display Area Information (TACSIT) Windows Alert Window Status Window 2. Alert is displayed in the window. Buzzer 1. Buzzer sounds when each alert enters the queue. A. Too many interruptions B. No operator control of alert queue C. No trend information D. No support for context recovery after interruption Variable Action Buttons REVIEW ALERTS Fixed Button Aegis needs the the HAIL technology to to remain Fully Mission Capable. 6/30/03-14
Research Foundation All HAIL publications are available on the HAIL-SS web site http://www.atl.lmco.com/projects/hail/ NASA SSC-SD TADMUS Product Development University Research NRL HAIL Other HAIL Engine ONR Manning Affordability MSR The results of McFarlane s NRL HAIL Research were published as a two-article stand-alone issue of the journal Human-Computer Interaction, 2002, 17(3). Results Interruption negatively impacts performance Negotiation support for interruptions gives biggest overall win except when an alert task is time-critical People perform better when primed for interruption People switch contexts more easily when given context recovery support HAIL is is founded on on proven empirical research. 6/30/03-15
HAIL Basic Research Goal: to invent a cognitive system to support a DSS to support increased human workload capability for alert-based activities Theoretical products A A General Definition of Human Interruption A A Taxonomy of Human Interruption Empirical findings Negotiation is the best overall interruption coordination solution except for cases where small differences in timelines of handling alerts are critical 6/30/03-16
Human-Subjects Experimentation Single-factor within-subjects Latin squares design 36 subjects (18 male, 18 female), avg. age 24.7 (min. 18, max. 47) 6 treatments 4 experimental + 2 control 24 trials; 4.5 minute each 1 hour of practice followed by 1 hour of experiment Pilot testing to set difficulty (59, 80) 6/30/03-17
Negotiated is best for primary task. 6/30/03-18
Immediate is worst for interruption task. 6/30/03-19
Small timeliness problem with Neg. and Sch. 6/30/03-20
Negotiated is efficient for interruption task. 6/30/03-21
Negotiated is best for keying effectiveness. 6/30/03-22
Negotiated is best for keying efficiency. 6/30/03-23
Cognitive System Development Product Overview Scope of HAIL technology HAIL technology capabilities Technical Approach HAIL Engine Capabilities Work Process Software Development Operational Impact Programmatic Overview HAIL Team Overview Operator Console Cognitive System Components HAIL Engine Weapon System
Product Overview Domain-independent independent decision-support support engine for alert management HAIL-SS is producing two products (1) A platform-independent software engine that delivers a intelligent alert management critical future naval capability for improved C2 decision-support support in dense information environments Before HAIL After HAIL HAIL-SS is is producing a reusable general component-based software product for for alert management. 6/30/03-25
Product Overview (cont.) Aegis is the first transition target HAIL-SS is producing two Products (cont.) (2) A HAIL Engine integrated, tested, and validated within an Aegis Weapon System HAIL-SS is is producing a validated integration of of the the HAIL Engine within the the Aegis platform. 6/30/03-26
Scope of HAIL-SS Technology Three Development Activities for a C2 Alerting Mechanism User Interface: Human Factors Eng. to dynamically balance multimedia cueing to match operators cognitive needs (flashing, alarm sounds, color, spatial layout) Dependent on domain Research complete: no Cognitive System Component [HAIL Engine]: DSS engineering to manage alerts (negotiation-based task management; interactive visualization; automatic trend inference; context recovery support) Independent of domain Research complete: yes Alert Generation: Systems engineering to generate alerts to support tactical action (hard real-time computing; operations; tactical role modeling) Dependent on domain Research complete: no Application The HAIL Engine will will enhance both current and future UI UI and Alert Generation technologies. 6/30/03-27
HAIL Technology Capabilities Cognitive System Capabilities Broker alert management Alert management Negotiation-based support for coordination Visualization support for SA Context recovery support Trend inference automation Purpose Break design gridlock Decouple alert rate from interruption rate Leverage human ability to dynamically prioritize and schedule Leverage human ability to assimilate information Minimize human weaknesses for context switching Minimize human weaknesses for monitoring change over time Warfighter Benefits Better alert SA Fewer interruptions Optimum prioritization and scheduling of interruptions Receive informational alert content without interruption Switch contexts without making errors Detect significant trends 6/30/03-28
Technical Approach Integrated technology definition/design/transition process Leverage existing research Reusable HAIL alerting engine Open architecture, components-based design and development Multiple domain applications driving requirements Use Navy domain application to Define requirements Test and evaluate in a realistic environment Support transition Transition Link with AOA schedule requirement Phase 1: Test and Evaluation of First Prototype Phase 2: Evaluation in Operational Setting using Baseline 7P1 Design Evaluation Capture Requirements A 2-cycle development approach ensures that the the HAIL product is is user-centered. Develop Integration and Test 6/30/03-29
HAIL Alerting Engine Capabilities Action alert management Supports multiple interruption strategies User Interface HAIL-Based Alerting Application HAIL Technology Presentation management Supports presentation of information-only only alerts in multiple modalities Context recovery support Supports domain-dependent dependent recovery mechanisms Automated assessment Supports inference of higher- level alerts and trends HAIL technology provides future naval capability for for action alert management, presentation management, context recovery support, and automated assessment. 6/30/03-30
Work Process Application Requirements Definition Human Cognitive Requirements Risk Definition Prototype Platform- Specific Application Design HAIL Research Core HAIL Technology Detailed design Architecture Evaluation Design Prototype Platform- Specific Application Integrate and Test Evaluation Human cognitive requirements and application requirements drive the the development process. 6/30/03-31 31
RUP-Guided S/W Development Process HAIL process and toolset is consistent with Aegis Open Architecture UML Documentation Use Cases Activity Diagrams Visio and Altia Design Mockups for supporting spiral iterations of user Interface (UI) design and usability testing. HAIL is is using industry standard development process and documentation artifacts. 6/30/03-32 32
Operational Impact (Estimated) Successfully Handle Higher Information Volume Better decision-making and situational awareness of naval warfighters through support for human alerting Product will have general utility potential for several platforms; examples are for Aegis Frequency of interruptions reduced 50-80% Button actions required reduced 50-80% Time and effort required for critical alerts improved 15-50% 50% Correlate alerts with tracks on the TACSIT improved 40-60% Efficiency in resuming primary task after interruption improved 10-30% Perceived workload reduced 10-30% Quality of situational awareness improved Perceived stressfulness reduced 20-30% The HAIL cognitive system will will support greatly improved human capabilities for for dealing with alerts in in complex multitasking work contexts. 6/30/03-33 33
Programmatic Overview Funded by ONR and founded on the results of interdisciplinary theory eory-based research from NRL NRL NRL/ONR HAIL 95-00 00 ONR KSA FNC HAIL-SS 02-04 04 PEOTSC 05-?? (existing TTA) Aegis Research Technology Maturity Product Development Measured in TRL s Integration Other Naval Systems Other; The HAIL-SS Web Site: http://www.atl.lmco.com/projects/hail/ HAIL-SS is: An ONR KSA FNC project to develop and test a generic alerting software component that actualizes the accepted NRL NCARAI alerting research findings for improving operator performance in the presence of alerts HAIL-SS is on contract with the ONR KSA FNC Program (N00014-02 02-C- 0014, begun 4/02), LM ATL lead The HAIL technology starts with NRL-proven solution to to a recognized future Navy problem and is is paying down the the risk for for acquisition. 6/30/03-34 34
HAIL-SS Team Overview Full-coverage team for creation of a successful HAIL product Surface Ship Combat System Engineering Agent LM NE&SS-SS, SS, CSC Receiving Command PEO-IWS, PMS 400 PMS 461, PMS 378 HAIL Technology LM ATL (lead), NRL HAIL Product Transition into Fleet Operational SMEs BCI, ATRC, CSEDS This integrated, diverse team provides insight and expertise covering all all areas relating to to Aegis. 6/30/03-35 35
R&D Process Process Operator Console Cognitive System Components Weapon System HAIL Engine 6/30/03-36 36
Process Publication in Public Domain ISART HAIL (NRL 95-00) 00) (ATL 00) Lab TRL 1 TRL 2 TRL 3 Research HAIL-SS (ONR KSA FNC 02-04) 04) TRL 4 TRL 5 Product Development Mature COTS Technology TRL 6 TRL 7 Integration and Test LM NE&SS Aegis TRL 8 Fielded Surface Combatant TRL 9 The theory- based empirical experiments Future Mission Requirements Fully mission capable Better Human Alerting Mechanisms (Lessons Learned 91-92) HAIL-SS is is an an ONR-funded product maturation effort. 6/30/03-37 37
Summary Summary Operator Console Cognitive System Components Weapon System HAIL Engine 6/30/03-38 38
Summary HAIL is producing a platform-independent technology composed of two things: (1) The HAIL Engine: A cognitive system that delivers cognition-based resources for human alerting activities (2) The HAIL System: An open architecture decision- support system (DSS) component built around the HAIL Engine HAIL supports increased human capability: (1) Decouples the alerting rate from the human interruption rate (2) Increases user control for coordination of interruption events and recovering context after interruption 6/30/03-39 39