Beyond ergonomics, beyond integration, The world behind the display -Ecological Interface Design for the Flight Deck- Max Mulder, Control and Simulation Division 26-5-2011 Delft University of Technology Challenge the future
Outline Introduction to the HMS group Cognitive systems engineering & ecological interface design Example : airborne separation assurance 2
Introduction to the Human-Machine Systems group 3
Organisation Delft University of Technology Faculty of Aerospace Engineering Control and Simulation Guidance, Control, and Navigation Human-Machine Systems 4
Knowledge clusters GNC and HMS Human-Machine Systems modeling Guidance, Control and Navigation simulation integration (avionics) laboratory aircraft human-machine interaction laboratory facilities UAV helicopter SIMONA flight simulator ATM simulator 5
People in the HMS cluster prof. dr ir Max Mulder dr ir René van Paassen dr ir Clark Borst dr ir Mark Mulder ir Xander in t Veld ir Olaf Stroosma division head associate professor assistant professor (TBD) research fellow researcher & Cessna Citation test pilot PhD candidate, noise abatement procedures researcher @ SIMONA PhD candidate, simulator motion systems Currently 15 PhD students (7 at external partners) Approximately 20-25 MSc graduate students each year 6
People in the HMS cluster prof. dr ir Max Mulder dr ir René van Paassen dr ir Clark Borst dr ir Mark Mulder ir Xander in t Veld ir Olaf Stroosma division head associate professor assistant professor (TBD) research fellow researcher & Cessna Citation test pilot PhD candidate, noise abatement procedures researcher @ SIMONA PhD candidate, simulator motion systems Currently 15 PhD students (7 at external partners) Approximately 20-25 MSc graduate students each year 7
Know-how Create, integrate and validate knowledge from various domains 1. Engineering sciences Systems and control theory Artificial intelligence Real-time simulation 2. Cognitive sciences (Ecological) psychology Cognitive systems engineering 8
MSc PhD Pubs Funding 9
cognitive systems engineering & ecological interface design 10
Why do we need to study humans in the aerospace domain? 11
Response Options Fire the pilot Improve training Better maintenance, improve reliability Adapt procedures Add automation/warning systems (TCAS, EGPWS) Improve the interface 12
Levels in interface design illumination, readability, colors, symbols integrated displays, configural displays, emergent features, principle of moving part? 13
Flight Deck Evolution Classic flight deck (1960s) Modern glass cockpit Yes, all information is presented to the pilot. But, in doing so, all cognition needs to be done by the human. High workload, low performance Yes, most tasks are automated. But, in doing so, only a small part of the cognition is done by the human. Low workload, low SA 14
The challenge of automation 15
The flight-deck is an open system (Vicente) extensive and complex interaction with the environment the airborne office... a workplace for cognitive (team)work 16
Is there a display format that helps pilots with their cognitive tasks? 17
Skimming the surface German: Schnittstelle = place where you cut = interface Suggests that tasks are to be split between pilot and automation Suggests that interface problems show up as communication bottlenecks 18
Problems in process control Jens Rasmussen investigated accidents in nuclear plants. He found that the big accidents were not foreseen in the hazard analyses He was surprised by the difference between theory (procedures and logic), and practice He studied bicycle/television repair men and the maintenance operators at work at RISØ, and came to the Skill, Rules and Knowledge (SRK) taxonomy 19
What are the big challenges We commonly design displays for the operator s task (as implemented: participative design, or as designed: task analysis) Our normative task models are not implemented by the operator local adaptation The really big accidents are unforeseen, and therefore not in the models. Ergo: We need displays that support operators in unforeseen situations 20
Levels in interface design illumination, readability, colors, symbols integrated displays, configural displays, emergent features, principle of moving part? support for cognitive work : Cognitive Systems Engineering 21
Ecological Interface Design Vicente and Rasmussen (1990, 1992) Framework for creating human-machine interfaces Starts out with a work domain analysis (WDA) Named after Ecological Psychology (Gibson), exploiting the idea of direct perception 22
Ecological Psychology Equipped with the theory of depth perception, tests were developed with the objective to predict the success or failure of a student pilot in tasks as landing an aircraft None of these were successful, leading James J. Gibson to believe that the whole theory of depth perception is false 23
Ecological Psychology Indirect perception. The camera analogy. The threedimensional information about the environment is lost due to the projection on the two-dimensional plane (retina). The third dimension is reconstructed via depth cues and knowledge about the world. Direct perception. All information is in the visual array of light entering the eyes. The visual array is structured and all spatial information is available directly through patterns in the visual array. Gibson's ecological approach to visual perception 24
Human capabilities Gibson s ``Direct Perception affording specifying perception-action coupling 25
A small example of direct perception 26
Ecological interface design (Vicente & Rasmussen, 1992) Basic idea: make visible the invisible Use technology to create an interface that provides meaningful information and that allows humans to directly act on the information to achieve their goal Transfer a cognitive process into a perceptual process Work Domain Analysis + Control task analysis Strategies analysis Social organization and cooperation Worker competencies analysis Interface design 27
Ecological interface design Rationale We create complex systems in which the affordances are not clear Analyse what is possible with a system (WDA), figure out boundaries and constraints Create an interface that matches The system, by showing the constraints The human, by matching his/her capabilities (SRK) 28
Ecological interface design EID archetype: DURESS Double feedwater system Laboratory task 29
Work domain analysis Consider the path taken by an ant on the beach. It looks irregular: the ant must be a complex organism to remember & follow this path If one instead studies the beach, the path seems obvious. The ant creates that path as it feels, sees and smells the beach We did not evolve to feel, see and smell process plants and airplanes. We should analyse the beach and create an appropriate representation 30
Work domain analysis The abstraction hierarchy Functional Purpose: survival, comfort Abstract Function: isolation (thermal, physical) Generalised Function: shelter Physical Function: rock cave Physical Form: dimensions, type of stone, construction, colour, weight 31
Work domain analysis AH for DURESS functional purpose abstract function generalized function physical function physical form 32
33
heating system mass flow heater control energy flow water energy flow heater funnel: level trend storage ( reservoirs ) flow goal temperature goal outflow valve 34
Hutchins Cognition in the wild 35
Cognition and interfaces old view new view 36
Cross-over to transport domain Postdoc in Kassel, on EID for a cement mill work on Multilevel Flow Modelling, Danmark transport ; functionality to move mass or energy from one place to another Limits / constraints: e.g. maximum flow, gravity Also: Dinadis & Vicente, Application of EID to aviation. Applied to aircraft systems 37
Driving, flying, sailing vs. process plants Extension of natural ecological perception Transport is the issue Interaction complex environment Control Build new ecology Transported stuff is (nearly) anonymous Limited (known) number of variables Functionality creation and selection 38
Can we get classical EID in an ( ship aircraft (car, Already have Ecological Perception Enhance, not substitute Time scale is different not always opportunity to visually explore an interface Controls are not co-located with the interface Interaction is already multi-modal process control EID could learn from us there We cannot measure everything in the outside world rely on humans to read signs etc. 39
Ecological Support Interface Design Analyze work domain (AH) Analyze control tasks (added cybernetics) Identify what affordances are not sufficiently specified Enhance 40
Example: Avoiding Aircraft 41
42
Meaningful physics: travel-space modeling It is a fact that we know the physics of an aircraft moving in space very well but what do these equations mean in a context like this? not much! 43
Airborne Separation Assurance System Calculation CPA: (CPA < look-ahead time) & (CPA dist < 5 NM) = conflict 44
Airborne Separation Assurance System Known problems Conflict location moves when maneuvering Affordance hit is clear, affordance avoidance is not Only heading, no speed New conflicts triggered by maneuvers Engineer s answer: Predictive ASAS 45
Airborne Separation Assurance System The heading band safe heading unsafe heading 46
But, what is the best maneuver? let s review a conflict situation 47
ATP Two aircraft 48
ATP Add protected zone 49
ATP Create forbidden beam zone in relative space 50
ATP Calculate relative speed Here: safe 51
ATP Calculate relative speed Here: unsafe 52
ATP Move FBZ to own velocity space 53
ATP Construction of the State Vector Envelope 54
ATP Add maximum speed 55
ATP Add minimum speed 56
ATP Add maximum angles 57
ATP Recap 58
ATP Recap Forbidden Beam Zone 59
ATP Recap V min and V max Multiple heading bands, depending on speed changes Incorporating the allowable changes in speed and heading yields the state vector envelope 60
ATP Recap keep away from the red dot, this will yield a very inefficient resolution maneuver move the black dot out of the forbidden beam zone the SVE provides the meaningful information we were aiming for transforming making cognition visible into the perception invisible 61
ATP Demos (1) 62
ATP Demos (2) 63
ATP Multiple intruders 64
ATP Multiple intruders demo 65
Meaningful physics p q r x, x f( x, u, t) u v w x y z If you start of with engineering: model of aircraft kinematics Led to ASAS, pasas Computer prediction of aircraft position, computer solution of problem State Vector Envelope: 2 representations Absolute motion: for approaching destination Relative motion: for separation from other aircraft 66
The analysis links up to the interface functional purpose production economy safety abstract function absolute (loco-)motion relative motion generalized function path control flight 67
flight and control safety locomotion production relative motion efficiency aircraft limits 68
Our approach to interface design Usually starts out with a lot of engineering calculations, modelling and describing the system Picking the right representation (state variables) is crucial to the success of the interface design We go through lots of iterations 69
Closing statements The goal of EID is to transform a cognitive task into a perceptual task by providing meaningful information that humans can directly perceive and act on accordingly. make visible the invisible Distribute the cognition between humans and the automated systems through the interface. Strive for a joint cognitive system. 70
Is there a display format that helps pilots with their cognitive tasks? 71
Closing statements Don t just continue automating, create joint cognition and keep the pilot involved 72