Perceptual Overlays for Teaching Advanced Driving Skills Brent Gillespie Micah Steele ARC Conference May 24, 2000 5/21/00 1
Outline 1. Haptics in the Driver-Vehicle Interface 2. Perceptual Overlays for Shared Control 3. Preliminary Experiments 5/21/00 2
3 Context Driver-Vehicle Interface Product Design Ergonomics Human-Computer Interface Haptic Visual Audio Psychophysics Dynamic Systems and Controls
4 Haptics Haptos: Greek for the sense of touch Kinesthesia Sense of body motion Taction Sense of Skin contact Haptic Interface: Mechanical or contact-based interface
5 Computer Displays Immersion Corp. TouchSense Technology Visual SenseAble Corp. The PHANToM Audio Haptic Interfaces
6 Haptics: The Field Conferences Annual Manual 1970s-80s ASME IMECE Haptics Symposium 1994- SPIE Telemanipulator and Telepresence Tech. 1994- Phantom User s Group 1995- Haptic Human-Computer Interaction 2000 Journals Haptics-e Presence Commercial Ventures Immersion Corp. San Jose, CA BMW License SensAble Devices. Boston, MA Haptics Technology Inc Montréal, Québec Ford Reseh Laboratory Interactive Conceptual Aided Design
7 Haptics: some reseh topics Stability/Passivity of the human/device coupled system Design Guidelines for Haptic Devices Use for sensory substitution Real-time (1 khz) simulation of hybrid systems
8 Why Haptics for Vehicles? Visual and Audio modalities already overloaded Increased attention available if shared among modalities Possibly more intuitive (allowing faster response..?)
9 Haptics in Driving Buttons and Knobs: transmit state information
10 Haptics in Driving Steering Wheel: transmits tire aligning moment and feel for the road (i.e. curbs)
11 Haptics in Driving Power Steering: must still transmit the tire aligning moment to be effective driver interface
Haptics in Driving Brake Pedal: reflects force applied to brakes 5/21/00 12
Driving Simulator Visual Display Audio Display Motion Display Force-Reflecting Steering Wheel (Haptic Interface) 5/21/00 13
14 Steering Interface θtire τ tire θ steering τ tire τ steering
15 Steer by Wire θtire τ tire τ tire Motors and Encoders θ steering τ steering
16 Outline 1. Haptics in Driver-Vehicle Interface 2. Perceptual Overlays for Shared Control 3. Preliminary Experiments
17 The Need Neatly share vehicle control between a driver and an automatic controller. The Idea Automatic guidance system Active safety system Use haptic interface to create perceptual overlays on the physical environment: e.g. Embodiments of the control system, recognizable to the driver as familiar objects on the road.
18 A possible extension Use perceptual overlays to teach advanced driving skills
Case Study Boeing/Airbus - Contrasting Pilot/Autopilot interface philosophy Mh 4, 2000 5/21/00 19
20 http://www.boeing.com/commercial/aeromagazine/aero_08/human_textonly.html#flight Appropriate degree of automation. Boeing flight decks are designed to provide automation to assist, but not replace, the flight crew member responsible for safe operation of the airplane. Flight crew errors typically occur when the crew does not perceive a problem and fails to correct the error in time to prevent the situation from deteriorating. Consequently, Boeing flight decks incorporate intuitive, easy-to-use systems. These systems support instrument displays with visual and tactile motion cues to minimize potential confusion about what functions are automated. In the flyby-wire 777, visual and tactile motion cues are provided by backdriven controls. These controls reinforce situational awareness and help keep the flight crew fully aware of changes occurring to the airplane s status and flight path during all phases of automated and manual flight.
21 Sharing Control with a Human Human Controller Plant Human Haptic Interface Controller Plant
22 Shared Control: Haptic Interface Human Haptic Interface Controller Plant
23 Distal Attribution The innate human tendency to assume that sets (or streams) of proximal stimuli are attributable to external, distal objects.
24 Model Fidelity 1. Ability to predict measurements In the context of driver-in-the-loop simulation: 2. Ability to inspire perception
25 Force-reflection in Steer-by-wire F curb Physical Fixture Controller τ steering
26 Virtual Fixtures in Steer-by-wire F curb Image of Virtual Fixture Controller Virtual Fixture τ steering
27 Virtual Fixtures in Materials Handling
28 Virtual Curbs for Shared Control? Happy Motoring Freeway
29 Training Wheels: Do they teach?
30 Virtual Fixtures for teaching?
31
32 Path Tracking for Agricultural Vehicles
Combine 5/21/00 33
34 Outline 1. Haptics in Driver-Vehicle Interface 2. Perceptual Overlays for Driver Training 3. Preliminary Experiments
35 Modeling Tractor-Trailer(s) Kinematic Model
36 Tractor-Trailer Kinematic Model Commanded steering β Actual steering φ β φ θ 0 θ 1
37 Simple steering linkage model F
38 Tracking a reference point on the planned path Form the error vector R planned path S P R P T P T s p R R s S actual path
39 Feedback Linearization to design a controller that maintains an appropriate reference point on the planned path R''= ν linear control decoupling control plant ν = Kx v U = M 1 ( ν b) U R' ' = MU + b x
40 Maintaining a reference on the planned path Planned path 1.5 1 0.5 Vehicle path 0-0.5-1 -1.5-2 -1.5-1 -0.5 0 0.5 1 1.5 2
Tracking a circular path (no driver input) 8 7 6 5 4 3 2 1 0-1 -10-8 -6-4 -2 0 5/21/00 41
42 Path Planning for Nonholonomic Systems Automatic Parallel Parking
43 Automatically generated Steering Input
44 Simulator Screen shot
45 Conclusions Perceptual Overlays can be used to divide the labor between a human and automatic control system in a neat and intuitive fashion. Applications Automated Highway Active Safety Teaching driving skill Human-Robot Collaboration Computer or Robotic Assisted Surgery