Designing & Deploying Multimodal UIs in Autonomous Vehicles

Transcription

1 Designing & Deploying Multimodal UIs in Autonomous Vehicles Bruce N. Walker, Ph.D. Professor of Psychology and of Interactive Computing Georgia Institute of Technology

2 Transition to Automation Acceptance We must provide drivers with information to make automation more transparent Vehicle decision making Vehicle situation awareness Acceptance and trust is required for us to benefit from the improved safety and traffic management of automated vehicles.

3 Automated Driving Displays Focus thus far has been presenting system performance information Reliability at levels 2-3. At Level 5, system performance information is not as helpful. What information is important?

4 Multimodal User Interfaces: Think Different(ly) Visual Auditory Tactile/haptic etc.

5 Sidebar: Auditory Interface Elements Speech Non-speech: Music Spindex Spearcons Alerts & warnings

6 Taxonomy of Sounds Alarms, Alerts, & Warnings Object, Item, & Status Indicators Auditory Menus Data Representation & Exploration 3D Audio Displays & Spatial Representations Soundscapes & Ambient Audio Arts & Entertainment

7 Think Differently Deeply: Auditory Menus Computers, phones, etc., have a menu of functions or options Simple or complex 1-Dimensional or hierarchical

8 Visual Menus are Sophisticated Menu Title Menu Item (grayed out) Divider Menu Item Shortcut Icon Submenu Dialog Box

9 Typical Auditory Menus = TTS Text-to-Speech (TTS) Renders text contents only Missed opportunity for rendering other details Slow Low ratings Allegra Seidner Print Preview

10 TTS Enhancements to Consider Change TTS to render available vs. unavailable items Normal vs. quieter Male vs. female Normal vs. whisper Performance & preference Whisper, same voice Print Preview Print Preview

11 Spearcons Brief sounds used to represent the menu item, before the TTS Generated from TTS Performance & preference Faster, more accurate Easier to learn Preferred over earcons, auditory icons, TTS-only Allegra Seidner (TTS) Allegra Seidner Deborah Joseph

12 Auditory Scrollbars Represent: Size of menu Location within it Performance & preference Faster navigation Better awareness of menu size Better situational awareness

13 Spindex Speech-based index Letter sounds ( A, B, etc.) Provides location in alphabetical list Like tabs on a reference book Performance & preference Especially in long lists Basic Type Attenuated Type Decreased Type Minimal Type

14 Case Study 1: Multimodal Situation Awareness Vision is occupied by driving or by not driving Audio can provide SA in either case Combine music, audio alert, lighting, HUD cues Think outside the box!

15 Vigilance Display

16 Case Study 2: Automation Reliability Displays Automation monitors its own status Convey reliability to driver? Or required driver engagement? Visual display or audio or multimodal?

17 Reliability or RDE? Multimodal?

18 Case Study 3: Personalized Displays in AD Defensive Driving profile I want to arrive as safely as possible! Transit profile Get me there as efficiently as possible. ETA, Time to arrival, fuel economy, route, impact Thrill-Seeking profile Driving as a hobby, fun, not a chore

19 Defensive Driving Provides insight into: Near navigation Recognized vehicles Following distance Speed

20 Transit Provides insight into: ETA Time to arrival Speed Fuel economy Route

21 Thrill-Seeking Provides insight into: Speed RPM Immediate lane change maneuvers Tire pressure

22 Summary of Multimodal Displays in AD Automated driving presents new tasks, requires new displays and controls Multimodal UIs can help with Situation Awareness and Trust Creative solutions and careful evaluation are crucial

23 Thank You! Bruce N. Walker Sonification Lab Georgia Tech Atlanta, USA