D34.1 HMI Solution Design Restricted Copyright DESERVE. HMI Solution Design. D34.1 HMI Solution Design

Transcription

1 HMI Solution Design Deliverable n. D34.1 HMI Solution Design Sub Project SP3 Driver Behaviour / HMI Workpackage WP34 Innovative Integrated HMI Task n. T3.4.1 HMI requirement analysis and design of solution Authors Elena Bianco, Elena Balocco Pablo Lopez, Virginia Sixto Pereiro Nora Von Egloffstein Matti Kutila, Pyykönen Pasi Hans Deragården, Erik Nordin Andrea Saccagno, Daniel Bande CRF CTAG DAIMLER VTT VOLVO FICOSA File name Status Distribution DESERVE_D341_HMI_Solution_Design_v1.4.doc Final Restricted (RE) Issue date Creation date Project start and duration 1st of September, months [Type text]

2 REVISION AND HISTORY CHART VER. DATE AUTHOR REASON Elena Balocco (CRF) Document template Elena Balocco (CRF) Draft 1 reaching ICOOR comments Elena Balocco (CRF) Draft Elena Bianco (CRF) Review of some paragraphs Pablo Lopez (CTAG) Document review Nora Von Egloffstein (DAIMLER) Matti Kutila, Pasi Pyykönen (VTT) Hans Deragården, Erik Nordin (VOLVO) Integration of use case and architecture description of the DAIMLER demonstrator Integration of use case and architecture description of the motorcycle demonstrator Integration of use case and architecture description of the truck demonstrator Andrea Saccagno, Driver drowsiness description Daniel Bande (FICOSA) Matti Kutila (VTT) Integration of motorcycle HMI components Elena Bianco (CRF) Version to be sent to the peer review process to ARMINES E INTEMPORA Gwenaël Dunand Peer review INTEMPORA (Intempora) Bogdan Stanciulescu Peer review ARMINES (ARMINES) David Daurenjou Driver distraction description (CONTI) Elena Bianco (CRF) Final version after the peer review process Version 1.5, 29/08/2014 Page 2 of 44

3 TABLE OF CONTENTS REVISION AND HISTORY CHART... 2 LIST OF FIGURES... 4 LIST OF TABLES... 4 LIST OF ACRONYMS... 4 EXECUTIVE SUMMARY INTRODUCTION OBJECTIVES AND SCOPE OF THE DOCUMENT STRUCTURE OF THE DELIVERABLE INPUT FROM PREVIOUS WORK HMI OVERALL STRUCTURE ARCHITECTURE HMI INPUTS HMI OUTPUTS Visual output Acoustic output Haptic output Multimodality GENERAL HMI REQUIREMENTS HMI GENERAL SOLUTION DESIGN SYSTEM OPERATING STATUS WARNINGS STRATEGIES Warning prioritization HMI GRAPHICS HMI SOLUTION DESIGN FOR THE TARGET DEMONSTRATORS CRF DEMONSTRATOR VEHICLES CRF demonstrator vehicles - architecture HMI components HMI graphic DAIMLER DEMONSTRATOR VEHICLE Daimler demonstrator vehicles architecture HMI components HMI graphic VOLVO DEMONSTRATOR VEHICLE Volvo demonstrator vehicles - architecture HMI components HMI graphic VTT - RAMBOLL DEMONSTRATOR VEHICLE VTT demonstrator vehicles architecture CONCLUSIONS REFERENCES Version 1.5, 29/08/2014 Page 3 of 44

4 LIST OF FIGURES FIGURE 1 OVERVIEW OF THE DESERVE PLATFORM 7 FIGURE 2 DESERVE GENERAL ARCHITECTURE 8 FIGURE 3 IWI CONTROLLER: MAIN INPUTS/OUTPUTS 9 FIGURE 4 SEQUENCE OF INTERACTION FOR LONGITUDINAL CONTROL SCHEME TAKEN FROM INTERACTIVE PROJECT 20 FIGURE 5 AEB INTERURBAN PROPOSED HMI (CRF CAR) 31 FIGURE 6 AEB PEDESTRIAN PROPOSED HMI (CRF CAR) 32 FIGURE 7 DRIVER STATUS PROPOSED HMI (CRF CAR) 34 FIGURE 8 FUNCTIONAL BLOCK DIAGRAM FOR INTER URBAN ASSIST (DAIMLER DEMO VEHICLE) 36 FIGURE 9 POSSIBLE AUGMENTED NIGHT-VIEW PICTURE (DAIMLER CAR) 37 FIGURE 10 ACC WITH AUTO BRAKE (VOLVO DEMONSTRATION TRUCK) 38 FIGURE 11 ACC STEERING WHEEL BUTTONS (VOLVO TRUCK) 41 FIGURE 12 INSTRUMENT CLUSTER CENTRE DISPLAY (VOLVO TRUCK) 41 FIGURE 13 HEAD UP DISPLAY (VOLVO TRUCK) 41 FIGURE 14 FUNCTIONAL BLOCK DIAGRAM FOR VTT - RAMBOLL DEMONSTRATOR VEHICLE 42 LIST OF TABLES TABLE 1 INPUT DEVICE REQUIREMENTS TABLE 2 VISUAL FEEDBACK MAIN REQUIREMENTS TABLE 3 ACOUSTIC FEEDBACK MAIN REQUIREMENTS TABLE 4 HAPTIC FEEDBACK MAIN REQUIREMENTS TABLE 5 LIST OF HMI FEATURES OF ADAS GROUPS IDENTIFIED IN D TABLE 6 LIST OF ADAS FUNCTIONS DEVELOPED ON THE CRF DEMONSTRATOR VEHICLES LIST OF ACRONYMS ABBREVIATION ADAS ESoP HMI HUD HW IWI NHTSA PTW SW DESCRIPTION Advanced Driver Assists Systems European Statement of Principles on HMI Human Machine Interface Head up display Hardware Information Warning Intervention National Highway Traffic Safety Administration Powered Two wheelers Software Version 1.5, 29/08/2014 Page 4 of 44

5 EXECUTIVE SUMMARY The main objective of the WP3.4 is to exemplify the general solution defined in the WP3.3 in order to develop a HMI prototype solution. It aims to complete the analysis of the requirements to define a concrete HMI solution that selects a limited number of hardware and software components from the DESERVE platform. In particular, the deliverable D3.4.1, main output of the task T3.4.1, complete the analysis of the requirements and specification for the HMI manager involved in the arbitration of the information to be provided to the driver, the modalities of the representation (visual, audio, haptic, etc ) and, in case, the sharing of the resources (among several displays in the vehicle, use of the audio speakers for the radio, phone call and audio messages from the vehicle, etc ). After this general overview, a short description of each HMI prototype solution that will be implemented in the final demonstrator vehicles is provided. The deliverable presents the results of the activities performed in the task T3.4.1 and constitutes the basis of the activities of the task T This deliverable is strictly connected to the previous deliverables of the SP3 Driver behaviour HMI D3.3.1 HMI Needs Analysis and Specifications, describing the HMI user needs, and D3.3.2 Definition of a general integrated HMI solution, analysing the main HMI solutions available on the market for the ADAS and highlighting design guidelines for the final DESERVE HMI solutions. Moreover, the D3.4.1 is connected to the deliverables of the SP1 Requirement and Specification D1.2.1 Development Platform Requirements and D1.3.1 Development platform specification, describing respectively the requirements and the specifications of the overall project platform, based on common software architecture suitable for the development and simulation of several ADAS functions, as identified in deliverable D1.1.1 Application Database. 0. INTRODUCTION 0.1 Objectives and scope of the document The main objective of the WP3.4 is to exemplify the general solution defined in the WP3.3 in order to develop a HMI prototype solution. It aims to complete the analysis of the requirements to define a concrete HMI solution that selects a limited number of hardware and software components from the DESERVE platform. The document is structured as followed: - Chapter 1 - INPUT FROM PREViouS WORK - Chapter 2 - HMI OVERALL STRUCTURE - Chapter 3 - HMI general solution design - Chapter 4 - HMI solution design for the TARGET demonstrators In particular, in chapter 1 a short description of the previous work done in several R&D project on ADAS function and their HMI solution design is reported. Version 1.5, 29/08/2014 Page 5 of 44

6 Chapter 2 shows the DESERVE general architecture, focalised the attention on HMI input and output modalities. HMI general requirements are also described. Chapter 3 describes the general HMI solution suggested to be implemented in the DESERVE project and its strict link with the previous experience developed in the InteractIVe project. Going through the warning strategies and prioritisation logics, the HMI graphic of InteractIVe is proposed as a common basis for the specific ones implemented in each demonstrator vehicle. At the end, a short description of the HMI developed in each demonstrator vehicle is provided as available at this stage of the DESERVE project itself. 0.2 Structure of the deliverable The document is divided in two main parts, one referred to the DESERVE HMI solution specifications and the other referred to the description of the HMI solution implemented in each demonstrator vehicle. The document starts highlighting the link with the previous European projects in particular with the Interactive one, and on the other side, with the previous work done in the DESERVE project itself. Then an overview of the project platform is presented, focusing the attention on the HMI module inputs in order to understand which information need to be presented to the driver through the visual, acoustic or haptic channels. Chapter HMI solution design for the TARGET demonstrators is the core of D In this chapter a description of the HMI of each demonstrator will be proposed, if any, starting from the architecture, going through the HW components and arriving at the general description of the graphics draft foreseen sketches. Thus partners are not due to provide the final version of the HMI solution, but a general representative idea of it. 1. INPUT FROM PREVIOUS WORK In the previous years, several R&D projects focalized their attention on ADAS functions and their HMI solution design. In particular, valuable inputs are provided from the following projects: - AIDE- D3COS for HMI aspects and driver behaviour modelling - SAFESPOT CVIS for the cooperative system ADAS - INTERACTIVE PREVENT for the most innovative autonomous ADAS functions - CityMobile CyberCars for the Autonomous driving functions - Have-it for sharing control between driver and vehicle - Cesar for the Platform Functional Safety As described in details in the deliverable D3.3.1 HMI Needs Analysis and Specification, each project made a step towards the development of a new generation of ADAS applications, with a new concept of sensor fusion modules and HMI systems. To manage the level of complexity, different ADAS function must co-exist and cooperate. Version 1.5, 29/08/2014 Page 6 of 44

7 In particular, focalizing the attention on the Human Machine Interface, it should be unified in order to implement an integrated, safe and harmonized interaction with the driver. Actually the DESERVE project is planning to use and extend the functionalities of the interactive platform, as displayed in Figure 1. Figure 1 Overview of the DESERVE platform The DESERVE project will progress beyond the state-of-the-art in the ADAS domain providing a low cost, highly reliable, standardized platform, for compositional development, sharing sensors, actuators and HMI, enabling the low cost development of complex driver assistance systems and applications. 2. HMI OVERALL STRUCTURE 2.1 Architecture The baseline for DESERVE is represented by the results of past and on-going research projects, and in particular of interactive project, addressing the development of a common perception framework for multiple safety applications with unified output interface from the perception layer to the application layer. In Figure 1 the DESERVE platform framework is shown, highlighting by a dashed line the different focus of DESERVE and interactive. The challenge of DESERVE is to go beyond interactive, targeting the standardization of a wider software architecture including the Application and the Information Warning Intervention (IWI) controller, in addition to the Perception platform already developed within interactive. Version 1.5, 29/08/2014 Page 7 of 44

8 In particular, a general DESERVE architecture is shown in the figure below. Figure 2 DESERVE general architecture The DESERVE common development platform is based on three main layers: - Perception platform - Application platform - Information - Warning Intervention (IWI) controller platform In this architecture, the Perception Platform processes the data received from the sensors that are available on the ego vehicle and sends them to the Application Platform in order to develop control functions and to decide the actuation strategies. A special focus is required for the IWI manager sub-module of the Application Platform, which is the central intelligence managing the communication between the driver and the invehicle systems, able to avoid critical effects of interdependences. It ensures that information is given to the driver at the right time and in the right way and that only functions that are relevant in the present driving context are active. This includes the selection of modality for presentation, the message prioritisation and scheduling and the general adaptability of the driver-vehicle interface. This module is composed of 3 sub-modules: - the Information Manager dealing with the information to be provided to the driver; - the Warning Manager analysing the results of the Threat Assessment and the Driving Strategy modules and making the final decision about when to issue a warning and when to suppress warnings; - the Intervention Manager analysing the results of the Threat Assessment and the Driving strategy modules and making the final decision about when to issue an intervention and when to suppress interventions. The outputs of the IWI manager together with the Vehicle module ones are sent to the IWI Controller Platform, which main aim is to inform the driver in case of warning conditions and to activate the systems related to the longitudinal and/or lateral dynamics. Version 1.5, 29/08/2014 Page 8 of 44

9 The aim of D3.4.1 is to analyse in detail the inputs and outputs of the IWI Controller platform (see Figure 3) Figure 3 IWI controller: main inputs/outputs Based on some conditions, like traffic, driver intention, distraction or drowsiness, the IWI manager module generates appropriate input to the IWI controller, so that the driver could be informed through the visual, acoustic, or haptic channel. 2.2 HMI inputs HMI inputs to the HMI module are principally driver inputs ones, that can change depending on the car segment and brand. Independently on the DESERVE project implementation, a possible list on input devices is hereafter reported: - Steering wheel buttons - Switches around steering wheel - Buttons in the central panel used for active/deactivate specific functions - Touch screen - Voice recognition - Multifunctional haptic controllers - Active brake pedal - Active steering wheel - Active throttle pedal Version 1.5, 29/08/2014 Page 9 of 44

10 It is important to highlight that some of these devices can be used as both input and output devices, depending on the context of use. In fact for example: - Touch screen can be used as input in order to select something but also as output in order to visualize text, graphics or video; - Active brake pedal can be used as braking command but also for generating artificial forces and/or vibrations; - Active throttle pedal can be used as acceleration command but can be also be equipped with a mechanism for generating artificial forces and/or vibrations; - Active steering wheel can be used as steering input command but can also be modified in order to give an artificial torque related to lateral control and/or give a vibration feedback In the chapter dedicated to the HMI demonstrator solutions, the input devices that will be installed will be described. 2.3 HMI outputs In this paragraph are collected the information related to the output channels that can be used to give feedbacks to the driver. In order to cover all the human senses involved during the driving experience, visual, acoustic as well as the haptic channels will be used to transmit the information and the warnings in the most appropriate way. In general, visual channel has to be considered as the priority feedback channel, followed by the acoustic and haptic one Visual output The principal visual output are: - instrument panel cluster display - central panel display - head-up display (HUD) Independently on the position of the display, it is important that it achieves a good reading visibility, contrast, resolution and field of view. In this way, the information provided to the driver is clear and legible. More in particular, for visual information, important properties would be: - brightness/intensity - visual acuity/spatial contrast sensitivity - colour specification - perceptual organisation: o figure-ground organisation, o grouping principles (proximity, similarity, continuity, closure, common elements with common motion tend to be grouped) etc. For safety related functions it would be preferred to be displayed in a frontal position, such as in the instrument panel cluster display or in the HUD, since they should minimize distraction from the road ahead. In general, using the visual output, the ADAS functions provide information and warnings to the driver using sketch, text as well as soft and hard telltales. Version 1.5, 29/08/2014 Page 10 of 44

11 2.3.2 Acoustic output The main advantage of using auditory feedback is to provide information and warnings without distracting the driver visually. Simple reaction time for auditory stimuli is shorter than for visual stimuli (Salvendy, 1997) and could have an effect on the overall response time. Moreover, taking advantage of the sound direction, acoustic feedback can attract driver s attention to the location of hazard even if the hazard is out of the driver s field of view. Like the visual output, also the acoustic output have to respect some general requirements in terms of sound pressure level, frequency range, and spatial resolution. First of all, it is important that the sounds used are well distinguishable from the other vehicle sounds (both informing sounds, like arrow sounds, and alerting sounds, like empty fuel level), secondly the sounds loudness level need to be calibrated in order to transmit to the driver the proper level of urgency without creating annoyance. Contrarily to the visual output, it is better that the acoustic feedback is given associated to other output, if it is not sure that the feedback itself is associated univocally to an event. Therefore it is common that the acoustic feedbacks are associated to a visual as well as to an haptic feedback. Tonal sounds (earcons) can typically be used for prompting or reinforcing other modalities while auditory icons can be used when a rapid response is required, provided that there is a suitable match between the auditory icon and the event/hazard to warn for. It should be noted that auditory icons may require a wider frequency range of the audio reproduction system than tonal sounds. Some example of signals that can be used are: - sweeping sounds for immediate acoustic feedback - patterns of segments with constant pitch for short-term acoustic feedback - two-times chimes, high-low non recurrent for long-term acoustic feedback From the drivers point of view, sometimes acoustic signals are more annoying than haptic feedback and have to be used properly when it is strictly needed to recall the driver attention. So a recommendation could be to provide auditory alerts only for items needing immediate attention Haptic output Haptic signals as output signals are mainly used for alerting the driver (e.g. vibration on the steering wheel in case of lane departure) or for displaying a specific range of operation (e.g. force on accelerator pedal indicating exceedance of the current speed limit). In some cases, haptic signals are also used as a display for the driver to monitor the actions of a Driver Assistance System (e.g. steering wheel is turning when lane keeping system is active). Haptic signals can appear in very different forms: tactile like vibrations, torque or forces, and kinaesthetic like accelerations, braking pulse, turning rates. In addition, haptic signals can be applied on different devices such as the steering wheel, brakes and accelerator pedals as well as on the seat or on the seatbelt. Haptic signals are most often displayed on the device where an action of the driver is required. Additionally, acceleration or brake pulses that the driver perceives as body motion (kinaesthetic perception) can also be classified as haptic output signals. Version 1.5, 29/08/2014 Page 11 of 44

12 In the next chapter General HMI requirements, haptic parameters that were identified in a literature search are presented. However, as the effects of haptic signals strongly depend on the specific hardware characteristics general requirements could not always be derived from literature. Therefore, the parameterization for the application needs to be specifically adapted for each hardware device Multimodality Visual channel has to be considered as the priority feedback channel, followed by the acoustic and haptic one. In some cases the three channels could work separately (or only one output channel is available), but it is better to think about them working together in order to give a more appropriate feedback to the driver: this case is called multimodality. In multimodality mode, based on the different level of information/warning to be provided to the driver, a sort of prioritization of the output channels have to be planned. In particular: - only one channel can be used to present cautionary crash avoidance warning and it is preferred to use the visual output; - two channels are at least required for imminent crash avoidance warnings, so visual channel can be associated to acoustic or haptic channel. Since imminent crash-avoidance warnings are of highest priority, redundancy is crucial. In this case, visual output shall only be supportive and not the primary source. It has been shown that when auditory information can be replaced with haptic information in certain use cases this may result in a higher level of acceptance (Brockman, et al., 2012). In case of multimodality, it is recommended to arbitrate with the output channels that should be used for performing the wanted action by the driver or the system. So a prioritization strategy has to be implemented. 2.4 General HMI requirements The following general HMI requirements have to be considered during the design of the DESERVE HMI. Requirements about the installation including layout, visibility and reach: - The system shall comply with relevant regulation and standards. - The system shall comply with automotive environmental requirement e.g. temperature, vibration, etc. - The system shall have standard vehicle interface or defined common DESERVE interface. - The system shall be able to communicate with DESERVE system. - The system shall support addition and removal of applications. - The system does not obstruct: - Forward view - View and reach of primary controls and displays - The button shall have right order, sensitivity and size. Version 1.5, 29/08/2014 Page 12 of 44

13 Interaction - The system shall to complaint with automotive environmental requirement. - The system shall support the driver and not increase driver distraction from driving task. - The system shall not require uninterruptible sequences of interaction. - The driver shall be able to continue the interrupted sequence of interaction with the system at the point of interruption or at another logical point. - The system response (e.g. feedback, confirmation) after driver s input shall be timely and clearly perceptible. - The system shall have a default predetermined setting. Certain system setting shall be adjustable or personalized. Important requirements to be also considered but not directly related to DESERVE work: - System check shall be automated. - The system must be failure tolerant. - The system must be user error tolerant. - The device shall have automatic calibration. If it is not possible, the procedure for calibration shall be simple and straightforward. In the following tables, general requirements for input / output devices are reported. IWI input requirement type General Mechanical Auditory Requirements short description Operation shall be simple and intuitive (ESoP Expansion) Shall be located where the driver can easily access and control (easy to each distance) (UMTRI 3.2; 3.3) The function of the buttons must be clear. (UMTRI 3.9) Manual control shall consider colour codes. (UMTRI 3.9) Manual control shall consider shape, position, texture and dimension codes. (UMTRI 3.10) Manual control shall consider movement compatibility and conventions. (Battelle Cap. 4) Shall have the right operating force, feedback and etc. for the operation. (UMTRI 3.8) Most of them shall be supported by backlighting. (ISO 15008) Speech based communication shall be hands free speaking and operation. (ESoP Expansion - ISO 15006) Shall have noise cancellation. (ESoP) Shall understand the speech input by driver in local region. (ESoP - ISO 15006) Table 1 Input device requirements IWI visual output Displays Requirements short description The visual indication should be located as close to the forward road scene as possible (SAE J2400 Display should be located within a 10-degree cone of the driver s line of sight). The visual warning should capture attention, and quickly communicate to the driver the situation context. Version 1.5, 29/08/2014 Page 13 of 44

14 IWI visual output Requirements short description Visual indications will be dependent on display capabilities, but must be designed such that drivers quickly understand if an action is required. ISO standards design requirements (brightness/intensity, visual acuity, spatial contrast sensitivity, colour specification, resolution and field of view) Compliance with ISO 2575 Road vehicle Symbols for controls, indicators and telltales. Visual elements should be organised according to a recognizable hierarchy (i.e more relevant and critical information must be more salient).figure-ground organisation, grouping principles (proximity, similarity, continuity, closure, ) Any piece of information presented through the visual channel should minimize distraction from the road scene ahead. ISO 2575, General guidelines with regards to symbols for controls, indicators and tell-tales: Colour indication Red: Danger Yellow/Amber: Caution Green: Safe, normal operating condition (ISO 16352, 2005) A given symbol may be shown in more than one colour in order to convey a change in the operating condition. Flashing indicators should not be used to indicate the operational status of a warning device. Because of their superior conspicuity, flashing visual displays should be reserved strictly for imminent crash-avoidance displays. (NHTSA) Table 2 Visual feedback main requirements IWI acoustic output Acoustic Requirement short description The sound pressure level has to be within acceptable range for human ears. The frequency range has to be within acceptable range for human ears. The audible source indication should be at least 75 db at the driver s ear. When designing a warning, map acoustic sources to location of the risk in space. Auditory indications should be distinct from all other audible indications present in the vehicle. Other audible sources, e.g. radio, navigation, phone, should be attenuated to at least 16 db below the ADAS warning. The loudness level of each sound needs to be calibrated so that a proper level of urgency is achieved without creating annoyance. Version 1.5, 29/08/2014 Page 14 of 44

15 IWI acoustic output Requirement short description The audible indication should capture attention, and communicate the urgency of the situation. Auditory feedback could be chime, speech or earcon, but more research is needed to provide a final recommendation Allows cross-modal matching with other feedback types such as visual and haptic output. Table 3 Acoustic feedback main requirements IWI haptic feeback Description Requirement short description Haptic feedback Haptic feedback has been shown to elicit faster responses relative to other modalities (DFSS P365, CRF Interactive). Haptic feedback may include pedal movement, seatbelt tug/tightening, brake jerk, seat vibration/movement, steering wheel feedback, etc. Haptic feedback through steering wheel Haptic feedback through throttle Haptic feedback through safety belt Provides feedback to the driver in form of steering torque or steering vibration Provides haptic feedback to the driver by the accelerator pedal. Provides feedback to the driver by tensioning the safety belt or generating a vibration through a vibrating pad in case of User acceptance: the signal should be clearly perceivable without being intrusive or and well accepted by the user. Redundancy on other channels (i.e telltales) is recommended in order to avoid ambiguity. The signal should combine an oscillation/vibration with directional information or torque lead Haptic accelerator pedal signals are most often applied for longitudinal support systems. Semantics can be associated to different physical patterns (i.e. vibration, knocking or pressure). The parameters for vibrations and torques have to be individually adapted to pedal type and position in combination with the seating position of the driver in the vehicle. The parameters for vibrations have to be adapted to the seating position of the driver in the vehicle. Version 1.5, 29/08/2014 Page 15 of 44

16 IWI haptic feeback Description Requirement short description warning condition, gives a feedback to the driver through a vibration or a tightening on the seat belt. This kind of feedback is used for frontal dangerous conditions. Table 4 Haptic feedback main requirements 3. HMI GENERAL SOLUTION DESIGN The general HMI solution suggested to be implemented in the DESERVE project is based on the previous experience developed in the InteractIVe project, considered as the last main project dealing with ADAS interface. The steps forward suggested in DESERVE project are the following: - Integration of HMI solution for driver distraction and drowsiness - Design of a specific HMI solution for motorcycles based on the proposed approach for passenger cars and commercial vehicles - Modular HMI approach able to cover the different modular ADAS functional SW, as foreseen to be validated by end users during the virtual testing phase In fact, the DESERVE project will propose a modular HMI able to cover several HMI requirements of different ADAS. The aim of this document is to describe an HMI solution that can be used in different platforms, even if only some demonstrator vehicles will implement this HMI in the final stage of the project (see chapter 4 - HMI solution design for the TARGET demonstrators). During the development of a human- machine- interface, an important aspect to be considered is the coexistence of different ADAS functions in the same demonstrator vehicle, which have different use cases but the same states of operation. The purpose of the functions proposed in DESERVE project is the integration of these systems, giving the driver the possibility to have a full driving assistance experience with a whole system. The driver does not need to perceive all the technology units that are behind the HMI, but he/she needs to have all the information that allows him/her to interact without risk of mistakes. In this view, it is foreseen to have a general sketch representing the ADAS functions status corresponding to their operating range. Version 1.5, 29/08/2014 Page 16 of 44

17 3.1 System operating status As described in the InteractIVe project, in general the system status can be divided in five operating status: - System On/Off Status - Available (Standby) Status - Available (Active) Status - Unavailable Status - Failure Status System On/Off status The system could be switched on or switched off using a specific control that could be a dedicated control on the steering wheel, on dashboard or an option on settings menu. The On/Off state has just to inform clearly the driver that the system is operating (or not), allowing the driver to understand if expect a support or not. For systems that are NOT default ON, the on-state needs to be indicated to the driver. On the contrary, for systems that are default ON, the off-state needs to be indicated to the driver. In general the visual output is the principal channel to communicate this status, through telltales or with led in the control button. However, for systems that are generally default ON it is also possible to use the acoustic channel to transmit to the driver the OFF status in order to highlight this status. Some functions, like the Driver drowsiness detection, have an additional status called calibrating between the ON status and the available (standby) status. This stage can be used by the system in order to monitor/learn the driver behavior or driving style. This stage can be provided using both visual and textual feedback, in order to inform the driver about the learning phase progress. Available (Standby) status The system is ON and ready to answer to the proper target, but there are no danger reasons to activate it. The Available (standby) state has to describe the environment and context of operation, as well as the level of operation and availability. It has to communicate how, when and where the system can support and how, when and where the driver has to take the control. This state can be provided generally using only a visual feedback. Available (Active) Status The system detects danger risky situations. The Available (Active) state has to describe the warning escalation sequence and provides information about the type and level of danger. This information are recommended to be displayed in a driver related display in order to enable the driver, on one side, to understand quickly the potential critical situation, and, on the other side, to became aware of the system limits and to avoid unrealistic or wrong expectations about the system detection capabilities. This state can be provided generally using multi modal feedback (haptic and acoustic too). The provided feedbacks have to alert the driver about a possible danger and, if it is necessary, stimulate a quick and correct action to avoid it. Version 1.5, 29/08/2014 Page 17 of 44

18 Therefore a multimodality feedback is preferable. The logic behind this state is described in details in the Warnings strategie chapter. Unavailable status The system is ON but the operational criteria are not satisfied. The Unavailable state has to alert when the system (or sub systems) is unable to detect a danger because the operational criteria are violated. When a system implies a high availability confusion risk for the driver the unavailable state should be clearly indicated. Acoustic feedback can be used to quickly focalize the driver attention to the unavailable status. Failure Status The system does not temporary or permanently work. Dedicated feedback could be provided to the driver explaining this failure situation. A multimodal feedback is preferable, due to the riskiness condition: so for example visual channel, using icons and telltales as well as clear text messages describing why the system is not working, can be supported by acoustic sounds able to attract the driver attention. 3.2 Warnings strategies When the system is in the available (active) status, if a risky situation occurs it has to be appropriate communicate to the driver. In general there is a sequence of events leading up to a crash: - normal driving - emerging potential risky situation - critical situation - crash unavoidable A classification of several warning/intervention degrees should be based on the criticality of the situation and consequently on the evaluation of urgency for each message. Therefore, a four-point scale is provided: - information only, no direct reaction required by the driver - response preparation needed (more than 10 seconds) - response required within 3-10 seconds - immediate response required (0-3 seconds) and as a consequence, time requirement for HMI interaction is also identified: - short interaction times (< 1 sec) - medium interaction times - long interaction times (> 2 sec) In literature, the time-scale is sometimes not so rigid for the response required phase and immediate response required phase, so sometimes it is possible to find 2 seconds as lower level for the first and higher for the second one. In general, following the InteractIVe approach, the Active status could be divided in 5 different sequences: Version 1.5, 29/08/2014 Page 18 of 44

19 - safe area, if there is no danger risk around the vehicle and only generic information about the system status are provided to the driver - sequence 0, also called pre-warning phase. The system pre-alerts the driver of a potential danger helping to forecast the situations and the risks. However the driver does not have to act but only to be ready. - sequence 1, also called warning phase. The driver enters an area of imminent danger. The warning provided has to persuade the driver to act quickly. - sequence 2, actuation/intervention phase. This phase occurs only for system with intervention. A feedback is provided to inform the driver of the initiation and type of the automatic manoeuvre. - sequence 3, post intervention phase. This phase occurs only for system with intervention. A visual feedback is expected which informs that a system intervention occurred and the kind of intervention. More in details, during the so-called normal driving phase, the driver is a safe area, with no danger detected by the ADAS functions. The information provided to the driver are the generic information about the system status (available (standby), available (active), unavailable, failure) see sub-section before. This information persists also if a danger situation occurs. If a potential danger occurs, a cautionary pre-warning will be given to the driver. It could happen, for example, that in the surrounding there are some elements that could lead the driver in a dangerous situation, but the driver with an appropriate maneuver can avoid it. The warning provided is called pre-warning, able to pre-alert the driver of the potential danger if he/she decides to perform specific actions, for example accelerate, and which would help the driver forecast the situations and the risks. In order to reach an appropriate response from the driver, the warning signal needs to attract the driver s attention and inform him/her of the situation (detection). After having understood the situation (identification), the driver can decide in which way react (decision and response phase). The warning itself could be a visual warning in the cluster display or head-up display using yellow colour display indication. For system with intervention, this phase could be considered as not mandatory. If the driver underestimates the situation, or the traffic conditions change suddenly, so that the potential danger became a critical risk, an imminent crash avoidance warning will be provided. The warning provided has to persuade the driver to act quickly. The information provided has to enhance the driver attention and clearly explain the risky situation. In this sequence for all system (with or without intervention) it is required to have a warning with dual modality, the first one must be visual (red colour is recommended) while the second one should be haptic, for example pedal vibration or brake pulse, or acoustic, for example onomatopoeic sounds. In systems without intervention, warning ends when the vehicle is outside the danger area. If the driver is not able to avoid the crash, an intervention is required. This situation occurs if the behavior of the driver is very critical l (the driver is distracted, is drowsy, or the driver is performing inappropriate maneuver) so that he/she or the system itself has to react immediately to avoid or to reduce the impact effects. During this sequence, the systems, which have implemented it, perform the avoidance and/or mitigation action via automatic intervention, and only after that the driver is Version 1.5, 29/08/2014 Page 19 of 44

20 informed about the occurred intervention. This information has to be displayed for a few seconds after the end of the intervention. For systems without intervention, the feedback to the driver can be increased with respect to the warning stage whether increasing the intensity of a warning or switching to a different modality to communicate an increase in criticality. In the following figure, an example of sequence of interaction for a longitudinal ADAS function is provided. Figure 4 Sequence of interaction for longitudinal control Scheme taken from Interactive project Warning prioritization If more than one warning occurs, a prioritisation warning strategies has to be considered. For simplicity, the space around the car can be divided in three main areas: - a frontal area - a lateral area - a rear area therefore we can speak about prioritization among the areas, or within the same area. Nevertheless, since the three areas are well defined and unambiguous among them, if a frontal warning occurs at the same time with a lateral dangerous situation, the driver can be informed about the two risks with appropriate feedback. On the contrary, if two or more risky situations occur in the same area, a prioritization strategy is needed. Version 1.5, 29/08/2014 Page 20 of 44

21 Based on the classification explained in the previous paragraph, sequence 2 has the priority on sequence 1, which in its turn, has the priority on sequence 0. In other words, an intervention feedback has the priority on a warning feedback that has the priority on the pre-warning one. However, among the pre-warnings and the warnings feedback themselves, it is also needed to define a hierarchy. For example, we can consider the following scenario where at the same time a dangerous curve, a dangerous frontal vehicle and a pedestrian are detected in front of the vehicle. In this case the nearest obstacle became the most dangerous one, since no more than one information can be given in a suitable way to the driver if referred to the same area. 3.3 HMI graphics A dedicated paragraph is required for the visual feedback regarding the ADAS function system status. In the DESERVE project it is proposed to have both telltales briefly informing the driver about the system status as well as having sketch drawing with few lines the surrounding environment (suggested as a unique solution in the InteractIVe project). However, depending on the demonstrator vehicles, it could happen that only one of the two before mentioned HMI graphics can be implemented, depending on the dimensions of the installed display. In fact telltales can replace the sketch in a small display, while in a large display, they can be not present since the sketch describes better the road situation. If implemented, a specific telltale has to be draw for each ADAS implemented in order to univocally identify the function. Graphics have not to be too much detailed or complex, on one hand due to the restricted dimensions and on the other hand due to the fact that the driver has to understand the system status in a quick look. In some case for standardized functions, ISO icons are still available. Moreover, icons/telltales can be used in case of ADAS functions that not required to transmit the idea of a safety shield around the vehicle, like the driver drowsiness and distraction. Using different colours code it is possible to transmit different status using the same graphics. Based on the previous experience, the colours suggested are: - White for active (ready) status - Green for active (available) status - Grey for unavailable status or calibrating undergoing A slightly different icon is instead suggested for transmitting failure conditions, for example with a bigger dimension or with a blinking effect. Since Fault state has to alert the driver that something does not properly work, permanently or temporarily, it is thought to use an icon together with a short text message. In some case also acoustic feedback can be present. For these conditions the colors suggested are yellow for temporary failure and red for permanent failure. Depending on the demonstrator vehicle HMI implemented, these telltales could also become pop-up that appear to the driver when the situation occurs. A more complex and detailed description of the system status is given by using sketches. Version 1.5, 29/08/2014 Page 21 of 44

22 Using this HMI graphics modality, the driver become aware of the overall situation around his/her vehicle. Take care that also in this case the graphics have not to be too much detailed or complex (only elements that give relevant information have to be provided; other elements like shadows should be avoided). As suggested in the interactive project, a safety shield can be designed around the egovehicle in order to transmit to the driver the idea of ADAS helping against accidents. The shield will be present in front of the ego-vehicle, in lateral side, or rear side, depending on the ADAS installed detection area (frontal, lateral or rear). The colours code suggested for the safety shields is: - None (shields not present) for the system status OFF - Grey for the system status available (standby) - Yellow for the system status available (active) sequence 0 - Red for the system status available (active) sequence 1 Moreover the safety shields can be solid for available systems and fade or missing for unavailable systems. 4. HMI SOLUTION DESIGN FOR THE TARGET DEMONSTRATORS In the subproject WP11 - Application needs, an application database was created, that identified 10 groups of ADAS with 33 applications that are currently available or will be soon introduced in the automotive market. This elaborated ADAS database will serve as a basis for the ADAS applications addressed, investigated and finally selected for further work within the DESERVE development framework. It should be noted that not all of the ADAS applications could be dealt with in the same manner and working depth throughout the project and a selection to a few demo cases, that will be examined and developed in more detail in WP4, is therefore needed. The database content is divided into 10 main ADAS groups: - Lane change assistance system - Pedestrian safety systems - Forward/Rearward looking system - Adaptive light control - Park assistant - Night vision system - Cruise Control System - Traffic sign and traffic light recognition - Map supported systems (Note: only ADAS scope, no driver information) - Vehicle interior observation The following table presents an initial list of HMI features (i.e. type of message for users) of the before mentioned ADAS groups, and a short description of the DESERVE proposed HMI. Version 1.5, 29/08/2014 Page 22 of 44

23 HMI interfaces HMI warnings Driver Assistance Application Groups Acoustic Visual Haptic On/Off commands DESERVE HMI proposed Lane change assistance system x x x safety shield with special lateral focus AND /OR telltales about the system status Pedestrian safety systems x x x safety shield with special frontal focus AND /OR telltales about the system status Forward/Rearward looking system x x x safety shield with special frontal/rear focus AND /OR telltales about the system status Adaptive light control x x telltales about the system status Park assistant x x safety shield with special rear focus AND /OR telltales about the system status Night vision system x safety shield with special frontal focus AND /OR telltales about the system status Cruise control system x x x safety shield with Version 1.5, 29/08/2014 Page 23 of 44

24 HMI interfaces HMI warnings Driver Assistance Application Groups Acoustic Visual Haptic On/Off commands DESERVE HMI proposed special frontal focus AND /OR telltales about the system status Traffic sign and traffic light recognition x x telltales about the system status Map supported systems (Note: only ADAS scope, no driver information) x no HMI foreseen Vehicle interior observation x x x telltales about the system status Table 5 List of HMI features of ADAS groups identified in D1.1.1 It is important to highlight the modularity and flexibility of the proposed HMI able to cover several HMI requirements of different ADAS both for passenger cars and commercial vehicles. Due to the specific requirements of the motorcycles applications, it is proposed to rescale the HMI graphics solution before mentioned to the only telltales informing the rider to the ADAS function status. However, as described in the following, not all the DESERVE demonstrator vehicles will foresee to implement this HMI, since the aim of the project is to develop a general modular HMI solution and not to implement it in a vehicle demonstrator due to possible existing constraints of the current available vehicles demonstrators (e.g. architectural limits, sensors configuration, ). Version 1.5, 29/08/2014 Page 24 of 44

25 4.1 CRF demonstrator vehicles Among the list of the ADAS functions, CRF develops the so called Driver impairment warning system for cars. This function is developed in two different ways, to cover two different scenarios of use (use cases), the urban scenario and the extra-urban one. Two different demo cars will be used in order to highlight better the various functionalities (see Table 6). Demo car ADAS function Passenger car Autonomous Emergency Braking (AEB) pedestrian Driver distraction Driver intention Sport utility vehicle (SUV) Autonomous Emergency Braking (AEB) interurban Driver distraction Driver drowsiness Driver intention Table 6 List of ADAS functions developed on the CRF demonstrator vehicles HMI feedbacks are foreseen for the following ADAS functions: - AEB interurban - AEB pedestrian - Driver drowsiness while Driver intention and Driver distraction will be considered as inputs to the IWI manager module, able to reinforce and adjust the warnings dedicated to a critical driving situation (pedestrian detected, driving fault, ) or driver situation, adding for example an acoustic and/or haptic feedback to focalize the driver attention to the primary driving task. For example, if the ADAS system detects a pedestrian in front of the vehicle, the warning level could be adjust using the driver distraction state (driver attentive: beep, driver inattentive: beep + haptic) CRF demonstrator vehicles - architecture In the following figures are reported the draft architectures (under development therefore some changes are possible), available in the stage of the project, of the two different demonstrators vehicle developed by CRF. Version 1.5, 29/08/2014 Page 25 of 44

26 Figure 3 AEB pedestrian, Driver Distraction and Driver Intention (CRF passenger demo car) Figure 4 AEB interurban, Driver Distraction / Drowsiness and Driver Intention (CRF SUV demo car) Version 1.5, 29/08/2014 Page 26 of 44

27 The following table highlights the HMI needs for each function, described according to a common template reported in the deliverable D3.3.1 Needs Analysis and Specifications. The template foresees the following information for each function: - type of function (information - warning intervention) - the HMI needs related to the type of message (acoustic, visual, haptic) - the presence of On/Off commands - the timing between different functions (e.g. contemporary or sequential), - the possibility that the driver overrides the warning, information, intervention function, - case dependencies (e.g. function dependent on day/night, road type ), - the possibility that a message / function is repeated (e.g. in case a warning message is ignored) - the possibility that different degrees of risks exist in relation to each function - need of an acknowledge from the driver This information had been requested to project partners at the time of finalizing the deliverable D3.3.1, but at that stage not all the information were available. Therefore, since this report is strictly connected and goes through the output of that deliverable, it was decided to sum up hereafter the results. Version 1.5, 29/08/2014 Page 27 of 44

28 Function AEB pedestrian AEB interurban Driver drowsiness Type (Information - warning - intervention) warningintervention informationwarning - intervention informationwarning - intervention (y/n) Y Acoustic Visual Haptic On/Off commands Needs description Acoustic channel has to be considered as a secondary channel. (y/n) Needs description (y/n) Needs description (y/n) Needs description Y Visual output could be considered as a primary channel. N An haptic feedback could be foreseen since it provides a faster responses with respect to the other modalities. However, no haptic feedback is foreseen in the CRF demonstrator vehicles. Y Y N Y/N Y Acoustic channel has to be considered as a primary channel. Y Visual output could be considered as a secondary channel when it is in green and yellow alarm, but primary when the system detects that the driver is in a drowsy state N An haptic feedback could be foreseen since it provides a faster responses with respect to the other modalities. However, no haptic feedback is foreseen in the CRF demonstrator vehicles. Y/N N Y = a general button for the ON/OFF of the DESERVE function is foreseen. N = a dedicated button for the switch on/off is not foreseen Y = a general button for the ON/OFF of the DESERVE function is foreseen. N = a dedicated button for the switch on/off is not foreseen Timing (e.g. contemporary w/other functions, before/after) Not applicable since it is the only ADAS function with an HMI output to the driver Not applicable since it is the only ADAS function with an HMI output to the driver Version 1.5, 29/08/2014 Page 28 of 44

29 Function AEB pedestrian AEB interurban Driver drowsiness Override permitted? Describe case dependencies (e.g. day/night, road type, interaction w/other ADAS) (y/n) remarks (y/n) Repetition of message / function? Description (same, different degrees, etc.) N all conditions Y SAME Y N all conditions Y SAME Y Only day conditions and highway scenarios (>75 km/h) for full configuration performance (PERCLOS, Driving behaviour analysis and driver biological data analysis ) Y SAME Y Are there different degrees of risk? Need of an acknowledge from driver? (y/n) Remarks (y/n) Remarks Depending on the level of risk (warning or intervention) different outputs will be foreseen. Depending on the level of risk (information, warning or intervention) different outputs will be foreseen. Depending on the level of risk (information, warning or intervention) different outputs will be foreseen. N N N Version 1.5, 29/08/2014 Page 29 of 44

30 4.1.2 HMI components Demonstrator vehicle CRF passenger car CRF SUV Functions AEB pedestrian AEB interurban Driver drowsiness Possible input devices Button to switch on the DESERVE system Button to switch on the DESERVE system No input foreseen. It is an automatic feature Possible visual output devices Dedicated display TFT possibly positioned in the instrument cluster Dedicated display TFT possibly positioned in the instrument cluster Possible audio output devices Buzzer Buzzer HMI graphic In this paragraph, is reported a short description of the HMI graphics solution that at this stage of the project is thought to be implemented in the two CRF demonstrator vehicles. AEB interurban HMI It can occur that a driver on a motorway or a dual carriageway might be distracted and fail to recognize that the traffic in front of him/her is coming to a stop. By the time he/she notices the danger, it is too late to apply the brakes and avoid the impact or he may misjudge the braking of the car in front and fail to apply sufficient braking force. The AEB system operates as follows: a warning signal is given to the driver to try to alert him/her to the danger. If the driver does not respond, a second warning may be given (for example a brake jerk or seatbelt tug) and the brakes will be pre-armed for maximum braking. Again, if there is no reaction from the driver, the system will itself apply heavy braking. The driver needs therefore to be informed about the overall sequence. In particular, considering the HMI suggestions described in chapter 3, the following visual output are foreseen: - Sketch representing the ego-vehicle, the frontal safety shield, the detected vehicle in front for the sequence 0 (pre-warning phase) - Sketch representing the ego-vehicle, the frontal safety shield, the detected vehicle in front (with a short distance with respect to the sequence 0) for the sequence 1 (warning phase) - Sketch representing the ego-vehicle, the frontal safety shield, the detected vehicle in front (with a short distance with respect to the sequence 1) and a text message Brake! for the sequence 2 (warning phase) - Text message AEB interurban intervention for sequence 3 (post intervention phase) Version 1.5, 29/08/2014 Page 30 of 44

31 The colour code is the following: - Yellow for the safety shield and detected car in sequence 0 - Red for the safety shield and detected car in sequence 1,2 and 3 In the following figure an example of HMI graphic that will be implemented for sequence 2 is reported. Figure 5 AEB interurban proposed HMI (CRF car) AEB pedestrian HMI The autonomous emergency braking pedestrian function helps the driver to detect a pedestrian crossing his/her vehicle trajectory. If the driver is not able to perform an adequate braking the function will start an emergency brake. The driver needs therefore to be informed about the overall sequence. In particular, considering the HMI suggestions described in chapter 3, the following visual output are foreseen: - Sketch representing the ego-vehicle, the frontal safety shield, the pedestrian detected for the sequence 1 (warning phase) - Sketch representing the ego-vehicle, the frontal safety shield, the pedestrian detected, text message Brake! for the sequence 2 (warning phase) - Text message AEB pedestrian intervention for sequence 3 (post intervention phase) The colour code is the following: - Red for the safety shield and detected car in sequence 1,2 and 3 In the following figure an example of HMI graphic that will be implemented for sequence 2 is reported. Version 1.5, 29/08/2014 Page 31 of 44

32 Figure 6 AEB pedestrian proposed HMI (CRF car) Driver drowsiness HMI When a driver alert system detects signs of driver fatigue or drowsiness, a number of things can happen. Some of these systems provide a multi-tiered method, which increases in severity as time passes. These systems will typically start off by sounding some type of buzzer or chime and illuminating a light on the dash. However, if the signs of fatigued driving continue, the driver alert system may sound a louder alarm that requires some sort of driver interaction to cancel. Some driver alert systems eventually progress to an alarm that can only be cancelled by pulling the vehicle over and either opening the driver s door or shutting the engine off. To indicate a dangerous level of drowsiness, the more effective type of feedback is preferable, in haptic, audible, visual modality, or a combination of them. When the estimation of the attention level of the driver will reach a level considered dangerous by this function, appropriate countermeasures could be taken. Several possibilities can be considered: - ask the user to perform an action within a timeout (for example to switch off the radio tuner) - reduce progressively the power of the engine - force the steering to keep the right lane of the road (if the LDW camera is present) - diffuse in the cockpit a particular fragrance - pop-up informing the driver about the necessity of having a brake The following HMI solutions and graphics is supposed at this stage of the project to be used: - Graphic for a start learning phase - Graphic indicating driver status (awake, quite tired, tired need a rest) Version 1.5, 29/08/2014 Page 32 of 44

33 The icons/telltales referred to the driver status is the same in the before mentioned stages; the colour code indicate the decline. In particular, in the start learning phase the icon in the dashboard will appear in grey colour until the system will be calibrated and then it will change to green. Text messages are also foreseen in this phase, informing the driver about the progress of the learning phase. Examples of text messages can be: Starting Learning phase, In progress and Learning phase completed. Instead regarding the icon indicating the increase drowsiness, the colour code is: - Green: driver awake - Yellow: driver quite tired - Red: driver tired, a rest is needed Figure 7 shows an example of driver drowsiness icon. Version 1.5, 29/08/2014 Page 33 of 44

34 Figure 7 Driver status proposed HMI (CRF car) 4.2 DAIMLER demonstrator vehicle Daimler develops the Inter-Urban Assist, which is an ADAS covering three main information functions: - an augmented night vision system - a predictive front light - a spotlight functionality as a warning system for vulnerable road users. The augmented night vision system will have an HMI display feedback to the driver, and both the predictive front light functionality and the spotlight functionality result in lighting control. The lighting control functions are classified also as information functions. The reaction is up to the driver and, in case of the vulnerable road user warning function, up to the vulnerable road user. Version 1.5, 29/08/2014 Page 34 of 44

35 Function Type (Information - warning - intervention) (y/n) Acoustic Visual Haptic On/Off commands Needs description Augmented night-vision system information N Y Adaptive light control Pedestrian warning information N Y information N Y (y/n) Needs description (y/n) Needs description (y/n) Needs description Visual output is considered as the only output for an augmented view. Adaptive light control results only in visual feedback. Pedestrian warning results only in light control. N N N N N Timing (e.g. contemporary w/other functions, before/after) Not applicable since augmentation is independent of other functions. Not applicable since light control is independent of other information functions. No timing restrictions, since pedestrian warning is only an information function. Consequences are up to the driver and/or the pedestrian. Function Augmented night-vision system Adaptive light control Pedestrian warning Override permitted? Describe case dependencies (e.g. day/night, road type, interaction w/other ADAS) (y/n) remarks (y/n) Repetition of message / function? Description (same, different degrees, etc.) Are there different degrees of risk? Need of an acknowledge from driver? (y/n) Remarks (y/n) Remarks N night, inter-urban Y same N N N night, inter-urban Y same N N N night, inter-urban Y same N N Since the augmented view only offers information by an enhanced view, there is no acknowledge from the driver needed. Since adaptive light control only offers information about the course of the road ahead, there is no acknowledge from the driver needed. As pedestrian warning results immediately in light control, no acknowledge from the driver is needed. Version 1.5, 29/08/2014 Page 35 of 44

36 4.2.1 Daimler demonstrator vehicles architecture The following Figure 8 Functional block diagram for Inter Urban Assist (Daimler demo vehicle) illustrates the architectures of the demonstrator vehicle developed by Daimler. Figure 8 Functional block diagram for Inter Urban Assist (Daimler demo vehicle) HMI components In the table below, HMI components are listed. A more detailed description of how they are used is provided in HMI graphic section below. Demonstrator vehicle DAIMLER car Functions Augmented night-vision system Adaptive light control Possible input devices Button to switch on the DESERVE system Possible visual output devices Dedicated display TFT positioned in the instrument cluster Dedicated display TFT positioned in the instrument cluster Possible audio output devices Not foreseen HMI graphic Regarding the Augmented night-vision system visual output is considered as the only output for an augmented view. A possible HMI graphic is reported in the figure below. For more details see D Version 1.5, 29/08/2014 Page 36 of 44

37 Figure 9 Possible augmented night-view picture (DAIMLER car) Regarding the Adaptive light control, it will show its off-status in a severe error case. 4.3 VOLVO demonstrator vehicle The Volvo demonstrator truck will feature an enhanced adaptive cruise control with emergency brake (auto brake) for heavy trucks. Compared to the normal adaptive cruise control with auto brake available in Volvo trucks, the main difference for the DESERVE implementation is increased vehicle speed range down to stand still and start inhibit functionality enabled by the close range object detection camera system. This means that the HMI solution will be very close to the standard HMI. More details about the HMI solutions are presented below Volvo demonstrator vehicles - architecture In the following figures are reported the draft architectures of the demonstrator truck developed by Volvo. Version 1.5, 29/08/2014 Page 37 of 44

38 Figure 10 ACC with auto brake (Volvo demonstration truck) The following table highlights the HMI needs for each function, described according to a common template reported in the deliverable D3.3.1 Needs Analysis and Specifications. Version 1.5, 29/08/2014 Page 38 of 44

39 Function Adaptive Cruise Control with auto brake Type (Information - warning - intervention) warningintervention (y/n) Y Acoustic Visual Haptic On/Off commands Needs description Acoustic channel used in collision warning phase, could be considered as a secondary channel. (y/n) Needs description (y/n) Needs description (y/n) Needs description Y Visual output could be considered as a primary channel. N The initial part of the auto brake could be considered as haptic information but this is not unique for the DESERVE implementation. N Y = a general button for the ON/OFF of the DESERVE function is foreseen. N = a dedicated button for the switch on/off is not foreseen Timing (e.g. contemporary w/other functions, before/after) Not applicable since standard acoustic- and visual resource allocation is used. Function Adaptive Cruise Control with auto brake Override permitted? Describe case dependencies (e.g. day/night, road type, interaction w/other ADAS) (y/n) remarks (y/n) Y all conditions Y Repetition of message / function? Description (same, different degrees, etc.) Forward Collision Warning escalation according to standard implementation Are there different degrees of risk? Need of an acknowledge from driver? (y/n) Remarks (y/n) Remarks Y Depending on the level of risk (warning or intervention) different outputs will be foreseen. N Version 1.5, 29/08/2014 Page 39 of 44

40 4.3.2 HMI components In the table below, HMI components are listed. A more detailed description of how they are used is provided in HMI graphic section below. Demonstrator vehicle Volvo Truck Functions Adaptive Cruise Control with Emergency Braking Possible input devices Steering wheel buttons for ACC settings Possible visual output devices System settings and warnings are displayed in the instrument cluster. Warnings are also issued via the head-up display Possible audio output devices Forward Collision Warning Buzzer HMI graphic The Adaptive Cruise Control (ACC) keeps a safe distance to the vehicle in front by controlling the accelerator and all available brakes. If there s a risk of collision, warning lights are projected on the windscreen. The Collision Warning with Emergency Brake takes this one step further. It automatically assists you in emergency braking if an impact is imminent. The following HMI solutions and graphics are used. HMI input: steering wheel buttons for ACC driver control (see Figure 11). HMI output: - Visual output is foreseen in a centre display positioned in the instrument cluster as well as in an head up display (see Figure 12 and Figure 13) - Acoustic output is foreseen in order to enforce the visual information present in the head up display - Haptic feedback is foreseen if there is no driver reaction in case of collision In particular in the centre display the cruising speed and ACC state are shown, while the head up display is used for indicating hazardous situations in front, e.g. when there is slow or stopped traffic ahead. If no reaction of the driver the brakes will be automatically applied to avoid or mitigate a collision with objects ahead. Version 1.5, 29/08/2014 Page 40 of 44

41 Figure 11 ACC Steering wheel buttons (VOLVO truck) Figure 12 instrument cluster centre display (VOLVO truck) Figure 13 Head up display (VOLVO truck) 4.4 VTT - RAMBOLL demonstrator vehicle The following uses cases has been selected to the motorcycle demonstration which partly bases on the driver monitoring and the implemented ADAS functionalities: Blind-Spot-Detection for motorcycles Blind-Spot-Detection in the motorcycles use case is related to the Rider drowsiness use case. The idea is to utilize VTT s gaze detection system and analyse if the rider is not paying attention for the blind spots i.e. when turning the motorcycle. In this use case signal of Version 1.5, 29/08/2014 Page 41 of 44

42 failed blind-spot-detection can be warned in HMI using i.e. signal on handle bars or with sound signal. Motorcycle occupant detection and classification system In the motorcycle occupant detection and classification system use case the idea is to monitor the user (occupant) who is riding with the motorcycle and adapt motorcycle settings accordingly. When system detects rider by classifying face between the pre-existing users, VTT s rider gaze direction monitoring system can be used to configure the settings according to his/her saved settings. This use case can also provide notification through to HMI by light or sound signal that the new rider has been detected. Rider drowsiness for PTW In the Rider drowsiness detection for PTW use case, the idea is to monitor rider drowsiness by using VTT s gaze direction monitoring system. In this use case, the analysis results from rider drowsiness detection are sent to HMI using i.e. signal on handle bars or with sound signal VTT demonstrator vehicles architecture The following Figure 14 illustrates the architectures of the demonstrator vehicle developed by VTT - RAMBOLL. Figure 14 Functional block diagram for VTT - RAMBOLL demonstrator vehicle Version 1.5, 29/08/2014 Page 42 of 44