In-cockpit dynamic lighting system for driving simulators and its impact on quality of driving immersion

Transcription

1 In-cockpit dynamic lighting system for driving simulators and its impact on quality of driving immersion Petr Bouchner, Stanislav Novotný, Roman Piekník Driving Simulation Research Group Department of Control Engineering and Telematics Faculty of Transportation Sciences Czech Technical University in Prague Konviktská Prague Czech Republic {bouchner novotny pieknik}@lss.fd.cvut.cz Abstract Our paper describes a design and realization of the system DYICOLIS (Dynamic in-cockpit Lighting System) which allows dynamical illumination projected on the driver and the cockpit of a driving simulator and its incorporation into the driving simulator systems. It describes the ways of how the visual databases for the system are designed and the ways how the reflected or transmitted light from the virtual scene affects the shadow world. It also presents the software and hardware realization of the system incorporated into our steady based driving simulator. Finally the paper proposes planned enhancements and improvements of DYICOLIS. Résumé

2 Introduction Everyone knows the situation when driving from the sharp summer light just for only several seconds through the shadow cast by the bridge or viaduct. The eyes accustomed to the bright light conditions cannot easily accommodate to this sudden change. In that moment the dashboard is poorly lit and without additional illumination it is hardly legible. This is one of very interesting (but basic) examples which testify for importance of realistic lighting of the car interior of driving simulators. There are a lot of more examples that demonstrate in general the importance of realistic illumination of driving simulator cockpit. How unrealistic it could impress when the scenery projected on the screens is dynamically being changed in the contrary to static illumination of the cockpit (and many times inner illumination is totally missing). Since we are involved in a research in the field of IVIS assessments (using driving simulators e.g. [1] and plenty of others) we need to guarantee adequate and realistic lighting inside a car cabin which could be adjustable to different kinds of day and/or season outer light conditions. One of the main topics of our research performed on our driving simulators is a subject fatigue. From that point it pays off to include the element of light changes impacting the driver during the measurements into the experiments. The light changes could either influence the driver s vigilance in a positive sense (waking up) or they can contribute to his/her discomfort. As it is known the drowsiness and discomfort contributes seriously to degradation of driver s attention on primary driving tasks, so those factors lead to fatal consequences. One of the topics of the research is an assessment of driver s behavior inside road tunnels. It is approved by many studies done on simulator ([2, 3] etc.) or in the real car (for example [4] or the project, which our laboratory is involved in [5]) that drivers behave differently inside the tunnels than in open country areas. The drivers seem to be more stressed and of course any accident inside the tunnel has much more serious consequences that in the open road environment. Inside the tunnel the influence of the fast changes of lights and shadows of different colors has indisputable impact on driver s behavior. Those light factors should be also considered in such a kind of experiments. There are very few works describing the problem of the light conditions inside the cockpit of the driving simulators. For example [6] deals with ambient light inside the car and offers a solution which uses light tubes placed just around the car mockup roof. This approach approved to be a good contribution to an over all fidelity of the driving simulation for driving under different light condition (night, day, fog ). Unfortunately it does not consider any dynamics either from the point of view of light color or the lights and shadows shapes. Our experiments as mentioned above utilize virtual environments which are naturally full of such light and shadow changes and it is necessary to take into account their influence on the inner cockpit and/or driver s body parts. Because of this fact and respecting requirements which we have for our experiments we decided to design a custom system that offers dynamic and realistic illumination of the cockpit of the driving simulators. We called it DYICOLIS since this is abbreviation of Dynamic In- Cockpit Lighting System.

3 Dynamic In-Cockpit Lighting System (DYICOLIS) for driving simulators Usually the designers of driving simulators are interested in motion cueing since this feature plays an indisputable role in the fidelity of the driving simulation and consequently it significantly affects the validity of performed experiment. If the drivers lack some important percepts, they could hardly believe that they drive a real car and they cannot immerse enough in the virtual reality so that they behavior is the same as in reality. When dealing with motion cueing it is not only the force which acts on the driver but also different stimuli influencing his/her immediate surroundings (close audio, haptic and visual percepts). Shadows and lights cast by various static or dynamic objects appearing on the scene (besides changeable light conditions) are just in touch distance to the driver and because of that fact they intensively influence driver s senses. From such information the driver can easily derive information about the self movement - speed, position, etc. but also about road or weather conditions for instance. Many works in the field of computer graphics applied on driving simulation deal with fidelity of the image of virtual reality (for example [7]). Shadows and lights influencing virtual objects contribute indisputably to their photorealistic look. The cast shadows and lights should be derived from the scenery the driver goes through but this cannot be used directly. The virtual world of the driving scenery should be now defined in two variants; one of which is projected on normal screens and the second which casts shadows and lights on a cockpit. Elements of light/shadow simulation For systematic creation of the shadow scenario it was necessary to categorize different elements of the light (shadow). For this purpose the objects are divided into following subcategories: 1. Ambient light a. Open spaces b. Tunnels c. Weather conditions d. Lighting 2. Elements of light a. Lamps b. Semaphores (traffic lights) c. Lighted sign tables d. Reflective planes e. Dynamic lights projected and/or reflected by surrounding cars 3. Elements of shadow a. Bridges, tunnels b. Shadows of buildings, track side objects c. Soft shadows of complex objects (trees, bushes fences ) d. Dynamic shadows cast by surrounding cars and other moving objects Casting maps Generally in the computer graphics those textures are made as secondary textures on the objects and they re called light maps or shadow maps. For our purposes there is no need

4 for secondary textures, the shadows and lights are projected on a real object mapped on its real appearance. Because of that fact we only use the primary textures with gray shadows and color lights mixed together, mapped on simple objects (for simplicity the called them casting maps ). The casting maps are in fact images obtained as a mixture of the actual ambient expected to persist in particular area and all the active lights which can affect volume of the car interior. To achieve a realistic look it is necessary to take into account also the ambient light (the contribution of lights, shadows and ambient lights needs to be computed properly). See the picture (Fig.1) for the reference. Fig. 1: Schematic process of creation of casting maps The following pictures (Fig. 2) demonstrate the use of casting maps in the shadow world and their corresponding objects as they appear in the common virtual reality scenery. Fig. 2: Examples of the objects, their projection on the real cockpit and their casting maps (from the top to the bottom: Tunnel driving, Traffic lights in the dark environment, Open scenery with trees)

5 Programming design When dealing with motion of light and shadow maps which are cast (or projected) on to the cockpit and some parts of the driver s body it is necessary to synchronize them correctly with the image which the drivers sees on the screens. The arrangement of the programming solution is described by means of the following figure (Fig. 3). CAR PHYSICS All motion data Rendered image GRAPHIC ENGINE PREDICTION, INTERPOLATOIN x, z, α, v SCREEN PROJECTION SHADOW-LIGHT SHADOWS ENGINE PROJECTION Shadow-light image Fig. 3: Schematic diagram of interconnection between the modules Data which are needed from the basic visualization system are as follows: Planar coordinates (X,Z) Heading angle of the virtual car Vector of car velocity (used for prediction) Rolling of the car and the elevation of the road are not taken into account since these values do not affect the overall dynamics of the light and shadow casting. The software performing this special kind of projection is written using OpenGL library. The communication between car physics engine and visualization software should be as fast as possible. The frame rate of the projected image is expected to be as high as possible because of the fact that the projected shadow and light maps appear in the immediate vicinity of the driver; hence he/she is very sensitive to jerking of the image. We approved by experiment and subjective evaluations that the frame rate should be at least 60 FPS and the latency between the visual image and the shadow image should be 20 milliseconds at maximum. Thanks to such arrangement the animation of the shadow world does not rely only on the supplied data but it also computes the prediction of the next movement from the history of previous steps and the vector of the instantaneous speed (as possible to suppose the car movement to be continuous). Realization of the system Our steady based simulator on which we applied the first version of DYICOLIS (with the one projector beaming from the front to back called type I.) is equipped with frontal projection and active reflective mirrors. The system consists of beamers projecting two parallel images with partial overlap offering observing angle about 100 horizontally and 40 vertically. The next two pictures describe the setting up the shadow lights system. The side view (Fig. 4 - left) shows the arrangement of primary and secondary projectors and the placement of a mirror reflecting the cast maps on to the cockpit parts. Meanwhile the top view (Fig. 4 - right) shows a range in which this projection influences the cockpit and the driver. The primary projectors beam on the flat screen from behind the simulator mockup while the shadow casting (secondary) projector is placed in front of the car

6 mockup reflecting the picture with a help of a mirror. (In fact it is possible to place the secondary projector directly but there is often no available space behind and above projecting screens.) MIRROR S C R E E N FRONTSCREEN PROJECTION S C R E E N SHADOWLIGHT PROJECTION Fig. 4: The side view (left) and the top (right) view on the steady based simulator equipped with planar projection which is enhanced with DYICOLIS installation DYICOLIS in vivo In the following picture it is possible to see our steady based simulator equipped with the DYICOLIS during the experiment. It shows exemplary situations as proposed in previous chapters (Fig. 5). Fig. 5: View on the simulator equipped with DYICOLIS during a real experiment

7 Notes to the realization of the system There are several traps and bottlenecks which one should overcome to obtain satisfactory results. First the placement of the mirror and the projector should be in such an arrangement that the common projection screen is not affected either by the beam from the shadow projector or by the reflections of primary projectors. Also the angle of projection should not impact the driver s head (for that purpose we used an adaptable shade visor, which each particular driver can fix according to his or her height). This is also hard to be done with common glass windshield since it could cause unforeseeable reflections (our simulator has no windshield). It is not recommended to use LCD projectors since those shine partially polarized light which after the secondary polarization caused by the mirror can result in ambiguously colored image instead of bright white (we used a high quality DLP projector). Also the lens of the projector is recommended to be slightly unsharpened so that the raster is not apparent and the shadow-light transitions are smooth. Validation experiments For the purpose of evaluation of the rate in which DYICOLIS contributes to overall fidelity of driving simulation we prepared simple experiments in which the experimentees subjectively assessed the immersion of the virtual reality. We divided the experiment into two experiments; the experimentees of the first one was asked for evaluation of the shadow and light casting system while experimentees of the second one were said to evaluate quality of overall immersion of fictive improvements in sound system. In both experiments we asked the drivers to evaluate contribution of the new feature to the overall quality of the immersion. The experiments consisted of two parts; the first one without and second one which new feature in action. We used a simple Likert scale for evaluation of the results. Majority of the respondents considered DYICOLIS as a significant improvement of the simulator fidelity (even during the blind tests - but almost a half of them discovered the change in visual appearance). System enhancements and improvements proposals Although the system which is described in this paper brought a new kind of experience into the driving simulator and experimentees approved it to be valuable enhancement to driving simulation fidelity, it still provides wide rage for particular improvements. Casting maps enhancements For the first version of DYICOLIS we created the casting maps manually mixing up all the surrounding shadow casting objects, light sources and ambient light. To be more precise we need to take into account a complex model of the visual environment which encompasses object materials reflections and absorbance (raytracing, radiosity etc. see also [7]). These maps can be obtained using professional 3D modeling packages (3D Max, Maya etc.) or specialized freeware software like for example LightMapper [8].

8 Dynamic light/shadows The world surrounding the driver in visual environment is not static but it moves and changes its properties. The shadow world should also reflect those changes. In fact even the contemporary version of DYICOLIS supports limited range of dynamics (for example it can take into account changes in traffic lights but not animations). Currently we are working on extension of the Shadow coulisse world to be capable of following dynamic features (points 2.e and 3.d in the chapter Elements of light/shadow simulation). It is further possible categorize the elements of the shadow world dynamics: Static with dynamic behaviour o Shadows of rotating or shivering objects (trees etc.) o Rotating lights or locally movable (light sirens) Dynamic objects o Shadows or reflections cast by moving cars o Lights of surrounding traffic (including non-car traffic) o Lights of emergency vehicles (police, ambulance) Fast changes in shadows When the car is moving at very high speeds the movement on the shadow-light maps (mostly shadows in this case) is so fast that it could not be projected continuously but it suffers from aliasing. For instance the shadow of the width about 0.5m (street lamp, pillar ), which the moving car on the highway (driving 130km/h) is displayed to, stays on the cockpit (supposing affected length is maximally 1m) for 13.8 milliseconds at maximum. This is less than (or on the limit) of the refresh rate of the projectors and can cause that these shadows are displayed improperly or not at all. For the speed over certain limits, a consideration, whether to display a real shadow or a flickering dummy substitute, should be done. Multi projector setup (2 and 3) Although the design described here was proven to a contributing to the realistic view of the overall simulation (and consequently fidelity of driving simulators), it affects only the frontal part of the cockpit. To be precise we need to spotlight the inner cockpit from arbitrary direction. It can be done a help of three projectors placed to shine from the front and side directions (or at least two the second heading from a driver s side window). Such a design however should overcome several problems connected with overlapping (masking) or three projected images which are moreover projected on very diverse (and if affecting the driver also moving) objects. We called these complex setups type I. and type II.. Motion base simulators There is no principal problem to adopt DYICOLIS for the motion based simulator systems. For the simulators with moving platform connected to the projection screens (or dome) a solution should be very similar to the proposed one. A problematic realization of the system will be in the case when the projection screens do not move with the simulator body. We recommend also in this situation to connect the projection of the shadows with

9 the motion platform of the simulator. (Moreover it would be hard to track very fast movements of the simulator body by rendered image and also the refresh rate of the DLP projectors may not be satisfactory). Conclusion Our system (DYICOLIS) as introduced in the paper was approved by experimental drivers as a significant enhancement of the simulator fidelity. The solution appears to be novel and brings new a kind of experience into the discipline of driving simulation which should lead to higher fidelity of the simulation and consequently to better immersion of the experimentees into virtual driving. Better immersion supports validity of the data obtained from the experiments performed on the driving simulators. This system can also significantly enhance steady based stimulators perception with certain kind of false motion cueing. Our goal now is to improve this system with respect to the stated proposals, primarily to equip the system with dynamics and support for motion based simulators. Acknowledgement The work on this project was supported by the research grant GAČR 102/07/1191, Analysis of driver observation area and reaction time changes in relation to attention degeneration. References [1] Bouchner P., Novotný S., System with driving simulation device for HMI measurements, 8th WSEAS International Conference on SYSTEMS (part of the 8th CSCC Multiconference), Athens,2005 [2] Ruggero Ceci, Lennart Högman and Christopher Patten, Measures of driver behavior and cognitive workload in a driving simulator and in a real traffic environment, Driving Assessment 2001, Snowmass Village at Aspen Colorado [3] Amundsen F. H., Studies of driver behavior in Norwegian road tunnels. Tunnelling and underground space technology, (1): p [4] Toernros J.: Driving behavior in a real and simulated road tunnel - a validation study. Accident analysis and prevention, (4): p [5] Bouchner P., Novák M., Přibyl P., Vysoký P., et al. Optimization of design and maintenance of road tunnels, annual reports 2004, 2005, 2006 (in Czech), National project supported by Czech ministry of transportation, online [6] Wenzel G., Dangelmaier M., Marbeger C., Becirovic S., Daylight simulation for driving simulator interior, Driving Simulation Conference 2004, Paris [7] Gavin A., Chalmers A., Generation of High Fidelity Graphics for Simulator Environments, Driving Simulation Conference 2005, Orlando [8] online: