Virtual Immersion Facility (VIF) Future Battle Commanders with Advanced Decision Making Capabilities 28 February 2008 Defence Research and Development Canada Recherche et développement pour la défense Canada Canada
Who we are System of Systems (SoS) Section in DRDC - Valcartier Simulation and Comprehension of Complex Situation (SCSC) Group Team composed of 3 defense scientists and 1 computer support Dr. Marielle Mokhtari François Bernier Éric Boivin Frédéric Drolet
Rational SCSC aims exploiting the Synthetic Environment (SE) for DND/CF (immersive environments, virtual reality, advanced visualization, simulation, serious gaming, haptic devices) We found out that there was no high-end general purpose immersive environment in the DND/CF Then we decided to create the Virtual Immersive Facility (VIF)
Virtual Immersive Facility Mission Explores the potential of advanced visualization, interactive simulation & immersive capabilities for the DND/CF Description Supports experimentations (or training), allows testing of applications, and provides computer infrastructures (140 m²) to many projects simultaneously The CAVE is the main component; the VIF also includes many other advanced devices Operational since August 2006 Investment of 1.5 M$
What is a CAVE? Developed in 1992 at the University of Illinois at Chicago Cave Automatic Virtual Environment (CAVE) Immersive virtual reality environment Room whose walls surround a viewer with projected images (Cruz-Neira et al., 1992) In the CAVE, you are no longer on the outside looking in but on the inside looking out (DeFanti)
Main advantages High degree of immersion by using advanced human machine interfaces (visual, haptic, tracking, odors, etc.) High degree of realism More natural interaction with the virtual environment High degree of presence (the sense of being in the virtual environment)
Visual Immersion - Full Field of View Synthetic Environment 1
Visual Immersion - Full Field of View Synthetic Environment 1 Synthetic Environment 2
Visual Immersion Full Field of View Synthetic Environment 1 Synthetic Environment 2
Visual Immersion - Continuous display Synthetic Environment 1 Synthetic Environment 2
Visual Immersion Perspective Matters Synthetic Environment 1 Synthetic Environment 1 Synthetic Environment 2
Visual Immersion Perspective Matters Synthetic Environment 1 Synthetic Environment 1 Synthetic Environment 2 Synthetic Environment 2
Visual Immersion - Continuous display Synthetic Environment 1 Synthetic Environment 1 Synthetic Environment 2 Synthetic Environment 2
Visual Immersion Perspective matters External perspective Immersed perspective
Interaction - Simulated Weapon (1)
Interaction Hands & Other Devices (2)
The presence Defined as the participant s sense of being there in the virtual environment (Bartfield et al., 1995) Participants feel they are part of the simulated world because they forget that their perceptions are mediated by technologies (perceptual illusion of non-mediation) The most important characteristic of immersive systems Facilitated by the immersion, the realism of the environment and natural interaction with the virtual environment It can be used to create stress, lost of equilibrium, fear, trigger reflex, increase concentration, motivation & interest, involvement in the situation, facilitate learning, better transmit the complexity of a situation, etc. Allows to integrate physical with cognitive aspects
Flex CAVE in Valcartier Reconfigurable "Holodeck" Immersive Mode Immersive Theatre Mode Flat Wall Mode Characteristics Virtual immersion system Flex by Fakespace: 3 walls and 1 floor of large size (2.4 m 3.3 m 3.3 m) High intensity (8000 Lumens) and high resolution (1400 1050 pixels) stereoscopic projectors 3D realistic sound effects Real-time user tracking
Approach Focus on performance, flexibility & robustness Exploit high-end multi-screen visualization & immersion system Emphasize on technologies integration: visualization, simulation & interaction Exploit methodologies to improve application development Rely on VR innovations Exploit gaming
Applications Improve team-based activities by facilitating the collaboration & the development of a shared perspective (IMAGE project: team supported with interactive simulation & visualization) Provide more realistic & flexible training capability at lower cost (e.g.: stress inoculation, sensitive site exploitation, tactical combat casualty care, foreign culture ) Recreate the battlefield in a realistic way which can help in the treatment of post traumatic stress disorder treatment Facilitate extraction of meaningful patterns from (massive) datasets, help to detect the expected & discover the unexpected Improve situational awareness, mission planning / rehearsal & after action review Facilitate engineering review
Future Battle Commanders with Advanced Decision Making Capabilities What if Combination if a commander could of of advanced virtually step visualization step into into an & interaction an actual operational techniques environment before and and simulation sending tools troops To in? To support in? What commander s if if he he could plan need plan his for for mission his entire mission by planning/rehearsal, by walking through each SA SA & step AARstep & AARand to manoeuvre, pointing to assist commanders out in out the in their the most their probable point of of retaliation? By By bringing resources such as as Google decision-making process. Earth & using the the latest in in gaming technology, it it is is possible to to develop the the future of of battle command planning...
Context Existing Limitations (1) (1) Applications can can generate easily 3D 3D models of of the the battlefield display on on 2D 2D monitors. (2) (2) Huge of of information to to understand, manipulate & structure display on on (small) 2D 2D monitors. (3) (3) No No easy way way to to interact with with a running simulation in in order to to change parameters value at at one one specific point to to rapidly see see the the impact Need of of an an advanced capability to to immerse commanders in in a SE SE supporting interaction visualization --simulation (& (& modeling). Idea Idea--Simultaneously increase the the overall comprehension of of situation information through simulation & intuitive visualization and and improve the the mechanisms for for user user perception interaction. Expected Outcome --Improve future commanders ability to to make (C2) (C2) decisions by by permitting interactive & dynamic virtual 3D 3D course of of action analysis, mission rehearsals, and and mission planning so so they they may may act act decisively in in the the face face of of quickly changing circumstances.
Adequacy (Large scale) immersive environments are are considered beneficial for for data data analysis & visualization. Increase the the volume of of data data available to to commander but butimprove perception of of these data. Immersive 3D 3D visualization stereoscopic technologies Improve the the commander s ability to to comprehend this this avalanche of of information and and act act upon it it quickly & decisively. Interactive simulation Human-in-the-loop Improve the the commander s ability to to control, to to act act upon the the simulation, then then to to control, to to act act upon scenario & models.
3D 3D virtual representations of of battle scenarios provide commanders with with a more reliable & accurate means of of viewing information on on the the battlefield (in (in RT). RT). Scenario simulation control adds a new new dimension in in the the understanding of of the the situation. Immersive view view of of a fictitious replica of of a specific site site enables commanders to to comprehend complex environment that that his his troops face face during operations (intuitive & efficient 3D 3D exploration of, of, navigation in, in, learning about). about). Using a combination of of technological innovations in in simulation, visualization & interaction, commanders can can quickly gain gain an an understanding of of evolving situations, explore their their options and and communicate their their plans.
U.S Activities CERDEC C2D CAVE Lab. The U.S. Army Communications Electronics Research Development & Engineering Command (CERDEC) Command & Control Directorate (C2D) has developed immersive C2 applications to support the future commander s ability to make decisions faster & more accurately. R&D in the application of immersive technologies & advanced visualization techniques To assist commanders in their decision-making process C2 Applications Basic functionalities Terrain models that provide the 3D simulation of the modeled area; User interface that permits selection of the operating mode: live or playback; Event Data Manager which receives, stores & forwards tactical messages; Presentation software which displays the scenes & permits the user to interact with the immersive environment
U.S Activities C4ISR On-The-Move 2006 Experiment Mission Planning & Execution - Soldiers response to technology Three leaders stated that the planning session was a great benefit to their planning. The Platoon Leader estimated that the technology improved his planning capabilities by 75%. The Platoon Sergeant stated that the technology is outstanding if it is accurate and up to date. He thought the colors & shapes of the buildings & the layout of the streets was very helpful. His estimate of how this improved their planning was 70%. The Robotics NCO estimated that the ROVR planning was 80% better than using a sand table. On the morning of the MOUT mission, the Platoon Sergeant was confident in the locations of the buildings of interest in the area & knew where his soldiers & unmanned assets were to be placed. This technology provided Defence R&D an Canada excellent R & tool D pour for la défense the Canada soldiers to become familiar with the layout of the MOUT site.
Dr Marielle Mokhtari Defence Scientist Defence R&D Canada Valcartier (418) 844-4000 (4282) Marielle.Mokhtari@drdc-rddc.gc.ca VIF Points of Contact François Bernier Defence Scientist Defence R&D Canada Valcartier (418) 844-4000 (4346) Francois.Bernier@drdc-rddc.gc.ca Frédéric Drolet Computer Support Defence R&D Canada Valcartier (418) 844-4000 (4351) Frederic.Drolet@drdc-rddc.gc.ca Éric Boivin Defence Scientist Defence R&D Canada Valcartier (418) 844-4000 (4394) Eric.Boivin@drdc-rddc.gc.ca
(Hypothesis) Mission Planning Session - MOUT Obtain a general view/orientation of the layout of the MOUT; Identify suspected enemy strongholds & safe houses; Determine the best locations to place UGS; Identify potential sniper positions; From an aerial view Identify routes leading into & out of the MOUT site; By freely navigating within the environment, use a bird-eye view of the MOUT street layout for improved orientation; Based on intelligence reports, predict where enemy forces would enter the site & able to determine approximate locations for UGS; Closer look at the terrain navigate down to a dismount or first person perspective street level for a closer inspection (buildings, streets, configuration of a square ) walk through the streets to determine best routes, identify specific buildings & identify any obstacles that may impede interrupt movements of troops walk around the suspected enemy safe houses and strongholds to determine entrance ways, windows & learn characteristics of each building able to identify a foothold, to learn the general Defence R&D layout Canada of this R & location D pour la défense and Canada to identify routes of ingress and egress to/from the building.
SANS CLASSIFICATION COTE DE SÉCURITÉ DE LA FORMULE (plus haut niveau du titre, du résumé ou des mots-clefs) FICHE DE CONTRÔLE DU DOCUMENT 1. PROVENANCE (le nom et l adresse) R&D pour le défense Canada - Valcartier 2. COTE DE SÉCURITÉ (y compris les notices d avertissement, s il y a lieu) Sans classification 3. TITRE (Indiquer la cote de sécurité au moyen de l abréviation (S, C, R ou U) mise entre parenthèses, immédiatement après le titre.) Virtual Immersive Facility - Future Battle Commanders with Advanced Decision-Making Capabilities 4. AUTEURS (Nom de famille, prénom et initiales. Indiquer les grades militaires, ex.: Bleau, Maj. Louis E.) BERNIER, F., Boivin, E., Mokhtari, M., 5. DATE DE PUBLICATION DU DOCUMENT (mois et année) 2008 6a. NOMBRE DE PAGES 28 6b. NOMBRE DE REFERENCES 0 7. DESCRIPTION DU DOCUMENT (La catégorie du document, par exemple rapport, note technique ou mémorandum. Indiquer les dates lorsque le rapport couvre une période définie.) Présentation 8. PARRAIN (le nom et l adresse) 9a. NUMÉRO DU PROJET OU DE LA SUBVENTION (Spécifier si c est un projet ou une subvention) 9b. NUMÉRO DE CONTRAT 10a. NUMÉRO DU DOCUMENT DE L ORGANISME EXPÉDITEUR 10b. AUTRES NUMÉROS DU DOCUMENT N/A 11. ACCÈS AU DOCUMENT (Toutes les restrictions concernant une diffusion plus ample du document, autres que celles inhérentes à la cote de sécurité.) Diffusion illimitée Diffusion limitée aux entrepreneurs des pays suivants (spécifier) Diffusion limitée aux entrepreneurs canadiens (avec une justification) Diffusion limitée aux organismes gouvernementaux (avec une justification) Diffusion limitée aux ministères de la Défense Autres 12. ANNONCE DU DOCUMENT (Toutes les restrictions à l annonce bibliographique de ce document. Cela correspond, en principe, aux données d accès au document (11). Lorsqu une diffusion supplémentaire (à d autres organismes que ceux précisés à la case 11) est possible, on pourra élargir le cercle de diffusion de l annonce.) dcd03f ver.(06-1999) SANS CLASSIFICATION COTE DE LA SÉCURITÉ DE LA FORMULE (plus haut niveau du titre, du résumé ou des mots-clefs)
SANS CLASSIFICATION COTE DE LA SÉCURITÉ DE LA FORMULE (plus haut niveau du titre, du résumé ou des mots-clefs) 13. SOMMAIRE (Un résumé clair et concis du document. Les renseignements peuvent aussi figurer ailleurs dans le document. Il est souhaitable que le sommaire des documents classifiés soit non classifié. Il faut inscrire au commencement de chaque paragraphe du sommaire la cote de sécurité applicable aux renseignements qui s y trouvent, à moins que le document lui-même soit non classifié. Se servir des lettres suivantes: (S), (C), (R) ou (U). Il n est pas nécessaire de fournir ici des sommaires dans les deux langues officielles à moins que le document soit bilingue.) This document presents the Virtual Immersive Facility in DRDC Valcartier, its technologies and how it can contribute to improve the commander tasks with advanced visualization, simulation and interactions. 14. MOTS-CLÉS, DESCRIPTEURS OU RENSEIGNEMENTS SPÉCIAUX (Expressions ou mots significatifs du point de vue technique, qui caractérisent un document et peuvent aider à le cataloguer. Il faut choisir des termes qui n exigent pas de cote de sécurité. Des renseignements tels que le modèle de l équipement, la marque de fabrique, le nom de code du projet militaire, la situation géographique, peuvent servir de mots-clés. Si possible, on doit choisir des mots-clés d un thésaurus, par exemple le Thesaurus of Engineering and Scientific Terms (TESTS). Nommer ce thésaurus. Si l on ne peut pas trouver de termes non classifiés, il faut indiquer la classification de chaque terme comme on le fait avec le titre.) Virtual Immersive Facility, Presence, Immersion,Commander, Mission Planning SANS CLASSIFICATION COTE DE SÉCURITÉ DE LA FORMULE (plus haut niveau du titre, du résumé ou des mots-clefs) dcd03f ver.(06-1999)