Defence Research and Development Canada R&D Activities in the Simulation & Comprehension of Complex Situation Group

Size: px

Start display at page:

Download "Defence Research and Development Canada R&D Activities in the Simulation & Comprehension of Complex Situation Group"

Georgina McLaughlin
6 years ago
Views:

1 Defence Research and Development Canada R&D Activities in the Simulation & Comprehension of Complex Situation Group Dr. François Bernier Dr. Marielle Mokhtari Éric Boivin Frédéric Drolet Systems of Systems September 2010

2 Outline Defence R&D Canada - Valcartier Virtual Immersion Lab (VIL) & Gaming and Emerging Technology Lab (GET) Project overview Complex Situation ti Understanding di (IMAGE) Stress Management Training (SMT) After Action Review (AAR) Sensitive Site Exploitation Training Medic in Combat Situation Training Immersive vs Non-Immersive Training Immersive Exploratory Perception Conclusion 2

3 Defence R&D Canada (DRDC) Defence R&D Canada - Agency of the Canadian Department of National Defence responding to the scientific & technological needs of the Canadian Forces (CF). Mission - To ensure that the CF remains scientifically and operationally relevant. The agency is made up of 7 research centres located across Canada with a corporate office in Ottawa. 3

4 Defence R&D Canada - Valcartier Virtual Immersion Laboratory Gaming and Emerging Technology Lab 4

5 5 Virtual Immersion Laboratory (VIL) Flexible multi-screen visualization infrastructure for exploitation and integration of virtual reality and simulation technologies; Immersive, collaborative, interactive visualization approach in support of complexity-characterized applications; Realistic environment for training and experimentation.

6 Gaming and Emerging Technology Laboratory (GET) Investigate how gaming & emergent technologies could be exploited to support the development of defence & security applications. This exploitation targets but is not limited to: Gaming genres & mechanics; Visualization & interaction techniques inspired from video game industry; Software components used in video game development; Display & interaction emergent devices; Video game culture 6

7 Dr. François Bernier, Dr. Marielle Mokhtari, Eric Boivin, and Frédéric Drolet Complex Situation Understanding After Action Review Stress Management Training Analysis Mission Rehearsal Planning Sensitive Site Exploitation Medic in Operational Situation Experimentation Training Decision-Making Individual Battle Task Standard Immersive Exploratory Perception 7

8 8 IMAGE : An Interactive ti Computer-aided d Cognition Capability for C4ISR Complexity Discovery

9 Complexity At the constituent level Many linked, entangled elements No centralized command Many scales in time, size and space At the behaviour level Exchange with their environment Adaptation (self-organisation) and evolution Emergence Non-linear, feedback loops Consequences Resilience, self-repair Operate at the edge-of-chaos Hard to predict and control 9 Source: Advanced Technology Assessment

public opinion) CF: Canadian Forces MSF:

10 Complex Situation Complexity is a important aspect of current and future defence operations (coalition, diverse issues, diverse power and influence, public opinion) CF: Canadian Forces MSF: Réf. Médecins B-GL /AF001 sans frontières UNAMA: United Nations Assistance Mission in Afghanistan CIDA: Canadian International ti Development Agency GoIRA: Government of the Islamic Republic of Afghanistan ISAF: International Security Assistance Force Afghanistan 10 «Afghanistan National Development Strategy, 2006»

11 Complex Situation? DM 11

12 IMAGE Objective Develop a toolset supporting collaboration of experts trying to reach a common comprehension of a complex situation ti Identify if this toolset can help to better/faster understand (increase the ratio: understanding / time) IMAGE Concept Many components work in synergy to produce a common representation 12

13 IMAGE EXPLORATION Explore the dataset to make sense of it REPRESENTATION Producing and sharing a common vocabulary and conceptual graphs SIMULATION Moving in the space of variables and walking along the simulation SCENARISATION Transforming conceptual graphs into executable models 13

14 Representation Produce and share a common vocabulary Produce comprehension models using conceptual graphs Recording any type of information Establishing any link Infer conclusions links and patterns Share individual and common understandings 14

15 Scenarisation Select and standardise concepts and relations that are relevant for simulation using the scenarisation vocabulary Add required details for simulation Isolate elements useless for simulation purposes Scenarisation (CoGUI + Eclipse) Comprehension Model Scenarisation Vocabulary JAVA simulator Executable Model 15

16 Simulation Multichronia - Visual interactive manipulation of the simulation Traveling in the solution space (variables such as time) Testing hypotheses Comparing multiple runs from different initial start points or bifurcations of the same one Generating results to be analysed using Exploration 16

17 17 Tactical Level Demonstration

18 Exploration (XPL) Offer a human-centric exploration environment XPL V1 - Experimentation XPL V2 - Proof of concept Propose a wide range of visualization and interaction concepts / metaphors for investigating large data sets dynamically y Support the common understanding process of the complex situation 18

19 Software Visualization Multichronia XPL V1 Desktop Mode Recording of User s Actions Hardware 3 LCD Software Hardware monitors Morae Head tracker Keyboard Mouse Desktop Mode Allows one objective measure of the user s actions, but This solution is non-immersive 19 Proposed solutions constrained by the cognition-driven evaluation strategy

20 Visualization i Software Multichronia Hardware 3 screens with retro projection Hardware 4 screens in stereo projection XPL V1 CAVE Mode Non Immersive CAVE Mode Offers partial immersion With a more rich visualization functionality than the Desktop mode Recording of User s Actions Hardware Tbl TabletPC Keyboard Mouse Hardware Head tracker Wand TabletPC Semi Immersive CAVE Mode Proposed solutions constrained by the cognition-driven evaluation strategy 20

21 XPL V2 Fully immersive human-centric exploration env. Hardware Software Components Components CAVE - Fakespace 4 Stereo Projection Screens Custom OO Architecture Cybergloves Immersion 22 sensors Selection Manipulation Navigation Humancentric interactions OSG-GIS Geographical Information Tracking System Intersense Inertial + Ultrasounds Visualisation Multichronia 21 Stereo Glasses CrystalEyes Graphics Engine Simulation Engine

22 Experimental Validation An experiment assessed the benefit of IMAGE 36 analyst/scientists were asked to understand a complex situation using one of the configuration: baseline, IMAGE desktop and IMAGE CAVE 22

23 Experimental Results - Performance Participants were asked to find the best strategy to optimise the outcome of a complex simulated situation. This value is a measure of their comprehension. We calculated the distribution of each group. Fréqu uency (dens sity) Integrity Halton Baseline Desktop CAVE 23

24 Stress Management Training (SMT) 24 Partner Canadian Forces Health Services Challenge Exposure to stressful events on the battlefield places soldiers at risk for adverse psychological and behavioral reactions Stress management strategies (coping strategies) should Reduce negative reactions to stress Improve operational effectiveness Protecting against psychological injuries (long-term) Existing programs: R2MR (CAN) and Resilience Training (US) Issues: Learning in theory transfers poorly to actual change in stressful operational situation Learning coping strategies is not considered as virile

stressful situations Coping Strategies Stressors Virtual Environment

25 Stress Management Training (SMT) Objective Investigate the potential of virtual reality (VR) to practice coping skills in progressive stressful situations Coping Strategies Stressors Virtual Environment Physiological Arousal Electrocardiogram Biofeedback Electrodermal activity 25

26 How to Create a Stressful Virtual Environment (1/2)? Use stressors found in theatre Combat-related stressors experienced in Afghanistan (CF) 26 *Raw data from CF Land Personnel Concept and Policies Office

27 How to Create a Stressful Virtual Environment (2/2)? Tangible Field of view Stereoscopy Graphic quality Lin et al. (2007) Ivory & Magee (2009) Rajae-Joordens (2008) Arriage et al. (2008) Persky & Blascovich (2006) Ivory & Kalyanaraman (2007) Slater et al. (2009) Interactivity ti it Mommer (2008) Music/Sounds Hebert et al. (2005) Grimshaw et al. (2008) Intangible 27 Presence Narrative/ Cinem matograp phic Violence Uncertainty, forewarning, surprise Ravaja et al. (2006) Wiederhold et al. (2003) Anderson & Bushman (2001) Fleming & Wood (2001) Arriaga et al. (2008) Weber et al. (2009) Perron (2005) Color features: high contrasts, Block (2001) User experience (appreciation) Jeong et al. (2008) Kuikkaniemi et al (2010)

28 Approach 1: Graphic Quality Unreal Tournament 2004 Unreal Tournament 3 (2007) Arma 2 (2009) 28 Far Cry 2 (2009) Virtual Battlespace 2 (2007)

29 Approach 2: Immersion and VG What level of immersion is sufficient? Does the type of video game matter? We exposed 56 soldiers to various virtual environments (2 video games x 3 immersive technologies) 29

30 Approach 2: Immersion and VG What level of immersion is sufficient? Does the type of video game matter? We exposed 56 soldiers to various virtual environments (2 video games x 3 immersive technologies) 22 3D LCD 73 3D TV CAVE Result: All virtual environments are equally stressful 30

31 Approach 3: Suffering Avatar Could we elicit stress with a virtual character of a known person (avatar) experiencing pain? 42 civilians has been exposed (in a 6- side CAVE-Like system) to familiar and unfamiliar avatars 6-side CAVE-Like (UQO) 31 Suffering Yes Known Not yet

After Action Review (AAR) Learning more from our training

- CMTC) Challenge Measure the OCT plus-value Improve OCT

collection data sharing & communication Provide data coming

.. Analyze data before the AARP conduct phase Provide the

32 After Action Review (AAR) Learning more from our training Partner Canadian Forces (Canadian Manoeuvre Training Centre - CMTC) Challenge Measure the OCT plus-value Improve OCT learning process During a training session Facilitate data collection data sharing & communication Provide data coming from system integration... Analyze data before the AARP conduct phase Provide the more suitable data representation during the AARP conduct phase Capture store distribute lessons learned 32

33 After Action Review (AAR) Learning more from

(high-level) needs related to : OCT; LVC training

level 7); level of trainee expertise; current &

improvement roadmap that could include : how to

fuse data; what data system to consider; how to

technologies to support this roadmap: advanced

33 33 After Action Review (AAR) Learning more from our training Study Objectives Identify (high-level) needs related to : OCT; LVC training sessions; level of training sessions (level 1 to level 7); level of trainee expertise; current & incoming training system capabilities Propose an improvement roadmap that could include : how to collect / display data; how to extract / analyze / fuse data; what data system to consider; how to structure historic Log DB Evaluate (emergent) technologies to support this roadmap: advanced 2D/3D visualization & natural interface devices; game-based learning / e-learning.

staff training by adding interactive experience inside diversified, rich & realistic context-driven environments?

34 Sensitive Site Exploitation (SSE) Partner Intelligence staff + SSE trainers Challenge Existing training mainly based on lecture classes Live simulation very limited Objective Since it is a highly spatial task, could the CAVE strengthen traditional Intelligence staff training by adding interactive experience inside diversified, rich & realistic context-driven environments? Verify the technical feasibility CAVE + gaming technology + SSE scenarios Demonstrate interaction metaphors, fast scenario development, task realism, etc. 34

35 35 SSE Some examples

36 Training: Medic in a Combat Context Could an immersive virtual simulation (CAVE) provide a realistic operational context for combat medic training? 36

37 Training: Individual Battle Task Standard (IBTS) Benefits of immersion for training IBTS (in support of a Applied Military Science Project - Kingston) Detachment t level l training i : 1 CAVE + 4 Desktops Mission: IED identification & vehicle checkpoint 37

38 Training: IBTS Fully immersive simulator tends to achieve training objectives slightly better than the desktop immersive i simulator Minor technological deficiencies identified Integration of cognitive, physical and affective aspects (e.g.: crouching when shot) Familiarization faster than with desktop Less a game than a real training environment Tiring but motivating Able to cause information overload 38

39 Immersive Exploratory Perception (Proof of concept) Partner Canadian Forces - Public Security - Search & Rescue Challenge

& surveillance - Intervention & assistance Perception & cartography - Tele-presence Objectives Virtually & safely explore the

Fuse, if required, data from different sources (2D, laser, stereo, ) Refine (locally and / or globally) the virtual model Receive

39 39 Immersive Exploratory Perception (Proof of concept) Partner Canadian Forces - Public Security - Search & Rescue Challenge Collect data information from unknown and / or hostile real environment ((semi)-urban areas, indoor / outdoor) for Reconnaissance & surveillance - Intervention & assistance Perception & cartography - Tele-presence Objectives Virtually & safely explore the environment Iteratively generate in near RT one scalable & interactive immersive 3D model of the real environment (and its parts) Fuse, if required, data from different sources (2D, laser, stereo, ) Refine (locally and / or globally) the virtual model Receive data and send commands (navigate to, manipulate ) Coordinate, for cooperation, several & different kinds of mobile platforms (UTV / UAV)

40 Conclusion Summary Immersive virtual environments can provide a realistic, interactive, engaging, and cognitively augmented setting for a range of applications We better understand the applicability of immersive virtual environments We are always looking for collaborations and new applications Challenges Lack of integration with the current CF training environment Many benefits of immersive virtual environments remain to be identified The cost is still an issue 40

41 Dr François Bernier Defence Scientist Defence R&D Canada Valcartier (418) (4346) Points of Contact Dr Marielle Mokhtari Defence Scientist Defence R&D Canada Valcartier (418) (4282) 41 Éric Boivin Defence Scientist Defence R&D Canada Valcartier (418) (4394) Frédéric Drolet Computer Support Defence R&D Canada Vl Valcartier (418) (4351)

ARMY RDT&E BUDGET ITEM JUSTIFICATION (R2 Exhibit)

Exhibit R-2 0602308A Advanced Concepts and Simulation ARMY RDT&E BUDGET ITEM JUSTIFICATION (R2 Exhibit) FY 2005 FY 2006 FY 2007 FY 2008 FY 2009 FY 2010 FY 2011 Total Program Element (PE) Cost 22710 27416