www.dlr.de Chart 1 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 VR-OOS System Architecture Workshop zu interaktiven VR-Technologien für On-Orbit Servicing Robin Wolff DLR, and Software Technology
www.dlr.de Chart 2 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 Overview - Motivation - Physical Mock-up - Servicing Tasks - System Overview - Components - Requirements - System Architecture - Distributed Framework - Loop - End-to-End Latency - Module Implementations - Physics - - Summary
www.dlr.de Chart 3 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 Motivation Physical Evaluation Mock-up - Tele-Presence System demonstrated at ILA 2010 in Berlin
www.dlr.de Chart 4 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 Motivation Physical Evaluation Mock-up Virtual Evaluation Mock-up - Tele-Presence System demonstrated at ILA 2010 in Berlin
www.dlr.de Chart 5 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 Motivation Servicing Tasks - Remove Multi-Layer Insulation (MLI) - Loosen / Tighten Screws - Remove / Insert Modules (e.g. using a Bayonet Handle) - Operate Switches - Take Measurements (e.g. using a Voltmeter) EVA Task Sheet Physical Satellite Mock-up Virtual Satellite Mock-up
www.dlr.de Chart 6 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 System Overview System Components Object Properties (Geometry, Material, Constraints, etc.) Virtual Service Robot Model Physics x user x object x user x object Collision Detection & Force Computation x hd F hd x tracking Image Haptic Device Virtual Target Satellite Model VR Display
www.dlr.de Chart 7 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 System Overview Requirements - Interactive Real-Time - Fast Response Times necessary - Complexity & Accuracy Trade-off - Dedicated Machines used for Computation Central Control / Manager Physics Collision Detection, Force Computation Rate: ~60-100Hz Rate: >1kHz Rate: ~30-120Hz Network
www.dlr.de Chart 8 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 System Architecture Distributed Framework - Distributes across dedicated Machines - Each runs at its own Rate and only sees its own World - s are synchronized via Network Messages - Wraps Common Functionality - Unified Scene Management - Asynchronous Message Passing - Transparent Processing Loop Manager Physics Haptics Scene Scene Scene Scene Communication Communication Communication Communication Server Client Client Client Network
www.dlr.de Chart 9 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 System Architecture Common Loop Processing Loop Recv. Process Updates Step Distribute Changes Send Read & Modify Interpret Updates Common Collect Changes Extra Scene Objects & State Load Scene Scene Description (COLLADA)
www.dlr.de Chart 10 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 System Architecture Common Loop Processing Loop Recv. Process Updates Step Distribute Changes Send Read & Modify Interpret Updates Common Collect Changes Extra Scene Objects & State Load Scene Scene Description (COLLADA)
www.dlr.de Chart 11 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 System Architecture Distributed Framework - Includes Optimizations for Network Throughput - Communicates via UDP/TCP/IPC Channels - Unlimited Number of Communication Channels - Configurable Queuing Schemes (e.g. FIFO, Most-Recent) - Optional Spatial and Temporal Threshold Filters - What about Network Issues? - Latency, Jitter, Loss, Bandwidth Manager Physics Haptics Scene Scene Scene Scene Communication Communication Communication Communication Server Client Client Client Network
www.dlr.de Chart 12 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 System Architecture End-to-End Latency Object Properties (Geometry, Material, Constraints, etc.) x user x object Collision Detection & Force Computation x hd F hd Virtual Service Robot Model Physics host.59, host.62 Haptic Device x user x tracking x object Image Virtual Target Satellite Model host.61 host.62 VR Display (RTT between all hosts = 0.2ms)
www.dlr.de Chart 13 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 System Architecture End-to-End Latency A. Time between Hand Movement and feeling the Surface of static, not moving Objects Object Properties (Geometry, Material, Constraints, etc.) Virtual Service Robot Model Physics x user x object Collision Detection & Force Computation (t hap = 0.18ms) host.59, host.62 x hd F hd Haptic Device x user x tracking x object Image Virtual Target Satellite Model host.61 host.62 VR Display Latency A = 0.80 ms (RTT between all hosts = 0.2ms)
www.dlr.de Chart 14 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 System Architecture End-to-End Latency B. Time between Hand Movement and seeing the Hand move Object Properties (Geometry, Material, Constraints, etc.) Virtual Service Robot Model Physics x user x object Collision Detection & Force Computation (t hap = 0.18ms) host.59, host.62 x hd x hd F hd Haptic Device x user x tracking x object (t vis = 3.30ms) Image Virtual Target Satellite Model host.61 host.62 VR Display Latency B = 6.09 ms (RTT between all hosts = 0.2ms)
www.dlr.de Chart 15 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 System Architecture End-to-End Latency C. Time until User sees pushed Object moving Object Properties (Geometry, Material, Constraints, etc.) Virtual Service Robot Model Physics (t phy = 0.02ms) x user x object x user Collision Detection & Force Computation (t hap = 0.18ms) host.59, host.62 x hd F hd x tracking Haptic Device x object (t vis = 3.30ms) Image Virtual Target Satellite Model host.61 host.62 VR Display Latency C = 7.15 ms (RTT between all hosts = 0.2ms)
www.dlr.de Chart 16 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 System Architecture End-to-End Latency D. Time between Hand Movement and feeling the Surface of moving Object Object Properties (Geometry, Material, Constraints, etc.) Virtual Service Robot Model Physics (t phy = 0.02ms) x user x object x user Collision Detection & Force Computation (t hap = 0.18ms) host.59, host.62 x hd F hd x tracking Haptic Device x object (t vis = 3.30ms) Image Virtual Target Satellite Model host.61 host.62 VR Display Latency D = 2.82 ms (RTT between all hosts = 0.2ms)
www.dlr.de Chart 17 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 Implemented Modules Haptics, Extensions - Haptics - Collision Detection and Force Computation A. Improved Voxmap-Point-Shell (VPS) B. OpenHaptics Toolkit - Interfaces - Bi-manual HMI, Phantom Omni, Falcon VPS Algorithm DLR Bi-manual HMI Phantom Omni Novint Falcon - Extensions - Finger Tracking - Speech Recognition - Logging - More coming ART Finger Tracking
www.dlr.de Chart 18 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 Implemented Modules Physics - Uses Bullet Physics Engine - Real-Time n-body - Optimized for Speed, rather than Accuracy - Support for Rigid Bodies - Simplified Collision Detection - Universal Constraints - Support for Soft Bodies - Alternatives:
www.dlr.de Chart 19 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 Implemented Modules - InstantReality - Developed at Fraunhofer IGD - Widely used at DLR RM - ViSTA VR-Toolkit - Developed at RWTH Aachen (+10 Years) - Now in Cooperation between RWTH & DLR - Open Source - Based on OpenSG (soon OpenSceneGraph) - Support for Scientific Methods using ViSTA VR-Toolkit
www.dlr.de Chart 20 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 Implemented Modules (Columbus EVA, ESA)
www.dlr.de Chart 21 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 Implemented Modules - OpenGL Shader Support - Simple Shadow Mapping - Realistic Star Background - Positions based on Hipparchos and Tycho-2 Star Catalogues - Realistic Earth - High-Resolution Textures - Night Lights - Clouds with Shadows - Atmospheric Scattering OpenGL Shader (top), Realistic Stars (bottom)
www.dlr.de Chart 22 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 Implemented Modules Rendering of Earth with High-Res. Textures, Night Lights, Atmospheric Scattering, Clouds with Shadows.
www.dlr.de Chart 23 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 Implemented Modules - NVIDIA OptiX (work in progress) - Toolkit for Real-Time Ray-Tracing - Allows accurate of Light Conditions - Creates High-Quality Shadows, Reflection, Refraction - Could also be used to generate virtual Camera Images to evaluate Image-Recognition Algorithms Rendering Ray-Tracing Photograph
www.dlr.de Chart 24 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 Summary - VR-OOS: a Virtual Reality Environment for On-Orbit Servicing - Software Framework provides a generic Architecture for a distributed interactive Real-Time Environment - Low-Latency Response Times - Several Modules implemented and in use - Live Demo Today! 15:15-16:30
www.dlr.de Chart 25 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012 Contact Dr Robin Wolff German Aerospace Center (DLR) and Software Technology Software for Space Systems and Interactive Lilienthalplatz 7 38108 Braunschweig Germany www.dlr.de/sc robin.wolff@dlr.de
www.dlr.de Chart 26 > VR-OOS System Architecture > Robin Wolff VR-OOS Workshop 09/10.10.2012