Using Hybrid Reality to Explore Scientific Exploration Scenarios

Transcription

1 Using Hybrid Reality to Explore Scientific Exploration Scenarios EVA Technology Workshop 2017 Kelsey Young Exploration Scientist

2 NASA Hybrid Reality Lab - Background Combines real-time photo-realistic virtual reality, tracking system, interaction with physical objects: tools, mockups, props, and other 3D printed objects. Also adds other technologies that stimulate the other senses along with deep learning, eye tracking, and full body avatar tracking to increase the level of immersion and overall usefulness of the system for a wide assortment of applications. The high level of immersion and other capabilities in NHRL systems will compliment Analog Mission locations and local Center Facilities/Labs NASA s Hybrid Reality Lab (NHRL) showing a user viewing a photorealistic PGT inside an HTC Vive headset, while manipulating a 3D printed PGT that allows for physical feedback. NASA Hybrid Reality Lab run by Frank Delgado and Matthew Noyes 2

3 NASA Hybrid Reality Lab System Specs Hardware VR Headset used is the HTC Vive 1080 x 1200 resolution image presented to each eye Uses Lighthouse for roomscale positional and rotational tracking Positional tracking accuracy to 0.3mm, and rotational accuracy to 0.1 degrees at an overall 1000Hz sampling rate and 500Hz reporting rate. Desktop portable computers Intel 7700K Kaby Lake CPU running at 4.2GHz NVIDIA 1080ti GPU MSI VR One Mobile Backpack Computer Software Intel core i7-7820hk CPU GTX 1070 GPU Unreal rendering engine Blueprint coding for legacy support and prototyping HTC Vive headset, 2 controllers, and 2 lighthouse base stations MSI VR One Backpack 3

4 NASA HYBRID REALITY LAB NASA Hybrid Reality Lab Relevant Technologies VISUALIZATION Display of Photo Realistic Visual Content o HARDWARE: Projectors Systems and Video Wall, Head Worn Goggles, Flexible Displays, Direct and Near Eye Projection Systems. o RENDERING ENGINES: Game Rendering Engines TACTILE / HAPTIC FEEDBACK Touch and Feel Sensation: Vibration, Pressure / Pneumatic, Direct Electrical Stimulation, Temperature ACOUSTIC HW: Omi Directional Speaker System, Headsets, Bone conduction, Directed Beam SW: Various Commercial/Research Solutions OLFACTORY Various COTS products that produce various smells Inside out & Outside In TRACKING SYSTEMS ENVIRONMENT Temperature, Wind, Fog Gravity Environments Different Surfaces with varied coefficient of frictions Low Visibility (glare, dust, fog, etc.) A.I. Automatic creation and adjustment of all Holodeck content and assets. Intelligent Procedure Assistant COMMAND AND CONTROL Dynamic Scene Interaction o Gesture Recognition, full body, limb tracking o Voice Recognition o Eye Tracking and Gaze Control o Omni Directional and traditional treadmills o Brain-Computer Interface using EEG Headsets o Exoskeleton, wearable prosthetics, and other E- Textile Technologies, glove technologies MODEL / ASSET DEVELOPMENT Additive Manufacturing / 3D Printing of objects 3D Scanning OPERATIONAL AND HUMAN PERFORMANCE Analog Missions, Vestibular/Vision/Physiology studies, Operation Concepts, etc. ARCHITECTURE / COMPUTING SW: Data Distribution HW: High Performance GPU / CPU

5 SHyRE Scientific Hybrid Reality Environments SHyRE builds off of high-fidelity science data collected over a number of years to build a platform capable of setting up repeatable testing scenarios. The HR flow can be used for ops con development, procedure development, and astronaut training. 5

6 SHyRE Scientific Hybrid Reality Environments Schematic of SHyRE operational environment Test integrated operational scenarios with multiple EV + IV crew Working from the Exploration EVA SKG document to ensure lessons learned are directly applicable into current ops planning and technology development 6

7 Future Capabilities of Hybrid Reality Operational concepts for lunar surface activities Teleoperation of robotic assets using VR Astronaut crew training 7