PHYSICS-BASED INTERACTIONS IN VIRTUAL REALITY MAX LAMMERS LEAD SENSE GLOVE

Transcription

1 PHYSICS-BASED INTERACTIONS IN VIRTUAL REALITY MAX LAMMERS LEAD SENSE GLOVE

2 Current Interactions in VR Input Device Virtual Hand Model (VHM)

3 Sense Glove

4 Accuracy (per category) Optics based hand tracking & controllers Sensor based hand tracking Force feedback Sense Glove Leap Motion ManusVR VR Gluv Oculus controllers Noitom Hi5 HTC controllers CaptoGlove Dexmo Level of immersion

5 Accuracy (per category) Optics based hand tracking & controllers Sensor based hand tracking Force feedback Sense Glove Leap Motion ManusVR These advanced hand tracking devices require an equally advanced interaction system VR Gluv Oculus controllers Noitom Hi5 HTC controllers CaptoGlove Dexmo Level of immersion

6 Physics-Based Interaction Input Device Virtual Hand Model (VHM)

7 Physics-Based Interaction Uses the physics engine to allow interactions through a VHM in a way that feels realistic and intuitive. Independent of predefined gestures or animations. Suited for advanced hand tracking devices.

8 Ingredients 1. Virtual Hand Model Controlled by an input device. 2. Up to 4 layers working together to determine which type of interaction takes place. 3. Objects with defined behaviors for these interactions.

9 Layers Pickup Layer Gesture Layer Feedback Layer Rigidbody Layer

10 Layers Pickup Layer Determines if the user CAN grab an object Colliders to check which object the fingers and hand palm are currently touching

11 Layers Pickup Layer Collider scripts to remember the last (grabable) object touched Overarching grab script to check which colliders are touching the same object

12 Layers Pickup Layer Finger-Thumb Finger-Palm Finger-Finger

13 Layers Gesture Layer Determines if the user INTENDS to grab an object, using data from the input device Checked separately for each finger

14 Layers Gesture Layer Open Hand In between / Grasp motion Fist

15 Layers Rigidbody Layer Actual collision in the Physics Engine Prevents the hand from passing though objects Can be omitted, resulting in a Ghost Hand

16 Layers Feedback Layer Calculates the material s response Force feedback based on material properties.

17 Layers Feedback Layer Entry Vector (OE) Collider position (OP) Collider Depth (OP )

18 Layers Feedback Layer

19 Layers - Combined

20 Interactable Objects The layers are used to initiate or to end and interaction The objects themselves determine what that entails:

21 Layers and objects - combined

22 Conclusion Physics-Based Interaction Uses collision of the physics engine Can we interact AND do we want to? More realistic than Traditional VR Interaction Works best with some form of tactile feedback

23 Extensions Mesh Deformation Hand model that does not not pass though objects Gestures as a replacement for buttons

24

25 Making Virtual Reality Real

26

27 The Sense Glove team and our partners are working to make the first commercial Sense Glove a reality in Q2 of 2018 Core Team Ir. Gijs den Butter CEO Ir. Johannes Luijten CTO (Hardware) Ir. Max Lammers Lead developer Development Partners like

28 Currently, 10 Pilot customers are using prototypes of the Sense Glove to develop content where Sense Glove is vital for interaction Gert-Jan Brok InMotionVR Although it is a prototype, the Sense Glove is currently the best system available

29 The Sense Glove is the most accurate wearable hand capturing device, and has the ability to simulate touch.

30 Types of feedback Visual Texture Deformation Object Behavior Auditory Tactile

31 Feedback Loop

32 Retention rate based on teaching style

33 Sense Glove Pricing Developers KIT 999: Set of Sense gloves with developers KIT 299: For a wireless & battery pack For customers from 2020: 599: Set of Sense gloves (Suggested retail price)

34 The VR market is growing rapidly. Base case expectations are that B2B applications will make up 46% of the total $45B hardware VR market in 2025 VR & AR hardware market Market segmentation $9 Billion For Input devices (Kzero 2015) $45 Billion in 2025 (Goldman Sachs 2016) $2,25 Billion For Gloves $33,75 Billion for head mounted displays Current focus 15% Healthcare (b2b) 13% Engineering (b2b) 4% Aero & Military (b2b) 2% Education (b2b) From % Video Games (b2c) Kzero expects that in 2018, 10M input devices for VR will be sold The remaining 33% falls outside the focus of Sense Glove and contains segments such as 3d video streaming

35 Sense Glove s customers can be divided into three segments: VR / AR developers, Corporates using VR / AR and research institutions & clinics