PHYSICS-BASED INTERACTIONS IN VIRTUAL REALITY MAX LAMMERS LEAD DEVELOPER @ SENSE GLOVE
Current Interactions in VR Input Device Virtual Hand Model (VHM)
Sense Glove
Accuracy (per category) Optics based hand tracking & controllers Sensor based hand tracking Force feedback Sense Glove Leap Motion ManusVR VR Gluv Oculus controllers Noitom Hi5 HTC controllers CaptoGlove Dexmo Level of immersion
Accuracy (per category) Optics based hand tracking & controllers Sensor based hand tracking Force feedback Sense Glove Leap Motion ManusVR These advanced hand tracking devices require an equally advanced interaction system VR Gluv Oculus controllers Noitom Hi5 HTC controllers CaptoGlove Dexmo Level of immersion
Physics-Based Interaction Input Device Virtual Hand Model (VHM)
Physics-Based Interaction Uses the physics engine to allow interactions through a VHM in a way that feels realistic and intuitive. Independent of predefined gestures or animations. Suited for advanced hand tracking devices.
Ingredients 1. Virtual Hand Model Controlled by an input device. 2. Up to 4 layers working together to determine which type of interaction takes place. 3. Objects with defined behaviors for these interactions.
Layers Pickup Layer Gesture Layer Feedback Layer Rigidbody Layer
Layers Pickup Layer Determines if the user CAN grab an object Colliders to check which object the fingers and hand palm are currently touching
Layers Pickup Layer Collider scripts to remember the last (grabable) object touched Overarching grab script to check which colliders are touching the same object
Layers Pickup Layer Finger-Thumb Finger-Palm Finger-Finger
Layers Gesture Layer Determines if the user INTENDS to grab an object, using data from the input device Checked separately for each finger
Layers Gesture Layer Open Hand In between / Grasp motion Fist
Layers Rigidbody Layer Actual collision in the Physics Engine Prevents the hand from passing though objects Can be omitted, resulting in a Ghost Hand
Layers Feedback Layer Calculates the material s response Force feedback based on material properties.
Layers Feedback Layer Entry Vector (OE) Collider position (OP) Collider Depth (OP )
Layers Feedback Layer
Layers - Combined
Interactable Objects The layers are used to initiate or to end and interaction The objects themselves determine what that entails:
Layers and objects - combined
Conclusion Physics-Based Interaction Uses collision of the physics engine Can we interact AND do we want to? More realistic than Traditional VR Interaction Works best with some form of tactile feedback
Extensions Mesh Deformation Hand model that does not not pass though objects Gestures as a replacement for buttons
www.senseglove.com max@senseglove.com + 31 6 21 96 35 01 Making Virtual Reality Real
The Sense Glove team and our partners are working to make the first commercial Sense Glove a reality in Q2 of 2018 Core Team Ir. Gijs den Butter CEO Ir. Johannes Luijten CTO (Hardware) Ir. Max Lammers Lead developer Development Partners like
Currently, 10 Pilot customers are using prototypes of the Sense Glove to develop content where Sense Glove is vital for interaction Gert-Jan Brok InMotionVR Although it is a prototype, the Sense Glove is currently the best system available
The Sense Glove is the most accurate wearable hand capturing device, and has the ability to simulate touch.
Types of feedback Visual Texture Deformation Object Behavior Auditory Tactile
Feedback Loop
Retention rate based on teaching style
Sense Glove Pricing Developers KIT 999: Set of Sense gloves with developers KIT 299: For a wireless & battery pack For customers from 2020: 599: Set of Sense gloves (Suggested retail price)
The VR market is growing rapidly. Base case expectations are that B2B applications will make up 46% of the total $45B hardware VR market in 2025 VR & AR hardware market Market segmentation $9 Billion For Input devices (Kzero 2015) $45 Billion in 2025 (Goldman Sachs 2016) $2,25 Billion For Gloves $33,75 Billion for head mounted displays Current focus 15% Healthcare (b2b) 13% Engineering (b2b) 4% Aero & Military (b2b) 2% Education (b2b) From 2020 33% Video Games (b2c) Kzero expects that in 2018, 10M input devices for VR will be sold The remaining 33% falls outside the focus of Sense Glove and contains segments such as 3d video streaming
Sense Glove s customers can be divided into three segments: VR / AR developers, Corporates using VR / AR and research institutions & clinics