VR in TSD. Frank Bahrmann Andre Minz Sven Hellbach Hans-Joachim Böhme. HTW Dresden - Artificial Intelligence Lab. 26th November 2016

Transcription

1 VR in TSD HTW Dresden - Artificial Intelligence Lab Frank Bahrmann Andre Minz Sven Hellbach Hans-Joachim Böhme 26th November 2016

2 Agenda F. Bahrmann VR in TSD 2 / 15

3 Current Workflow Current Workflow Current Workflow 1. Implement new features, hunt old bugs 2. Drive to Museum 3. Deploy new version 4. Prepare Robot for experiments (a) Attach new/exclusive Hardware (b) Charge Batteries (c) Drive to Starting Position 5. Prepare environment for experiments (a) Wizard of Oz Workstation (b) Network Connections (c) (Prepare special exhibits) 6. Wait for visitors that are willing to cooperate with the robot / us 7. Copy Log Files and Pack Up 8. Drive to HTW 9. Analyze the data and draw conclusions 10. GOTO 1 F. Bahrmann VR in TSD 3 / 15

4 Current Workflow Current Workflow Current Workflow 1. Implement new features, hunt old bugs 2. Drive to Museum 3. Deploy new version 4. Prepare Robot for experiments (a) Attach new/exclusive Hardware (b) Charge Batteries (c) Drive to Starting Position 5. Prepare environment for experiments (a) Wizard of Oz Workstation (b) Network Connections (c) (Prepare special exhibits) 6. Wait for visitors that are willing to cooperate with the robot / us 7. Copy Log Files and Pack Up 8. Drive to HTW 9. Analyze the data and draw conclusions 10. GOTO 1 F. Bahrmann VR in TSD 4 / 15

5 Already available Already available Near Future Impact F. Bahrmann VR in TSD 5 / 15

6 Near Future Already available Near Future Impact 1. Wireless \o/ (a) GiFi - (60 Ghz-Wifi-Adapter by TPCast 1st Quarter 2017) - 8gbs (real world conditions) (b) LiFi - Visible Light Communications gbs (lab conditions) 2. Higher Resolution (a) OLED (b) 4k Mobile screens 3. Built In Eye Tracking (a) FOVE Eyetracking VR Headset available for preorder (b) SMI Mobile Eye Tracking HMD (Sensor Motoric Instruments) F. Bahrmann VR in TSD 6 / 15

7 Impact Already available Near Future Impact F. Bahrmann VR in TSD 7 / 15

8 What is done... What is done... What is to do... Greyboxing Environment with robot dummy navigational scripts (motion + path planning) Preparation of Points of Interest (later exhibits) Remodelled 3d-printer-ready CAD robot as usable 3D game object Speech synthesis with same backend as real world robot HTML5 based interactive robot display (based on awesomnium) Locomotion via armswinger/vrtk script First steps to technical photogrammetry F. Bahrmann VR in TSD 8 / 15

9 What is to do... What is done... What is to do... More steps towards an easy accessible technical photogrammetry Implement interfaces to use our real world algorithms within VR Path planning, obstacle avoidance People tracking Projection onto exhibits Refinements of existing solutions Stepwise modelling of museum environment User tests F. Bahrmann VR in TSD 9 / 15

10 Physical Properties Intrinsic Parameters I Intrinsic Parameters II F. Bahrmann VR in TSD 10 / 15

11 Physical Properties Physical Properties Intrinsic Parameters I Intrinsic Parameters II User position in space Gaze Estimation Position of Points of Interest and of robot Speech Recognition with acoustic and semantic language processing interest / emotion estimation F. Bahrmann VR in TSD 11 / 15

12 Intrinsic Parameters I Physical Properties Intrinsic Parameters I Intrinsic Parameters II Wheel of Emotions, by Robert Plutchik..., by Robin Nixon F. Bahrmann VR in TSD 12 / 15

13 Intrinsic Parameters II Physical Properties Intrinsic Parameters I Intrinsic Parameters II TiltBrush Color Picker by Google F. Bahrmann VR in TSD 13 / 15

14 Amazingly easy to create software for/in/with VR with aid of modern game engines New VR hardware can immerse people in a virtual scene Challenges: Cope with photogrammetry problems Usability Testing the usability tests Find ways to apply VR to elderly care F. Bahrmann VR in TSD 14 / 15

15 Thanks for listening and enjoy the video. F. Bahrmann VR in TSD 15 / 15