Building a bimanual gesture based 3D user interface for Blender

Size: px

Start display at page:

Download "Building a bimanual gesture based 3D user interface for Blender"

Colleen Houston
5 years ago
Views:

1 Modeling by Hand Building a bimanual gesture based 3D user interface for Blender Tatu Harviainen Helsinki University of Technology Telecommunications Software and Multimedia Laboratory

2 Content 1. Background 2. Prototypes and testing with Blender 1. Virtual reality (VR) system setup 2. Blender to VR 3. Usability study 4. Building bimanual UI 3. Future work and goals

3 Background HandsOn research project On-going since August 2006 to the end of 2008 Participating research units Helsinki University of Technology Tampere University of Technology Helsinki University of Art and Design Funding Finnish Funding Agency for Technology and Innovation, TEKES Finnish companies ranging from Visual effects and film post production to industrial manufacturing

4 Background HandsOn research project Focus on studying 3D user interfaces in the context of computer-aided design and animation tasks Technologies involved Optical tracking Haptic force feedback Stereoscopic displays and VR installations Three dimensional user interfaces, interaction models and embodied interaction Analysis of design work processes

5 Background Research motivation Study how VR technologies can be applied in context of 3D modeling and animation Study alternative interaction models to support creative 3D design Create more intuitive and easier to learn user interfaces Increase overall understanding of 3D user interfaces

6 Background This presentation focuses on prototype and Blender related development VR system setup First prototype Existing 3D modeling application use in VR Usability study comparing modeling with traditional 2D desktop user interface with working in immersive VE Bimanual user interface design Iterative implementation of designed UI with Blender

7 VR System Setup Laboratory has previously developed a light weight CAVE-like VR system called Upponurkka (Lokki et al. 2006)

8 VR System Setup New, more generic rendering framework was needed for using existing software Chromium (Humphreys et al, 2002) captures OpenGL stream by replacing the OpenGL driver on PC running the application Application OpenGL OpenGL driver Chromium Render server Render server Render server Render server

9 VR System Setup New modules were implemented for Chromium Viewpoint modification for each rendering process according to the head tracking Keystone correction of projected images Unmodified OpenGL application can be rendered on VR system Blender required constant 3D view redrawing Interfaced Blender with optical tracking system 3D pointer device for input Additional thread for receiving tracker data Bypass the blocking event queue wait

10 System setup Blender to VR Application PC (WinXP / Linux) Head & 3D cursor location (UDP) Render PC 1 (Linux) Tracker Camera 1 Camera 2 Application (Blender 3D) OpenGL Chromium Render server Projector Projector Chromium (OpenGL32.dll) OpenGL + view / projection matrix Render PC 2 (Linux) Chromium Render server Projector Projector

11 Usability Study Compare the use of Blender with original 2D UI with the use in VR with 3D UI Automatic instrumentation to collect user performance statistics Additional visualization for the vertex locations to guide test users Results published in Intuititon 2007 conference T. Harviainen,, L. Svan,, T. Takala Usability Testing of Virtual Reality Aided Design: Framework for a Prototype Development and a Test Scenario 4 th INTUITION International Conference on Virtual Reality and Virtual Environments proceedings, October 2007, Athens, Greece, ISBN

12 Test Scenario Users were asked to shape a 3D object to match an object shown for reference Only direct vertex manipulation used. Shaping done by translating one vertex at a time View control In 3D UI automatically by head tracking In 2D UI with normal viewport controls Three test cases

13 Test Scenario

14 Data Collection Quantitative data All user actions collected with timestamps Times to complete each task Qualitative data Structured questionnaire Interview Small scale pilot test 6 test users with varying CAD experience

15 Preliminary Results 3000 Time to complete the task (s) User 1 User 2 User 3 User 4 User 5 User 6 Task 2 2D 186, , ,156 89,969 88,25 264,047 Task 3 2D 220, , ,438 38,86 137, ,5 Task 2 3D 475, , , , , ,828 Task 3 3D 373, , , , , ,719 Task 2 2D Task 3 2D Task 2 3D Task 3 3D

16 Preliminary Results Qualitative results: Overall users regarded 3D UI to be more intuitive and easier to learn Users still preferred to work on 2D UI No conclusive quantitative results can be given at this point Pilot test points out needs for improvement in our test setup Technical issues concerning tracking and stereoscopic rendering Longitudinal study to minimize the effect of previous experience on 2D UI

17 Building a Bimanual 3D UI Currently we are implementing a bimanual gesture based 3D UI for modeling and character animation using Blender First version of the 3D UI hopefully soon ready for preliminary testing

18 3D UI Design Focus on improving creative use of CAD More suitable for initial drafting phase when rough and arbitrary overall shapes are designed Overall for designing organic free-form form shapes First tools for polygonal modeling Aim to provide tools that have same expressive power as polygonal modeling tools found commonly in current 3D modeling software

19 3D UI Design Small set of hand gestures and hand movements used as an input Reduce the use of menus or keyboard shortcuts Context sensitive gesture commands and direct manipulation

20 3D UI Implementation Hand tracking interfaced with Blender 6 degrees of freedom input with both hands Bend angles for all fingers Generic UDP interface for receiving data New state management for handling bimanual operations Additional hand data polling and processing loop Concurrent modeling commands, edit modes and selections Several individual on-going operations can be executed in parallel Original 2D UI can be used in conjunction with the 3D UI

21 3D UI Implementation

22 Future Work Usability studies Iterative development (design-implementation implementation-testing) testing) of 3D UI Additional features and interaction models Sub-division surfaces Sculpting Testing with various VR system setups Provide access to 3D UI technology to wider audience We hope to spark interest also outside the project group

23 Thank You! contact: handson.uiah.fi

6 System architecture

6 System architecture is an application for interactively controlling the animation of VRML avatars. It uses the pen interaction technique described in Chapter 3 - Interaction technique. It is used in