Modeling by Hand Building a bimanual gesture based 3D user interface for Blender Tatu Harviainen Helsinki University of Technology Telecommunications Software and Multimedia Laboratory
Content 1. Background 2. Prototypes and testing with Blender 1. Virtual reality (VR) system setup 2. Blender to VR 3. Usability study 4. Building bimanual UI 3. Future work and goals
Background HandsOn research project On-going since August 2006 to the end of 2008 Participating research units Helsinki University of Technology Tampere University of Technology Helsinki University of Art and Design Funding Finnish Funding Agency for Technology and Innovation, TEKES Finnish companies ranging from Visual effects and film post production to industrial manufacturing
Background HandsOn research project Focus on studying 3D user interfaces in the context of computer-aided design and animation tasks Technologies involved Optical tracking Haptic force feedback Stereoscopic displays and VR installations Three dimensional user interfaces, interaction models and embodied interaction Analysis of design work processes
Background Research motivation Study how VR technologies can be applied in context of 3D modeling and animation Study alternative interaction models to support creative 3D design Create more intuitive and easier to learn user interfaces Increase overall understanding of 3D user interfaces
Background This presentation focuses on prototype and Blender related development VR system setup First prototype Existing 3D modeling application use in VR Usability study comparing modeling with traditional 2D desktop user interface with working in immersive VE Bimanual user interface design Iterative implementation of designed UI with Blender
VR System Setup Laboratory has previously developed a light weight CAVE-like VR system called Upponurkka (Lokki et al. 2006)
VR System Setup New, more generic rendering framework was needed for using existing software Chromium (Humphreys et al, 2002) captures OpenGL stream by replacing the OpenGL driver on PC running the application Application OpenGL OpenGL driver Chromium Render server Render server Render server Render server
VR System Setup New modules were implemented for Chromium Viewpoint modification for each rendering process according to the head tracking Keystone correction of projected images Unmodified OpenGL application can be rendered on VR system Blender required constant 3D view redrawing Interfaced Blender with optical tracking system 3D pointer device for input Additional thread for receiving tracker data Bypass the blocking event queue wait
System setup Blender to VR Application PC (WinXP / Linux) Head & 3D cursor location (UDP) Render PC 1 (Linux) Tracker Camera 1 Camera 2 Application (Blender 3D) OpenGL Chromium Render server Projector Projector Chromium (OpenGL32.dll) OpenGL + view / projection matrix Render PC 2 (Linux) Chromium Render server Projector Projector
Usability Study Compare the use of Blender with original 2D UI with the use in VR with 3D UI Automatic instrumentation to collect user performance statistics Additional visualization for the vertex locations to guide test users Results published in Intuititon 2007 conference T. Harviainen,, L. Svan,, T. Takala Usability Testing of Virtual Reality Aided Design: Framework for a Prototype Development and a Test Scenario 4 th INTUITION International Conference on Virtual Reality and Virtual Environments proceedings, 4-54 5 October 2007, Athens, Greece, ISBN- 978-960 960-254-665-9
Test Scenario Users were asked to shape a 3D object to match an object shown for reference Only direct vertex manipulation used. Shaping done by translating one vertex at a time View control In 3D UI automatically by head tracking In 2D UI with normal viewport controls Three test cases
Test Scenario
Data Collection Quantitative data All user actions collected with timestamps Times to complete each task Qualitative data Structured questionnaire Interview Small scale pilot test 6 test users with varying CAD experience
Preliminary Results 3000 Time to complete the task (s) 2500 2000 1500 1000 500 0 User 1 User 2 User 3 User 4 User 5 User 6 Task 2 2D 186,656 188,766 222,156 89,969 88,25 264,047 Task 3 2D 220,172 368,406 274,438 38,86 137,125 362,5 Task 2 3D 475,547 2447,828 664,829 228,625 609,782 111,828 Task 3 3D 373,922 677,281 415,016 172,875 248,407 146,719 Task 2 2D Task 3 2D Task 2 3D Task 3 3D
Preliminary Results Qualitative results: Overall users regarded 3D UI to be more intuitive and easier to learn Users still preferred to work on 2D UI No conclusive quantitative results can be given at this point Pilot test points out needs for improvement in our test setup Technical issues concerning tracking and stereoscopic rendering Longitudinal study to minimize the effect of previous experience on 2D UI
Building a Bimanual 3D UI Currently we are implementing a bimanual gesture based 3D UI for modeling and character animation using Blender First version of the 3D UI hopefully soon ready for preliminary testing
3D UI Design Focus on improving creative use of CAD More suitable for initial drafting phase when rough and arbitrary overall shapes are designed Overall for designing organic free-form form shapes First tools for polygonal modeling Aim to provide tools that have same expressive power as polygonal modeling tools found commonly in current 3D modeling software
3D UI Design Small set of hand gestures and hand movements used as an input Reduce the use of menus or keyboard shortcuts Context sensitive gesture commands and direct manipulation
3D UI Implementation Hand tracking interfaced with Blender 6 degrees of freedom input with both hands Bend angles for all fingers Generic UDP interface for receiving data New state management for handling bimanual operations Additional hand data polling and processing loop Concurrent modeling commands, edit modes and selections Several individual on-going operations can be executed in parallel Original 2D UI can be used in conjunction with the 3D UI
3D UI Implementation
Future Work Usability studies Iterative development (design-implementation implementation-testing) testing) of 3D UI Additional features and interaction models Sub-division surfaces Sculpting Testing with various VR system setups Provide access to 3D UI technology to wider audience We hope to spark interest also outside the project group
Thank You! contact: tatu.harviainen@tkk.fi http://handson.uiah.fi handson.uiah.fi