Introduction to Virtual Environments - Spring Wernert/Arns. Lecture 5.2 Overview of VR Development Methods

Size: px

Start display at page:

Download "Introduction to Virtual Environments - Spring Wernert/Arns. Lecture 5.2 Overview of VR Development Methods"

Blaise Andrews
5 years ago
Views:

Introduction to Virtual Environments - Spring 2004 - Wernert/Arns Lecture 5.2 Overview of VR Development Methods Outline 1. Additional Rendering Issues 2. Overview of Software Tools for VR 3.

1 Introduction to Virtual Environments - Spring Wernert/Arns Lecture 5.2 Overview of VR Development Methods Outline 1. Additional Rendering Issues 2. Overview of Software Tools for VR 3. Development Case Studies from IU's AVL 1. Additional Realtime Rendering Issues 1.1 Draw Style styles include: points (vertices), lines (edges), hidden line, faces, textured faces, faces+lines APIs/viewers that support switching between representations at runtime are helpful for debugging and analysis 1.2 Aliasing and Anti-aliasing aliasing occurs as a result of a discrete sampling and representation of a continuous world Aliasing Types: spatial aliasing - "the jaggies"; moire effects temporal aliasing - the "wagon wheel" effect color aliasing - e.g. 24-bit image on an 8-bit display Anti-aliasing methods: spatial anti-aliasing supersampling - render the scene several times from slightly different (jittered) camera locations; average the results 1 of 10

2 temporal anti-aliasing motion blur - average (blend) several consecutive frames 2 of 10

color anti-aliasing rarely a problem with most systems being 24-bit or more many high-level APIs may compress textures to save memory; may result in color artifacts in texture maps; can often force

3 color anti-aliasing rarely a problem with most systems being 24-bit or more many high-level APIs may compress textures to save memory; may result in color artifacts in texture maps; can often force higher quality in Inventor/VRML1 - texturecomplexity in range ; 0.5 default in VRML2 - most browsers have texture quality/speed option dithering methods allow for color approximation try to avoid GIF as a texture format; watch compression levels with JPEG 1.3 Gamma Correction most monitors and projectors do not have a linear response to linear input this is responsible for many of the differences you may notice between displays and operating systems gamma correction adjusts the relative intensities of the input to achieve the desired output a simple gamma-correction test image 3 of 10

1.4 Rendering Efficiencies Back-face Culling - don't draw any faces that face away from the viewer; assumes solid shapes VRML IndexedFaceSet - use Solid field to allow browser to enable back-face

4 1.4 Rendering Efficiencies Back-face Culling - don't draw any faces that face away from the viewer; assumes solid shapes VRML IndexedFaceSet - use Solid field to allow browser to enable back-face culling Visibility Culling - don't draw any objects that fall completely outside the view frustum bounding boxes & bounding spheres are easy to compute and provide a quick rejection test BSP trees - works well for interior environments - wall serve as natural dividers to segment the scene (behind vs. in front) was originally used as a hidden-surface removal technique (e.g. Doom) instead of relying on z-buffering portal techniques - use current viewpoint and room-to-room visibility information to determine when parts of the scene need to be drawn. 2. Overview of Software Tools for VR 2.1 Desirable Traits for VR Software Robust and Reliable - you don't often get a chance to make a second impression Fast - realtime performance, low latency for tracking and networking Scalable - VE complexity, simulation complexity, compute system complexity (multi-processing) Flexible/Configurable - display devices, input devices Multiple user/developer interfaces - transition from application -> development tool -> scripting -> programming Modular and Extensible General - applicable to any content area Rapid Prototyping Multi-platform - Windows, Linux, MacOS X first; other Unixes second Free - remove adoption hurdle Open-source - protect investment in learning time and content development 2.2 VR Tool Categories Modeling and Animation Tools Media Tools Graphics Libraries High-level (scene graph) Low-level (immediate-mode rendering) VR Libraries VR Display & Interaction libraries Device drivers Integrated, Very-High-Level Tools Audio libraries Haptic libraries Collaboration libraries 4 of 10

5 2.2.1 Modeling and Animation Tools Categories Commercial: 3D Studio Max, Maya, Cinema4D, FormZ, etc. Shareware/Inexpensive: AC3D (multi-platform), Free: gmax Open Source: Blender3D (multi-platform), Wings3D, Art of Illusion Issues polygonal conversion (for NURBS-based modelers) tessellation control model export formats: Inventor (VRML1), VRML2, Wavefront (.obj), 3DS (3D Studio Max) model export accuracy: materials, textures, texture coordinates, smoothing groups (normals) polygon reduction modes animation export: format (VRML2?), interpolation assumptions, etc Media Tools painting and image editting tools - for textures 3D painting tools (e.g. Alias Studio Paint 3D) allow real-time painting onto 3D objects video/movie tools - for animated textures or previews/mockups audio processing/editting - for sounds issues: file formats, adjustable quality Graphics Libraries Low-level Graphics Libraries immediate-mode graphics: address the graphics hardware more or less directly examples: OpenGL DirectX pros: more flexibility; access latest hardware features and capabilities cons: more effort in programming (maintain data structures, track states, etc.); may be inefficient use of the hardware High-level Graphics Library retained-mode or scene-graph library examples: Open Inventor - SGI (IRIX), Template Graphics (Windows, other platforms) Coin3D - Inventor work-alike from Systems in Motion OpenGL Performer - SGI - IRIX, Windows, Linux (commercial) OpenSG - German Consortium - multiplatform Java3D - Sun - Solaris, Windows OpenSceneGraph - Unix (IRIX, Solaris), Windows, Linux, OSX OpenRM - R3Vis - Unix, Windows, Linux pros: easier development with higher-level abstractions (object-oriented); API optimizes use of graphics hardware cons: may limit flexibility or access to advanced techiques or features Graphics Library Issues (tend to be Technology-driven) multi-processor support multi-pipe (graphics card) support loaders/utility libraries for specific 3D file formats image file format support advanced rendering/interaction features (e.g. particle system support, collision detection, etc.) supported platforms 5 of 10

6 2.2.4 VR Libraries VR Display & Interaction libraries tasks: display configuration and abstraction: number of screens, size, orientation, etc. setup graphics: create windows (full screen), create graphics context (stereo, buffering), input device and tracking interface/abstraction update view frustums for each frame based on head position/orientation multi-process configuration & interprocess communication synchronize multiple drawing processes support development without VR display device (simulator mode) examples: CAVElib - VRCO (out of EVL) - all platforms; OpenGL & Performer bindings; commercial VRJuggler - Iowa State - multiple platforms; OpenGL + several scene graph APIs; open-source FreeVR - CAVElib workalike from NCSA; OpenGL; all Unixes Syzergy - PC cluster library from NCSA Diverse - Virginia Tech - Irix & Linux - OpenGL & Performer bindings; open source World Toolkit - Windows, Irix, Solaris, Linux; commercial EON SDK - Windows only; commercial Device drivers tasks: support multiple input devices: wands, gloves, etc. support multiple tracking devices convert input into normalized spaces support network access to input devices examples: trackd (VRCO) VRPN (UNC) Gadgeteer (VRJuggler) Diverse Toolkit (Diverse, VaTech) Issues - (tend to be User-driven) what display(s) what platform(s) what input Very High-Level or Integrated Tools Tasks: creation & playback of VEs through an integrated suite of tools Examples: VRML (loosely integrated) EON Studio (EON Reality) World-Up (Sense8) Virtools Blender (realtime packaging model) Shockwave3D Game Engines Pros: rapid prototyping; accessible to non-programmers; nice visual effects Cons: limited platforms (usually Windows); often commercial; limited flexibility; limited scalability; development "wall" (beyond which simple things become more kludgey) Collaborative Libraries Tasks - facilitate shared virtual worlds establish network connections 6 of 10

7 ensure data consistency between sites send realtime updates of tracking, navigation, and manipulation data avatar support support persistence Examples: CAVERNsoft/QUANTA - EVL additional high-level modules built on CAVElib and Performer Octopus - Iowa State, VRJuggler - defunct? DIVE (Distributed Interactive VE)- Swedish Inst. of Computer Science - old? Audio Libraries Tasks: sounds from samples or synthesis control volume and pitch position sounds in space (localize) support client-server configuration Examples: VSS - Vanilla Sound Server - NCSA - support samples and synthesis; lots of features for sonification; client-server model bergen - EVL (Dave Pape) - sample-based only; client-server model OpenAL - open, multi-platform software interface to audio hardware Sonix - Audio layer on top of OpenAL for VRJuggler Haptics Libraries Tasks: serve as device driver for haptic output optionally, serve as driver for device input create and position forces, create a force field, or create a haptic scene graph optionally, may also create a graphical scene graph Examples: Ghost - low-level API for the Phantom e-touch 3D - open source; multiple haptic devices; integrated visual scene graph Reachin API - commercial; multiple haptic devices; integrated visual scene graph (VRML2 superset); collocated haptics and graphics 3. IU Examples & Case Studies Solar Journey NASA-sponsored collaboration between IU, U. Chicago, Adler Planetarium IU Participants: Prof. Andrew Hanson, Philip Fu, AVL Goal: to visualize the local galactic neighborhood of the Sun Deliverables: desktop editing tools desktop visualization tools movies (SIGGRAPH 2000 Theater) Adler Elumens dome, interactive VR (CAVE) non-interactive VR (7-projector dome for Beijing Planetarium) Tools used: VRML for early proof of concepts and rapid prototyping OpenGL & GLUI/GLUT for scalable, cross-platform performance of desktop tools custom OpenGL applications for movie-creation 7 of 10

CAVElib and Performer for interactive VR Virtual

Ray Burke in the Marketing Department of the Kelley

marketing and in-store technologies May 1998 - IMRA

application with multiple worlds Tools used: CAVElib

description file) Lots of free content from 3D Cafe

8 CAVElib and Performer for interactive VR Virtual Display Case Collaboration with Prof. Ray Burke in the Marketing Department of the Kelley School of Business Goal: Show prototypes of new marketing and in-store technologies May IMRA conference in Orlando Deliverable: ImmersaDesk application with multiple worlds Tools used: CAVElib and Performer for interactive VR (custom scene description file) Lots of free content from 3D Cafe Modeling tools and file format converters Image tools (screen captures and GIMP) to create billboard imposters 8 of 10

Simple MIDI audio playback for atmosphere

production Problem: Realtime was not possible

minutes of 1280x1024 content = 12 Gb AVI file!

3D Studio Max FormZ for mall post-processing

Pro to edit focus group video and create

tools to record voice-over and to edit music

9 Simple MIDI audio playback for atmosphere Performer portal tool to tie worlds together March Retail Technology Show in Chicago Deliverable: Elumens Visiondome production Problem: Realtime was not possible Solution: Use pre-rendered content (15 minutes of 1280x1024 content = 12 Gb AVI file!) Tools used: Modeling and Rendering Tools: 3D Studio Max FormZ for mall post-processing program to warp images Director and Final-Cut Pro to edit focus group video and create overlays and transitions Miscellaneous audio tools to record voice-over and to edit music soundtrack True-motion compression and playback codec for final encoding 9 of 10

10 10 of 10

LOOKING AHEAD: UE4 VR Roadmap. Nick Whiting Technical Director VR / AR

LOOKING AHEAD: UE4 VR Roadmap. Nick Whiting Technical Director VR / AR LOOKING AHEAD: UE4 VR Roadmap Nick Whiting Technical Director VR / AR HEADLINE AND IMAGE LAYOUT RECENT DEVELOPMENTS RECENT DEVELOPMENTS At Epic, we drive our engine development by creating content. We