3D User Interfaces. Using the Kinect and Beyond. John Murray. John Murray

Transcription

1 Using the Kinect and Beyond // Center for Games and Playable Media // John Murray John Murray Expressive Title Here (Arial) Intelligence Studio

2 Introduction to Interfaces User interface design (HCI) focuses on use cases and user studies Have only begun investigating 3D interfaces with the same level of rigor But have been actively researching them for the past two decades Divided into input devices, manipulation techniques, navigation techniques, and

3 Input Devices Joysticks 3D Mice 2D Mice Buttons Brain Input Gloves

4 Special Purpose Game Controllers Playstation Move Wiimote & Nunchuk Steel Battalion controller Wii Fit 360 Controller

5 Playstation Move Uses optical tracking to locate sphere Color distinguishes from environment Uses 6dof (accelerometer & gyroscope) to provide rotation around point Up to 4 controllers Very fast

6 Wiimote Began it all Uses IR LEDs in a tracker to handle exact position, but must be pointed toward sensor bar. Has accelerometer, but not gyroscope Extension adds additional resolution/detail to rotation

7 Steel Battalion

8 New wave of input devices Touche $? Leap $70 Sparkfun $400+ Emotiv EPOC neuroheadset $300

9 Output Augmented reality (Foresight, Google Glass) 3D TVs and Monitors getting cheaper Tactile (Vibration, haptic feedback) Heat/cold

10 3D MANIPULATION

11 3D Selection & Manipulation Nonstandard manipulation Deform Puppet Meshes 3D Manipulation Spatial Rigid Object Manipulation Selection Positioning: Rotation: Changing orientation of the object

12 Precision Grasp Small muscle groups (thumbs, fingers Thumbsticks on 360 and wiimote Better performance in 3d, faster, more accurate Reduce clutching Attached to hand (Kinect) Provides kinesthetic sense Uses larger muscle groups Exercise & imitates actual actions

13 Isomorphic vs Isometric Think thumbpad vs mouse or Kinect. Measuring force vs position Relative vs absolute mapping of rotation and transform Think its more natural, but isomorphic can be limited by our own physical limitations. We do not have extendable arms, for instance.

14 Selection Technique Indication of object Occlusion Object touching (list, voice set, automatic, iconic) Pointing (2d, 3d gaze, 3d hand) Indirect selection (1-1, pos->pos, vel->pos, pos->vel) Confirmation of selection (Event, gesture, voice command, no explicit command) Feedback (Text/Symbolic,Aural, visual, force/tacticle)

15 Manipulation Techniques Exocentric Metaphors World in miniature Automatic scaling Egocentric Metaphors Virtual Hand Metaphors Classical Go-Go Indirect Go-Go Virtual Pointer Metaphors Ray Casting Aperture Flashlight Image plane

16 TRAVEL AND WAYFINDING

17 Maneuvering Similar to search task: destination known. Need to have precise local changes to reach destination How far do you need to go? How many turns in path? Turning torse when less frequent, other method when they are frequent Visibility of target from start location

18 Travel Techniques Active vs Passive Active is when the user moves their viewport consciously Passive is when the user directs travel Direction/Target Selection Gaze, pointing, orientation, Stop moving Velocity/Acceleration Selection Constant, gesture, explicit, adaptive Input Conditions Constant, continuous input, start and stop inputs, automatic start/stop