Using the Kinect and Beyond // Center for Games and Playable Media // http://games.soe.ucsc.edu John Murray John Murray Expressive Title Here (Arial) Intelligence Studio
Introduction to Interfaces User interface design (HCI) focuses on use cases and user studies Have only begun investigating 3D interfaces with the same level of rigor But have been actively researching them for the past two decades Divided into input devices, manipulation techniques, navigation techniques, and
Input Devices Joysticks 3D Mice 2D Mice Buttons Brain Input Gloves
Special Purpose Game Controllers Playstation Move Wiimote & Nunchuk Steel Battalion controller Wii Fit 360 Controller
Playstation Move Uses optical tracking to locate sphere Color distinguishes from environment Uses 6dof (accelerometer & gyroscope) to provide rotation around point Up to 4 controllers Very fast
Wiimote Began it all Uses IR LEDs in a tracker to handle exact position, but must be pointed toward sensor bar. Has accelerometer, but not gyroscope Extension adds additional resolution/detail to rotation
Steel Battalion
New wave of input devices Touche $? Leap $70 Sparkfun $400+ Emotiv EPOC neuroheadset $300
Output Augmented reality (Foresight, Google Glass) 3D TVs and Monitors getting cheaper Tactile (Vibration, haptic feedback) Heat/cold
3D MANIPULATION
3D Selection & Manipulation Nonstandard manipulation Deform Puppet Meshes 3D Manipulation Spatial Rigid Object Manipulation Selection Positioning: Rotation: Changing orientation of the object
Precision Grasp Small muscle groups (thumbs, fingers Thumbsticks on 360 and wiimote Better performance in 3d, faster, more accurate Reduce clutching Attached to hand (Kinect) Provides kinesthetic sense Uses larger muscle groups Exercise & imitates actual actions
Isomorphic vs Isometric Think thumbpad vs mouse or Kinect. Measuring force vs position Relative vs absolute mapping of rotation and transform Think its more natural, but isomorphic can be limited by our own physical limitations. We do not have extendable arms, for instance.
Selection Technique Indication of object Occlusion Object touching (list, voice set, automatic, iconic) Pointing (2d, 3d gaze, 3d hand) Indirect selection (1-1, pos->pos, vel->pos, pos->vel) Confirmation of selection (Event, gesture, voice command, no explicit command) Feedback (Text/Symbolic,Aural, visual, force/tacticle)
Manipulation Techniques Exocentric Metaphors World in miniature Automatic scaling Egocentric Metaphors Virtual Hand Metaphors Classical Go-Go Indirect Go-Go Virtual Pointer Metaphors Ray Casting Aperture Flashlight Image plane
TRAVEL AND WAYFINDING
Maneuvering Similar to search task: destination known. Need to have precise local changes to reach destination How far do you need to go? How many turns in path? Turning torse when less frequent, other method when they are frequent Visibility of target from start location
Travel Techniques Active vs Passive Active is when the user moves their viewport consciously Passive is when the user directs travel Direction/Target Selection Gaze, pointing, orientation, Stop moving Velocity/Acceleration Selection Constant, gesture, explicit, adaptive Input Conditions Constant, continuous input, start and stop inputs, automatic start/stop