Interactive Tables ~Avishek Anand Supervised by: Michael Kipp Chair: Vitaly Friedman
Tables of Past
Tables of Future metadesk Dialog Table Lazy Susan Luminous Table Drift Table Habitat Message Table Reactive Table Ping Pong Plus EnterTaible Tangible Viewpoints Microsoft Surface
Today s Focus STARS: A Multimodal interactive framework for Pervasive gaming applications Multi Modal Multiplayer Tabletop Gaming Interaction Techniques for a musical performance on a table top Interface
STARS: A Multimodal interactive framework for Pervasive gaming applications
Goals of the System A platform for developing computer augmented board games. Novel hardware Unifying Traditional and Computer Table Top gaming. Computing without compromising the Human centered Interaction.
STARS Components Setup The Game Table InteracTable : plasma screen with no back projection. Overhead axis camera for detection of hand gestures and extra tracking of playing objects. Integrated RF-ID antenna for pawn/token detection.
STARS Components Setup Vertical Displays Public display about game status, game logic and display. PDA and Audio Devices Administering private data of game components. Private interaction between players. Verbal commands to the game.
STARS Setup Public Display Audio Devices PDA Interactive Table
Pool of Modes Large number of modes Audio, touch, PDA, gestures. Distinction between Input and Output modes. Complex Interactions Multiple Input modes Parallel Output modes
Interaction Framework Application level translates the multimodal interactions to Interaction Requests and sends them to the Interaction Manager. These Interaction requests consist of hints using which the Interaction Manager maps them to the respective devices.
Interaction Manager Maps Interaction Requests from the higher level of the game logic to the lower level of the so called Interface Services implemented for each device. Uses a rule-based system to determine how to map an Interaction Request to one or more input or output modes.
Interface Selection by Interaction Manager
STARS Software Architecture Built on.net architecture to allow a flexible distribution of components among different computers. Multimodal inputs may be offshored to different computers for better performance.
Advantages and Takeaways Social interaction only enhances with new set of interactions surfacing. Feature rich Gaming experience with augmented Game logic and Visualization. Easier for the application developer to formulate IR in a mode independent way. Program behavior can be tuned during run time without touching any code.
MULTIMODAL MULTIPLAYER TABLETOP GAMING
Aim of the Paper Multimodal gesture and speech input benefits collaborative interaction over such a digital table Designed a multimodal, multiplayer gaming environment that allows players to interact directly atop a digital table via speech and rich whole hand gestures.
Behavioral Foundations Summarization from research to out to find effectiveness of the following multimodal ways: Gestures and Hand positions Speech and Aloud Combination of Speech and Gestures Gaze Awareness
Application of Foundations Redesign of 2 multiplayer games: Warcraft III and The Sims
Use of Hand Gestures 5-finger grabbing gesture to reach, pick up, move and place items on a surface a fist gesture mimicking the use of a physical stamp to paste object instances on the terrain
Use of Hand Gestures Selecting six friendly units within a particular region of the screen using a two-handed selection gesture. One handed panning gesture similar to how one moves a paper map on a table
Use of Hand Gestures Pointing for item selection Hand gestures coupled with speech to define other meaningful custom actions
Use of Meaningful Speech A player should be able to rapidly issue commands to the game table. Its meaning should be easily understood by other players within the context of the visual landscape and the player s gestures. Speech to low level action mapping is easy. Speech and Gestures integration should be easy.
Feedback and Gaze Feedback and feedthrough are important in reflection of the translation of the gestures. E.g. Highlighted area shown by arrows or audio feedback as Yes Master. Awareness and Gaze are instrumental in understanding the modes, actions and consequences. E.g. understand from feedback what the player interprets.
Conclusion and Advantages This paper contributes multimodal co-located tabletop interaction as a new genre of home console gaming. Gaze and awareness hugely improved and provides rich gaming insight. Single user games can be easily repurposed for different game genres on similar lines.
Interaction Techniques for a musical performance on a table top Interface
Goals of the System - AudioPad Tracking object positions and translating them into meaningful commands for the music synthesizer. Define a vocabulary for Tangible User Interfaces (TUI), by spatial rearrangement of objects on the interface. Promote visual participation of onlookers during performances.
Interaction Techniques Hardware AudioPad is a tabletop RF tracking system with video projection. The tracked objects used here are pucks. The selector puck is used to select properties and others represent individual tracks.
Hierarchical Item Browsing and The non-dominant hand holds the puck to be modified, and the dominant hand holds the modifier puck. One can also select items from the tree using one hand, since it depends only on relative positions. Selection
Floating Menus Defining the selection area of the puck where the Menu it should not move. Re-centering of the Menu around the puck when moved When should it move?
Changing Continuous Parameters The value of the parameter is determined by the distance between the puck and another master puck. Defining a transfer function: Function which maps the distance to the value being configured
Setting Two-dimensional Parameters Use of effect zones on the table where the two dimensional motion of the puck controlled the two parameters Relatively adjusting pucks (two dimensional motions of the puck ) to effect changes in twodimensional parameters.
Conclusion and Advantages Making the interaction legible to the observers. Use of relative mapping based on the positions of other pucks for setting continuous parameters. Using both absolute and relative positioning and other TUI interaction techniques this system can be scaled to other domains as well.
References: Patten, Recht, Ishii (2006) Interaction Techniques for Musical Performance with Tabletop Tangible Interfaces Magerkurth, Memisoglu, Engelke, Streitz (2004) Towards the Next Generation of Tabletop Gaming Experiences Magerkurth, Stenzel, Streitz, Neuhold (2003) A Multimodal Interaction Framework for Pervasive Game Applications. Tse et al. (2006) MULTIMODAL MULTIPLAYER TABLETOP GAMING. http://wikipedia.org/ http://social.cs.uiuc.edu/papers/
Thank You Questions Suggestions Discussions