User Interface Agents

Size: px
Start display at page:

Download "User Interface Agents"

Transcription

1 User Interface Agents Roope Raisamo Department of Computer Sciences University of Tampere

2 User Interface Agents Schiaffino and Amandi [2004]: Interface agents are computer programs that have the ability to learn a user s preferences and working habits, and (the ability) to provide him/her proactive and reactive assistance in order to increase the user s productivity.

3 User Interface Agents A user interface agent guides and helps the user Many user interface agents observe the activities of the user and suggest better ways for carrying out the same operations They can also automate a series of operations based on observing the users (e.g., Eager) Many user interface agents are based on the principles of Programming by Example (PBE)

4 VIDEOS: two examples of user interface agents Allen Cypher: Eager (2:08) Henry Lieberman: Letizia (1:36)

5 Eager automated macro generator Allen Cypher, Observes the activities of the user and tries to detect repeating sequences of actions. When such a sequence is detected, Eager offers a possibility to automate that task. like an automated macro generator this kind of functionality is still not a part of common applications, even if it could be.

6 Eager Eager observes repeating sequences of actions When Eager finds one, it jumps on the screen and suggests the next phase

7 Eager When all the phases suggested by Eager have been shown and accepted, the user can give Eager the permission to carry out the automated task.

8 Letizia a browser companion agent [Lieberman, 1997] Letizia observes the user and tries to preload interesting web pages at the same time as the user browses through the web

9 Letizia

10 Letizia Traditional browsing leads the user into doing a depth first search of the Web Letizia conducts a concurrent breadth-first search rooted from the user's current position Another example: [Marais and Bharat, 1997]

11 Questions?

12 The appearance of agents The appearance of an agent is a very important feature when a user tries to find out what a certain agent can do. It is a bad mistake to use such an appearance that makes the user believe an agent to be more intelligent than it really is. The appearance must not be disturbing.

13 Computer-generated talking heads and moving bodies one of the most demanding forms of agent presentation a human head suggests the agent to be rather intelligent a talking head probably is the most natural way to present an agent in a conversational user interface. FaceWorks

14 Drawn or animated characters the apperance of the agent has a great effect on the expectations of the user a paper clip vs. a dog vs. Merlin the Sorceror Continuously animated, slowly changing or static presentation

15 VIDEO: an example of a conversational interface agent [Cassel et al., 1999] Embodiment in Conversational Interfaces: Rea. CHI 99 Video Proceedings, (2:08)

16 Textual presentation Textual feedback of the actions of an agent Textual input should normally be avoided if it is not a part of the main task of the agent. Chatterbots e.g., Julia that is a user in a MUD (multi user dungeon) world. It can also answer to questions concerning this world. julia.html so called NPCs (non-person characters) in multiplayer role-playing computer games.

17 Auditory presentation An agent can also be presented only by voice or sound, the auditory channel: ambient sound beeps, signals melodies, music recorded speech synthetic speech

18 Haptic presentation In addition to auditory channel, or to replace it, an agent can present information by haptic feedback Haptic simulation modalities force and position tactile vibration thermal electrical

19 Haptic feedback devices Inexpensive devices: The most common haptic devices are still the different force-feedback controllers used in computer games, for example force-feedback joysticks and wheels. In 1999 Immersion Corporation s force feedback mouse was introduced as Logitech Wingman Force Feedback Gaming Mouse In 2000 Immersion Corporation s tactile feedback mouse was introduced as Logitech ifeel Tactile Feedback Mouse

20 No direct presentation at all An agent helps the user by carrying out different supporting actions e.g., prefetching needed information from the web, automatic hard disk management, An indirectly controlled background agent The question: How to implement this indirect control? Multisensory input: the agent is observing a system, an environment, or the user. Related to ubiquitous (intelligent) environments

21 Related user interface metaphors: Conversational User Interfaces Multimodal User Interfaces

22 Conversational User Interfaces Why conversation? a natural way of communication learnt at quite a young age to fix the problems of direct manipulation interfaces Conversation augments, not necessarily replaces a traditional user interface the failure of Microsoft Bob Microsoft Office Assistant

23 Microsoft Office Assistant Microsoft Office assistant tries to help in the use of Microsoft Office software with a varying rate of success. The user can choose the appearance of the agent unfortunately, this has no effect on the capabilities of the agent A paper clip is most likely a better presentation for the present assistant than Merlin the sorceror.

24 Multimodal User Interfaces Multimodal interfaces combine many simultaneous input modalities and may present the information using synergistic representation of many different output modalities.

25 Multimodal User Interfaces An agent makes use of multimodality when observing the user, e.g.: speech recognition reacts on speech commands, or observes the user without requiring actual commands machine vision, pattern recognition: recognizing facial gestures recognizing gaze direction recognizing gestures

26 Multimodal User Interfaces a specific problem in multimodal interaction is to combine the simultaneous inputs. this requires a certain amount of domain knowledge and intelligence this way every multimodal user interface is at least in some respect a user interface agent that tries to find out what the user wants based on the available information

27 A high-level architecture for multimodal user interfaces Adapted from [Maybury and Wahlster, 1998] Input processing - motor - speech - vision - Output generation - graphics - animation - speech - sound - Media analysis - language - recognition - gesture - Media design - language - modality - gesture - Interaction management - media fusion - discourse modeling - plan recognition and generation - user modeling - presentation design Application interface

28 Put That There [Bolt, 1980]

29 Combining inputs [Nigay and Coutaz, 1993] [Nigay and Coutaz, 1995]

30 Combining inputs Agents fit well in handling multimodal interaction there can be specific agents for each input and feedback channel the raw input from the lower-level input agents is then processed and combined with others by the higher-level agents working in a higher abstraction level there can be any necessary amount of agent levels in a given system finally, the root agent has all the available information from different input devices and sensors, and acts based on this information

31 Example: : DEC Smart Kiosk Smart Kiosk was a research project at Compaq-Digital (now HP) Cambridge Research Laboratory in which easy-touse information kiosks were built to be used by all people Combines new technology: machine vision, pattern recognition speech synthesis (DECtalk) speech recognition animated talking head (DECface) [Christian and Avery, 1998]

32 Example: : DEC Smart Kiosk Vision DECface Netscape Navigator Touchscreen Active vision zone

33 Example: : DEC Smart Kiosk

34 Example: : DEC Smart Kiosk

35 The Roles of Agents Fully automatic, active observation Agent results observes User collaborative and passive agent Agent user groups results User gives feedback Both active and passive gathering of information Agent results observes gives feedback User 35

36 The Roles of Agents Conversational, anthropomorphic agents talking head results conversation User Background maintaining assistant Agent System Network User observes/ adjusts Both active and passive gathering of information, collaborative Agent user groups observes gives feedback User

37 Questions?

38 References [Bolt, 1980] Richard A. Bolt, Put-that-there. SIGGRAPH 80 Conference Proceedings, ACM Press, 1980, [Christian and Avery, 1998] Andrew D. Christian and Brian L. Avery, Digital Smart Kiosk project. Human Factors in Computing Systems, CHI 98 Conference Proceedings, ACM Press, 1998, [Lieberman, 1997] Henry Lieberman, Autonomous interface agents. Human Factors in Computing Systems, CHI 97 Conference Prodeedings, ACM Press, 1997, [Maybury and Wahlster, 1998] Mark T. Maybury and Wolfgang Wahlster (Eds.), Readings in Intelligent User Interfaces. Morgan Kaufmann Publishers, [Marais and Bharat, 1997] Supporting cooperative and personal surfing with a desktop assistant. Proceedings of UIST 97, ACM Symposium on User Interface Software and Technology, ACM Press, 1997, [Nigay and Coutaz, 1993] Laurence Nigay and Joëlle Coutaz, A design space for multimodal systems: concurrent processing and data fusion. Human Factors in Computing Systems, INTERCHI 93 Conference Proceedings, ACM Press, 1993, [Nigay and Coutaz, 1995] A generic platform for addressing the multimodal challenge. Human Factors in Computing Systems, CHI 95 Conference Prodeedings, ACM Press, 1995, [Schiaffino and Amandi, 2004] Silvia Schiaffino and Analía Amandi, User interface agent interaction: personalization issues. International Journal on Human- Computer Studies 60, Elsevier Science, 2004,

Introduction to Haptics

Introduction to Haptics Introduction to Haptics Roope Raisamo Multimodal Interaction Research Group Tampere Unit for Computer Human Interaction (TAUCHI) Department of Computer Sciences University of Tampere, Finland Definition

More information

Multi-modal Human-Computer Interaction. Attila Fazekas.

Multi-modal Human-Computer Interaction. Attila Fazekas. Multi-modal Human-Computer Interaction Attila Fazekas Attila.Fazekas@inf.unideb.hu Szeged, 12 July 2007 Hungary and Debrecen Multi-modal Human-Computer Interaction - 2 Debrecen Big Church Multi-modal Human-Computer

More information

Short Course on Computational Illumination

Short Course on Computational Illumination Short Course on Computational Illumination University of Tampere August 9/10, 2012 Matthew Turk Computer Science Department and Media Arts and Technology Program University of California, Santa Barbara

More information

Multi-Modal User Interaction

Multi-Modal User Interaction Multi-Modal User Interaction Lecture 4: Multiple Modalities Zheng-Hua Tan Department of Electronic Systems Aalborg University, Denmark zt@es.aau.dk MMUI, IV, Zheng-Hua Tan 1 Outline Multimodal interface

More information

HAPTIC USER INTERFACES Final lecture

HAPTIC USER INTERFACES Final lecture HAPTIC USER INTERFACES Final lecture Roope Raisamo and Jukka Raisamo Tampere Unit for Computer-Human Interaction (TAUCHI) Department of Computer Sciences University of Tampere, Finland Passing the Course

More information

LCC 3710 Principles of Interaction Design. Readings. Sound in Interfaces. Speech Interfaces. Speech Applications. Motivation for Speech Interfaces

LCC 3710 Principles of Interaction Design. Readings. Sound in Interfaces. Speech Interfaces. Speech Applications. Motivation for Speech Interfaces LCC 3710 Principles of Interaction Design Class agenda: - Readings - Speech, Sonification, Music Readings Hermann, T., Hunt, A. (2005). "An Introduction to Interactive Sonification" in IEEE Multimedia,

More information

Perceptual Interfaces. Matthew Turk s (UCSB) and George G. Robertson s (Microsoft Research) slides on perceptual p interfaces

Perceptual Interfaces. Matthew Turk s (UCSB) and George G. Robertson s (Microsoft Research) slides on perceptual p interfaces Perceptual Interfaces Adapted from Matthew Turk s (UCSB) and George G. Robertson s (Microsoft Research) slides on perceptual p interfaces Outline Why Perceptual Interfaces? Multimodal interfaces Vision

More information

Virtual Reality Calendar Tour Guide

Virtual Reality Calendar Tour Guide Technical Disclosure Commons Defensive Publications Series October 02, 2017 Virtual Reality Calendar Tour Guide Walter Ianneo Follow this and additional works at: http://www.tdcommons.org/dpubs_series

More information

Multimodal Human- Computer Interaction: a constructive and empirical study

Multimodal Human- Computer Interaction: a constructive and empirical study Roope Raisamo Multimodal Human- Computer Interaction: a constructive and empirical study ACADEMIC DISSERTATION To be presented, with the permission of the Faculty of Economics and Administration of the

More information

HUMAN-COMPUTER INTERACTION: OVERVIEW ON STATE OF THE ART TECHNOLOGY

HUMAN-COMPUTER INTERACTION: OVERVIEW ON STATE OF THE ART TECHNOLOGY HUMAN-COMPUTER INTERACTION: OVERVIEW ON STATE OF THE ART TECHNOLOGY *Ms. S. VAISHNAVI, Assistant Professor, Sri Krishna Arts And Science College, Coimbatore. TN INDIA **SWETHASRI. L., Final Year B.Com

More information

ENHANCED HUMAN-AGENT INTERACTION: AUGMENTING INTERACTION MODELS WITH EMBODIED AGENTS BY SERAFIN BENTO. MASTER OF SCIENCE in INFORMATION SYSTEMS

ENHANCED HUMAN-AGENT INTERACTION: AUGMENTING INTERACTION MODELS WITH EMBODIED AGENTS BY SERAFIN BENTO. MASTER OF SCIENCE in INFORMATION SYSTEMS BY SERAFIN BENTO MASTER OF SCIENCE in INFORMATION SYSTEMS Edmonton, Alberta September, 2015 ABSTRACT The popularity of software agents demands for more comprehensive HAI design processes. The outcome of

More information

HUMAN-COMPUTER INTERACTION: OVERVIEW ON STATE OF THE ART

HUMAN-COMPUTER INTERACTION: OVERVIEW ON STATE OF THE ART HUMAN-COMPUTER INTERACTION: OVERVIEW ON STATE OF THE ART Author: S. VAISHNAVI Assistant Professor, Sri Krishna Arts and Science College, Coimbatore (TN) INDIA Co-Author: SWETHASRI L. III.B.Com (PA), Sri

More information

Mario Romero 2014/11/05. Multimodal Interaction and Interfaces Mixed Reality

Mario Romero 2014/11/05. Multimodal Interaction and Interfaces Mixed Reality Mario Romero 2014/11/05 Multimodal Interaction and Interfaces Mixed Reality Outline Who am I and how I can help you? What is the Visualization Studio? What is Mixed Reality? What can we do for you? What

More information

Introduction: What are the agents?

Introduction: What are the agents? Introduction: What are the agents? Roope Raisamo (rr@cs.uta.fi) Department of Computer Sciences University of Tampere http://www.cs.uta.fi/sat/ Definitions of agents The concept of agent has been used

More information

HAPTIC USER INTERFACES Final lecture

HAPTIC USER INTERFACES Final lecture HAPTIC USER INTERFACES Final lecture Roope Raisamo School of Information Sciences University of Tampere, Finland Content A little more about crossmodal interaction The next steps in the course 1 2 CROSSMODAL

More information

Definitions and Application Areas

Definitions and Application Areas Definitions and Application Areas Ambient intelligence: technology and design Fulvio Corno Politecnico di Torino, 2013/2014 http://praxis.cs.usyd.edu.au/~peterris Summary Definition(s) Application areas

More information

Interface Design V: Beyond the Desktop

Interface Design V: Beyond the Desktop Interface Design V: Beyond the Desktop Rob Procter Further Reading Dix et al., chapter 4, p. 153-161 and chapter 15. Norman, The Invisible Computer, MIT Press, 1998, chapters 4 and 15. 11/25/01 CS4: HCI

More information

Project Multimodal FooBilliard

Project Multimodal FooBilliard Project Multimodal FooBilliard adding two multimodal user interfaces to an existing 3d billiard game Dominic Sina, Paul Frischknecht, Marian Briceag, Ulzhan Kakenova March May 2015, for Future User Interfaces

More information

Pinch-the-Sky Dome: Freehand Multi-Point Interactions with Immersive Omni-Directional Data

Pinch-the-Sky Dome: Freehand Multi-Point Interactions with Immersive Omni-Directional Data Pinch-the-Sky Dome: Freehand Multi-Point Interactions with Immersive Omni-Directional Data Hrvoje Benko Microsoft Research One Microsoft Way Redmond, WA 98052 USA benko@microsoft.com Andrew D. Wilson Microsoft

More information

Universidade de Aveiro Departamento de Electrónica, Telecomunicações e Informática. Interaction in Virtual and Augmented Reality 3DUIs

Universidade de Aveiro Departamento de Electrónica, Telecomunicações e Informática. Interaction in Virtual and Augmented Reality 3DUIs Universidade de Aveiro Departamento de Electrónica, Telecomunicações e Informática Interaction in Virtual and Augmented Reality 3DUIs Realidade Virtual e Aumentada 2017/2018 Beatriz Sousa Santos Interaction

More information

Mission Space. Value-based use of augmented reality in support of critical contextual environments

Mission Space. Value-based use of augmented reality in support of critical contextual environments Mission Space Value-based use of augmented reality in support of critical contextual environments Vicki A. Barbur Ph.D. Senior Vice President and Chief Technical Officer Concurrent Technologies Corporation

More information

CS277 - Experimental Haptics Lecture 1. Introduction to Haptics

CS277 - Experimental Haptics Lecture 1. Introduction to Haptics CS277 - Experimental Haptics Lecture 1 Introduction to Haptics Haptic Interfaces Enables physical interaction with virtual objects Haptic Rendering Potential Fields Polygonal Meshes Implicit Surfaces Volumetric

More information

Craig Barnes. Previous Work. Introduction. Tools for Programming Agents

Craig Barnes. Previous Work. Introduction. Tools for Programming Agents From: AAAI Technical Report SS-00-04. Compilation copyright 2000, AAAI (www.aaai.org). All rights reserved. Visual Programming Agents for Virtual Environments Craig Barnes Electronic Visualization Lab

More information

NCCT IEEE PROJECTS ADVANCED ROBOTICS SOLUTIONS. Latest Projects, in various Domains. Promise for the Best Projects

NCCT IEEE PROJECTS ADVANCED ROBOTICS SOLUTIONS. Latest Projects, in various Domains. Promise for the Best Projects NCCT Promise for the Best Projects IEEE PROJECTS in various Domains Latest Projects, 2009-2010 ADVANCED ROBOTICS SOLUTIONS EMBEDDED SYSTEM PROJECTS Microcontrollers VLSI DSP Matlab Robotics ADVANCED ROBOTICS

More information

COMS W4172 Design Principles

COMS W4172 Design Principles COMS W4172 Design Principles Steven Feiner Department of Computer Science Columbia University New York, NY 10027 www.cs.columbia.edu/graphics/courses/csw4172 January 25, 2018 1 2D & 3D UIs: What s the

More information

Alternative Interfaces. Overview. Limitations of the Mac Interface. SMD157 Human-Computer Interaction Fall 2002

Alternative Interfaces. Overview. Limitations of the Mac Interface. SMD157 Human-Computer Interaction Fall 2002 INSTITUTIONEN FÖR SYSTEMTEKNIK LULEÅ TEKNISKA UNIVERSITET Alternative Interfaces SMD157 Human-Computer Interaction Fall 2002 Nov-27-03 SMD157, Alternate Interfaces 1 L Overview Limitation of the Mac interface

More information

Interacting within Virtual Worlds (based on talks by Greg Welch and Mark Mine)

Interacting within Virtual Worlds (based on talks by Greg Welch and Mark Mine) Interacting within Virtual Worlds (based on talks by Greg Welch and Mark Mine) Presentation Working in a virtual world Interaction principles Interaction examples Why VR in the First Place? Direct perception

More information

Exploring Passive Ambient Static Electric Field Sensing to Enhance Interaction Modalities Based on Body Motion and Activity

Exploring Passive Ambient Static Electric Field Sensing to Enhance Interaction Modalities Based on Body Motion and Activity Exploring Passive Ambient Static Electric Field Sensing to Enhance Interaction Modalities Based on Body Motion and Activity Adiyan Mujibiya The University of Tokyo adiyan@acm.org http://lab.rekimoto.org/projects/mirage-exploring-interactionmodalities-using-off-body-static-electric-field-sensing/

More information

* Intelli Robotic Wheel Chair for Specialty Operations & Physically Challenged

* Intelli Robotic Wheel Chair for Specialty Operations & Physically Challenged ADVANCED ROBOTICS SOLUTIONS * Intelli Mobile Robot for Multi Specialty Operations * Advanced Robotic Pick and Place Arm and Hand System * Automatic Color Sensing Robot using PC * AI Based Image Capturing

More information

ARMY RDT&E BUDGET ITEM JUSTIFICATION (R2 Exhibit)

ARMY RDT&E BUDGET ITEM JUSTIFICATION (R2 Exhibit) Exhibit R-2 0602308A Advanced Concepts and Simulation ARMY RDT&E BUDGET ITEM JUSTIFICATION (R2 Exhibit) FY 2005 FY 2006 FY 2007 FY 2008 FY 2009 FY 2010 FY 2011 Total Program Element (PE) Cost 22710 27416

More information

3D User Interfaces. Using the Kinect and Beyond. John Murray. John Murray

3D User Interfaces. Using the Kinect and Beyond. John Murray. John Murray Using the Kinect and Beyond // Center for Games and Playable Media // http://games.soe.ucsc.edu John Murray John Murray Expressive Title Here (Arial) Intelligence Studio Introduction to Interfaces User

More information

Virtual Environments. Ruth Aylett

Virtual Environments. Ruth Aylett Virtual Environments Ruth Aylett Aims of the course 1. To demonstrate a critical understanding of modern VE systems, evaluating the strengths and weaknesses of the current VR technologies 2. To be able

More information

Graphical User Interfaces for Blind Users: An Overview of Haptic Devices

Graphical User Interfaces for Blind Users: An Overview of Haptic Devices Graphical User Interfaces for Blind Users: An Overview of Haptic Devices Hasti Seifi, CPSC554m: Assignment 1 Abstract Graphical user interfaces greatly enhanced usability of computer systems over older

More information

Access Invaders: Developing a Universally Accessible Action Game

Access Invaders: Developing a Universally Accessible Action Game ICCHP 2006 Thursday, 13 July 2006 Access Invaders: Developing a Universally Accessible Action Game Dimitris Grammenos, Anthony Savidis, Yannis Georgalis, Constantine Stephanidis Human-Computer Interaction

More information

Keywords: Human-Building Interaction, Metaphor, Human-Computer Interaction, Interactive Architecture

Keywords: Human-Building Interaction, Metaphor, Human-Computer Interaction, Interactive Architecture Metaphor Metaphor: A tool for designing the next generation of human-building interaction Jingoog Kim 1, Mary Lou Maher 2, John Gero 3, Eric Sauda 4 1,2,3,4 University of North Carolina at Charlotte, USA

More information

CSC 2524, Fall 2017 AR/VR Interaction Interface

CSC 2524, Fall 2017 AR/VR Interaction Interface CSC 2524, Fall 2017 AR/VR Interaction Interface Karan Singh Adapted from and with thanks to Mark Billinghurst Typical Virtual Reality System HMD User Interface Input Tracking How can we Interact in VR?

More information

HAPTICS AND AUTOMOTIVE HMI

HAPTICS AND AUTOMOTIVE HMI HAPTICS AND AUTOMOTIVE HMI Technology and trends report January 2018 EXECUTIVE SUMMARY The automotive industry is on the cusp of a perfect storm of trends driving radical design change. Mary Barra (CEO

More information

Modalities for Building Relationships with Handheld Computer Agents

Modalities for Building Relationships with Handheld Computer Agents Modalities for Building Relationships with Handheld Computer Agents Timothy Bickmore Assistant Professor College of Computer and Information Science Northeastern University 360 Huntington Ave, WVH 202

More information

Human Computer Interaction Lecture 04 [ Paradigms ]

Human Computer Interaction Lecture 04 [ Paradigms ] Human Computer Interaction Lecture 04 [ Paradigms ] Imran Ihsan Assistant Professor www.imranihsan.com imranihsan.com HCIS1404 - Paradigms 1 why study paradigms Concerns how can an interactive system be

More information

The Mixed Reality Book: A New Multimedia Reading Experience

The Mixed Reality Book: A New Multimedia Reading Experience The Mixed Reality Book: A New Multimedia Reading Experience Raphaël Grasset raphael.grasset@hitlabnz.org Andreas Dünser andreas.duenser@hitlabnz.org Mark Billinghurst mark.billinghurst@hitlabnz.org Hartmut

More information

INTERACTION AND SOCIAL ISSUES IN A HUMAN-CENTERED REACTIVE ENVIRONMENT

INTERACTION AND SOCIAL ISSUES IN A HUMAN-CENTERED REACTIVE ENVIRONMENT INTERACTION AND SOCIAL ISSUES IN A HUMAN-CENTERED REACTIVE ENVIRONMENT TAYSHENG JENG, CHIA-HSUN LEE, CHI CHEN, YU-PIN MA Department of Architecture, National Cheng Kung University No. 1, University Road,

More information

Advancements in Gesture Recognition Technology

Advancements in Gesture Recognition Technology IOSR Journal of VLSI and Signal Processing (IOSR-JVSP) Volume 4, Issue 4, Ver. I (Jul-Aug. 2014), PP 01-07 e-issn: 2319 4200, p-issn No. : 2319 4197 Advancements in Gesture Recognition Technology 1 Poluka

More information

Integrated Driving Aware System in the Real-World: Sensing, Computing and Feedback

Integrated Driving Aware System in the Real-World: Sensing, Computing and Feedback Integrated Driving Aware System in the Real-World: Sensing, Computing and Feedback Jung Wook Park HCI Institute Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA, USA, 15213 jungwoop@andrew.cmu.edu

More information

DESIGN FOR INTERACTION IN INSTRUMENTED ENVIRONMENTS. Lucia Terrenghi*

DESIGN FOR INTERACTION IN INSTRUMENTED ENVIRONMENTS. Lucia Terrenghi* DESIGN FOR INTERACTION IN INSTRUMENTED ENVIRONMENTS Lucia Terrenghi* Abstract Embedding technologies into everyday life generates new contexts of mixed-reality. My research focuses on interaction techniques

More information

Arbitrating Multimodal Outputs: Using Ambient Displays as Interruptions

Arbitrating Multimodal Outputs: Using Ambient Displays as Interruptions Arbitrating Multimodal Outputs: Using Ambient Displays as Interruptions Ernesto Arroyo MIT Media Laboratory 20 Ames Street E15-313 Cambridge, MA 02139 USA earroyo@media.mit.edu Ted Selker MIT Media Laboratory

More information

ACTIVE, A PLATFORM FOR BUILDING INTELLIGENT OPERATING ROOMS

ACTIVE, A PLATFORM FOR BUILDING INTELLIGENT OPERATING ROOMS ACTIVE, A PLATFORM FOR BUILDING INTELLIGENT OPERATING ROOMS D. GUZZONI 1, C. BAUR 1, A. CHEYER 2 1 VRAI Group EPFL 1015 Lausanne Switzerland 2 AIC SRI International Menlo Park, CA USA Today computers are

More information

ISCW 2001 Tutorial. An Introduction to Augmented Reality

ISCW 2001 Tutorial. An Introduction to Augmented Reality ISCW 2001 Tutorial An Introduction to Augmented Reality Mark Billinghurst Human Interface Technology Laboratory University of Washington, Seattle grof@hitl.washington.edu Dieter Schmalstieg Technical University

More information

Vision-based User-interfaces for Pervasive Computing. CHI 2003 Tutorial Notes. Trevor Darrell Vision Interface Group MIT AI Lab

Vision-based User-interfaces for Pervasive Computing. CHI 2003 Tutorial Notes. Trevor Darrell Vision Interface Group MIT AI Lab Vision-based User-interfaces for Pervasive Computing Tutorial Notes Vision Interface Group MIT AI Lab Table of contents Biographical sketch..ii Agenda..iii Objectives.. iv Abstract..v Introduction....1

More information

Essay on A Survey of Socially Interactive Robots Authors: Terrence Fong, Illah Nourbakhsh, Kerstin Dautenhahn Summarized by: Mehwish Alam

Essay on A Survey of Socially Interactive Robots Authors: Terrence Fong, Illah Nourbakhsh, Kerstin Dautenhahn Summarized by: Mehwish Alam 1 Introduction Essay on A Survey of Socially Interactive Robots Authors: Terrence Fong, Illah Nourbakhsh, Kerstin Dautenhahn Summarized by: Mehwish Alam 1.1 Social Robots: Definition: Social robots are

More information

Dipartimento di Elettronica Informazione e Bioingegneria Robotics

Dipartimento di Elettronica Informazione e Bioingegneria Robotics Dipartimento di Elettronica Informazione e Bioingegneria Robotics Behavioral robotics @ 2014 Behaviorism behave is what organisms do Behaviorism is built on this assumption, and its goal is to promote

More information

Gameplay as On-Line Mediation Search

Gameplay as On-Line Mediation Search Gameplay as On-Line Mediation Search Justus Robertson and R. Michael Young Liquid Narrative Group Department of Computer Science North Carolina State University Raleigh, NC 27695 jjrobert@ncsu.edu, young@csc.ncsu.edu

More information

A SURVEY OF SOCIALLY INTERACTIVE ROBOTS

A SURVEY OF SOCIALLY INTERACTIVE ROBOTS A SURVEY OF SOCIALLY INTERACTIVE ROBOTS Terrence Fong, Illah Nourbakhsh, Kerstin Dautenhahn Presented By: Mehwish Alam INTRODUCTION History of Social Robots Social Robots Socially Interactive Robots Why

More information

Applying the Wizard-of-Oz Framework to Cooperative Service Discovery and Configuration

Applying the Wizard-of-Oz Framework to Cooperative Service Discovery and Configuration Applying the Wizard-of-Oz Framework to Cooperative Service Discovery and Configuration Anders Green Helge Hüttenrauch Kerstin Severinson Eklundh KTH NADA Interaction and Presentation Laboratory 100 44

More information

Direct Manipulation. and Instrumental Interaction. CS Direct Manipulation

Direct Manipulation. and Instrumental Interaction. CS Direct Manipulation Direct Manipulation and Instrumental Interaction 1 Review: Interaction vs. Interface What s the difference between user interaction and user interface? Interface refers to what the system presents to the

More information

Outline. Paradigms for interaction. Introduction. Chapter 5 : Paradigms. Introduction Paradigms for interaction (15)

Outline. Paradigms for interaction. Introduction. Chapter 5 : Paradigms. Introduction Paradigms for interaction (15) Outline 01076568 Human Computer Interaction Chapter 5 : Paradigms Introduction Paradigms for interaction (15) ดร.ชมพ น ท จ นจาคาม [kjchompo@gmail.com] สาขาว ชาว ศวกรรมคอมพ วเตอร คณะว ศวกรรมศาสตร สถาบ นเทคโนโลย

More information

CONTROLLING METHODS AND CHALLENGES OF ROBOTIC ARM

CONTROLLING METHODS AND CHALLENGES OF ROBOTIC ARM CONTROLLING METHODS AND CHALLENGES OF ROBOTIC ARM Aniket D. Kulkarni *1, Dr.Sayyad Ajij D. *2 *1(Student of E&C Department, MIT Aurangabad, India) *2(HOD of E&C department, MIT Aurangabad, India) aniket2212@gmail.com*1,

More information

Interactive Simulation: UCF EIN5255. VR Software. Audio Output. Page 4-1

Interactive Simulation: UCF EIN5255. VR Software. Audio Output. Page 4-1 VR Software Class 4 Dr. Nabil Rami http://www.simulationfirst.com/ein5255/ Audio Output Can be divided into two elements: Audio Generation Audio Presentation Page 4-1 Audio Generation A variety of audio

More information

A Brief Survey of HCI Technology. Lecture #3

A Brief Survey of HCI Technology. Lecture #3 A Brief Survey of HCI Technology Lecture #3 Agenda Evolution of HCI Technology Computer side Human side Scope of HCI 2 HCI: Historical Perspective Primitive age Charles Babbage s computer Punch card Command

More information

Heads up interaction: glasgow university multimodal research. Eve Hoggan

Heads up interaction: glasgow university multimodal research. Eve Hoggan Heads up interaction: glasgow university multimodal research Eve Hoggan www.tactons.org multimodal interaction Multimodal Interaction Group Key area of work is Multimodality A more human way to work Not

More information

With a New Helper Comes New Tasks

With a New Helper Comes New Tasks With a New Helper Comes New Tasks Mixed-Initiative Interaction for Robot-Assisted Shopping Anders Green 1 Helge Hüttenrauch 1 Cristian Bogdan 1 Kerstin Severinson Eklundh 1 1 School of Computer Science

More information

Building Spatial Experiences in the Automotive Industry

Building Spatial Experiences in the Automotive Industry Building Spatial Experiences in the Automotive Industry i-know Data-driven Business Conference Franz Weghofer franz.weghofer@magna.com Video Agenda Digital Factory - Data Backbone of all Virtual Representations

More information

Glasgow eprints Service

Glasgow eprints Service Hoggan, E.E and Brewster, S.A. (2006) Crossmodal icons for information display. In, Conference on Human Factors in Computing Systems, 22-27 April 2006, pages pp. 857-862, Montréal, Québec, Canada. http://eprints.gla.ac.uk/3269/

More information

Chapter 2 Introduction to Haptics 2.1 Definition of Haptics

Chapter 2 Introduction to Haptics 2.1 Definition of Haptics Chapter 2 Introduction to Haptics 2.1 Definition of Haptics The word haptic originates from the Greek verb hapto to touch and therefore refers to the ability to touch and manipulate objects. The haptic

More information

Advanced User Interfaces: Topics in Human-Computer Interaction

Advanced User Interfaces: Topics in Human-Computer Interaction Computer Science 425 Advanced User Interfaces: Topics in Human-Computer Interaction Week 04: Disappearing Computers 90s-00s of Human-Computer Interaction Research Prof. Roel Vertegaal, PhD Week 8: Plan

More information

Geo-Located Content in Virtual and Augmented Reality

Geo-Located Content in Virtual and Augmented Reality Technical Disclosure Commons Defensive Publications Series October 02, 2017 Geo-Located Content in Virtual and Augmented Reality Thomas Anglaret Follow this and additional works at: http://www.tdcommons.org/dpubs_series

More information

TECHNOLOGICAL COOPERATION MISSION COMPANY PARTNER SEARCH

TECHNOLOGICAL COOPERATION MISSION COMPANY PARTNER SEARCH TECHNOLOGICAL COOPERATION MISSION COMPANY PARTNER SEARCH The information you are about to provide in this form will be distributed among GERMAN companies matching your company profile and that might be

More information

Development of a telepresence agent

Development of a telepresence agent Author: Chung-Chen Tsai, Yeh-Liang Hsu (2001-04-06); recommended: Yeh-Liang Hsu (2001-04-06); last updated: Yeh-Liang Hsu (2004-03-23). Note: This paper was first presented at. The revised paper was presented

More information

EXPERIENTIAL MEDIA SYSTEMS

EXPERIENTIAL MEDIA SYSTEMS EXPERIENTIAL MEDIA SYSTEMS Hari Sundaram and Thanassis Rikakis Arts Media and Engineering Program Arizona State University, Tempe, AZ, USA Our civilization is currently undergoing major changes. Traditionally,

More information

Prospective Teleautonomy For EOD Operations

Prospective Teleautonomy For EOD Operations Perception and task guidance Perceived world model & intent Prospective Teleautonomy For EOD Operations Prof. Seth Teller Electrical Engineering and Computer Science Department Computer Science and Artificial

More information

VMD: Biomolecular Visualization and Analysis

VMD: Biomolecular Visualization and Analysis VMD: Biomolecular Visualization and Analysis John E. Stone Beckman Institute University of Illinois VMD Highlights Available on all major platforms. Displays large biomolecules and simulation trajectories

More information

Interactive Tables. ~Avishek Anand Supervised by: Michael Kipp Chair: Vitaly Friedman

Interactive Tables. ~Avishek Anand Supervised by: Michael Kipp Chair: Vitaly Friedman Interactive Tables ~Avishek Anand Supervised by: Michael Kipp Chair: Vitaly Friedman Tables of Past Tables of Future metadesk Dialog Table Lazy Susan Luminous Table Drift Table Habitat Message Table Reactive

More information

CSE 165: 3D User Interaction. Lecture #14: 3D UI Design

CSE 165: 3D User Interaction. Lecture #14: 3D UI Design CSE 165: 3D User Interaction Lecture #14: 3D UI Design 2 Announcements Homework 3 due tomorrow 2pm Monday: midterm discussion Next Thursday: midterm exam 3D UI Design Strategies 3 4 Thus far 3DUI hardware

More information

Perception in Immersive Virtual Reality Environments ROB ALLISON DEPT. OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE YORK UNIVERSITY, TORONTO

Perception in Immersive Virtual Reality Environments ROB ALLISON DEPT. OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE YORK UNIVERSITY, TORONTO Perception in Immersive Virtual Reality Environments ROB ALLISON DEPT. OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE YORK UNIVERSITY, TORONTO Overview Basic concepts and ideas of virtual environments

More information

ACTIVE, A PLATFORM FOR BUILDING INTELLIGENT SOFTWARE

ACTIVE, A PLATFORM FOR BUILDING INTELLIGENT SOFTWARE ACTIVE, A PLATFORM FOR BUILDING INTELLIGENT SOFTWARE Didier Guzzoni Robotics Systems Lab (LSRO2) Swiss Federal Institute of Technology (EPFL) CH-1015, Lausanne, Switzerland email: didier.guzzoni@epfl.ch

More information

NICE: Combining Constructionism, Narrative, and Collaboration in a Virtual Learning Environment

NICE: Combining Constructionism, Narrative, and Collaboration in a Virtual Learning Environment In Computer Graphics Vol. 31 Num. 3 August 1997, pp. 62-63, ACM SIGGRAPH. NICE: Combining Constructionism, Narrative, and Collaboration in a Virtual Learning Environment Maria Roussos, Andrew E. Johnson,

More information

Multi-modal System Architecture for Serious Gaming

Multi-modal System Architecture for Serious Gaming Multi-modal System Architecture for Serious Gaming Otilia Kocsis, Todor Ganchev, Iosif Mporas, George Papadopoulos, Nikos Fakotakis Artificial Intelligence Group, Wire Communications Laboratory, Dept.

More information

From Conversational Tooltips to Grounded Discourse: Head Pose Tracking in Interactive Dialog Systems

From Conversational Tooltips to Grounded Discourse: Head Pose Tracking in Interactive Dialog Systems From Conversational Tooltips to Grounded Discourse: Head Pose Tracking in Interactive Dialog Systems Louis-Philippe Morency Computer Science and Artificial Intelligence Laboratory at MIT Cambridge, MA

More information

Multimodal Interaction and Proactive Computing

Multimodal Interaction and Proactive Computing Multimodal Interaction and Proactive Computing Stephen A Brewster Glasgow Interactive Systems Group Department of Computing Science University of Glasgow, Glasgow, G12 8QQ, UK E-mail: stephen@dcs.gla.ac.uk

More information

PROJECT FINAL REPORT

PROJECT FINAL REPORT PROJECT FINAL REPORT Grant Agreement number: 299408 Project acronym: MACAS Project title: Multi-Modal and Cognition-Aware Systems Funding Scheme: FP7-PEOPLE-2011-IEF Period covered: from 04/2012 to 01/2013

More information

Lecture Overview. c D. Poole and A. Mackworth 2017 Artificial Intelligence, Lecture 1.1, Page 1 1 / 15

Lecture Overview. c D. Poole and A. Mackworth 2017 Artificial Intelligence, Lecture 1.1, Page 1 1 / 15 Lecture Overview What is Artificial Intelligence? Agents acting in an environment Learning objectives: at the end of the class, you should be able to describe what an intelligent agent is identify the

More information

Guidelines for choosing VR Devices from Interaction Techniques

Guidelines for choosing VR Devices from Interaction Techniques Guidelines for choosing VR Devices from Interaction Techniques Jaime Ramírez Computer Science School Technical University of Madrid Campus de Montegancedo. Boadilla del Monte. Madrid Spain http://decoroso.ls.fi.upm.es

More information

DESIGN AGENTS IN VIRTUAL WORLDS. A User-centred Virtual Architecture Agent. 1. Introduction

DESIGN AGENTS IN VIRTUAL WORLDS. A User-centred Virtual Architecture Agent. 1. Introduction DESIGN GENTS IN VIRTUL WORLDS User-centred Virtual rchitecture gent MRY LOU MHER, NING GU Key Centre of Design Computing and Cognition Department of rchitectural and Design Science University of Sydney,

More information

Definitions of Ambient Intelligence

Definitions of Ambient Intelligence Definitions of Ambient Intelligence 01QZP Ambient intelligence Fulvio Corno Politecnico di Torino, 2017/2018 http://praxis.cs.usyd.edu.au/~peterris Summary Technology trends Definition(s) Requested features

More information

Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms

Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms Published in the Proceedings of CHI '97 Hiroshi Ishii and Brygg Ullmer MIT Media Laboratory Tangible Media Group 20 Ames Street,

More information

Distributed Robotics: Building an environment for digital cooperation. Artificial Intelligence series

Distributed Robotics: Building an environment for digital cooperation. Artificial Intelligence series Distributed Robotics: Building an environment for digital cooperation Artificial Intelligence series Distributed Robotics March 2018 02 From programmable machines to intelligent agents Robots, from the

More information

Haptics Technologies: Bringing Touch to Multimedia

Haptics Technologies: Bringing Touch to Multimedia Haptics Technologies: Bringing Touch to Multimedia C2: Haptics Applications Outline Haptic Evolution: from Psychophysics to Multimedia Haptics for Medical Applications Surgical Simulations Stroke-based

More information

Drumtastic: Haptic Guidance for Polyrhythmic Drumming Practice

Drumtastic: Haptic Guidance for Polyrhythmic Drumming Practice Drumtastic: Haptic Guidance for Polyrhythmic Drumming Practice ABSTRACT W e present Drumtastic, an application where the user interacts with two Novint Falcon haptic devices to play virtual drums. The

More information

Public Displays of Affect: Deploying Relational Agents in Public Spaces

Public Displays of Affect: Deploying Relational Agents in Public Spaces Public Displays of Affect: Deploying Relational Agents in Public Spaces Timothy Bickmore Laura Pfeifer Daniel Schulman Sepalika Perera Chaamari Senanayake Ishraque Nazmi Northeastern University College

More information

SMART EXPOSITION ROOMS: THE AMBIENT INTELLIGENCE VIEW 1

SMART EXPOSITION ROOMS: THE AMBIENT INTELLIGENCE VIEW 1 SMART EXPOSITION ROOMS: THE AMBIENT INTELLIGENCE VIEW 1 Anton Nijholt, University of Twente Centre of Telematics and Information Technology (CTIT) PO Box 217, 7500 AE Enschede, the Netherlands anijholt@cs.utwente.nl

More information

GestureCommander: Continuous Touch-based Gesture Prediction

GestureCommander: Continuous Touch-based Gesture Prediction GestureCommander: Continuous Touch-based Gesture Prediction George Lucchese george lucchese@tamu.edu Jimmy Ho jimmyho@tamu.edu Tracy Hammond hammond@cs.tamu.edu Martin Field martin.field@gmail.com Ricardo

More information

Phantom-X. Unnur Gretarsdottir, Federico Barbagli and Kenneth Salisbury

Phantom-X. Unnur Gretarsdottir, Federico Barbagli and Kenneth Salisbury Phantom-X Unnur Gretarsdottir, Federico Barbagli and Kenneth Salisbury Computer Science Department, Stanford University, Stanford CA 94305, USA, [ unnurg, barbagli, jks ] @stanford.edu Abstract. This paper

More information

Babak Ziraknejad Design Machine Group University of Washington. eframe! An Interactive Projected Family Wall Frame

Babak Ziraknejad Design Machine Group University of Washington. eframe! An Interactive Projected Family Wall Frame Babak Ziraknejad Design Machine Group University of Washington eframe! An Interactive Projected Family Wall Frame Overview: Previous Projects Objective, Goals, and Motivation Introduction eframe Concept

More information

Design and evaluation of Hapticons for enriched Instant Messaging

Design and evaluation of Hapticons for enriched Instant Messaging Design and evaluation of Hapticons for enriched Instant Messaging Loy Rovers and Harm van Essen Designed Intelligence Group, Department of Industrial Design Eindhoven University of Technology, The Netherlands

More information

COMMUNICATING WITH TEAMS OF COOPERATIVE ROBOTS

COMMUNICATING WITH TEAMS OF COOPERATIVE ROBOTS COMMUNICATING WITH TEAMS OF COOPERATIVE ROBOTS D. Perzanowski, A.C. Schultz, W. Adams, M. Bugajska, E. Marsh, G. Trafton, and D. Brock Codes 5512, 5513, and 5515, Naval Research Laboratory, Washington,

More information

Investigating Phicon Feedback in Non- Visual Tangible User Interfaces

Investigating Phicon Feedback in Non- Visual Tangible User Interfaces Investigating Phicon Feedback in Non- Visual Tangible User Interfaces David McGookin and Stephen Brewster Glasgow Interactive Systems Group School of Computing Science University of Glasgow Glasgow, G12

More information

INDE/TC 455: User Interface Design

INDE/TC 455: User Interface Design INDE/TC 455: User Interface Design Module 13.0 Interface Technology 1 Three more interface considerations What is the best allocation of responsibility between the human and the tool? What is the best

More information

Virtual Chromatic Percussions Simulated by Pseudo-Haptic and Vibrotactile Feedback

Virtual Chromatic Percussions Simulated by Pseudo-Haptic and Vibrotactile Feedback Virtual Chromatic Percussions Simulated by Pseudo-Haptic and Vibrotactile Feedback Taku Hachisu The University of Electro- Communications 1-5-1 Chofugaoka, Chofu, Tokyo 182-8585, Japan +81 42 443 5363

More information

User Interface Software Projects

User Interface Software Projects User Interface Software Projects Assoc. Professor Donald J. Patterson INF 134 Winter 2012 The author of this work license copyright to it according to the Creative Commons Attribution-Noncommercial-Share

More information

Sensors & Systems for Human Safety Assurance in Collaborative Exploration

Sensors & Systems for Human Safety Assurance in Collaborative Exploration Sensing and Sensors CMU SCS RI 16-722 S09 Ned Fox nfox@andrew.cmu.edu Outline What is collaborative exploration? Humans sensing robots Robots sensing humans Overseers sensing both Inherently safe systems

More information

MIN-Fakultät Fachbereich Informatik. Universität Hamburg. Socially interactive robots. Christine Upadek. 29 November Christine Upadek 1

MIN-Fakultät Fachbereich Informatik. Universität Hamburg. Socially interactive robots. Christine Upadek. 29 November Christine Upadek 1 Christine Upadek 29 November 2010 Christine Upadek 1 Outline Emotions Kismet - a sociable robot Outlook Christine Upadek 2 Denition Social robots are embodied agents that are part of a heterogeneous group:

More information