Alex Olwal, Ph.D. Interaction Technology, Interfaces and The End of Reality

Size: px
Start display at page:

Download "Alex Olwal, Ph.D. Interaction Technology, Interfaces and The End of Reality"

Transcription

1 Alex Olwal, Ph.D. Interaction Technology, Interfaces and The End of Reality

2 Alex Olwal, Ph.D. Human computer interaction Interaction technologies & techniques Augmented reality Medical & health applications

3 Alex Olwal, Ph.D. MIT Media Lab Cambridge, MA Columbia University New York, NY Royal Institute of Technology Stockholm, Sweden Microsoft Research Redmond, WA University of California Santa Barbara, CA Rhode Island School of Design Providence, RI

4

5 The Ultimate Display & The Sword of Damocles (Sutherland)

6 The Ultimate Display & The Sword of Damocles (Sutherland)

7 UX = SENSING DISPLAY INTERACTION 1 Unobtrusive 2 Seamless display 3 Embedded sensing 4 Hybrid & symbiotic Augmented Reality, Ubiquitous Computing, Spatially Aware Handhelds, Interactive Surfaces,

8 ASTOR Transparent window with autostereoscopic 3D overlays SPIE 2008 [Olwal, Gustafsson& Lindfors] ISMAR 2005 [Olwal, Lindfors, Gustafsson, Kjellberg& Mattson] SIGGRAPH 2004 Sketches [Olwal, Lindfors& Gustafsson]

9 ASTOR Multi-view display through HOE

10 ASTOR Flexible configuration

11

12 Process data Workspace

13 Workspace + Process data

14

15

16 SpaceTop Hybrid 2D + 3D workspaces CHI 2013 [Lee, Olwal, Boulanger& Ishii]

17 Immaterial displays Face-to-face, reach-through & walk-through interaction 3DTV Book 2008 [DiVerdi, Olwal, Rakkolainen& Höllerer] VR 2006 [Olwal, DiVerdi, Rakkolainen& Höllerer] SIGGRAPH 2005 Emerging Technologies [Rakkolainen, DiVerdi, Olwal, Candussi& Höllerer]

18 SurfaceFusion Sensing & augmenting everyday objects GI 2008 [Olwal& Wilson]

19

20 Temporal fusion Computer vision + RFID

21 sum(a) >> sum(d) sum(d) >> sum(a) sum(a) sum(d) added removed move

22 Spatially aware handhelds Enhancing interaction with large displays TEI 2009 [Olwal& Feiner] INTERACT 2009 [Olwal]

23

24

25 SpeckleSense: 2D + 3D optical motion sensing Fast, precise, low-cost & compact UIST 2011 [Zizka, Olwal& Raskar]

26 SpeckleSense: Optical motion sensing Fast (10000 fps) Precise (50 µm) Lensless Tiny (1 2 mm) Low power Minimal computation minimal latency

27 Handheld Speckle Projector Handheld Speckle Sensor Reflection Diffuser Self-tracked Device Speckle Tracking of Hands

28 SpeckleEye: Gesturalinteraction for Ubicomp Open source software + hardware platform for embedded electronics CHI 2012 Extended Abstracts [Olwal, Bardagjy, Zizka& Raskar]

29 Sublimate Switchable physical / virtual rendering CHI 2013 [Leithinger, Follmer, Olwal, Luescher, Hogge, Lee & Ishii]

30

31

32 Alex Olwal, Ph.D. UX = Sensing display interaction Unobtrusive Embedded sensing Seamless display Hybrid & symbiotic

33 The End of Reality (?) Alex Olwal olwal.com Jamie Zigelbaum jamiezigelbaum.com

34

35 Cambridge Jake Bernstein: Ph.D. candidate, MIT Media Lab Eric Gunther: Founder, Sosolimited Lauren McCarthy: Designer, programmer, artist Dan Paluska: Artist, roboticist Nadya Peek: Ph.D. candidate, MIT Center for Bits and Atoms David Robert: Ph.D. candidate, MIT Media Lab New York Christine Creamer: Product developer Amanda Parks: Media designer & technologist James Patten: Founder, Patten Studio Marko Tandefelt: Director of technology & research, Eyebeam Richard The: Senior designer, Google Creative Lab

36

37

38 Expectations of cognitive capabilities

39 Moore s law Buxton s law Man s law Technology Features Human capacity

40

41

42 Why are remote people so much more important?

43

44

45

46

47 Part man. Part machine. All cop.

48 Now vs. Later Local vs. Remote Present vs. Absent Focused vs. Distracted Unified vs. Fractured

49 Alex Olwal, Ph.D. UX = Sensing display interaction human Unobtrusive Embedded sensing Seamless display Hybrid & symbiotic

50 olwal.com

51 Alex Olwal, Ph.D. UX = Sensing display interaction human Unobtrusive Embedded sensing Seamless display Hybrid & symbiotic

52 TANGIBLE UIS& MOBILE AR SurfaceFusion RFID + vision ID + track everyday objects LightSense Dynamic augmentation of printed media GI 2008 [Olwal& Wilson] ISMAR 2006 [Olwal] LUMAR Hybrid 2D + 3D handheld AR Hybrid surface interaction Distributed interaction with large displays ICAT 2007 [Olwal& Henrysson] INTERACT 2009 [Olwal]

53 HYBRID UIS& INTERACTION TECHNIQUES Spatially aware handhelds Mobile devices expanded capabilities Collaborative interfaces Multiple users, devices & locations TEI 2009 [Olwal& Feiner] INTERACT 2011 [Olwal, Frykholm, Groth& Moll] Touch-screen techniques Minimal gestures for precise interaction 3D interaction Mobile, gestures, touch, eye tracking, CHI 2008 [Olwal, Feiner& Heyman] NordiCHI 2008 [Olwal]

54 ACTUATION & SENSING Jamming user interfaces Stiffness & sensing for malleable devices Multimodal motion guidance Dynamic tactile feedback for motor training UIST 2012 Best paper [Follmer, Leithinger, Olwal, Cheng & Ishii] SpeckleSense 2D + 3D optical motion sensing ICMI 2012 [Schönauer, Fukushi, Olwal, Kaufmann, & Raskar] SpeckleSense/ SpeckleEye Embedded motion sensing & gestural interaction UIST 2011 [Zizka, Olwal& Raskar] CHI 2012 Extended Abstracts [Olwal, Bardagjy, Zizka& Raskar]

55 MEDICAL & HEALTH Eye tracking for radiology 2D tracking 3D visualization 3D interaction with 2D X-ray imagery Spatiotemporal visualizations to minimize radiation In progress [Olwal, Drew & Wolfe] Mobile eye exams Portable cataract screening & retinal imaging CBMS 2011 [Ioakeimidou, Olwal, Nordberg & Holst] Low-cost telemedicine for prosthetics Real-time collaboration for remote advice SIGGRAPH 2012 Talks [Lawson, Boggess, Khullar, Olwal, Wetzstein& Raskar] MobileHCI workshop 2011 [Sengeh & Olwal]

56 TACTAM (company) Exhibit Surface Multi-touch + RFID Hybrid Surface Sensing & interaction for objects & mobile devices Museum of Naval History, Stockholm Expressive weather Robotic façade display with weather sayings Ericsson: HQ, Mobile World Congress, Volvo Ocean Race, PeopleBandit Oversized slot machine remixes 1000 locals Tom Tits Experiment Lunagallerian

SpeckleEye: Gestural Interaction for Embedded Electronics in Ubiquitous Computing

SpeckleEye: Gestural Interaction for Embedded Electronics in Ubiquitous Computing SpeckleEye: Gestural Interaction for Embedded Electronics in Ubiquitous Computing Alex Olwal MIT Media Lab, 75 Amherst St, Cambridge, MA olwal@media.mit.edu Andy Bardagjy MIT Media Lab, 75 Amherst St,

More information

Mario Romero 2014/11/05. Multimodal Interaction and Interfaces Mixed Reality

Mario Romero 2014/11/05. Multimodal Interaction and Interfaces Mixed Reality Mario Romero 2014/11/05 Multimodal Interaction and Interfaces Mixed Reality Outline Who am I and how I can help you? What is the Visualization Studio? What is Mixed Reality? What can we do for you? What

More information

Unobtrusive Augmentation of Physical Environments

Unobtrusive Augmentation of Physical Environments Unobtrusive Augmentation of Physical Environments Interaction Techniques, Spatial Displays and Ubiquitous Sensing ALEX OLWAL Avhandling som med tillstånd av Kungliga Tekniska högskolan framlägges till

More information

ISCW 2001 Tutorial. An Introduction to Augmented Reality

ISCW 2001 Tutorial. An Introduction to Augmented Reality ISCW 2001 Tutorial An Introduction to Augmented Reality Mark Billinghurst Human Interface Technology Laboratory University of Washington, Seattle grof@hitl.washington.edu Dieter Schmalstieg Technical University

More information

Short Course on Computational Illumination

Short Course on Computational Illumination Short Course on Computational Illumination University of Tampere August 9/10, 2012 Matthew Turk Computer Science Department and Media Arts and Technology Program University of California, Santa Barbara

More information

Meaning, Mapping & Correspondence in Tangible User Interfaces

Meaning, Mapping & Correspondence in Tangible User Interfaces Meaning, Mapping & Correspondence in Tangible User Interfaces CHI '07 Workshop on Tangible User Interfaces in Context & Theory Darren Edge Rainbow Group Computer Laboratory University of Cambridge A Solid

More information

Beyond: collapsible tools and gestures for computational design

Beyond: collapsible tools and gestures for computational design Beyond: collapsible tools and gestures for computational design The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation As Published

More information

Ubiquitous Computing MICHAEL BERNSTEIN CS 376

Ubiquitous Computing MICHAEL BERNSTEIN CS 376 Ubiquitous Computing MICHAEL BERNSTEIN CS 376 Reminders First critiques were due last night Idea Generation (Round One) due next Friday, with a team Next week: Social computing Design and creation Clarification

More information

Augmented Reality And Ubiquitous Computing using HCI

Augmented Reality And Ubiquitous Computing using HCI Augmented Reality And Ubiquitous Computing using HCI Ashmit Kolli MS in Data Science Michigan Technological University CS5760 Topic Assignment 2 akolli@mtu.edu Abstract : Direct use of the hand as an input

More information

Augmented and Virtual Reality 6.S063 Engineering Interaction Technologies. Prof. Stefanie Mueller MIT CSAIL HCI Engineering Group

Augmented and Virtual Reality 6.S063 Engineering Interaction Technologies. Prof. Stefanie Mueller MIT CSAIL HCI Engineering Group Augmented and Virtual Reality 6.S063 Engineering Interaction Technologies Prof. Stefanie Mueller MIT CSAIL HCI Engineering Group AR supplements the real world VR replaces the real world mixed reality real

More information

Future Directions for Augmented Reality. Mark Billinghurst

Future Directions for Augmented Reality. Mark Billinghurst Future Directions for Augmented Reality Mark Billinghurst 1968 Sutherland/Sproull s HMD https://www.youtube.com/watch?v=ntwzxgprxag Star Wars - 1977 Augmented Reality Combines Real and Virtual Images Both

More information

Proposal - Diploma Thesis Looking Through Time

Proposal - Diploma Thesis Looking Through Time Proposal - Diploma Thesis Looking Through Time Torsten Palm June 15, 2009 1 Initial project idea The initial idea of my diploma thesis was to realize certain parts of an AR application Looking Through

More information

The Mixed Reality Book: A New Multimedia Reading Experience

The Mixed Reality Book: A New Multimedia Reading Experience The Mixed Reality Book: A New Multimedia Reading Experience Raphaël Grasset raphael.grasset@hitlabnz.org Andreas Dünser andreas.duenser@hitlabnz.org Mark Billinghurst mark.billinghurst@hitlabnz.org Hartmut

More information

Ubiquitous Computing. michael bernstein spring cs376.stanford.edu. Wednesday, April 3, 13

Ubiquitous Computing. michael bernstein spring cs376.stanford.edu. Wednesday, April 3, 13 Ubiquitous Computing michael bernstein spring 2013 cs376.stanford.edu Ubiquitous? Ubiquitous? 3 Ubicomp Vision A new way of thinking about computers in the world, one that takes into account the natural

More information

HCI Outlook: Tangible and Tabletop Interaction

HCI Outlook: Tangible and Tabletop Interaction HCI Outlook: Tangible and Tabletop Interaction multiple degree-of-freedom (DOF) input Morten Fjeld Associate Professor, Computer Science and Engineering Chalmers University of Technology Gothenburg University

More information

preface Motivation Figure 1. Reality-virtuality continuum (Milgram & Kishino, 1994) Mixed.Reality Augmented. Virtuality Real...

preface Motivation Figure 1. Reality-virtuality continuum (Milgram & Kishino, 1994) Mixed.Reality Augmented. Virtuality Real... v preface Motivation Augmented reality (AR) research aims to develop technologies that allow the real-time fusion of computer-generated digital content with the real world. Unlike virtual reality (VR)

More information

ITS '14, Nov , Dresden, Germany

ITS '14, Nov , Dresden, Germany 3D Tabletop User Interface Using Virtual Elastic Objects Figure 1: 3D Interaction with a virtual elastic object Hiroaki Tateyama Graduate School of Science and Engineering, Saitama University 255 Shimo-Okubo,

More information

Theory and Practice of Tangible User Interfaces Tuesday, Week 9

Theory and Practice of Tangible User Interfaces Tuesday, Week 9 Augmented Reality Theory and Practice of Tangible User Interfaces Tuesday, Week 9 Outline Overview Examples Theory Examples Supporting AR Designs Examples Theory Outline Overview Examples Theory Examples

More information

Portfolio. Swaroop Kumar Pal swarooppal.wordpress.com github.com/swarooppal1088

Portfolio. Swaroop Kumar Pal swarooppal.wordpress.com github.com/swarooppal1088 Portfolio About Me: I am a Computer Science graduate student at The University of Texas at Dallas. I am currently working as Augmented Reality Engineer at Aireal, Dallas and also as a Graduate Researcher

More information

mixed reality mixed reality & (tactile and) tangible interaction (tactile and) tangible interaction class housekeeping about me

mixed reality mixed reality & (tactile and) tangible interaction (tactile and) tangible interaction class housekeeping about me Mixed Reality Tangible Interaction mixed reality (tactile and) mixed reality (tactile and) Jean-Marc Vezien Jean-Marc Vezien about me Assistant prof in Paris-Sud and co-head of masters contact: anastasia.bezerianos@lri.fr

More information

AUGMENTED REALITY: PRINCIPLES AND PRACTICE (USABILITY) BY DIETER SCHMALSTIEG, TOBIAS HOLLERER

AUGMENTED REALITY: PRINCIPLES AND PRACTICE (USABILITY) BY DIETER SCHMALSTIEG, TOBIAS HOLLERER AUGMENTED REALITY: PRINCIPLES AND PRACTICE (USABILITY) BY DIETER SCHMALSTIEG, TOBIAS HOLLERER DOWNLOAD EBOOK : AUGMENTED REALITY: PRINCIPLES AND PRACTICE (USABILITY) BY DIETER SCHMALSTIEG, TOBIAS HOLLERER

More information

CSC 2524, Fall 2017 AR/VR Interaction Interface

CSC 2524, Fall 2017 AR/VR Interaction Interface CSC 2524, Fall 2017 AR/VR Interaction Interface Karan Singh Adapted from and with thanks to Mark Billinghurst Typical Virtual Reality System HMD User Interface Input Tracking How can we Interact in VR?

More information

Computer-Augmented Environments: Back to the Real World

Computer-Augmented Environments: Back to the Real World Computer-Augmented Environments: Back to the Real World Hans-W. Gellersen Lancaster University Department of Computing Ubiquitous Computing Research HWG 1 What I thought this talk would be about Back to

More information

Head Mounted Display Optics II!

Head Mounted Display Optics II! ! Head Mounted Display Optics II! Gordon Wetzstein! Stanford University! EE 267 Virtual Reality! Lecture 8! stanford.edu/class/ee267/!! Lecture Overview! focus cues & the vergence-accommodation conflict!

More information

X11 in Virtual Environments ARL

X11 in Virtual Environments ARL COMS W4172 Case Study: 3D Windows/Desktops 2 Steven Feiner Department of Computer Science Columbia University New York, NY 10027 www.cs.columbia.edu/graphics/courses/csw4172 February 8, 2018 1 X11 in Virtual

More information

EMPOWERING THE CONNECTED FIELD FORCE WORKER WITH ADVANCED ANALYTICS MATTHEW SHORT ACCENTURE LABS

EMPOWERING THE CONNECTED FIELD FORCE WORKER WITH ADVANCED ANALYTICS MATTHEW SHORT ACCENTURE LABS EMPOWERING THE CONNECTED FIELD FORCE WORKER WITH ADVANCED ANALYTICS MATTHEW SHORT ACCENTURE LABS ACCENTURE LABS DUBLIN Artificial Intelligence Security SILICON VALLEY Digital Experiences Artificial Intelligence

More information

Building Spatial Experiences in the Automotive Industry

Building Spatial Experiences in the Automotive Industry Building Spatial Experiences in the Automotive Industry i-know Data-driven Business Conference Franz Weghofer franz.weghofer@magna.com Video Agenda Digital Factory - Data Backbone of all Virtual Representations

More information

clayodor: Retrieving Scents through the Manipulation of Malleable Material

clayodor: Retrieving Scents through the Manipulation of Malleable Material clayodor: Retrieving Scents through the Manipulation of Malleable Material Cindy Hsin-Liu Kao* cindykao@media.mit.edu Ermal Dreshaj* ermal@media.mit.edu Judith Amores* amores@media.mit.edu Sang-won Leigh*

More information

Welcome, Introduction, and Roadmap Joseph J. LaViola Jr.

Welcome, Introduction, and Roadmap Joseph J. LaViola Jr. Welcome, Introduction, and Roadmap Joseph J. LaViola Jr. Welcome, Introduction, & Roadmap 3D UIs 101 3D UIs 201 User Studies and 3D UIs Guidelines for Developing 3D UIs Video Games: 3D UIs for the Masses

More information

NUI. Research Topic. Research Topic. Multi-touch TANGIBLE INTERACTION DESIGN ON MULTI-TOUCH DISPLAY. Tangible User Interface + Multi-touch

NUI. Research Topic. Research Topic. Multi-touch TANGIBLE INTERACTION DESIGN ON MULTI-TOUCH DISPLAY. Tangible User Interface + Multi-touch 1 2 Research Topic TANGIBLE INTERACTION DESIGN ON MULTI-TOUCH DISPLAY Human-Computer Interaction / Natural User Interface Neng-Hao (Jones) Yu, Assistant Professor Department of Computer Science National

More information

AUGMENTED REALITY FOR COLLABORATIVE EXPLORATION OF UNFAMILIAR ENVIRONMENTS

AUGMENTED REALITY FOR COLLABORATIVE EXPLORATION OF UNFAMILIAR ENVIRONMENTS NSF Lake Tahoe Workshop on Collaborative Virtual Reality and Visualization (CVRV 2003), October 26 28, 2003 AUGMENTED REALITY FOR COLLABORATIVE EXPLORATION OF UNFAMILIAR ENVIRONMENTS B. Bell and S. Feiner

More information

mixed reality & (tactile and) tangible interaction

mixed reality & (tactile and) tangible interaction mixed reality & (tactile and) Anastasia Bezerianos & Jean-Marc Vezien mixed reality & (tactile and) Jean-Marc Vezien & Anastasia Bezerianos Anastasia Bezerianos 1 about me Assistant prof in Paris-Sud and

More information

MIXED REALITY BENEFITS IN BUSINESS

MIXED REALITY BENEFITS IN BUSINESS MIXED REALITY BENEFITS IN BUSINESS Denise E. White Founder, Digital Nomadic Living Slide 1: Introduction Hi, Good Morning! [pause] I m Denise White. I live a Mixed Reality life, today or as I like to say,

More information

Localized Space Display

Localized Space Display Localized Space Display EE 267 Virtual Reality, Stanford University Vincent Chen & Jason Ginsberg {vschen, jasong2}@stanford.edu 1 Abstract Current virtual reality systems require expensive head-mounted

More information

Job Description. Commitment: Must be available to work full-time hours, M-F for weeks beginning Summer of 2018.

Job Description. Commitment: Must be available to work full-time hours, M-F for weeks beginning Summer of 2018. Research Intern Director of Research We are seeking a summer intern to support the team to develop prototype 3D sensing systems based on state-of-the-art sensing technologies along with computer vision

More information

Multi-Modal User Interaction

Multi-Modal User Interaction Multi-Modal User Interaction Lecture 4: Multiple Modalities Zheng-Hua Tan Department of Electronic Systems Aalborg University, Denmark zt@es.aau.dk MMUI, IV, Zheng-Hua Tan 1 Outline Multimodal interface

More information

AUGMENTED REALITY APPLICATIONS USING VISUAL TRACKING

AUGMENTED REALITY APPLICATIONS USING VISUAL TRACKING AUGMENTED REALITY APPLICATIONS USING VISUAL TRACKING ABSTRACT Chutisant Kerdvibulvech Department of Information and Communication Technology, Rangsit University, Thailand Email: chutisant.k@rsu.ac.th In

More information

INTERACTION AND SOCIAL ISSUES IN A HUMAN-CENTERED REACTIVE ENVIRONMENT

INTERACTION AND SOCIAL ISSUES IN A HUMAN-CENTERED REACTIVE ENVIRONMENT INTERACTION AND SOCIAL ISSUES IN A HUMAN-CENTERED REACTIVE ENVIRONMENT TAYSHENG JENG, CHIA-HSUN LEE, CHI CHEN, YU-PIN MA Department of Architecture, National Cheng Kung University No. 1, University Road,

More information

DepthTouch: Using Depth-Sensing Camera to Enable Freehand Interactions On and Above the Interactive Surface

DepthTouch: Using Depth-Sensing Camera to Enable Freehand Interactions On and Above the Interactive Surface DepthTouch: Using Depth-Sensing Camera to Enable Freehand Interactions On and Above the Interactive Surface Hrvoje Benko and Andrew D. Wilson Microsoft Research One Microsoft Way Redmond, WA 98052, USA

More information

Paint with Your Voice: An Interactive, Sonic Installation

Paint with Your Voice: An Interactive, Sonic Installation Paint with Your Voice: An Interactive, Sonic Installation Benjamin Böhm 1 benboehm86@gmail.com Julian Hermann 1 julian.hermann@img.fh-mainz.de Tim Rizzo 1 tim.rizzo@img.fh-mainz.de Anja Stöffler 1 anja.stoeffler@img.fh-mainz.de

More information

3D and Sequential Representations of Spatial Relationships among Photos

3D and Sequential Representations of Spatial Relationships among Photos 3D and Sequential Representations of Spatial Relationships among Photos Mahoro Anabuki Canon Development Americas, Inc. E15-349, 20 Ames Street Cambridge, MA 02139 USA mahoro@media.mit.edu Hiroshi Ishii

More information

Investigating Gestures on Elastic Tabletops

Investigating Gestures on Elastic Tabletops Investigating Gestures on Elastic Tabletops Dietrich Kammer Thomas Gründer Chair of Media Design Chair of Media Design Technische Universität DresdenTechnische Universität Dresden 01062 Dresden, Germany

More information

Mission Space. Value-based use of augmented reality in support of critical contextual environments

Mission Space. Value-based use of augmented reality in support of critical contextual environments Mission Space Value-based use of augmented reality in support of critical contextual environments Vicki A. Barbur Ph.D. Senior Vice President and Chief Technical Officer Concurrent Technologies Corporation

More information

Organic UIs in Cross-Reality Spaces

Organic UIs in Cross-Reality Spaces Organic UIs in Cross-Reality Spaces Derek Reilly Jonathan Massey OCAD University GVU Center, Georgia Tech 205 Richmond St. Toronto, ON M5V 1V6 Canada dreilly@faculty.ocad.ca ragingpotato@gatech.edu Anthony

More information

HUMAN COMPUTER INTERACTION 0. PREFACE. I-Chen Lin, National Chiao Tung University, Taiwan

HUMAN COMPUTER INTERACTION 0. PREFACE. I-Chen Lin, National Chiao Tung University, Taiwan HUMAN COMPUTER INTERACTION 0. PREFACE I-Chen Lin, National Chiao Tung University, Taiwan About The Course Course title: Human Computer Interaction (HCI) Lectures: ED202, 13:20~15:10(Mon.), 9:00~9:50(Thur.)

More information

Spatial augmented reality to enhance physical artistic creation.

Spatial augmented reality to enhance physical artistic creation. Spatial augmented reality to enhance physical artistic creation. Jérémy Laviole, Martin Hachet To cite this version: Jérémy Laviole, Martin Hachet. Spatial augmented reality to enhance physical artistic

More information

Sublimate: State-Changing Virtual and Physical Rendering to Augment Interaction with Shape Displays

Sublimate: State-Changing Virtual and Physical Rendering to Augment Interaction with Shape Displays Sublimate: State-Changing Virtual and Physical Rendering to Augment Interaction with Shape Displays Daniel Leithinger Sean Follmer Alex Olwal Samuel Luescher Akimitsu Hogge Jinha Lee Hiroshi Ishii MIT

More information

reactive boxes

reactive boxes Since June 1999, I have been working and studying in the Aesthetics and Computation Group at the MIT Media Laboratory. During this time, my work shifted from experiments in interface and information design

More information

Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms

Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms Published in the Proceedings of CHI '97 Hiroshi Ishii and Brygg Ullmer MIT Media Laboratory Tangible Media Group 20 Ames Street,

More information

! Computation embedded in the physical spaces around us. ! Ambient intelligence. ! Input in the real world. ! Output in the real world also

! Computation embedded in the physical spaces around us. ! Ambient intelligence. ! Input in the real world. ! Output in the real world also Ubicomp? Ubicomp and Physical Interaction! Computation embedded in the physical spaces around us! Ambient intelligence! Take advantage of naturally-occurring actions and activities to support people! Input

More information

VR/AR Concepts in Architecture And Available Tools

VR/AR Concepts in Architecture And Available Tools VR/AR Concepts in Architecture And Available Tools Peter Kán Interactive Media Systems Group Institute of Software Technology and Interactive Systems TU Wien Outline 1. What can you do with virtual reality

More information

Augmented and mixed reality (AR & MR)

Augmented and mixed reality (AR & MR) Augmented and mixed reality (AR & MR) Doug Bowman CS 5754 Based on original lecture notes by Ivan Poupyrev AR/MR example (C) 2008 Doug Bowman, Virginia Tech 2 Definitions Augmented reality: Refers to a

More information

COMS W4172 Design Principles

COMS W4172 Design Principles COMS W4172 Design Principles Steven Feiner Department of Computer Science Columbia University New York, NY 10027 www.cs.columbia.edu/graphics/courses/csw4172 January 25, 2018 1 2D & 3D UIs: What s the

More information

ACTUI: Using Commodity Mobile Devices to Build Active Tangible User Interfaces

ACTUI: Using Commodity Mobile Devices to Build Active Tangible User Interfaces Demonstrations ACTUI: Using Commodity Mobile Devices to Build Active Tangible User Interfaces Ming Li Computer Graphics & Multimedia Group RWTH Aachen, AhornStr. 55 52074 Aachen, Germany mingli@cs.rwth-aachen.de

More information

Advanced User Interfaces: Topics in Human-Computer Interaction

Advanced User Interfaces: Topics in Human-Computer Interaction Computer Science 425 Advanced User Interfaces: Topics in Human-Computer Interaction Week 04: Disappearing Computers 90s-00s of Human-Computer Interaction Research Prof. Roel Vertegaal, PhD Week 8: Plan

More information

Beyond Visual: Shape, Haptics and Actuation in 3D UI

Beyond Visual: Shape, Haptics and Actuation in 3D UI Beyond Visual: Shape, Haptics and Actuation in 3D UI Ivan Poupyrev Welcome, Introduction, & Roadmap 3D UIs 101 3D UIs 201 User Studies and 3D UIs Guidelines for Developing 3D UIs Video Games: 3D UIs for

More information

By: Celine, Yan Ran, Yuolmae. Image from oss

By: Celine, Yan Ran, Yuolmae. Image from oss IMMERSION By: Celine, Yan Ran, Yuolmae Image from oss Content 1. Char Davies 2. Osmose 3. The Ultimate Display, Ivan Sutherland 4. Virtual Environments, Scott Fisher Artist A Canadian contemporary artist

More information

T(ether): spatially-aware handhelds, gestures and proprioception for multi-user 3D modeling and animation

T(ether): spatially-aware handhelds, gestures and proprioception for multi-user 3D modeling and animation T(ether): spatially-aware handhelds, gestures and proprioception for multi-user 3D modeling and animation The MIT Faculty has made this article openly available. Please share how this access benefits you.

More information

Advanced Interaction Techniques for Augmented Reality Applications

Advanced Interaction Techniques for Augmented Reality Applications Advanced Interaction Techniques for Augmented Reality Applications Mark Billinghurst 1, Hirokazu Kato 2, and Seiko Myojin 2 1 The Human Interface Technology New Zealand (HIT Lab NZ), University of Canterbury,

More information

Interaction Design. Chapter 9 (July 6th, 2011, 9am-12pm): Physical Interaction, Tangible and Ambient UI

Interaction Design. Chapter 9 (July 6th, 2011, 9am-12pm): Physical Interaction, Tangible and Ambient UI Interaction Design Chapter 9 (July 6th, 2011, 9am-12pm): Physical Interaction, Tangible and Ambient UI 1 Physical Interaction, Tangible and Ambient UI Shareable Interfaces Tangible UI General purpose TUI

More information

Tangible Lenses, Touch & Tilt: 3D Interaction with Multiple Displays

Tangible Lenses, Touch & Tilt: 3D Interaction with Multiple Displays SIG T3D (Touching the 3rd Dimension) @ CHI 2011, Vancouver Tangible Lenses, Touch & Tilt: 3D Interaction with Multiple Displays Raimund Dachselt University of Magdeburg Computer Science User Interface

More information

My Accessible+ Math: Creation of the Haptic Interface Prototype

My Accessible+ Math: Creation of the Haptic Interface Prototype DREU Final Paper Michelle Tocora Florida Institute of Technology mtoco14@gmail.com August 27, 2016 My Accessible+ Math: Creation of the Haptic Interface Prototype ABSTRACT My Accessible+ Math is a project

More information

Simulation of Tangible User Interfaces with the ROS Middleware

Simulation of Tangible User Interfaces with the ROS Middleware Simulation of Tangible User Interfaces with the ROS Middleware Stefan Diewald 1 stefan.diewald@tum.de Andreas Möller 1 andreas.moeller@tum.de Luis Roalter 1 roalter@tum.de Matthias Kranz 2 matthias.kranz@uni-passau.de

More information

ExTouch: Spatially-aware embodied manipulation of actuated objects mediated by augmented reality

ExTouch: Spatially-aware embodied manipulation of actuated objects mediated by augmented reality ExTouch: Spatially-aware embodied manipulation of actuated objects mediated by augmented reality The MIT Faculty has made this article openly available. Please share how this access benefits you. Your

More information

Corey Pittman Fallon Blvd NE, Palm Bay, FL USA

Corey Pittman Fallon Blvd NE, Palm Bay, FL USA Corey Pittman 2179 Fallon Blvd NE, Palm Bay, FL 32907 USA Research Interests 1-561-578-3932 pittmancoreyr@gmail.com Novel user interfaces, Augmented Reality (AR), gesture recognition, human-robot interaction

More information

synchrolight: Three-dimensional Pointing System for Remote Video Communication

synchrolight: Three-dimensional Pointing System for Remote Video Communication synchrolight: Three-dimensional Pointing System for Remote Video Communication Jifei Ou MIT Media Lab 75 Amherst St. Cambridge, MA 02139 jifei@media.mit.edu Sheng Kai Tang MIT Media Lab 75 Amherst St.

More information

HCI Outlook: Tangible and Tabletop Interaction

HCI Outlook: Tangible and Tabletop Interaction Tangible and Tabletop UIs Pushing the edge of interactive technology... HCI Outlook: Tangible and Tabletop Interaction multiple degree-of-freedom (DOF) input Morten Fjeld Associate Professor Dept. of Computer

More information

Virtual Reality in Neuro- Rehabilitation and Beyond

Virtual Reality in Neuro- Rehabilitation and Beyond Virtual Reality in Neuro- Rehabilitation and Beyond Amanda Carr, OTRL, CBIS Origami Brain Injury Rehabilitation Center Director of Rehabilitation Amanda.Carr@origamirehab.org Objectives Define virtual

More information

Virtual/Augmented Reality (VR/AR) 101

Virtual/Augmented Reality (VR/AR) 101 Virtual/Augmented Reality (VR/AR) 101 Dr. Judy M. Vance Virtual Reality Applications Center (VRAC) Mechanical Engineering Department Iowa State University Ames, IA Virtual Reality Virtual Reality Virtual

More information

Augmented Reality. ARC Industry Forum Orlando February Will Hastings Analyst ARC Advisory Group

Augmented Reality. ARC Industry Forum Orlando February Will Hastings Analyst ARC Advisory Group Augmented Reality ARC Industry Forum Orlando February 2017 Will Hastings Analyst ARC Advisory Group whastings@arcweb.com Agenda Digital Enterprise: Set the stage Augmented Reality vs. Virtual Reality Industry

More information

VIRTUAL REALITY Introduction. Emil M. Petriu SITE, University of Ottawa

VIRTUAL REALITY Introduction. Emil M. Petriu SITE, University of Ottawa VIRTUAL REALITY Introduction Emil M. Petriu SITE, University of Ottawa Natural and Virtual Reality Virtual Reality Interactive Virtual Reality Virtualized Reality Augmented Reality HUMAN PERCEPTION OF

More information

CSE Tue 10/09. Nadir Weibel

CSE Tue 10/09. Nadir Weibel CSE 118 - Tue 10/09 Nadir Weibel Today Admin Teams Assignments, grading, submissions Mini Quiz on Week 1 (readings and class material) Low-Fidelity Prototyping 1st Project Assignment Computer Vision, Kinect,

More information

WaveForm: Remote Video Blending for VJs Using In-Air Multitouch Gestures

WaveForm: Remote Video Blending for VJs Using In-Air Multitouch Gestures WaveForm: Remote Video Blending for VJs Using In-Air Multitouch Gestures Amartya Banerjee banerjee@cs.queensu.ca Jesse Burstyn jesse@cs.queensu.ca Audrey Girouard audrey@cs.queensu.ca Roel Vertegaal roel@cs.queensu.ca

More information

Haplug: A Haptic Plug for Dynamic VR Interactions

Haplug: A Haptic Plug for Dynamic VR Interactions Haplug: A Haptic Plug for Dynamic VR Interactions Nobuhisa Hanamitsu *, Ali Israr Disney Research, USA nobuhisa.hanamitsu@disneyresearch.com Abstract. We demonstrate applications of a new actuator, the

More information

Recent Progress on Wearable Augmented Interaction at AIST

Recent Progress on Wearable Augmented Interaction at AIST Recent Progress on Wearable Augmented Interaction at AIST Takeshi Kurata 12 1 Human Interface Technology Lab University of Washington 2 AIST, Japan kurata@ieee.org Weavy The goal of the Weavy project team

More information

Mohammad Akram Khan 2 India

Mohammad Akram Khan 2 India ISSN: 2321-7782 (Online) Impact Factor: 6.047 Volume 4, Issue 8, August 2016 International Journal of Advance Research in Computer Science and Management Studies Research Article / Survey Paper / Case

More information

Augmented and Virtual Reality

Augmented and Virtual Reality CS-3120 Human-Computer Interaction Augmented and Virtual Reality Mikko Kytö 7.11.2017 From Real to Virtual [1] Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS

More information

Spatial Augmented Reality: Special Effects in the Real World

Spatial Augmented Reality: Special Effects in the Real World Spatial Augmented Reality: Special Effects in the Real World Ramesh Raskar MIT Media Lab Cambridge, MA Poor Man s Palace Spatial Augmented Reality Raskar 2010 Poor Man s Palace Augment the world, projectors

More information

Interactive Multimedia Contents in the IllusionHole

Interactive Multimedia Contents in the IllusionHole Interactive Multimedia Contents in the IllusionHole Tokuo Yamaguchi, Kazuhiro Asai, Yoshifumi Kitamura, and Fumio Kishino Graduate School of Information Science and Technology, Osaka University, 2-1 Yamada-oka,

More information

PhonePaint: Using Smartphones as Dynamic Brushes with Interactive Displays

PhonePaint: Using Smartphones as Dynamic Brushes with Interactive Displays PhonePaint: Using Smartphones as Dynamic Brushes with Interactive Displays Jian Zhao Department of Computer Science University of Toronto jianzhao@dgp.toronto.edu Fanny Chevalier Department of Computer

More information

MIRACLE: Mixed Reality Applications for City-based Leisure and Experience. Mark Billinghurst HIT Lab NZ October 2009

MIRACLE: Mixed Reality Applications for City-based Leisure and Experience. Mark Billinghurst HIT Lab NZ October 2009 MIRACLE: Mixed Reality Applications for City-based Leisure and Experience Mark Billinghurst HIT Lab NZ October 2009 Looking to the Future Mobile devices MIRACLE Project Goal: Explore User Generated

More information

2. Introduction to Computer Haptics

2. Introduction to Computer Haptics 2. Introduction to Computer Haptics Seungmoon Choi, Ph.D. Assistant Professor Dept. of Computer Science and Engineering POSTECH Outline Basics of Force-Feedback Haptic Interfaces Introduction to Computer

More information

Perceptual Characters of Photorealistic See-through Vision in Handheld Augmented Reality

Perceptual Characters of Photorealistic See-through Vision in Handheld Augmented Reality Perceptual Characters of Photorealistic See-through Vision in Handheld Augmented Reality Arindam Dey PhD Student Magic Vision Lab University of South Australia Supervised by: Dr Christian Sandor and Prof.

More information

Interface Design V: Beyond the Desktop

Interface Design V: Beyond the Desktop Interface Design V: Beyond the Desktop Rob Procter Further Reading Dix et al., chapter 4, p. 153-161 and chapter 15. Norman, The Invisible Computer, MIT Press, 1998, chapters 4 and 15. 11/25/01 CS4: HCI

More information

Augmented reality as an aid for the use of machine tools

Augmented reality as an aid for the use of machine tools Augmented reality as an aid for the use of machine tools Jean-Rémy Chardonnet, Guillaume Fromentin, José Outeiro To cite this version: Jean-Rémy Chardonnet, Guillaume Fromentin, José Outeiro. Augmented

More information

Social and Spatial Interactions: Shared Co-Located Mobile Phone Use

Social and Spatial Interactions: Shared Co-Located Mobile Phone Use Social and Spatial Interactions: Shared Co-Located Mobile Phone Use Andrés Lucero User Experience and Design Team Nokia Research Center FI-33721 Tampere, Finland andres.lucero@nokia.com Jaakko Keränen

More information

Touch Feedback in a Head-Mounted Display Virtual Reality through a Kinesthetic Haptic Device

Touch Feedback in a Head-Mounted Display Virtual Reality through a Kinesthetic Haptic Device Touch Feedback in a Head-Mounted Display Virtual Reality through a Kinesthetic Haptic Device Andrew A. Stanley Stanford University Department of Mechanical Engineering astan@stanford.edu Alice X. Wu Stanford

More information

MRT: Mixed-Reality Tabletop

MRT: Mixed-Reality Tabletop MRT: Mixed-Reality Tabletop Students: Dan Bekins, Jonathan Deutsch, Matthew Garrett, Scott Yost PIs: Daniel Aliaga, Dongyan Xu August 2004 Goals Create a common locus for virtual interaction without having

More information

Booklet of teaching units

Booklet of teaching units International Master Program in Mechatronic Systems for Rehabilitation Booklet of teaching units Third semester (M2 S1) Master Sciences de l Ingénieur Université Pierre et Marie Curie Paris 6 Boite 164,

More information

Using Mixed Reality as a Simulation Tool in Urban Planning Project for Sustainable Development

Using Mixed Reality as a Simulation Tool in Urban Planning Project for Sustainable Development Journal of Civil Engineering and Architecture 9 (2015) 830-835 doi: 10.17265/1934-7359/2015.07.009 D DAVID PUBLISHING Using Mixed Reality as a Simulation Tool in Urban Planning Project Hisham El-Shimy

More information

Immersive Training. David Lafferty President of Scientific Technical Services And ARC Associate

Immersive Training. David Lafferty President of Scientific Technical Services And ARC Associate Immersive Training David Lafferty President of Scientific Technical Services And ARC Associate Current Situation Great Shift Change Drive The Need For Training Conventional Training Methods Are Expensive

More information

Andriy Pavlovych. Research Interests

Andriy Pavlovych.  Research Interests Research Interests Andriy Pavlovych andriyp@cse.yorku.ca http://www.cse.yorku.ca/~andriyp/ Human Computer Interaction o Human Performance in HCI Investigated the effects of latency, dropouts, spatial and

More information

IT and Systems Science Transformational Impact on Technology, Society, Work, Life, Education, Training

IT and Systems Science Transformational Impact on Technology, Society, Work, Life, Education, Training IT and Systems Science Transformational Impact on Technology, Society, Work, Life, Education, Training John S. Baras Institute for Systems Research and Dept. of Electrical and Computer Engin. University

More information

Sensing in Ubiquitous Computing

Sensing in Ubiquitous Computing Sensing in Ubiquitous Computing Hans-W. Gellersen Lancaster University Department of Computing Ubiquitous Computing Research HWG 1 Overview 1. Motivation: why sensing is important for Ubicomp 2. Examples:

More information

Improvisation and Tangible User Interfaces The case of the reactable

Improvisation and Tangible User Interfaces The case of the reactable Improvisation and Tangible User Interfaces The case of the reactable Nadir Weibel, Ph.D. Distributed Cognition and Human-Computer Interaction Lab University of California San Diego http://hci.ucsd.edu/weibel

More information

Mobile and Pervasive Game Technologies. Joel Ross ICS 62 05/19/2011

Mobile and Pervasive Game Technologies. Joel Ross ICS 62 05/19/2011 Mobile and Pervasive Game Technologies Joel Ross ICS 62 05/19/2011 jwross@uci.edu Reading Summary! Please answer the following questions: on a piece of paper: What do Ross et al. conclude about the relationship

More information

Service Cooperation and Co-creative Intelligence Cycle Based on Mixed-Reality Technology

Service Cooperation and Co-creative Intelligence Cycle Based on Mixed-Reality Technology Service Cooperation and Co-creative Intelligence Cycle Based on Mixed-Reality Technology Takeshi Kurata, Masakatsu Kourogi, Tomoya Ishikawa, Jungwoo Hyun and Anjin Park Center for Service Research, AIST

More information

Physical Computing: Hand, Body, and Room Sized Interaction. Ken Camarata

Physical Computing: Hand, Body, and Room Sized Interaction. Ken Camarata Physical Computing: Hand, Body, and Room Sized Interaction Ken Camarata camarata@cmu.edu http://code.arc.cmu.edu CoDe Lab Computational Design Research Laboratory School of Architecture, Carnegie Mellon

More information

Research Topics in Human-Computer Interaction

Research Topics in Human-Computer Interaction stanford hci group / cs376 Research Topics in Human-Computer Interaction Scott Klemmer 28 September 2004 http://cs376.stanford.edu Who am I? New professor in computer science MS/PhD in CS from UC Berkeley

More information

Augmented Reality Lecture notes 01 1

Augmented Reality Lecture notes 01 1 IntroductiontoAugmentedReality Lecture notes 01 1 Definition Augmented reality (AR) is a live, direct or indirect, view of a physical, real-world environment whose elements are augmented by computer-generated

More information