ubiquitous computing and augmented realities

Size: px

Start display at page:

Download "ubiquitous computing and augmented realities"

Eustacia Payne
6 years ago
Views:

1 ubiquitous computing and augmented realities chapter 20 ubiquitous computing and augmented realities ubiquitous computing filling the physical world with computers virtual and augmented reality making the physical world in a computer! 1/90 2/90 Challenging HCI Assumptions What do we imagine when we think of a computer? The most profound technologies are those that disappear. Weiser 1990 s: this was not our imagined computer! Ubiquitous Computing Any computing technology that permits human interaction away from a single workstation Implications for Technology defining the interactive experience Applications or uses Underlying theories of interaction 3/90 4/90 Scales of devices Weiser proposed Inch Foot Yard Implications for device size as well as relationship to people 5/90 Device scales Inch PDAs PARCTAB Voice Recorders smart phones Individuals own many of them and they can all communicate with each other and environment. 6/90 1

2 Device scales Device scales Foot notebooks tablets digital paper Individual owns several but not assumed to be always with them. Yard electronic whiteboards plasma displays smart bulletin boards Buildings or institutions own them and lots of people share them. 7/90 8/90 Defining the Interaction Experience Implicit input Sensor-based input Extends traditional explicit input (e.g., keyboard and mouse) Towards awareness Use of recognition technologies Introduces ambiguity because recognizers are not perfect Different Inputs Capacitive sensing on a table Sensors on a PDA 9/90 10/90 Multi-scale and distributed output The output experience Screens of many sizes (very) small (very) large Distributed in space, but coordinated 11/90 More than eye-grabbing raster displays Ambient: use features of the physical environment to signal information Peripheral: designed to be in the background Examples: The Dangling String The Water Lamp (shown) 12/90 2

3 Merging Physical and Digital Worlds Application Themes How can we remove the barrier? Actions on physical objects have meaning electronically, and vice versa Output from electronic world superimposed on physical world 13/90 A digital desk An augmented calendar Context-aware computing Sensed phenomena facilitate easier interaction Automated capture and access Live experiences stored for future access Toward continuous interaction Everyday activities have no clear begin-end conditions 14/90 virtual reality technology Virtual and Augmented Reality (VR and AR) VR - technology & experience web, desktop and simulators AR mixing virtual and real headsets allow user to see the virtual world gesture recognition achieved with DataGlove (lycra glove with optical sensors that measure hand and finger positions) eyegaze allows users to indicate direction with eyes alone whole body position sensed, walking etc. 15/90 16/90 VR headsets small TV screen for each eye slightly different angles 3D effect immersion VR computer simulation of the real world mainly visual, but sound, haptic experience life-like situations too dangerous, too expensive see unseen things: too small, too large, hidden, invisible e.g. manipulating molecules the experience aim is immersion, engagement, interaction 17/90 18/90 3

4 on the desktop VRML VR on the web headset VR expensive, uncomfortbale desktop VR use ordinary monitor and PC cheap and convenient in games and on the web VRML virtual reality markup language 19/90 #VRML V1.0 ascii Separator { Separator { # for sphere Material { emmissivecolor # blue } Sphere { radius 1 } } Transform { translation } Separator { # for cone Texture2 { filename "big_alan.jpg" } Cone { radius 1 # N.B. width=2*radius height 3 } } } COSMO, CORONA, plug-in s to view vrml models in web-browsers 20/90 command and control scenes projected on walls realistic environment hydraulic rams! real controls other people for: flight simulators ships military 21/90 Augmented Reality (AR) Definition Typical applications Forms of Display AR Examples AR-Research Focus I and II Research project overview: 22/90 Augmented Reality (AR) Definition of Augmented Reality 1. Combines real and virtual 2. Is interactive in real time 3. Is registered in three dimensions (Azuma, 1997) 23/90 Augmented Reality (AR) Typical applications: Medical Manufacturing and repair Outdoor and Construction Annotation and visualization Robot path finding Entertainment 24/90 4

5 Kinds of Displays for AR Head-up Display - Car Head-up Display Head Mounted Display (HDM) 25/90 26/90 Head-up Display - Car Head-up Display-Taxiway Navigation 27/90 28/90 Head-up Display-Taxiway Navigation Head Mounted Display (HMDs) 29/90 30/90 5

6 Head Mounted Display (HDM) are either Head Mounted Display (HDM) video display, or see-through display, or look-around display video-based see-through HMD 31/90 32/90 Head Mounted Display (HDM) Head Mounted Display (HDM) video display 33/90 look-around display 34/90 Real World Wide Web (RWWW), HMD 35/90 RWWW: Glare and gaze selection 36/90 6

7 RWWW: Glare and gaze selection 37/90 Manufacturing and repair 38/90 Manufacturing and repair 39/90 Design 40/90 Design 41/90 Outdoor and Construction 42/90 7

8 Outdoor and Construction 43/90 Outdoor and Construction 44/90 Stereoscopic telerobotic control 45/90 Robotic application 46/90 Entertainment 47/90 Entertainment 48/90 8

9 High Degr.-of-Freedom Curve Input Museum 49/90 Tovi Grossman, Ravin Balakrishnan, Karan Singh. (2003). An interface for creating and manipulating curves using a high degree-of-freedom input device. ACM CHI Conference on Human Factors in Computing Systems. ACM CHI Letters, 5(1). pp Paper: Video: 50/90 AR-Research Focus I: BUILD-IT, Tangible Interface for Collaborative Design Activity Theory: Complete Task Regulation -- 1 Fjeld, Rauterberg, Bixel ETH Zurich Video: 51/90 52/90 Activity Theory: Complete Task Regulation /90 54/90 9

10 55/90 56/90 Tangible User Interfaces (TUIs) are often seen as... AR-Research Focus II: Chemistry Education A Tangible Interaction Approach P. Juchli, B. Voegtli, M. Fjeld, HyperWerk FHBB, ETH Zurich Video: 57/90 57 Intuitive Easy to learn and to handle Direct Natural mapping of task Immediate feedback Task specific Require considerable developement ressources Are difficult to re-use in a different use context.. but, non-compliant with standard formats CAD Interface is rare Intergration with Word/OSX etc. difficult 58/90 For educational use of TUIs, particular challenges are but TUI for education also promise significant benefits Combined spatial representation and interaction Adaptation to a standard curriculum,e.g. science Adaptation to personal learning strategies Collaborative learning and human communication Information pieces attached to physical objects help memorize content Limitations of time multiplexed interfaces may be overcome through bi-manual (and collaborative) use Enable non-linear, explorative learning strategies 59/90 60/90 10

11 Educational Context in Focus: Molecular models and modelling; intermediate level Established tools for molecular modelling Physical ball-stick kit 61/90 Digital spacefill/wireframe GUI 62/90 The Augmented Chemistry (AC) System: Our strategy in interface design: Combine the strengths of physical and digital interface Offer direct manipulation of virtual molecular models Combine 3D representation and spatial interaction Give users audiovisual feedback Observe and keep track of user s interactive steps A virtual mirror as an augmentation of a tabletop learning environment The mirror is a design metaphor. An alternative would be a user s point-of-view. 63/90 64/90 Interactive Tools Booklet Cube Gripper 65/90 Platform 66/

12 Interaction Sequence 1. Element selection 2. Load an element with the Gripper Specialized tools 3. Place the atom onto the platform 4. Select binding atom 5. Determine singledouble or trible binding 6. User feedback after sucessful completion 67/ /90 68 Recent advances: Electronegativity; dipole-moment (1/4) Recent advances: Electronegativity; dipole-moment (2/4) H O H H O H 69/90 70/90 Recent advances: Electronegativity; dipole-moment (3/4) Recent advances: Electronegativity; dipole-moment (4/4) 71/90 72/90 12

13 Work in progress: Learning Studies (1/2) Work in progress: Learning Studies (2/2) Same task, three different conditions Measurement of: - Task completion - User satisfaction - Task load (NASA task load index) 73/90 74/90 scientific and technical data information and data visualisation VR, 3D and 2D displays scientific and complex data interactivity central 75/90 number of virtual dimensions that are real three dimensional space visualise invisible fields or values e.g. virtual wind tunnel two dimensional space can project data value up from plane e.g. geographic data N.B. viewing angle hard for static visualisation no real dimensions 2D/3D histograms, scatter plots, pie charts, etc. 76/90 virtual wind tunnel structured informnation fluid dynamics to simulate air flow virtual bubbles used to show movements better than real wind tunnel no disruption of air flow cheaper and faster scientific data just numbers information systems lots of kinds of data hierarchies file trees, organisation charts networks program flow charts, hypertext structure free text documents, web pages 77/90 78/90 13

14 visualising hiererchy wide hierarchies use 3D? 2D organisation chart familiar representation what happens when it gets wide? sales manager managing director marketing manager production manager managing director F. Bloggs J. Smith F. Bloggs A. Jones R.Carter K. West P. Larkin B. Firth sales manager marketing manager production manager cone trees (Xerox) levels become rings overlap OK in 3D F. Bloggs F. Bloggs A. Jones R.Carter K. West B. Firth J. Smith 79/90 P. Larkin 80/90 visualising knowledge and hiererchy visualising and interacing with relations a) Walden's Path b) File System Navigator c) Cheops d) VR-VIBE e) Information Pyramides f) The Web Book (invalid) g) WWW3D h) The Task Gallery /90 82/90 Scientific Visualisation: City O Scope» Test Details, Chapters from Textbook Test questions which are based on the textbook (Dix et al., 3rd Edition.), will come from these chapters and sub-chapters: 83/ Not relevant Not relevant Not relevant Not relevant , Not relevant Not relevant 17. Not relevant 18. Not relevant Not relevant 84/90 14

15 » Test Details The written test is compulsory ("obligatorisk") and will take place on Monday 18th October 2004, 08:30-12:30. The place is V-Huset, please follow indications in V-Huset. The written test consist of a compulsory ("obligatorisk") part and a voluntary ("frivillig") part. The compulsory part: -presents questions which have a rather clear answer, -reflects the miminum requirement to have the test accepted ("godkänt"), and -must be accepted ("godkänt") for the voluntary part to be corrected at all. 85/90» Test Details The voluntary part is for participants who have answered the compulsory part AND belive they will have the compulsory part accepted ("godkänt") AND aim for a higher score (4, 5, or VG) The voluntary part consists of a few advanced questions/tasks. Suggestion: Answer at least 7 of the 9 questions in Part I before proceeding to Part II. 86/90» Test Details Please answer in Swedish or English Max points per question is indicated after each question, total max is 60. PART I: Max 42 points, 25 points NECESSARY FOR PASS (GODKÄNT)» Test-/tente-granskning Oral feedback on written test, Q&A ("tentegranskning"). Monday 8th November 2004, 16:00-18:00, restaurant in Linsen. PART II: Max 18 points. This part is only corrected if 25 points or more were obtained in part I. 87/90 88/90» Assignment-granskning IMPORTANT Assignment file name IMPORTANT Oral feedback on assignment, Q&A ( assignment granskning"). Tuesday 30th November and Thursday 2nd December 2004, 13:00-15:00. Restaurant in Linsen The file submitted electronically for first draft and for final version MUST be named with the 40 first characters (including spaces) in the title of your paper, nothing else is accepted. E.g. if the title of your paper is Hand-held devices for whale hunting in Hudson Bay, then the file name becomes: hand_held_devices_for_whale_hunting_in_h.pdf 89/90 90/90 15

Ubiquitous Computing (Ubicomp) Virtual Reality (VR) Augmented Reality (AR)

Ubiquitous Computing (Ubicomp) Virtual Reality (VR) Augmented Reality (AR) Textbook sections 6.3.2-6.3.3 Textbook pages 240-285 1/61 Ubiquitous computing, VR, and AR ubiquitous computing filling the physical