Recent Progress on Augmented-Reality Interaction in AIST Takeshi Kurata ( チョヌン ) ( イムニダ ) Augmented Reality Interaction Subgroup Real-World Based Interaction Group Information Technology Research Institute, AIST http://itri.aist.go.jp/rwig/ari/ 1
Real world context & AR If a mobile computer can autonomously comprehend the user s situation and the surroundings? If the computer can provide information services just in time & just in place? Face 3D Object Position Orientation Hand Gesture 2
Remote Collaboration if a remote expert gives appropriate instructions to the company in a remote place while understanding what is going on in the remote place? efficient remote-collaboration tool Demonstration-based Interface for a Mixed Reality Authoring Environment Wearable Active Cam/Laser (WACL) 3-D Pedestrian Guide System Usable Indoors and Outdoors Tangible Tabletop Interface (TTT) Body-worn Laser (BOWL) ProCam 3
Aim of Research Realize intuitive and direct interaction with real/virtual environments and remote person by employing Multi-sensor fusion Wearable/tangible interface technologies Alleviate social problems Digital divide Baby boomer retirement Safety compliance, etc. 4
Headlines Personal Positioning 3D pedestrian guide & AR reliving tool Head/Eye-free Wearable I/O interfaces Tangible Tabletop Interface 5
Personal Positioning (PP) Dead-reckoning method Self-contained sensors (3-axis accelerometers, gyrosensors and magnetometers) Walking-locomotion analysis Stride length: estimated from amplitude of stabilized acceleration Motion recognition: walking horizontally/upstairs/downstairs, elevator Walking orientation: compensated by the Earth magnetic field to counter drifting effect of the gyrosensors 6
Video: Our System vs. GPS Complementary compensation for the GPS/RFID localization and the dead reckoning Each sensor: Each error model to use in a Kalman filter 7
How it works indoors (with map matching) 8
2F During ISMAR2007 demo session Trajectory (2F > 1F > outdoors > 1F > 2F) 9
2F Trajectory (2F > 1F > outdoors > 1F > 2F) 1F 10
2F Trajectory (2F > 1F > outdoors > 1F > 2F) 1F 11
2F Trajectory (2F > 1F > outdoors > 1F > 2F) 1F 12
2F Trajectory (2F > 1F > outdoors > 1F > 2F) 1F 13
2F Trajectory (2F > 1F > outdoors > 1F > 2F) 14
Demo in ISMAR2007 Indoor and outdoor location-based information service can be provided not only seamlessly, but pervasively without excessive dependence on infrastructure-side sensors Makes a ubiquitous information society more sustainable 15
Pilot User Study on 3D Museum Guide Participants recruitment advertising No study of 3-D indoor guide systems comparable in scale to this study (five floors, 2500 to 2700 m2/floor) Ensure pros/cons of the 3-D map Ensure characteristics of HMD/Hand-held Display for Museum Guide System 16
AR tool for Reliving Visitors Experience Capable of retrieving and visualizing the trajectory, voice, and video of each subject in a cross-sectional way using queries such as time, place, subject ID, keyword, etc Visitors, sociologist, service-engineering researchers: Browse other visitors activities with a sensation of realism on the spot 17
Tangible TableTop (TTT) Interface Physical tags & Virtual GUI objects Tightly coupled Tangible Windows Flexible GUI object size Key technologies Tabletop display: Large LCD with a SAW (Surface acoustic wave) touchscreen Physical tags: Small ultrasonic transmitters 18
Pick up a tag Set focus to a GUI object associated with the tag As if picking up the GUI object Pick2Focus 19
Group Copy-N-Paste Gather all worker tags Put them together in one place Use Copy-N-Paste gesture in the same manner as in single Copy-N-Paste Group Copy Group Paste 20
Tangible TableTop (TTT) Interface Remarkable features Affordances of the physical tags Natural tag gesture Asymmetric bimanual manipulation with tag gesture and touchscreen Problems: 21
2 nd Generation TTT Complementary fusion of two types of sensors Photo sensors + fiducial marker patterns shown on the display precise position/orientation measurement proposed marker pattern can alleviate the influence of ambient light and unevenness of display luminance Accelerometers keep tracking the 3-D motion 10Yen=100Won in size 22
Conclusion Contributions: Personal positioning 3D museum guide system WACL & BOWL ProCam TTT Capturing Real-World Objects into Virtual World BOWL (BOdy-Worn Laser) ProCam Interesting? Too diversified? Our focus: how to couple virtual worlds with the real world and how to connect person to person who does not coexist at the same place meaningfully 23
Future Works Bring out those technologies and findings into the actual real-world problems. Develop a PP-based task support system in cooperation with a major plant manufacturer Release a multi-purpose PP system 24
Thanks! ( カムサハムニダ ) 25