MOBILE AND UBIQUITOUS HAPTICS Jussi Rantala and Jukka Raisamo Tampere Unit for Computer-Human Interaction School of Information Sciences University of Tampere, Finland
Contents Haptic communication Affective haptics Haptic navigation Haptics in cars Other applications 1
2 Haptic communication
Haptic communication (1/2) Most work on virtual haptic communication simulate physical interaction between a human being and an object often lacks the subtle and expressive emotional elements natural for human-human communication 3 Smith & MacLean (2005) Using haptic metaphor to communicate emotion: A structured approach
Haptic communication (2/2) Haptic languages Braille (Louis Braille, 1824) Tadoma (Sophie Alcorn, ~1850) tactile lip reading : person places their thumb on the speaker's lips while the other fingers touch the speaker's face and neck Vibratese (Geldard, 1957) like tactile Morse code 45 basic elements in the back elements coded by amplitude, duration and location 60 wpm (~3 times better than Morse code) 4
intouch (video) Probably the first prototype for two-way haptic communication (from 1997) provides a physical link between users separated by distance one can both move the rollers and feel them moving http://tangible.media.mit.edu/projects/intouch/ 5
Haptic icons Haptic icons in communication structured, abstract messages that provide contextual information coded by, e.g., amplitude, frequency, duration, rhythm, etc. should be practical, reliable, quick to identify, pleasant 6
Tactile communication (1/2) Examples of tactile communication devices ComTouch (Chang et al., 2007) [ video ] Shake2Talk (Brown and Williamson, 2007) 7
Tactile communication (2/2) More examples of tactile communication CheekTouch (Park et al., 2009) Rantala et al., 2011 8
9 Affective haptics
Affective haptics (1/3) Can haptics evoke emotions? touch is often private touch tends to increase trust for many interactions touch may be too intimate or socially inappropriate Humans can express a range of emotions through interpersonal touch the effectiveness of remote touching depends on the design and technology used touch communication can be altered and used in ways that are not possible in the real world 10
Affective haptics (2/3) Touch me, hit me and I know how you feel: A design approach to emotionally rich interaction (Wensveen et al., 2000) emotionally rich interaction = interaction that heavily relies on emotion expressed through action 3-step design method What are the relevant emotional aspects? How can the product provide feedback? How should the product adapt its behavior? 11
Affective haptics (3/3) Also, affective responses to haptics have been studied by instructing users to rate their experienced emotional state after stimulation Stimuli with high intensity or amplitude are often felt as arousing and unpleasant Vice versa, more subtle stimuli are felt as relaxing and pleasant 12
Affective haptics: examples [ video ] PARO (http://paro.jp/english/) a mental commitment robot psychological (for relaxation and motivation) physiological (for improvement in vital signs) social effects (to promote communication among patients and caregivers) The Haptic Creature (Yohanan et al., 2011) 3 actuation types: ears, lungs, purr presents 9 different emotional states when interacting with it 13
Hug-over-distance (video) http://www.floydmueller.com/portfolio/hug_over_a_distance.htm 14
Other examples Touch Me (Vaucelle et al., 2009) one-way communication input: keyboard output: vibrations with varying amplitude & location The Hug (Gemperle et al., 2003) 15 input: squeeze, stroke, hug or pet (pressure & acceleration sensors) output: light, vibration, sound
Haptic navigation 16
Haptic navigation (1/2) Waypoint Navigation with a Vibrotactile Waist Belt (van Erp et al., 2005) 12 pedestrians, eight tactors in a belt distance translated to rhythm, direction into vibration location after about 30 min (five routes), the participants demonstrated acceptable effective walking speeds near to normal walking speeds usefulness shown in two case studies with a helicopter and a fast boat only directional information was provided vibration were well recognized in both cases navigation by tactile cues only proved to be successful 17
Haptic navigation (2/2) Application of tactile displays in sports: where to, how and when to move (van Erp, 2006) presenting tactical information (in soccer) belt indicates the direction where to move four other vibration signals: high up the back keep your head up left shoulder look left right shoulder look right middle of the chest stop posture information (skating, cycling, rowing) corrective instructions (e.g., shoulders & back position...) motion co-ordination pattern (posture plus timing) 18
Lead-Me Interface (video) http://www.tiresias.org/research/researchers/projects/int erfaces/haptic/perceptual_attrac.htm 19 Generates a pulling sensation by oscillating a small mass back and forth small acceleration forward, large backward Can be used to indicate direction (picture above), or to help navigation (below)
Skin stretch for direction Communicates direction with a small rubber cylinder pressed against a subject s fingertip moved 0.05 to 1 mm at constant speed can be moved to four directions (have to be moved back to center after the signal, though) Gleeson et al. (2009) Communication of direction through lateral skin stretch at the fingertip 20
CabBoots (video) http://www.freymartin.de/en/projects/cabboots 21
Haptic radar (video) http://www.k2.t.u-tokyo.ac.jp/perception/hapticradar/index-e.html 22
Haptics in cars 23
Haptic in cars (1/2) Using spatial vibrotactile cues to direct visual attention in driving scenes (Ho et al., 2005) vibrotactile stimuli presented on either front or back to inform of the rapid approach of a car Influence of steering wheel torque feedback in a dynamic driving simulator (Toffin et al., 2003) drivers on the simulator control their vehicles in curves with quite different torque feedback strategies zero torque or inverted torque feedback makes driving almost impossible 24
Haptics in cars (2/2) A pneumatic tactile alerting system for the driving environment (Enriquez & MacLean, 2001) a steering wheel with pneumatic pockets to produce pulsations at around 5 Hz lowers reaction time significantly using three levels of frequency provided extra information that helps to identify a problem 25
Commercial example Citroën Lane Departure Warning System (LDWS) on motorways and fast roads (above 80 km/h) the system will alert the driver if the vehicle drifts out of a lane infra-red sensors detect the change in road and warn the driver by vibrating the seat separate vibrators for left and right side http://www.psa-peugeot-citroen.com/en/fonctionnelle/player_afil.php 26
Extreme example (video) http://www.force-dynamics.com/media/ 27
Other applications 28
Telerehabilitation through mobile device (video) Kinesthetic therapy for patients with arm motion co-ordination disorder grounded haptic device attached to the patients arms + virtual therapist uses a handheld device to monitor and control the tasks Gutierrez et al. (2004) Telerehabilitation: Controlling haptic virtual environments through handheld interfaces 29
HapticWalker (video) http://www.hapticwalker.de/ Full foot guidance for rehabilitation gait restoration is often necessary after neurological injuries freely programmable walking trajectories motor learning favor a task specific training Virtual reality applications head mounted display (HMD) for visual immersion virtual ground (e.g., floor, staircase up/down, inclined plane) different ground conditions (e.g., concrete, wood = hard contact, carpet = soft contact). 30
Robotic Walker An assistive device for elderly people with cognitive impairment three walker modes: passive, active, forced robot localization (map) and navigation (laser, ultrasonic & infrared sensors) shared-control haptic interface (force sensors) Morris et al. (2003) A robotic walker that provides guidance 31
Balance aids Passive: vibrating insoles (on the left) improved the balance of elderly people when they stood on a pair of randomly vibrating insoles Active: tilt sensors & vibrotactile actuators (on the right) gives feedback to the user when to move one s leg Priplata et al. (2003) Vibrating insoles and balance control in elderly people Wall et al. (2001) Balance prosthesis based on micromechanical sensors using vibrotactile feedback of tilt 32
Training gaits Sense users normal gait and provide tactile feedback in real-time for learning a new gait that strains the knees less Shull et al. (2011) Training multi-parameter gaits to reduce the knee adduction moment with data-driven models and haptic feedback 33