Introduction to Haptics Roope Raisamo Multimodal Interaction Research Group Tampere Unit for Computer Human Interaction (TAUCHI) Department of Computer Sciences University of Tampere, Finland
Definition (Webster) hap tic 1. relating to or based on the sense of touch 2. characterized by a predilection for the sense of touch <a haptic person> 1
Terminology tactile: pertaining to sensory information derived from cutaneous inputs (i.e., via skin sensors) kinesthetic: pertaining to sensory information concerning limb movement haptic: pertaining to sensory information derived from both cutaneous and kinesthetic receptors, typically involves active touch 2
Terminology haptic interface: a user interface permitting human to have haptic interaction with real or virtual environments haptic device, haptic display: an interaction device actively producing haptic feedback 3
Haptic stimulation (feedback) Haptic properties of an object can be either passive or active in nature: passive haptics: the haptic properties of physical objects, such as a keyboard or a cup of coffee tangible interaction active haptics: haptics is actively generated by the device, based on haptic actuators and software haptic interaction 4
Why haptics? Touch is one of the most important human senses in the real world why not in user interfaces? Sense of touch is a parallel sensory channel to sight and hearing. It can be used to give additional or redundant information to the user. 5
Why haptics? Active or passive touching can easily be done while speaking or listening as it has been important in many activities during the evolution of humans. Currently haptic feedback is an underused modality. There are many new possibilities to improve human technology interaction. 6
Multimodal interaction Multimodal interaction makes use of several input and/or feedback modalities in interacting with a computer system. Examples of modalities: manual gestures, gaze, touch, speech, head and body movements Modality: human sensory channel, different representation modality, or different input method 7
Crossmodal interaction Crossmodal interaction makes use of a different human sensory modality to present information typically presented through another modality. 8
Multimodal humancomputer interaction Computer input modalities Computer cognition Computer output media / modalities Human output channels Human Cognition Human input channels Interaction information flow Intrinsic perception/action loop [MIAMI, 1995] 9
Senses and modalities Sensory perception Sense organ Modality Sense of sight Eyes Visual Sense of hearing Ears Auditive Sense of touch Skin Tactile Haptic Sense of smell Nose Olfactory Sense of taste Tongue Gustatory Sense of balance Organ of equilibrium Vestibular 10 [Silbernagel, 1979]
Potential benefits A list by Maybury and Wahlster [1998, p. 15]: Efficiency Redundancy Perceptability Naturalness Accuracy Synergy Mutual disambiguation of recognition errors [Oviatt, 1999a] 11
Two results from combining modalities 1. The combination of human output channels effectively increases the bandwidth of the human machine channel. 4 This has been discovered in many empirical studies of multimodal human computer interaction [e.g., Oviatt, 1999b]. 12
Two results from combining modalities 2. Adding extra output modality requires more neurocomputational resources and will lead to deteriorated output quality, resulting in reduced effective bandwidth. 4 Two types of effects are usually observed: 4 a slow down of all output processes, and 4 interference errors due to the fact that selective attention cannot be divided between the increased number of output channels. 4 Two examples of this: writing when speaking, and speaking when driving a car. 13
A design space for multimodal user interfaces Fusion Combined Independent Use of modalities Sequential Parallel ALTERNATE SYNERGISTIC EXCLUSIVE CONCURRENT Meaning No Meaning Meaning No Meaning Levels of abstraction 14 [Nigay and Coutaz, 1993]
Sequential: exclusive Sequential use of modalities that are not linked together in a same task. For example, recording speech and then closing all the unnecessary windows with a mouse. 15
Sequential: alternate The use of modalities is sequential, but they are linked together in the same task. For example, pointing an object with a mouse and giving a voice command to copy it. 16
Parallel: concurrent The use of modalities is parallel, but they are not linked together. Usually the hardest kind of multimodal systems, but may result in a better efficiency with some training. For example, driving a car while speaking in a mobile phone 17
Parallel: synergistic The use of modalities is parallel and they are linked together. For example, pointing and speech command at the same time. 18
Put That There [Bolt, 1980] 20