TACTILE SENSING & FEEDBACK

Transcription

1 TACTILE SENSING & FEEDBACK Jukka Raisamo Multimodal Interaction Research Group Tampere Unit for Computer-Human Interaction Department of Computer Sciences University of Tampere, Finland

2 Contents Tactile sensing in detail Tactile feedback Feedback technologies & displays 1

3 2 Tactile sensing

4 Tactile sensing There s two different types of receptors responsible for tactile sensing found in the skin free nerve endings encapsulated nerve endings, i.e., mechanoreceptors Most tactile information is delivered via mechanoreceptors but, e.g., hair receptors also affect the sensations Bent hair Indented skin Skin Bent hair RA receptor Indented skin RA receptor Sustained pressure SA receptor 3

5 Mechanoreceptors 1/3 Mechanoreceptors are sensitive to mechanical pressure or deformation of the skin four types: Meissner s corpuscles, Pacinian corpuscles, Merkel s disks and Ruffini endings differ in size, receptive fields, rate of adaptation, location in the skin, and physiological properties 4

6 Mechanoreceptors 2/3 Thresholds of different receptors overlap perceptual qualities of touch are determined by the combined inputs from different types of receptors operating range for the perception of vibration about 0.04 to 500 Hz frequencies over 500 Hz are felt more as textures, not vibration skin surface temperature affects perceiving tactile sensations 5

7 Mechanoreceptors 3/3 Receptor Merkel s disks Ruffini endings Meissner s corpuscles Pacinian corpuscles Rate of adaptation SA-I SA-II RA-I PC (RA-II) Stimulus frequency 0 30 Hz 0 15 Hz Hz Hz Receptive field 2 3 mm >10 mm 3 5 mm >20 mm Location High Deep High Deep Function Pressure; edges and intensity Directional skin stretch, tension Local skin deformation, low frequency vibratory sensations Unlocalized high frequency vibration; tool use Mechanoreceptors are generally specialized to certain stimuli contact forces are detected by Merkel s discs and Ruffini endings vibration primarily stimulates the Meissner s corpuscles and Pacinian corpuscles 6

8 Hairy vs. hairless skin Hairy skin is generally less sensitive to vibration compared to glabrous skin there seems to be no PC receptors in the hairy skin, however, they are present in the deeper underlying tissue surrounding joints and bones Hairy skin has poorer absolute threshold for both vibration & pressure still about the same capacity for discriminating vibrotactile frequencies 7

9 Tactile dimensions Tactile acuity (vibration & pressure) Spatial acuity Temporal acuity 8

10 Tactile acuity for vibration Vibration primarily stimulates the Pacinian corpuscles and Meissner s corpuscles pacinian channel (high frequency, from about 60Hz) non-pacinian channel (low frequency, below 60Hz) Human thresholds for detecting vibration: sensitivity for mechanical vibration increases above 100 Hz and decreases above 320 Hz (250 Hz being optimum) The spatial acuity and pattern perception is better for skin deformation compared to vibrotactile stimuli 9

11 Tactile acuity for pressure Sensitivity for pressure is largely dependant on the area of stimulation discrimination has higher resolution at those parts of the body with a low threshold (e.g. fingertips) Discrimination is not constant throughout the entire intensity scale, as with vision and auditory senses amplitude indentation discrimination is low at low intensities 10

12 Tactile acuity Threshold responses for pressure (bars) and vibration (dots) for 15 body sites human body is highly sensitive for vibration thresholds correlate with the density of cutaneous mechanoreceptors 11

13 Deterioration of tactile acuity There appears to be no significant reduction in vibrotactile detection at the fingertips in older subjects. reflects either the high receptor density of the area, or the functional importance of vibrotactile sensibility of the fingertips (or some combination of both of these factors) Pressure sensitivity reduces as a function of age Training can be used to improve sensimotor performance 12

14 Spatial acuity 1/3 Fingertips are the most sensitive part of the human hand in texture & vibrotactile perception corresponding to the largest density of PC receptors the more spatially distant two stimuli are, the more difficult it is to discriminate them Tactile texture perception is mediated by vibrational cues for fine textures, and by spatial cues for coarse textures discrimination of spatial information is considerably more accurate than their temporal interval when using hand, exploration of spatially varying surfaces is done with the entire fingertip (increased sensitivity by active touch) 13

15 Spatial acuity 2/3 Threshold = the point at which an effect begins to be produced detection threshold (the smallest detectable level of stimulus; a.k.a absolute threshold) difference threshold (the smallest detectable difference between stimuli; a.k.a just noticeable difference (jnd)) Successful method to reduce the detection threshold is either to increase the duration of the tactile stimulation, or the interval of two consecutive stimuli Why do people do better with gratings than twopoint discrimination? active vs. passive touch 14

16 Spatial acuity 3/3 Spatial dimension for touch 2-point discrimination (1 mm at fingertips, mm in the back) localization texture detection (depends on the surface) grating discrimination (detectable distance between two gratings) pressure sensitivity temporal discrimination 15

17 Spatial acuity for pressure 1/2 Spatial acuity for two-point thresholds (bars) and errors of localization (dots) for 14 body sites smallest resolution in facial area & hands differences due to both task demands & neural activity 16

18 Spatial acuity for pressure 2/2 Variation in pressure threshold over the body smallest in facial area fingers have about the same acuity as trunk the right side seems to have slightly better acuity on average 17

19 Temporal acuity Resolution of temporal frequency discrimination is finer at lower frequencies Thresholds for tactile sensations are lowered with increased duration and interval 18

20 Thermotactile interactions Eventhough being separate modalities, temperature and touch have interactions thermal adaptation cooling degrades tactile sensitivity warming sometimes enhances thermal intensification cold objects feel heavier warm objects feel heavier but less than cold ones thermal sharpening the warmer or colder the two points are, the easier they are to discriminate Thermal cues are very important in the identification of textures 19

21 Touch is not an absolute sense Several factors affect the sensitivity age individual differences, habits attention, fatigue, mood, stress diseases, disabilities training... scalability is important factor for tactile interfaces 20

22 Tactile feedback technologies 21

23 Methods for tactile stimulation Types of skin sensory stimulation: skin deformation vibration electric stimulation skin stretch friction (micro skin-stretch) heat Possible actuator configurations: single element multiple elements (array) 22

24 Tactile actuators Some technologies used in tactile interfaces vibrating motors linear motors solenoids piezoelectric actuators pneumatic systems shape memory alloys electrorheological fluids thermoelectric elements 23

25 Actuators: vibrating motors 24

26 Vibrating motors How they work: provides relatively small-amplitude vibration (linear or rotary) applies motion either directly to the skin or through mediating structure used singly or in arrays Most common types DC-motors with eccentric rotating mass voice coils 25

27 Vibrating motors: eccentric rotating mass DC-motor rotates an offcenter spinning mass inexpensive & exsisting technology poor resolution: it takes time to start and stop Frequency control only (amplitude = freq 2 ) amplitude fixed by the size & the weight of the rotating mass Used in various devices mobile phones, pagers, gaming devices, etc. 26

28 Vibrating motors: voice coils Voice coil basics current driven through the movable coil created magnetic field interacts with the field of the permanent magnet (one-way movement) vibrations created by switching the current on/off Both frequency and amplitude can be controlled somewhat independently however, the motor has always peak at certain frequencies (e.g. 250 Hz) 27

29 Vibrating motors: overview Advantages: simple, existing technology relatively inexpensive easily powered and controlled quite small power consumption Disadvantages: not very expressive feedback vibration can be irritating sometimes hard to miniaturize efficiently 28

30 Example: vibrotactile devices Logitech ifeel mouse & Kensington Orbit 3D trackball These ones use the TouchSense technology by Immersion Corporation ( Have a small rotating DCmotor inside the device which applies the vibration through the structure 29

31 Actuators: linear motors 30

32 Linear motors: pin displays How they work: pins in an array are actuated independently the actuated pins contact the surface of the skin Advantages: simple, readily available continuously positionable versatile: static pressure, vibration; shapes or force display relatively fast Disadvantages: very difficult to pack tightly relatively high cost (lots of motors/device) 31

33 Example: tactile array Mimics complex tactile sensations Exeter arrays stimulate the fingertips each pin has piezoelectric actuator Array 1: 100 pins over 1 cm 2, frequency range Hz Array 2: 24 pins with 2 mm spacing, Hz 32

34 Example: tactile arrays in a mouse ( Allows the user to scan the of an image the pins rise and fall dynamically delivering a tactile stimuli to the fingertips can be used to code patterns and colours into tactile data VTMouse (2001) three 4x8 matrix (32 pins) put in the place of the buttons VTPlayer (2003) two 4x4 matrix with 16 pins 33

35 Actuators: solenoids 34

36 Solenoids Multi-modal mouse by Akamatsu & MacKenzie (1996) solenoid driven pin under the right index finger that rises and falls Haptic Pen by Lee et al. (2004) solenoid shakes the pen by moving up and down in top of the pen 35

37 Example: solenoids in Braille displays Braille = tactile language for sensory substitution Traditionally Braille displays use solenoids to push up the pins (nowadays mostly piezoelectric actuators are used) Solenoids have poor power consumption 36

38 Actuators: piezoelectric actuators 37

39 Piezoelectric actuators 1/2 How they work: single or multilayer ceramic elements an element expands/bends when voltage is applied multiple layers can be used to amplify the effect Properties: very large forces but small motions one element typically around mm thick resolution for frequencies ~0.01 Hz 38

40 Piezoelectric actuators 2/2 Electromechanical device that converts electrical energy into mechanical motion Typically very compact as only few components are used in a complete system actuator itself can be very small 39

41 Example: STReSS & Virtual Braille Display (Hayward et al.) 2D tactile display with an array of miniature actuators stimulate the fingertip at about 1 cm 2 in area elements can be bended in two directions to increase the forces applied to the fingertip 40

42 Example: Tactile Handheld Miniature Bidirectional (THMB) THMB is an improved version of VBD miniaturized to fit inside a PDA-size case The handheld device comprises an LCD screen that allows combining tactile and visual feedback THMB stimulates the user's thumb and is mounted on a vertical slider so that it can be dragged up and down along the left side of the case ( 41

43 Piezoelectric actuators: overview Advantages: small in size potentially inexpensive in large volumes high frequency and static modes very fast response time low power consumption Disadvantages: dynamics: small displacements require accurate amplification high voltage 42

44 Actuators: pneumatic systems 43

45 Pneumatic systems Two possible output modes based on skin indentation (and vibration) suction air-pressure How it works: technologies: fillable air-pockets, air jets, suction holes vibratory rates: typically Hz static pressure with sealed pockets 44

46 Pneumatic systems: suction Draws air from a suction hole creating an illusion that the skin is pushed Very low spatial resolution (only appropriate for the palm) two basic patterns of stimulation (large holes and small holes) Need for regulation of air pressure (=lots of equipment) 45

47 Pneumatic systems: air-pressure DataGlove with pneumatics (Sato et al., 1991) Teletact II (Stone, 1992) DataGlove bandwidth of 5 Hz, amplitude & frequency modulated Teletact II 29+1 air pockets (40 tubes to control the air-pressure) object slippage (fingers) + force feedback (palm) 46

48 Pneumatic systems: overview Advantages: tubing make it possibly to take the bulky part away from point of application pressure can be more appropriate for some applications than pins or vibrating motors can mimic skin-slip (with multiple adjacent inflated pockets) Disadvantages: requires bulky parts (air compressor or motor-driven pistons) not really portable can be very noisy difficult to display sharp edges or discontinuities 47

49 Actuators: shape-memory alloys 48

50 Shape-memory alloys Metals that "remembers" their geometry restores its original geometry when heated usually temperature change of about 10 C is necessary to initiate the phase change How it works: expands (and heats up) when current runs through it contracts when cools down stimulates the skin when vibrates (expandcontract cycles) 49

51 Shape memory alloys Wearable Tactile Displays (MIT Touchlab) Tactile Display based on Shape Memory Alloy Tactile Display based on Elastomer Actuators 50

52 Actuators: electrorheological fluids 51

53 Electrorheological fluids Liquid which viscosity changes into semi-solid when electric current is applied change in viscosity feels as more resistive surface usually packed in 2-3mm bubbles can change from liquid to gel, and back, within milliseconds The change in viscosity is proportional to the applied current 52

54 Electrorheological fluids: overview Advantages: low power consumption no moving parts controlled electrically very compact performance improves as size decreases Disadvantages: high voltage required can t control force, only viscosity sharp edges and discontinuities difficult to render 53

55 Tactile displays: skin stretch 54

56 Skin-stretch Two main methods: rotational skin stretch lateral skin stretch What happens: forces are applied to skin for displacement contact forces are perceived as stretching of the skin Applying skin stretch is being investigated as an alternative method to vibrotactile feedback 55

57 Friction: skin-slip display Micro skin-stretch motor driven smooth cylinder strapped against finger (Chen and Marcus, 1994) when rotates, stimulates the mechanoreceptors Felt as a sensation of slip grasp simulations: causes the user to increase grip force often used to append force feedback displays 56

58 Tactile displays: electrotactile stimulation 57

59 Electrotactile stimulation Electrical stimulation is not widely accepted to consumer use often sudden bursts give an "invasive" impression square waves can be easily felt as too strong stimuli and they keep tickling the nerves the sensitivity to electrical stimulation varies greatly between and within individuals (e.g., sweating & pressure affect the sensation) Used mostly in research prototypes and for rehabilitation purposes 58

60 Example: SmartTouch Tactile display to present realistic skin sensation a thin electrotactile display and a sensor mounted Two layers top layer: 4x4 array of stimulating electrodes bottom layer: optical sensors Visual information is captured by the sensors and displayed through electrical stimulation e.g. the black stripes are perceived as bumps ( 59

61 Example: Electric mouse Array of small electrodes placed to fit fingertip 64 electrodes, 1mm in diameter Pressure sensor located under the electrodes to measure finger pressure. electrical current is controlled as a function of pressure creates more stable vibratory sensations compared to traditional displays ( 60

62 Example: Bioforce by Mad Catz (2001) A game controller that delivers mild cramps to the user An electrical shock is delivered by wired pads attached to the forearm 3x1.5V batteries provide 16 ma shocks similar to the shocks used for years by physical therapists 61