Tactile Sensing - From Humans to Humanoids

Transcription

1 1 Tactile Sensing - From Humans to Humanoids Ravinder S. Dahiya, Student Member, IEEE, Giorgio Metta, Maurizio Valle, Member, IEEE and Giulio Sandini Abstract Starting from human sense of touch, this work reviews the state of tactile sensing in robotics. Following a brief discussion on the physiology, coding, tactile data transfer and perceptual importance of sense of touch in humans, various design hints for robotic tactile sensing are derived. Various technologies and transduction methods used to improve the touch sense capability of robots are presented, followed by the trend and methods of developing tactile sensing arrays - for various body sites. Tactile sensing, focussed to fingertips and hands until last decade or so, has now been extended to whole body, even though many issues remain open. A discussion on various sysytem issues that keep tactile sensing away from widespread utility is also presented. Index Terms Tactile Sensing, Humanoid Robots. I. INTRODUCTION ROBOTIC devices, limited to the structured environment of manufacturing plants until few years ago, are slowly entering into human life in one form or another, thus, leading to emergence of interaction and learning issues - more so for humanoid robots. Humanoid robots, introduced as mechanical knight by Leonardo da Vinci in 1495 A.D. [1], will eventually work along humans if they can understand human intelligence, reason and act like humans. Since they are expected to simulate the human structure and behavior, they are more complex than other kinds of robots. For example, unlike industrial robots, a humanoid robot is expected to reach its goal while adapting to the changes in its environment - which requires autonomous learning and safe interaction, among many others. In this context, it is important to study the ways and means of humanoid robot s interaction with the environment. To perceive an object, humans use multiple sensory information from several sensory modalities like touch, vision, hearing etc. [2] - absence of any of which, would widen the gap between what is sensed and what is perceived. What happens if the humans have all sense modalities other than sense of touch? A simple experiment of exploring the objects after putting hands on an ice block for a while can answer this question. One such experiment performed by anesthetizing the skin on hands of a group of volunteers, demonstrates the difficulty in maintaining a stable grasp of objects even when they could see what they were doing [3]. The movements become inaccurate and unstable when sense of touch is lost. In another, rather unusual, experiment performed Ravinder S. Dahiya is with Robotics, Brain and Cognitive Sciences Department of Italian Institute of Technology, Genoa, & with University of Genoa, 16145, Italy. (ravinder.dahiya@iit.it). Giorgio Metta is with Robotics, Brain and Cognitive Sciences Department of Italian Institute of Technology, Genoa, & with University of Genoa, Italy. Maurizio Valle is with DIBE, University of Genoa, 16143, Italy. Giulio Sandini is with Robotics, Brain and Cognitive Sciences Department of Italian Institute of Technology, Genoa, & with University of Genoa, Italy. Manuscript received October 02, on astronauts in the International Space Station (ISS), the vibrotactile cues provided via sense of touch were found to be highly indicative of the direction and spatial disorientation [4]. Sense of touch allows assessing of objects properties like size, shape texture; detecting slip; rolling an object between fingers without dropping it; developing awareness of the body and in differentiating me from not me. Besides these, it is a powerful conduit for emotional connectedness. As in humans, the touch sensing in humanoid robots would help in knowing the interaction behaviors of a real world object - which depend on its weight and stiffness; on how its surface feels when touched; how it deforms on contact and how it moves when pushed. Despite being important, most humanoid projects have not paid any major attention to tactile sensing vis-a-vis other sensory modalities and more so in case of whole body skin - thereby strongly limiting their real world interaction and restricting their cognitive capabilities. This could be partly attributed to the complex and distributed nature of tactile sensing and partly also to the absence of satisfactory tactile sensors or taxels that can be incorporated into the system. In last two decades or so, the pursuit of improving tactile sense capability has resulted in many touch sensors, exploring nearly all modes of transduction [5] [36], but failed to produce something like CMOS (Complimentary Metal Oxide Semiconductor) optical array. Production of tactile sensors with innovative designs still continues, but, they largely remain unsatisfactory for robotics either because they are single big sized touch elements and are too big to be used without sacrificing the dexterity of robot, or because they are slow, or fragile, or due to lack of elasticity, flexibiity and mechanical robustness and also in some cases due to their digital nature i.e. all or none. Some other reasons for the neglect of tactile sensing in a general mechatronic systems are discussed in [37]. Design of a meaningful robotic tactile sensing system must be guided by a broad but integrated knowledge of how tactile information is encoded and transmitted at various stages of interaction. In this context, various studies on human sense of touch provide a good starting point. Starting from human sense of touch, this work presents the role, importance and current state of tactile sensing in robotics. This work is organized as follows: Section II describes various terms associated with sense of touch. Following a brief discussion on the physiology of human sense of touch, its role and perceptual importance are presented in section III. Based on these studies, various design hints for robotic tactile sensing are also presented. Various technologies, that have been used to improve touch sensing capability of robots, are presented in section IV. Trends and methods for the development of tactile sensing arrays, suitable for various body sites, are discussed in section V. Various issues needed to be considered

2 2 Stimulus mediated by Receptors in skin Stimulus mediated by Receptors in muscles, tendon, joints Cutaneous/Tactile Sensing CNS (Central Nervous System) Kinesthetic Sensing Tactual Perception Cutaneous/Tactile Perception Kinesthetic Perception Haptic Perception Control of pick up Information PassiveCutaneous Perception PassiveKinesthetic Perception Active Kinesthetic Perception PassiveHaptic Perception ActiveHaptic Perception Fig. 1: Components of tactual perception [38]. Dotted line represents the partial dependence of kinesthetic perception on stimulus mediated by receptors in the skin. for the effective utility of tactile sensing in robotics have been highlighted in section VI. Various open issues concerned with robotic tactile sensing have been presented at appropriate places through out the text and summarized in the conclusion. II. SENSE OF TOUCH - DEFINITIONS AND CLASSIFICATION Sense of touch is used as a laymen term in previous section and before moving further it is imperative to define various terms associated with the touch sense modality. The sense of touch in humans comprises of two main submodalities, Cutaneous and Kinesthetic, characterized on the basis of the site of sensory inputs. Cutaneous sense receives sensory inputs from the receptors embedded in the skin and kinesthetic sense receives sensory inputs from the receptors within muscles, tendons and joints [38], [39]. It should be noted that the sensory inputs are not only mechanical stimulations but also heat, cooling and various stimuli that produce pain. In context with the submodalities mentioned above, most researchers have distinguished among three sensory systems - Cutaneous, Kinesthetic and Haptic. According to Loomis and Lederman [38], [40], cutaneous system involves physical contact with the stimuli and provides the awareness of the stimulation of the outer surface of body by means of receptors in the skin and associated somatosensory area of Central Nervous System (CNS). The kinesthetic system provides information about the static and dynamic body postures (relative positioning of the head, torso, limbs and end effectors) on the basis of: (a) Afferent information originating from the muscles, joints and skin, and (b) Efference copy, which is the correlate of muscle efference available to the higher brain. The involvement of afferent information from skin in kinesthetic sensing, indicates its dependence on cutaneous sensing. The haptic system uses significant information about objects and events both from cutaneous and kinesthetic systems [38], [40]. The perception of a stimulus can be categorized as cutaneous perception, kinesthetic perception and haptic perception - on the basis of sensory systems discussed above. The perception of stimulus from cutaneous part is called cutaneous or tactile perception. In terms of Loomis and Lederman [38], the tactile perception refers to the perception mediated solely by variations in cutaneous stimulation. Kinesthetic perception is mediated exclusively or nearly so by the variations in kinesthetic stimulation. All perceptions mediated by cutaneous sensibility and/or kinesthesis are referred as tactual perception. Haptic perception is the tactual perception in which both cutaneous and kinesthetic systems convey significant information. Investigation of the properties of peripheral nervous system has been done in two ways: first, in which observer is touched by moving objects and second, which involves the purposive exploration of objects by observer. Accordingly, the sense of touch is classified as passive and active. Loomis and Lederman [38] made distinction between passive and active touch by adding the motor control inputs to the afferent information, as shown in Fig. 1. In everyday context, the touch is active as the sensory apparatus is present on the body structures that produce movements. Using various terms associated with human sense of touch, a parallel can be drawn for robotic tactile sensing. Generally, robotic tactile sensing is associated with the detection and measurement of forces in a predetermined area. Jayawant [41] defined it as the continuous detection of forces in an array. Crowder [42] defined it as the detection and measurement of perpendicular forces in a predetermined area and subsequent interpretation of the spatial information. Following human sense of touch, this definition is narrow for not including contact parameters other than perpendicular forces, and broad for including the interpretation of spatial information, which is basically perception - that includes the role of both cutaneous sensing and the corresponding area of analysis in somatosensory cortex in CNS. The tactile or cutaneous sensing is associated with the detection and measurement of contact parameters which can be a mechanical stimulation or temperature or moistness etc. In this context, definition of tactile sensor by Lee [14] is more complete as the tactile sensor is defined as a device or system that can measure a given property of an object through contact in the world. Studies on cutaneous sensing in humans, also indicate towards partial coding or pre - processing of stimulus at the receptor level - before transmitting signals to higher levels [43], [44]. In view of above facts, tactile sensing can be defined as the process of detecting and measuring a given contact parameter in a predetermined area and subsequent pre-processing of the signals - before sending them to higher levels for perceptual interpretation. On similar lines, touch sensing can be termed as tactile sensing at single contact point. The analogous terms, for cutaneous and kinesthetic sensing, in robotics are termed as extrinsic and intrinsic tactile sensing respectively. At system level, various components are involved in the perception of a contact event. For example, like cutaneous sensing system (Fig. 1), the extrinsic tactile sensing and the computational unit of robots can be termed as extrinsic tactile sensing system and similarly the intrinsic tactile sensing system and haptic system can also be defined for robotic applications. A broad classification of robotic tactile sensing is given in Fig. 2. Based on the tasks to be accomplished, the robotic tactile sensing may be grouped into two categories - first, Perception for Action as in grasp control and dexterous

3 3 Robotic Tactile Sensing Based on the Task to be Done Based on the Location of the Sensor Perception for Action Grasp Control, Dexterous Manipulation, Contact point estimation, Slip Detection Action for Perception Exploration, Object recognition, Surface properties, Hardness, Softness, Stiffness, Temperature Intrinsic Tactile Sensing Joint angle sensors, Force/Torque Sensors Extrinsic Tactile Sensing Distributed pressure/force/stress sensors, Temperature sensors High density Tactile Sensing Arrays for fingertips Large area Tactile Sensing arrays for artificial skin Fig. 2: Classification of Robotic Tactile Sensing. manipulation and second, Action for Perception as in object recognition, modeling and exploration. In addition to these two functional categories, a third category - not shown in Fig. 2 - could be haptics, involving both action and reaction or in other words, two way transfer of touch information. Based on the place where tactile sensor is located, the robotic tactile sensing can be categorized as extrinsic and intrinsic tactile sensing. While intrinsic sensors are placed within the mechanical structure of the system and derive the contact data like magnitude of force using force sensors; the extrinsic tactile sensors or sensing arrays are mounted at or near the contact interface and deal with the data from localized regions. The extrinsic tactile sensing is further categorized in two parts - first, for highly sensitive parts (e.g. fingertips) and second, for less sensitive parts (e.g. palm or large area skin). Whereas former requires a high density tactile sensing arrays or a large number of touch sensors in a small space ( 1mm spatial resolution) and fast response (of the order of few milliseconds); such constraints can be relaxed for the latter. Both, extrinsic and intrinsic tactile sensing can also be further classified - not shown in Fig. 2 - on the basis of the working principle of sensors and on the basis of physical nature of the sensors. On the basis of working principle, tactile sensors can be resistive, capacitive, inductive, optical, magnetic, piezoelectric, ultrasonic, magneto electric etc. On the basis of the physical nature, the sensors can be flexible, compliant, stiff and rigid etc. A detailed discussion on these classifications, is given in the following sections. This study is primarily focused on the extrinsic or cutaneous tactile sensing. Hereafter, for simplicity, the term tactile sensing is used for extrinsic tactile sensing in robotic applications. III. HUMAN TACTILE SENSING - A BASIS FOR ROBOTIC TACTILE SENSING Scientific studies on humans, like movements for optimum exploration of material properties, object recognition, active and passive perception, selective attention, sensory guided motor control etc., have addressed many issues that are challenging to roboticists as well. In absence of any rigorous robotic tactile sensing theory, such studies may be helpful in specifying important parameters like sensor density, resolution, location, bandwidth etc. and may also bring new ideas to raise the level of tactile sensitivity and acuity of robots to humans range. For centuries, biological systems have inspired engineers [45] and now roboticists [46] [48]. Following a brief discussion on cutaneous/tactile sensing in humans, this sections presents some design issues for robotic tactile system. For detailed study on touch sense modality and its perceptual importance in humans, one may refer to [2], [40], [49]. A. Neurophysiology and human touch system Human sense of touch deals with the spatiotemporal perception of external stimuli through a large number of receptors for different stimuli e.g. mechanoreceptors - for pressure/vibration, thermo receptors - for temperature and nocioceptors - for pain/damage [56]. They are distributed all over the body with variable density. For example, the number of mechanoreceptors per cm 2 area is estimated to be 241 in the fingertips and 58 in the palm of adult humans [57]. The response to mechanical stimulus is mediated by mechanoreceptors, embedded at various levels in the skin. The classification, functions and location of these receptors are shown in Fig. 3. They have different receptive fields - the extent of body area to which a receptor responds, and also different rates of adaptation. A fast-adapting (FA) receptor responds with bursts of action potentials when its preferred stimulus is first applied and when it is removed. It does not respond during the steady state between stimulus onset and offset. In contrast, a slow-adapting (SA) receptor remains active throughout the period during which the stimulus is in contact with its receptive field. SA-I mechanoreceptors exhibit fully tunable Stochastic Resonance [58] - a process whereby a non linear system is able to detect an otherwise undetectable signal (e.g. subthreshold stimulus) by adding a random stimulus or noise, to the input. The response to thermal stimulus is believed to be mediated by separate warm and cold thermo receptor population in the skin. Nociceptor units in the skin are primarily responsible for sensation of pain, but, they also respond to extremes in temperature and sometimes to mechanical stimulation [40]. The nature of electrical discharge from various receptors in response to the external stimuli (e.g. fast temperature changes and pressure pulse) - studied in vitro and in vivo on human skin samples - has been found to be pyroelectric and piezoelectric [59]. A comparative experimental evaluation of piezoelectric polymer, PVDF, with epidermis samples of skin shows a marked phenomenological analogy [60].

4 4 Primary Sensory Cortex Perception of the stimuli Thalamus Information in the form of neural codes. Spinothalamic tract (pain & crude touch) MedialLemniscus Midbrain Medulla Intermediate Ridges Information in the form of action potentials. Information in the form of spatio - temporal stress/strain in skin. Neural signal transmission Distortion of a population of Mechanoreceptors Dorsal Column (preeise touch; kinaesthesia). Information in the form of spatio- temporal force distribution. Skin deformation at contact point Sensory Nerves Stimulus (Skin - Object contact) Classification Basis Pacinian Corpuscle Ruffini Corpuscle Merkel Cells Meissner s Corpuscle Type FA II SA II SA I FA I Adaptation Rate Fast Slow Slow Fast Spatial Acuity (mm) Vibration/rapid Best(µm) indent. threshold Mean(µm) Stimuli Frequency (Hz) Conduction Velocity (m/s) Effective Stimuli Temporal changes in the Sustained downward Pressure; Spatial deformation; Sustained Temporal changes in skin Sensory Function skin deformation High frequency vibration detection; Tool use. Lateral skin stretch; Skin slip. Finger position; Stable grasp; Tangential Force; Motion direction pressure; Curvature, edge, corners. Pattern/form detection; texture perception; Tactile flow perception. deformation Lowfrequency vibration & motion detection; Grip control; Tactile flow perception. Fig. 3: (center-bottom) Section of glabrous skin (modified from [50]) showing physical location and classification of various mechanoreceptors [2], [49], [51] [54]; (left) Tactile signal Transmission - from fingertips to somatosensory area of brain (modified from [55]); (right) Functional events during tactile signal transmission from contact point to the brain. For simplicity, the signal flow is unidirectional. In general, the information transfer is bidirectional as same path is used by motor signals. B. Tactile Information encoding and transfer From the moment the skin is stimulated to the resulting perception, a variety of complex mechanical, perceptual and cognitive phenomena take place. A sequence of events during tactile signal transfer from contact point to brain is shown in Fig. 3. On contact with an object, the skin conforms to its surface and maintains the same local contour. The same deformation is projected to a large number of mechanoreceptors in the skin, each representing a small portion of the object and encoding the spatio-temporal tactile information as spikes of action potentials - voltage pulses generated when the stimulus is greater than a threshold. The amplitude of the stimulus is transformed by mechanoreceptors in the form a train of action potentional [2] - somewhat similar to digitizing and coding of analog signals in an analog to digital convertor. The contact event related information is transmitted to the CNS for higher level processing and interpretation via multiple nerves up to the spinal cord and via two major pathways: Spinothalamic and Dorsal-Column-Medial-Lemniscal (DCML), as shown in Fig. 3. The Spinothalamic Pathway is slower and carries temperature and pain related information and DCML Pathway is faster and conveys pressure/vibration related tactile information more quickly to brain and is important for the spatial and temporal comparisons of the stimuli. CNS does not always get the raw touch information as some pre-processing of the signals is done at various stages. As an example, during natural manipulations humans can perceive independently the curvature and the direction of force from first spikes of the ensembles of primary sensory neurons or mechanoreceptorsthe in the terminal phalanx [43]. In such a event, the CNS must only perform some higher level processing to disentangle the interactions between such information and other parameters like magnitude and rate of change of contact force, temperature, change in viscoelastic properties of fingertip etc. [61]. The tactile information transfer to brain is also subjected to an intense process of selection [62]. For example, the tactile information is transferred when attention is paid to which part of the body is being stroked. How CNS combines the information from the large number of receptors to get a coherent image of objects, is not discussed here and one can refer to [2], [49], [63] for such details.

5 5 C. Spatio-temporal properties and sensitivities of human tactile sensing The spatio-temporal limits on tactile acuity and sensitivity to mechanical stimulus in humans also needs attention as such parameters are known to have a direct affect the pattern/object recognition [38], directional sensitivity [64] etc. The pattern sensing capability of cutaneous sense is limited by both its spatial and temporal sensitivities as they quantify the information loss or blurring of stimulus by spatio-temporal filtering at early stage of cutaneous processing [38]. Study of such effects can help in defining cross-talk limits of robotic tactile sensors. Spatial acuity - the smallest separation at which one can tell if he/she has been touched at two points - gives information about spatial resolution. Studies using two points threshold [65] and grating orientation method [66], show that the spatial acuity varies across the body - being highest at fingertips, face and toes and lowest at thigh, shoulders and belly. The spatial resolution in the palm is about seven times smaller than that of the fingertips [67]. One can resolve two points as close as 3 mm on the fingertips, and up to 30 mm on the belly [49]. More sensitive psychophysical methods show that the spatial resolution on the fingertips is about 1 mm [68], [69]. These results place the tactile acuity somewhere between vision and audition - being worse than vision, but better than audition [49]. Besides body site, the ability to perceive a fine spatial structure, also depends on the temporal properties of stimulus (namely, frequency at which it vibrates) and the skin structure. Studies show that the spatial acuity decreases at higher vibratory frequencies [70]. The spatial acuity of torso measured with vibro-tactile stimuli has been reported to be mm [71]. When it comes to skin structure, the presence of intermediate ridges in the skin, shown in (Fig. 3, are known to enhance the tactile spatial acuity by transmitting magnified signals from surface of skin to the mechanoreceptors [72]. When it comes to temporal resolution, humans are capable of detecting vibrations up to 700 Hz, i.e. they can detect a single temporal interval of about 1.4 milliseconds [40]. Comparing temporal acuity of touch with that of vision (upper limit of 50 Hz for a flickering light) and audition (20,000 Hz), touch again lies between vision and audition, but this time audition is better [49]. Temporal separation of two contact events at different locations is also needed for detecting the presence of multiple events. The critical temporal separation for two events at different locations on fingertips has been found to be in the order of ms [73]. The knowledge of amplitude resolution - both in terms of pressure and deformation of skin - is also important in context with cutaneous sensing. A controlled pressure sensitive study shows that pressure thresholds (higher the threshold, lower is the sensitivity of the body part) vary with body site. For example on a scale of log0.1mg, the mean pressure threshold is 2.2 for fingers and 1.6 for upper lip, nose and cheek [49]. D. Tactile sensing in perception Humans are excellent at recognizing common objects by touching alone and cues like material properties, shape etc. are critical to this endeavor. Both cutaneous and Stroking or Vertical Indentation of object with different shapes S K I N Firing ratesf SA andf FA ofsaand FA receptors respectively. Fig. 4: Model of skin for coding of shape and orientation by the mechanoreceptors. kinesthetic sensing contribute to the perception of such cues. Tactile sensing in humans is better adapted to feeling the material properties of objects than is to feeling their shapes, particularly when the object is large enough to extend beyond the fingertip [49]. Perhaps this is the reason why most of the studies on tactile sensibility in humans and other primates have reported sensory perception in the context of exploratory tasks [44]. Shape detection of the objects - small enough to be within the contact area (7 12 mm) of fingertips, is an important function of the mechanoreceptors present in the skin. A number of experiments involving stroking and vertical indentation, with the force equal to that exerted by humans during natural manipulation (15 90 gm wt.), indicate that the object shape and orientation are signaled by the spatio-temporal responses of the afferent fiber populations, particularly those of the SAs [74] [78]. Further, humans are able to perceive independently curvature and force direction from these signals [61]. These experiments reveal that the firing rate of an SA is a function of the vertical displacement, vertical velocity, and the amount and the rate of change of curvature of the skin. However, SAs become silent in the event of negative rate of change of curvature. In case of FA, the firing rate is a function of the vertical velocity and the rate of change of curvature at the most sensitive part of the receptive field. These studies give a direct relation between the stimuli and neural signal used to code it - irrespective of the intervening mechanisms, such as the stresses and strain at the receptor site. Thus, assuming skin to be a blackbox, the relation between the inputs i.e. the shape stimuli and the output i.e. the firing rate of afferent fibers can be represented as in Fig. 4. f SA = a 1 R 1 + a 2 dr 1 dt + a 3 Z + a 4 dz dt dr 1 dz f FA = b 2 + b 4 (2) dt dt Where, f SA and f F A are the firing rates of SA and FA receptors respectively; R 1 is the skin curvature at contact point; Z is the vertical displacement and a 1, a 2, a 3, a 4, b 2, and b 4 are the constants. Firing rates are zero in case of negative rate of change of the curvature. The edge sensitivity is a special case of sensitivity to changes in skin curvatures. As can be noticed from (1) (2) that FA and SA receptors respond simultaneously at edges and boundaries and at other points FA receptors are silent. The response of SA (1)

6 6 receptors is higher at edges than at uniform surface because of high compressive strain at such points. The edge detection sensitivity of SA I receptors has also been attributed to the presence of Merkel cells on the tips of epidermal part of intermediate ridges - undulating epidermal tissues that descend into the epidermal-dermal junction, shown in Fig. 3, and believed to magnify the tactile signals from the surface of the skin to the mechanoreceptors by way of micro-lever action [79], [80]. The role of intermediate ridges studied through continuum mechanics or finite element modeling also show that the concentration of stress on the ridge tips improves the capability to distinguish different test indenters and enhances the capability to differentiate their finer details [81]. It is surprising that tactile receptors are located near to the points where stress is concentrated. The fact that receptors are sensitive to the rate of change of curvature - in addition to the curvature - enhances the contrast at the edges of objects where curvature changes abruptly. From robotics point of view, these results highlight the importance of having sensors that respond to both static and dynamic stimuli. A combination of capacitive and piezoelectric transduction could be one of many possible solutions. Roughness-smoothness is another important perceptual dimension. Neurophysiological studies suggest that the tactile roughness perception is accurately predicted by spatial variations of discharge of SA afferents and hence it is a function of multiple tactile elements. Roughness also varies with the spatial distribution, form and shape of the surface elements (e.g. pointed, round or flat) and material properties of the surface (e.g. stiffness, elasticity) can also contribute to roughness [82] [84]. As opposed to the general belief that the temporal parameters have little effect on roughness perception [84], recent studies show that they do indeed contribute to tactile surface texture perception [85]. Like primates, in rodents also the roughness perception depends on both the spatial and temporal signals [86]. Discrimination of surface roughness is also enhanced when there is tangential movement between the surface and skin [87] and this is independent of the mode (active or passive) of touch [88], in other words, this property is salient to cutaneous/tactile sensing. Roughness of objects has also been found to be significantly correlated with friction and that the correlation is much stronger when the variations and rate of change of the tangential forces are considered. This is evident from the experiments where subjects maintained a steady normal force rather than reducing it, in order to allow the tangential force to initiate and maintain sliding while scanning a surface with higher friction [89], [90]. These facts point out the importance of tangential forces and that the knowledge of these forces - in addition to the normal forces - can be useful in robotics, both for exploratory and manipulation tasks. Detection of slip can be viewed as the coding of motion by the receptors in the skin. Slip or relative movement between a surface and the skin is important for perception of roughness [84], [90], [91], hardness [92] and shape [93], [94]. Slip plays an important role in grip force control by acting as an error signal. All these, except static contact associated with thermal sensing, involve finger movements and thus highlight the importance of dynamic tactile sensing [95]. Tactile feedback from the contact surface of an object influences the perception of force used to support it. An experiment to study the effect of tactile sensing on the perception of force demonstrates underestimation of forces produced by muscles, when tactile sensory feedback from hand is constrained [96]. Interestingly, complete elimination of tactile feedback by anesthetizing skin results in an opposite perception of force i.e. increase in the perceived force or heaviness [97] and decrease in the maximum force that the fingers can produce [98]. Further, the effect of eliminating the tactile sensing from various fingers is also different. Elimination of cutaneous sensing from thumb and index finger results in an increase of perceived heaviness by 40% and 13% respectively [97]. In addition to magnitude, the direction of force is also important. It is critical for handling objects with irregular shapes while maintaining the desired orientation. Tactile afferents from the terminal phalanx of finger contribute to the encoding of direction of fingertip forces. The directionality is also thought to be due to different strains produced at the receptor site by forces (equal to those used in natural manipulation) applied in different directions [44]. In context with motor control, tactile information plays an important role in the controlling the execution of reaching to grasp movements. The contribution of cutaneous receptors for controlling prehensile force during object manipulation has been studied extensively [56], [99], [100]. Tactile sensory information from fingers helps in ascertaining the actual shear or load force which is useful in optimally adjusting the grip force [56], [98], [99]. During grasp, the cutaneous feedback is needed to have the actual state of the system, in absence of which, the internal models (of objects) underlying anticipatory control mechanisms are no longer updated [98], [101]. Touch information (along with kinesthetic, vision and motor-feedback signals) is needed to obtain the body schema, which is an internal representation of body s structure [39]. The correct grasp of an object requires fine control of not only the strength of finger muscle activation, but also of its temporal course or the duration, in various phases of grasp. Lack of tactile sensing has been shown to lengthen the duration of the finger-opening phase of the grasp and thereby impairing the control of grasp [102]. These findings show that the tactile information is possibly used in getting minimal duration for various phases, or in other words, for the time optimization of various phases of the movement. The discharge from specific receptors at the beginning and the end of a movement can be used to compute grasp time for various phases and to optimize the grasp temporal parameters [56]. In this context, taxels that are able to record dynamic events could be helpful in robotics. Tactile information from fingertips has also been shown to contribute to the control of timing in sequential actions such as playing a piano or tapping in synchrony to an external signal [103]. Thus, a variety of information about real world objects is obtained through cutaneous sensing and study of way they are sensed and processed can be used to derive some design cues for an

7 7 artificial tactile sensing system. However, it should be noted that humans system is complete, multi-level integrated system and hence sense of touch does not work alone. Different sense modalities operate simultaneously - each contributing to the perception of stimulus. Sometimes they compete (e.g. in presence of attention) and at other times the whole is an integrated combination of the different inputs. Even if a single modality is involved, the perception of an object can be due to combined contribution of its sub modalities. The combination and integration of sensory information from multiple sources is key to robust perception as it maximizes the information derived from the different sensory modalities and improves the reliability of the sensory estimate. As an example, the perception of size [104] and shape [105] is obtained by integrating visual and haptic information in a statistically optimal fashion and that the reliability of integrated estimate is higher than unimodal estimate. Similarly, the perception of roughness and moistness of surfaces is modulated by the frequency content of auditory feedback [106]. Both vision and proprioception provide information about position of hand in space [107]. In view of these facts, the design of any artificial tactile sensing for robots should also take into account the presence of other sensing modalities and their combined contribution in achieving a common goal. E. Skin Mechanics and tactile sensing Skin acts as a medium through which contact indentations are converted into stresses/strains and further coded as neural signals. Skin is multi-layered, non-linear, non-homogeneous, viscoelastic and a geometrically and structurally complex mechanical system supported on a deformable system of muscles and fat [80]. With such properties, the skin mechanics is bound to play an important role in the tactile perception. The stiffness of various layers in skin varies, with the base layer of epidermis being considerably stiffer than the dermis - having young s modulus times that of dermis [81]. Presence of physical interlocking between the epidermis and dermis layers of skin helps in resisting any tendency of their relative sliding over each other and also creates a filtering mechanism that distributes forces and stresses from their point of application [108]. Such a filtering mechanism also has considerable impact on the spatial resolution. The presence of intermediate ridges and their role in magnifying the tactile signals by way of micro-lever action has already been discussed. Intermediate ridges should not be confused with papillary ridges which are basically the external parallel whorls, also known as fingerprints. However, the center of each papillary ridge protuberance lies directly above the center of each intermediate ridge [81]. Though papillary ridges, together have also been suggested to improve the tactile acuity by micro-lever action [79], [80], finite element studies indicate very little involvement of papillary ridges in such mechanism [109]. A number of attempts have been made to model and study the mechanical behaviors of the skin and one can refer to [51], [81], [110], [111]. F. Hints for the design of robotic tactile sensing system With the above background on human tactile sensing, one can formulate some basic design criteria for tactile sensing in a general robotic system that can be used in a real-world environment. A few such studies have been reported in the literature [13] [16], [112] and their findings are also included in the following design hints for robotic tactile sensing: The presence of varied and distributed receptors with sharp division of functions calls for using different kinds of miniaturized sensors each having optimal response for a particular contact parameter (though each may also contribute to detection of other parameters as well). It is desirable to have multi-functional sensors, like contact force + hardness detection [113] and tactile + thermal sensor [114], that can measure more than one contact parameter. Number of such sensing elements may depend on the body site where they are intended to be placed. The spatial resolution of the tactile sensors (arranged in an array) should be based on the body site. For fingertips, it should be about 1 mm - which translates to an approximately 15 x 10 elements grid on a fingertip sized area - and for less sensitive parts like palm and shoulders it can be as high as 5 mm. The sensors should demonstrate high sensitivity and broad dynamic range. Normal manipulation involves forces in the range of gm wt [74], [75]. Considering involvement of taxels in various exploratory tasks, a force sensitivity range of gm. wt. and a dynamic range of 1000:1 are desirable [115]. In addition to the magnitude, the touch sensor should also be able to measure the direction of force. This is important because robots, in general, do not have a prior model of real world objects. taxels must be able to detect and measure both static and dynamic contact events. More than one mode of transduction may be required to meet such requirements. The robotic tactile sensors should be fast. This is particularly important, if tactile feedback is used in robotic control. The need for involving tactile sensing in control loop of robotic applications has been felt greatly due to insufficient contact information available from artificial muscles or kinesthetic sense alone. The signal frequency range to which different mechanoreceptors in human skin respond, can be used to set the response time requirements of sensors. In general, for real time contacts, each touch element should be as fast as 1 msec. The same is also true for an array of tactile sensing elements. However, such conditions can be somewhat relaxed in case of whole body skin type of distributed taxels. In human skin, the information collected by every mechanoreceptor is not directly sent to the brain for processing. Instead some complex pre-processing is takes place to fit the limited throughput of the nervous system. In order to reduce the amount of information to be transmitted to the central decision unit, it is important for large tactile arrays or modules to have some level of preprocessing (data selection, local computation etc.) at the

8 8 sensory location. Such an architecture would free robot s brain for more intelligent works. Otherwise, it would allow scaling up the system to practically any number of sensors. The contact information should be transferred via different paths with different transfer rates. The signals (mechanical) that require urgent attentions (e.g. in feedback control) can be transferred via faster path. However, such an arrangement would probably increase the number of wires - which is undesirable in robotics. The taxels may also be embedded into or covered with elastic material just like the receptors in the skin that lie under different layers of skin. Although embedding the sensors in elastic material may introduce some blurring or filtering effects; the increase in contact area, as a result of such elastic covering, is helpful in manipulation. The elastic covering of the sensors may be designed to have ridge structure similar to that of intermediate and papillary ridges. Besides concentrating the stresses on the sensing elements, structures like intermediate ridges will help in lowering the blurring effect of elastic cover. A textured pattern like papillary ridges on the surface of elastic material increases detectability [116]. Biological sensors can derive information like detailed contours of objects, because the skin is compliant and conforms to object. Thus, the robotic taxels should be robust, flexible, conformable, stretchable and soft so that it can withstand harsh conditions of temperature, humidity, chemical stresses, electric field, sudden force or shock etc. When distributed - as in whole body skin - the taxels and skin should not significantly increase the diameter/thickness of robot link/part. Linearity and low hysteresis are also desired. Although non-linearity can be dealt with by inverse compensation, the same is difficult for hysteresis. Output of taxels should be stable, monotonic and repeatable. It is interesting to note that the human tactile sensing is hysteric, nonlinear, time varying and slow. But, the presence of large number of technologically poor receptors enables the central nervous system to extract useful information. However, above requirements are also application dependent and thus should not be considered as definitive. Some of the above design cues seem to be technologically challenging and hence technological and manufacturing issues like production of sensing devices with similar performance (repeatability across different fabrications), type and number of interconnects, and repeatability of response over time etc. should also be considered while designing robotic tactile sensors. IV. TACTILE SENSOR TYPES Tactile information is useful in robotics in a number of ways. In manipulative tasks, tactile information is used as a control parameter [117] [119] and the required information typically includes contact point estimation, surface normal and curvature measurement and slip detection [120] through measurement of normal static forces. A measure of the contact forces allows the grasp force control, which is essential for maintaining stable grasps [121]. The Grasp force along with manipulator displacement is also needed in compliant manipulators [122]. In addition to magnitude, the direction of force is also critical in dexterous manipulation to regulate the balance between normal and tangential forces to ensure grasp stability - the so-called friction cone [123]. For full grasp force and torque determination, shear information is also required [124], [125]. The need for shear stress information is also supported by Finite Element Analysis (FEA) [126], [127]. Shear information is useful in determining coefficient of friction and in getting a unique surface stress profile when the sensor is covered with elastomeric layer [128]. Importance of shear force in humans has already been discussed. Although during interaction with the environment, a significant portion of the information about objects e.g. shape [129] [131], surface texture [17], [132], slip [132] [135], etc. comes through the detection of normal and shear forces; a real world interaction, involving both manipulation and exploration, should also measure material properties such as hardness [113], temperature [18] etc. taxels based on design hints presented in previous section, can possibly help in achieving above objectives. Even though few sensors have been implemented using similar design guidelines, the number and type of measures are still insufficient for a complete humanoid application. As an example, the interaction of robots with environment through tactile sensing has largely been limited to the measurement of static interaction forces whereas real world interaction involves both static and dynamic forces. Similarly, most of the sensors are designed to measure static pressure or forces, from which, it is difficult to obtain information like friction, stickiness, texture, hardness and elasticity. Recently, the importance of dynamic events has been recognized and sensors are being developed for detecting stress changes [10], [95], slip and other temporal contact events. A range of sensors that can detect object shape, size, position, forces and temperature have been reported in [13] [15], [136]. Few examples of sensors that could detect surface texture [17], [132], hardness or consistency [19], [113] are described in the literature. Very few examples of sensors that can detect force as well its direction have been reported [5], [137]. Nearly all modes of transduction namely Resistive/Piezoresistive, Tunnel Effect, Capacitive, Optical, Ultrasonic, Magnetic, Piezoelectric etc. have been reported in literature for developing taxels for robotic applications. Working of taxels based on various methods of transduction is described in [138] and relative advantages and disadvantages of some of them are given in [139]. Selected examples of robotic tactile sensors based on various transduction methods and the physical/mechanical nature are discussed below. A. Tactile Sensors based on Various Transduction Principle 1) Resistive Sensors: Tactile sensors based on resistive mode of transduction are of two types: i) with resistance that depends on contact location and ii) with resistance that depends on the applied force or, in other words, piezoresistive

9 9 taxels. Resistive touch sensors are generally sensitive and cheaper, but, they are expensive in terms of power consumption. Another limitation of resistive taxels is that they generally measure only one contact location. An improved design of taxel using resistive sensing technology is reported in [20]. The design involves arranging the sensors in an array and hence enables the measurement of many contact points. But, the lack of contact force measurement still remains a critical problem. Piezoresistive touch sensors are made of materials whose resistance changes with force/pressure. Touch sensing system using this mode of transduction have been reported for use in anthropomorphic hands [11]. Piezoresistive tactile sensing is particularly popular among the MEMS based and silicon based tactile sensors [21], [22]. FSRs (Force Sensing Resistors), widely used in pointing and position sensing devices such as joysticks, are also based on piezoresistive sensing technology. The FSR sensors are appealing, because of the low cost, good sensitivity, low noise and simple electronics. Commercially available from Interlink [23], FSRs have been used in many experimental tactile systems and advanced robotic hands [140], [141]. Requirement of serial or manual assembly, relatively stiff backing, high non-linear response and large hysteresis are some of the drawbacks of FSRs. 2) Tunnel Effect Tactile Sensors: Tactile sensors based on Quantum Tunnel Composites (QTC) have come up recently and are commercially available from Peratech [142]. QTC s have the unique capability of transforming from a virtually perfect insulator to a metal like conductor when deformed by compressing, twisting or stretching of the material. In QTCs the metal particles never come into contact. Rather they get so close that quantum tunneling (of electrons) takes place between the metal particles. Robot hands with QTC based taxels have also been reported in literature [143], [144]. A highly sensitive sensor based on electron tunneling principle is reported in [17]. The device directly converts stress into electroluminescent light and modulation in local current density, both being linearly proportional to local stress. Use of thin film with metal and semiconducting nanoparticles allows producing a single sensor (2.5 cm 2 size) that is larger than typical human fingertip, with the spatial resolution better than that of the human fingertip ( 40µm). The use of CCD (Charge- Coupled Device) camera, in this work, also adds to the sensor size and makes it difficult to integrate on the robot fingertip. 3) Capacitive Sensor: Capacitive taxels have been widely used in robotics [7], [10], [24] and they can be made very small, which allows the construction of dense sensor arrays. An array of capacitive sensors which couples to the object by means of little brushes of fibers is reported in [10]. The sensor elements on the array are reported to be very sensitive (with a threshold of about 5 mn) and robust enough not to be damaged during grasping. An 8x8 capacitive tactile sensing array with 1 mm 2 area and spatial resolution at least 10 times better than the human limit of 1 mm is reported in [7]. Capacitive sensing technology is also popular among the tactile sensors based on MEMS and silicon micromachining [5], [7], [8], [10]. Commercially available touch sensors such as RoboTouch and DigiTacts from Pressure Profile Systems [145] and ipodtouch [146], are based on capacitive sensing technology. Availability of commercial capacitance to digital convertor chip like AD7147: CapTouch T M from Analog Devices [147] has made it easier to design thin and reliable contemporary touch controls for sensors that use capacitive technology. Utility of such a chip in getting the digitized data corresponding to change in capacitance at the contact point, has been demonstrated in [148]. Touch sensors based on capacitive mode of transduction are very sensitive but stray capacity and severe hysteresis are major drawbacks. 4) Optical Sensors: Tactile sensors with optical mode of transduction use the properties of optical reflection between media of different refractive index. The change in light intensity when contact is made gives the measure of the pressure. Optical fiber based taxel capable of measuring normal forces is reported in [12]. The sensor can measure forces as low as 0.001N with the spatial resolution of 5 mm. An optical three axial taxel capable of measuring normal and shear forces is reported in [9]. Some cases of large area skin based on LEDs (Light-Emitting Diodes) has also been reported [149], [150]. Commercial taxels using optical mode of transduction are also available e.g. KINOTEX [151]. Optical based taxels are immune to electromagnetic interference, are flexible, sensitive and fast, but at times they are bulky. As an example, even after miniaturization, the optical taxel reported in [25] has 32 mm diameter, is 60 mm long and weighs 100 g. Issues like loss of light by micro bending and chirping, which causes distortion in the signal, are also associated with optical sensors. 5) Ultrasonics based Sensors: Acoustic ultrasonic sensing is yet another technology that has been used for the development of tactile sensors. Microphones are known to be useful for detecting surface noise that occurs at the onset of motion and during slip. A device that senses contact events from their ultrasonic emission at the contact point is described in [152]. A Polyvinylidene Fluoride (PVDF) polymer is used in a 2 x 2 array of receivers to localize the contact point on a silicone rubber sensing dome. The sensor is reported to be very effective in detecting slip and surface roughness during movement. The change in resonance frequency of Lead Zirconate Titanate (PZT), according to contact object s acoustic impedance, has been reported for detecting hardness and/or softness [19] and force/pressure [26]. Taxels based on ultrasonic approach have fast dynamic response and good force resolution, but materials like PZT are difficult to handle in miniaturized circuits. Using piezoelectric polymers can greatly simply the difficulties associated with PZT. 6) Magnetism based Sensors: Such tactile sensors measure the change in flux density caused by applied force. The flux measurement can be made by either a Hall Effect [137], [153] or a magneto resistive device [27]. The taxels based on magnetic principle have a number of advantages that include high sensitivity and dynamic range, no measurable mechanical hysteresis, a linear response, and physical robustness. But, their usage is limited to non-magnetic mediums. 7) Piezoelectric Sensors: The piezoelectric materials have the property of generating charge/voltage proportional to the applied force/pressure. Piezoelectric materials like PZT, PVDF etc. are suitable for dynamic tactile sensing. Though quartz and some ceramics (PZT) have better piezoelectric properties;