Biomimetic whiskers for shape recognition

Transcription

1 Robotics and Autonomous Systems 55 (2007) Biomimetic whiskers for shape recognition DaeEun Kim a,, Ralf Möller b a Max Planck Institute for Human Cognitive and Brain Sciences, Amalienstr. 33, Munich, D-80799, Germany b Computer Engineering Group, Faculty of Technology, Bielefeld University, D Bielefeld, Germany Received 20 July 2005; received in revised form 27 July 2006; accepted 4 August 2006 Available online 18 September 2006 Abstract Rodents demonstrate an outstanding capability of tactile perception with their whiskers. Mechanoreceptors surrounding the whisker shaft in their follicle structure measure deflection of the whisker. We designed biomimetic whiskers following the basic design of the follicle. In experiments with the artificial whiskers, we have explored tactile perception based on active whisking where the deflection angle or velocity provides the localization information which is the basis of shape recognition. Measuring contact distances at varying protraction angles allows discrimination of round objects with a varying curvature, or objects with different lateral shapes, such as square and round objects. We show the capabilities and limitations of a single whisker for shape recognition as well as the usefulness of multiple whiskers. In addition, measuring both vertical and horizontal deflection of a single whisker allows detection of the vertical shape for objects with a smooth surface. Two or more whiskers stacked vertically can recognize the vertical shape by observing the difference of their deflection amplitudes or the time shift of deflection velocity peak. The results provide a clue on how autonomous robots could improve their sensory capabilities with mechanical probes. c 2006 Elsevier B.V. All rights reserved. Keywords: Biomimetic whisker; Biorobotics; Tactile sensors; Shape recognition; Active perception 1. Introduction Recognizing shapes of objects with only a mechanical probe is a challenging problem. There have been several methods to use mechanical probes or antenna to recognize objects in the robotics field. The engineering approaches evaluate deflection with vision, torque sensors, strain gages, or potentiometers [13, 25,27,21]. The angle sweeped by an actuator was observed at the base of a mechanical probe and the contact distance was determined from the torque or force measurement. Russell [21] designed a tactile sensor array, each sensor consisting of a potentiometer and a long inflexible beam, and the potentiometer sensor at the whisker root measured the rotational angle proportional to the contact force applied to the antenna tip. The tactile system thus obtained the surface profile of an object. Alternatively, a flexible beam can be used as a probe. Such an antenna probe system, including a piano wire, a sweeping actuator and strain gages, has been tested for object Corresponding address: University of Leicester, Leicester, LE1 7RH, UK. Fax: address: daeeun.kim@le.ac.uk (D. Kim). detection [27]. To obtain localization information of an object, the contact point on objects can be determined by observing the fundamental resonant frequency of the vibration at the contact moment [26]. A combination of distance information based on force/moment sensors and a variety of force directions by an active antenna was used to extract the shape information of an object [25]. Kaneko et al. [14] presented a geometrical analysis for the relation between a probe and the curved shape of objects. They developed an active antenna system to determine the contact location as well as detect the slip of the antenna depending on the force direction. Their active antenna system consists of a flexible beam, actuators to move the beam, a torque sensor and angular position sensors to measure the rotational angle of the beam [14]. Recently a biomimetic whisker system, consisting of a capacitor microphone and a real rat whisker has been studied [18,8,7], and it was shown that it can be a useful tool to discriminate textures of target objects. Later Bovet and Pfeifer [4] proposed a robot control architecture which learns cross-modal correlations between visual, tactile or motor modalities. Seth et al. [22] developed artificial whiskers to provide signals proportional to the bending magnitude. Their /$ - see front matter c 2006 Elsevier B.V. All rights reserved. doi: /j.robot

2 230 D. Kim, R. Möller / Robotics and Autonomous Systems 55 (2007) we present the experimental results with single and multiple whiskers to discriminate various shapes of objects. 2. Method Fig. 1. Mouse whiskers (photo: Wolfram Schenck). whiskers consist of polyamid strips with 20 resistive areas embedded regularly along the longitudinal length, and the resistance of each whisker changes depending on the amount of bend. As a result, the whiskers can measure bending in only one direction. Our earlier work reported that a biomimetic whisker with steel shaft and magnetic sensors can measure the vibration of the whisker beam as well as the deflection angle [16]. We have explored tactile perception based on active whisking, and found the deflection amplitude or velocity provides localization information of a target object. Here, we will show that based on this localization approach, a collection of contact distances at varying protraction angles can provide shape information. Real rodents demonstrate an outstanding capability of shape recognition and texture discrimination with their whiskers [6,5, 3]. Rodents have tactile whisker arrays in the face as shown in Fig. 1 and they can process spatial information with laterally oriented vibrissae through active whisking. Our work on shape recognition is motivated by their whisking function and sensory abilities. In the biological whisker system, the whisker shaft is embedded in a follicle structure. Mechanoreceptors surrounding the shaft measure deflection in all directions, some of them with tonic, others with phasic response [17,20, 24,23,10]. Our artificial whisker system imitates this design in a very simplified way. In all our designs, the whisker is surrounded by sensors, and whisker deflection either affects these sensors without contact (magnetic sensors affected by magnets on the whisker shaft) or with contact (piezoelectric sensors). Previously we reported that the artificial whiskers can detect the deflection angle and the deflection direction for both low frequencies (including static deflection) and high frequencies (texture-related signals) [16]. Here, we will use low-frequency signals for shape recognition. In this paper, we suggest a novel method to recognize the shape of target objects based on deflection angles. The deflection amplitude or velocity signal directly provides the localization information, the distance and angular position of an object. Based on the localization approach, we describe several possibilities to detect the shape of a target object with a single whisker or multiple whiskers. We start with a theoretical analysis of deflection amplitude for our artificial whisker. Then We mounted two arrays of whiskers on a Koala platform (K-team) see Fig. 2(a). For active whisking, each array is mounted on a plate which can be rotated around a vertical axis by a DC motor. Both rotation angle and sweeping speed can be controlled, and its maximum rotation angle is about 120. Whisker sensors can be mounted in arbitrary position and orientation on the plate. All whiskers on the plate share the same angular movement. Since the whiskers are moved by the same DC motor, their protraction and retraction movements are synchronous. The sensor signals are amplified and transmitted into the on-board computer (PC104+) mounted on the robot, via a multi-channel data acquisition board (PCM-9112+). In each individual whisker, the whisker shaft is a steel beam with diameter 0.5 mm and is clamped at the aluminium base. Here we use artificial whiskers based on hall-effect sensors. The hall-effect magnetic sensors can measure the difference of magnetic flux at two sensors on opposite sides caused by the movement of the whisker to which a permanent magnet is attached. Directional sensitivity was achieved by using two pairs of sensors arranged orthogonally to each other. To test shape recognition, we can build several types of objects consisting of circular or square discs (10 mm height for each disc) see Fig. 2(b) Theoretical estimation of deflection We will first provide the theoretical model of deflection of our artificial whiskers and show how the deflection signal during active whisking is influenced by the shape of target objects. In an active antenna system, if the torque τ and the angular displacement θ at the antenna base are given, the distance can be easily estimated by the equation 3E I θ/τ [15, 14]. E is the Young s modulus of elasticity and I is the cross-sectional area momentum of inertia. Thus, the distance estimation with torque measurement depends on the material and thickness of the whisker shaft. In fact, rodents have slowly adapting and rapidly adapting mechanoreceptors around the whisker shaft in the follicle, and there has been physiological evidence that the deflection amplitude and velocity signals are coded at ganglion cells [23,24]. In this paper, we will focus on measuring the deflection amplitude and velocity, and investigate how they can serve to extract shape information. The artificial whiskers that we designed do not measure torque but record the deflection angle which is proportional to the contact force or torque at a given distance. The term deflection angle will be synonymously used with bending angle, which corresponds to the transverse movement at the sensor position. For a given torque τ in a contact phase of active whisking, we can derive the bending displacement or slope of the whisker from the Bernoulli Euler equation, which is given as: E I tan θ = τ 2d x2 τ x + 1 τd (1) 3

3 D. Kim, R. Möller / Robotics and Autonomous Systems 55 (2007) Fig. 2. Experiments with artificial whiskers (a) artificial whiskers on a Koala robot (four magnetic sensors and two microphone-based sensors at one side, and four microphone-based sensors at front) (b) different types of objects formed by discs. Fig. 3. Our active whisking system (a) active whisking and bending angle (the longitudinal axis of some whisker may not go through the centre of rotation) (b) varying deflection distance with a given torque (if we take the reference axis from the center of rotation to the contact position, the angle β is the angle between the surface slope and the reference axis, and the angle θ d represents the angle of the whisker beam at contact position relative to the reference axis. The relation between the two angles influences the deflection angle θ 0 θ 1 ). where tan θ is the slope at a position x (0 x d) along the whisker bar, starting at the clamped position, θ is measured with an axis connecting from the whisker base to the contact point, and d is the distance of an object from the whisker base at the onset of contact. The magnetic sensors in the artificial whisker measure the deflection of the whisker 14 mm away from the whisker base. In our artificial whisker system displayed in Fig. 3, the whisker base (clamped position) is 45 mm away from the rotational axis of the DC motor, and the position of the whisker base moves by the rotation of the DC motor. Now we define the term contact distance as the distance to the contact position from the center of rotation, and deflection distance as the distance to the contact point from the whisker base. Thus, the contact distance will be constant for a fixed contact position, but the deflection distance changes in a sweeping period even with a fixed contact location while the motor is rotating the support plate shown in Fig. 3(a). The position of whisker base for a rotational angle θ 0 is denoted by a vector c with the centre of rotation as a reference point see Fig. 3. Then c = ( c cos θ 0, c sin θ 0 ) where c is the distance between the centre of rotation and the whisker base, and θ 0 is measured relative to the axis from the center of rotation to the contact position. The whisker shaft is clamped at the whisker base and so the whisker base will be alternatively called the clamped position, which provides one of the boundary conditions of the Bernoulli Euler equation. We also define p for a position vector on the contact point ( p will be the distance from the centre of rotation to the contact point, simply denoted as p in the text), and the motor axis will be the reference point. Then p and d = p c represent contact distance and deflection distance, respectively. If we take the

4 232 D. Kim, R. Möller / Robotics and Autonomous Systems 55 (2007) Fig. 4. Theoretical deflection amplitudes depending on contact distance (with the assumption that the whisker bar is long enough to be bent with the corresponding distance) (a) varying distances (γ = 0) (b) varying γ s (160 mm distance). reference axis from the centre of rotation to the contact position, the angles at the whisker base and at the sensor position relative to the axis will be (θ 0 + α), (θ 1 + α), respectively, where α = cos 1 [ p ( p c)/( p p c )]. The instantaneous torque at the whisker base depends on the distance between the clamped position and the contact location. Thus, Eq. (1) can be re-formulated as below: E I tan(θ + α) = τ 2d x2 τ x + 1 τd. (2) 3 By applying the sensor position x = h = 14 mm and the clamped position x = 0 to the above equation, we can cancel out τ and obtain the following equation ( 3h 2 tan(θ 1 + α) = 2d 2 3h ) d + 1 tan(θ 0 + α) (3) where h is the distance 14 mm between sensor position and clamped position, and λ = θ 0 θ 1 is the deflection angle or bending angle that the sensors measure, which is distinguished from the protraction angle θ 0. The angle α between the two vectors, c and p c, is updated for each protraction angle θ 0. The above Eq. (3) implies that the bending angle alone is sufficient to estimate the contact distance 1 as well as the deflection distance. The longitudinal axis of a whisker may not go through the center of rotation. That is, the angle between c and p c at the onset time of contact may not be zero. Then the angle can be defined as γ = cos 1 [ c ( p c)/( c p c )]. The angle γ is the rotational angle of c counterclockwise relative to the reference vector p c, and we obtain θ 0 + α = γ at the onset of contact. With γ 0, θ 0 and θ 1 should be replaced by θ 0 γ and θ 1 γ in Eq. (3), respectively. Then we have ( 3h 2 tan(θ 1 γ + α) = 2d 2 3h ) d + 1 tan(θ 0 γ + α) (4) 1 The contact distance p can be calculated as p = d cos(α) + c cos(θ0 ) where c is 45 mm in our case. where the angles θ 0, θ 1 and α change during the protraction period, but γ is constant. From the above analysis for our whisker design, the deflection angle is inversely proportional to the contact distance as shown in Fig. 4(a), and smaller distances increase the deflection angle more rapidly in proportion to protraction angle. To determine the slope at contact position in active whisking (θ d in Fig. 3), we can substitute x = 0 and x = d in Eq. (2). Then we obtain tan[θ d (t) γ + α(t)] = 1 2 tan[θ 0(t) γ + α(t)] (5) from E I tan[θ 0 (t) γ + α(t)] = τd/3 and E I tan[θ d (t) γ + α(t)] = τd/6, where θ d (t), θ 0 (t) is the angle at contact position and clamped position at time t, respectively (θ 0 is set to 0 at the onset time of contact) Shape recognition We now apply our biomimetic whiskers to the problem of recognizing the shape of a target object. To estimate slope or curvature of an object, the whiskers need to touch many contact points around the object. Theoretically, we can estimate the shape of an object by collecting a set of distance values while the whisker is sliding over the surface of an object. This may need free angular control of the whisker beam in 3-dimensional space. 2 In our experiments, we will test a restricted domain of shape recognition based on just sweeping movements in the horizontal plane. We will observe the capabilities and limitations of a single whisker for shape recognition and also evaluate the usefulness of multiple whiskers. First, we will investigate discriminating a round object and an edged-surface object, which will be called lateral shape recognition. Second, 2 Rodents use horizontal-plane sweeping to recognize the shape or texture with a series of head movement sequences in which increasingly refined information is collected [5,11].

5 D. Kim, R. Möller / Robotics and Autonomous Systems 55 (2007) Fig. 5. Round and square objects and their change of contact position (α 1 is the angle protracted before contact, and it determines the angular position of an object). we will test discriminating vertical shapes (the shapes vary in the perpendicular direction relative to the whisking plane). Localization information, the contact distance and angular position of a target object, is obtained by assuming that the contact point is fixed. However, the contact point can vary depending on the surface shape along the whisking direction. To disambiguate the situations with fixed or varying contact positions, we define a point contact as the situation in which the contact position is fixed during the whisking period. A surface contact is defined as the situation in which the contact point varies. Normally the surface contact depends on the surface slope of target objects. A square object often experiences one fixed contact position with the artificial whisker during the whisking period, but if there is a narrow angle between the object side and the neutral axis from the contact position to the whisker base, it may happen that the whisker can touch two edge points as shown in Fig. 5. In contrast, a round object has a varying surface slope starting with zero on the lateral surface (surface along the whisking direction), and the contact position continues to change along the surface. The distance decreases in the protraction mode and the whisker system will experience larger deflection angles than with point contact for each given protraction angle. 3. Experiments In this paper, we do not intend to recognize an arbitrarily shaped object. Instead we will try to discriminate regular shapes such as a cube, a cylinder, a pyramid and a cone, which are formed by circular and square discs. That is, the target objects are vertical stacks of square or circular discs. First, we will observe the deflection angle that the whisker sensors measure, and analyze its relevance for the estimation of contact distance. Second, we will investigate how the deflection signal changes with a round object or a square object, and see if the time course of deflection signal can be a cue to discriminate round and square objects. Third, we will show why multiple whiskers have advantages over a single whisker especially for shape recognition. Fourth, we test if the vertical displacement of a single whisker as well as the time difference of contact onsets or the difference of deflection amplitudes among multiple whiskers can discriminate different vertical shapes Deflection amplitude and velocity To analyze temporal deflection information, we collected the time course of the deflection signal of a whisker during Fig. 6. Time course of the deflection amplitude for a fixed distance (thick curve: low-pass filtered signal, dotdashed: time course of protraction angle, small oscillations represent natural frequencies). active whisking. The sensor signals (sampling rate 200 Hz) were pre-processed by a low-pass filter; a 5th order lowpass Butterworth filter was used with a cutoff frequency 5 Hz to remove signals related to the natural frequencies of the whisker beam. Fig. 6 shows a typical time course of the deflection amplitude for one fixed contact position. The protraction angle was measured by the encoder of the DC motor. The deflection signal that the sensors measure was almost linearly proportional to the deflection angle in theory, and the signal is a dominant feature for object localization. In the curve of deflection angle vs. protraction angle in Fig. 4(a), we can see that the monotonic region is in the range of protracting angles from 0 up to 30 for small distances, for example, 100 mm. To determine the distance from the observed deflection angles, we need to use the monotonic region to avoid ambiguity in the mapping; therefore, the robot restricts the protraction angle after contact to less than 30. To check the relationship between contact distance and deflection angle, we collected temporal sensor signals during whisker sweeps for different distances ( mm with 20 mm steps). It was assumed that the whisker does not pass by the object in the sweeping period. To see the influence of contact distance, we tested varying distances of a square object placed with a large side angle relative to the center of rotation such that the whisker has a point contact with it. The sensor signals were pre-processed by a low-pass filter. We can easily observe different deflection amplitudes depending on contact

6 234 D. Kim, R. Möller / Robotics and Autonomous Systems 55 (2007) Fig. 7. Deflection signal for a cube (a) deflection amplitude (b) deflection velocity. Fig. 8. Deflection signals depending on contact distance (top: deflection amplitude, bottom: deflection velocity) (a) square object for point contact (b) round object with diameter 100 mm (c) square object with side length 50 mm and a side angle β = 16 (the protraction angle after contact was set to 21, dotted: 0.62 Hz, dotdashed: 0.81 Hz, solid: 1.07 Hz sweeping frequency). distance see Fig. 7. As expected, a shorter contact distance produces a larger deflection amplitude. The deflection velocity, which is calculated as a derivative of deflection amplitude, is another cue to determine the contact distance. The deflection amplitude and speed in Fig. 7 are inversely proportional to the contact distance, as predicted in the theoretical estimation. To investigate the influence of lateral shapes on deflection angles, we measured the time course of deflection signals for two different objects, a square object and a round object, and determined the peaks of deflection amplitude and velocity. The peak of deflection amplitude is found at the end of protraction. The motor control circuit for the DC motor can adjust the sweeping frequency and protraction/retraction angle of the whisker. The protraction angle after the onset of contact and the total sweeping angle of the whisker were set to about 21 and 51, respectively. Fig. 8 displays the average over the peaks of deflection amplitude and velocity for 16 whisking cycles for each distance of the initial contact position ( mm with

7 D. Kim, R. Möller / Robotics and Autonomous Systems 55 (2007) Fig. 9. Deflection amplitude vs. protraction angle (a) square object (b) round object with diameter 110 mm (arrow: protraction angle 10 after contact, the deflection amplitudes for the two different types of objects are indistinguishable at this point). 10 mm steps). The error bars show 99% confidence intervals by assuming a t-distribution. The deflection amplitudes for contact distances of 180 mm or less are easily distinguishable. The deflection velocity also differentiates the contact distance. For point contact with a square object, the deflection amplitude and velocity have a relatively small magnitude, although they can discriminate contact distances see Fig. 8(a). When an artificial whisker sweeps past a round object such as a cylinder, the contact distance constantly changes along the curved surface. This is reflected in the deflection amplitude and velocity as shown in Fig. 8(b). For a square object placed with a side angle (see Fig. 3) β = 16 relative to the reference axis, the whisker touched two edge-points consecutively. Fig. 8(c) shows more pronounced deflection amplitude and velocity, since the contact distance is rapidly decreasing when the whisker touches the second edge point. The experiments imply that the type of a target object influences the deflection amplitude. If the type and distance of a target object are unknown, the peak values of deflection amplitude or velocity may not be sufficient to localize the object precisely, or recognize the shape. Thus, we need a systematic approach to solve the problem Lateral shape Recognizing the lateral shape of a target object is closely related to distance estimation. Discriminating surface contact and point contact often needs prior information of contact distance at the onset of contact. Here, we will explore several approaches to estimate the contact distance or to determine the contact property. For the experiments, we placed a round or square object at an angular position of about 40 in the sweeping area of whiskers, and collected deflection amplitudes of whiskers in the same horizontal plane Observing deflection amplitudes of a single whisker A set of deflection amplitudes for a sequence of protraction angles is observed with a single whisker sweep. The contact distance is estimated at small protraction angles and the discrimination test between surface contact and point contact is made at large protraction angles. In Fig. 9, we plot the deflection amplitude depending on the protraction angles. The deflection signal from zero up to approximately 40 of protraction angle is almost zero in amplitude, which indicates no deflection at all. For protraction angles larger than 40, the deflection amplitude increases in proportion to the protraction angle. Thus, we can estimate the angular position of a target object as well as the contact distance p in the curve of deflection amplitude vs. protraction angle. For large protraction angles, significant differences are visible for varying contact distances and also for lateral shapes. The deflection amplitude at protraction angle 50, which has a protraction of 10 after contact, may provide a better estimate of contact distance, because deflection signals at small protraction angles are less strongly influenced by the shape of a target object. In this way, a set of deflection signals for small protraction angles can determine the contact distance. Once the distance information is given, we can distinguish based on large protractions between a surface contact and a point contact. This method only needs the time course of deflection for a single whisker, but the distance estimation at small protraction angles may have a precision problem, when we consider the influence of noise or natural frequencies on deflection signals Observing deflection amplitudes of multiple whiskers When multiple whiskers in the same horizontal plane contact a target object, the advanced-touch whisker will have a large variation in the deflection amplitude depending on the shape of a target object when contact distance is fixed. In contrast, the whisker with a delayed touch has almost the same deflection level for a given contact distance over different types of objects since it has a relatively small protraction angle after contact. Thus, for a given contact distance, the difference of the deflection amplitudes or distance estimation between the two whiskers can discriminate point contact and surface contact. Surface contact has a larger difference of deflection between

8 236 D. Kim, R. Möller / Robotics and Autonomous Systems 55 (2007) Fig. 10. Deflection amplitude of whisker-1 and whisker-2 (a) square object with point contact (b) round object with diameter 110 mm (arrows indicate the contact onset moment of the delayed whisker (whisker-2), upward: distance 120 mm, downward: distance 200 mm). the whiskers than point contact. This method will be useful if the time course of motor position or protraction angle is not available. Fig. 10 shows the X Y plot of the two whisker signals over time. We will attend to the moment that the delayed-touch whisker starts to contact the object, and take the moment as a reference time for the measurement of deflection angles of the advanced whisker. Arrows in the figure show the onset time of contact of the delayed whisker (whisker-2). The deflection amplitude of the advanced whisker (whisker-1) at the reference time is inversely proportional to the contact distance. We can see in Fig. 10 that there are several branching curves which rise at different deflection angles from the horizontal base line. To discriminate surface contact from point contact, we can use the relative slope of deflection amplitudes of the two whiskers. A round object produces a lower slope in the curve of whisker-2 vs. whisker-1 than a square object with point contact; the rate of the amplitude increase of the advanced whisker is higher for a surface contact than that for a point contact, while the delayed whisker has a similar rate of the amplitude change for the two types of objects. Therefore, the X Y plot of the two whisker signals includes two kinds of information, the contact distance and the shape of a target object. At least the plot provides a cue to determine if the whisker has point contact or surface contact. We can consider another approach with multiple whiskers. The time-shift or angular difference of the contact onsets between a pair of whiskers can be examined from the whisker signals and it can determine the contact distance; larger timeshift indicates closer distance. The contact discrimination test may be done based on the contact distance. The robot knows what deflection levels would be expected at each protraction for point contact, and can compare the measured signal with the expected signal; if the measured signal is larger than expected, it can be regarded as a surface contact. However, in the experiments, the distance estimation with the time-shift often yielded a very low precision result. The whiskers are affected by the vibration in their natural frequencies, 3 which may cause a variation of contact onset time in the measurement. A more refined sensor design including damping material could improve the result. So far we introduced methods for object localization and lateral shape recognition. Now we describe the experiments to identify square or round objects using the above methods Square object For an edged-surface object, the deflection amplitudes depend on how the edged surface is placed in the sweeping area of the whisker. We adjusted the angle of a square object relative to the axis from the rotational axis to the contact position, which is denoted as β in Fig. 3. Fig. 11 shows an example of the effect of varying β s. For β s smaller than 30, the whisker touches two-edge points and the contact distance for the second edge point increases as β increases. Thus, small β s result in high deflection amplitudes. For large β s, for instance, β = 30 or greater, the whisker experiences a point contact, because it touches only the first edge point. The second whisker, which has a delayed touch, has almost constant deflection amplitude with varying β s, because it has a point contact with the first edge point. The graph of deflection amplitudes over the two whiskers can determine the rotation angle of the square object relative to the reference axis for a given contact distance. We can use the methods described above to estimate the contact distance. Generally we can easily discriminate point contact and surface contact by observing deflection amplitude or velocity. How can we determine if a square object is rotated? If the surface slope of the object is large, it is not possible to estimate 3 The distribution of natural frequencies depends on the boundary conditions of the whisker beam. Conspicuous natural frequencies observed in the whisker sensor signal correspond to the fixed-free boundary condition in which one tip of the whisker shaft is clamped at the whisker base and the other tip is free in space.

9 D. Kim, R. Möller / Robotics and Autonomous Systems 55 (2007) Fig. 11. Deflection amplitudes with a square object (side length 40 mm and contact distance 160 mm) (a) whisker-1 deflection vs. protraction angle (b) deflection curve for whisker-1 vs. whisker-2. the rotated angle with a single whisker. It will have a point contact with only one fixed contact position. If a higher level of deflection is observed than expected with point contact, we can conclude that it is a surface contact. In the time course of deflection (or the curve of deflection vs. protraction), we can find the protraction angle where there is a transition from single-edge to two-edge point contact. In Fig. 11(a), there are many branches in the upward direction from the curve for β = 30. The branches start at different protraction angles. The curve for β = 30 corresponds to point contact with a square object, and the branches with smaller β s reflect a contact with the second edge point. That is, the branching point indicates the moment when the whisker touches the second edge point. Also we can observe the deflection amplitude at other protraction angles after the branching angle. This will determine the contact distance of the second edge point. From the angular information and the contact distance, we can estimate the side length of a square object as well as the rotated angle. For example, β = 12 has a branching point at around 11 of protraction after contact. By the geometrical relation in Fig. 3, we can first find α = 4.2. Using Eq. (5), we can estimate θ d = 11.9, which is very close to the rotated angle 12 of the square object. If we calculate the difference between the deflection amplitude for β = 12 and the amplitude for β = 30 at a specific reference protraction, we can obtain a pure deflection amplitude that the second edge point with β = 12 contributes to. The branching curve has a deflection amplitude of 1 V at a protraction angle of 10 after the branch angle of protraction, while β = 30 has 0.6 V at the protraction angle. Thus, the second edge point has a contribution of about 0.4 V at a protraction angle of 10 after the contact, which corresponds to the contact distance 120 mm. Assuming the contact distance of the first edge point is estimated as 160 mm (by observing the deflection signals of the two whiskers), we can determine the side length of a square object by ( )/ cos(12 ) = 40.9 mm, which approximates the original size of 40 mm Round object Now we investigate the deflection signal of a single whisker for the case of round objects. Unlike edged-surface objects, round objects always have surface contact, and also the curvature of objects influences the deflection angles as shown in Fig. 5. Objects with larger curvature have a shorter contact distance for a given protraction angle, and as a result, they have higher deflection amplitudes in time course. For round objects with varying curvature, we measured the deflection amplitudes with respect to the protraction angle in the artificial whisker system. Fig. 12(a), (b) show examples of the deflection amplitudes which are averaged over five sweeps of whisking. For a given contact distance, a round object with a larger diameter size produces a significantly higher deflection amplitude than a round object with smaller diameter. Fig. 12(c), (d) show the effect of varying curvature of round objects on deflection amplitudes for a fixed contact distance. Round objects with large diameter do not only produce larger amplitudes, but also experience more variation of contact distances along the surface. The difference of deflection amplitudes over varying size of round objects is more prominent with large protraction angles. At small protraction angles up to 10 after contact, the deflection amplitudes for various round objects are very similar and the influence of the curvature is minute. This provides further support for using small protraction angles to estimate the distance for the initial contact position. The above deflection experiments reveal that a single whisker, within the sensor resolution, has the potential of distinguishing arbitrary pairs of contact distance and diameter size for round objects. The robot can estimate the contact distance of a round object at small protraction angles and determine the diameter at large protraction angles, for instance, at 20 of protraction after contact Discriminating square and round objects How can we discriminate a round object from a square object with two edge points when their deflection curve is similar?

10 238 D. Kim, R. Möller / Robotics and Autonomous Systems 55 (2007) Fig. 12. Deflection amplitude vs. protraction angle for varying contact distances and varying size of round objects (a) circular disc with diameter 45 mm (b) circular disc with diameter 100 mm (c) varying diameters of circular discs with contact distance 140 mm (D: diameter size) (d) varying diameters of circular discs with contact distance 120 mm. It would be a difficult task to discriminate a curved surface and an edged surface including more than one point contact using the deflection amplitude alone, if the size and distance of objects are unknown. The whiskers may not discriminate the two objects easily, if they have a similar transition of contact distances along the surface. The task needs high precision sensors to measure the transition of deflection. Theoretically their curves of deflection angle over time are different (an edged surface will produce a more sudden change of deflection as well as high deflection velocity at the transition between edges), but noisy sensor signals and the presence of the whisker beam s natural frequencies make these differences hard to detect. An alternative solution to the problem is that we can change the robot position and collect another set of deflection signals. For instance, after further moving the robot towards the target object by 10 mm, we can again collect the deflection signals for the two whiskers. Then we can estimate the size and shape of the object by observing the sequence of X Y plot of the whisker signals. Fig. 13 shows the X Y plot of two whiskers in the same horizontal plane for varying distances. A square object at a distance of 150 mm has very similar deflection curve to a round object at a distance of 140 mm and thus the shape information cannot be retrieved although we may determine if the touch is a point contact or surface contact. If the robot changes the contact distance, the response curves of deflection for the two objects are completely different. Thus, a collection of deflection signals at different distances can provide the appropriate shape information. It is an advantage of multiple whiskers over a single whisker that a round object and an edged-surface object can be discriminated more precisely. In principle, a single whisker can distinguish point contact from surface contact, using the deflection signal. However, adding localization information by another whisker will provide more reliable information of the contact distance and shape of a target object Vertical shape by the slip of a single whisker Kaneko et al. [14] provides a geometrical analysis on the relationship between varying force directions and the corresponding slip. In their experiments, the force direction of the beam was iteratively changed towards a normal direction

11 D. Kim, R. Möller / Robotics and Autonomous Systems 55 (2007) Fig. 14. Slope test configuration; the angle of aluminium bar is adjusted for the slope test, and the slope angle is measured counter-clockwise from the horizontal line. Fig. 13. Deflection amplitudes of two whiskers for round and square objects (the side length of the square object was 35 mm, which was rotated with a side angle 15 and the diameter of round object was 110 mm, arrow: indistinguishable deflection signals for a square object at a distance of 150 mm and a round object at a distance of 140 mm). with respect to the surface with the help of slip information. However, there was no process to translate the whisker slip into the shape of objects directly and the method required a set of contact positions to build a contour of an object. In our active whisker system, we use two channels of sensor signals, horizontal and vertical, together to evaluate the slip of the whisker. The deflection of both channels is directly correlated with the surface slope of target objects. Fig. 14 shows how we tested this method of slope recognition. We first use an aluminium bar whose slope angle can be adjusted freely. The slope angle is measured anticlockwise from the horizontal line and the whisker touches the aluminium bar from the side. Normally the whisker shaft follows the angle of slope and the sensors at the whisker base can estimate the slope angle from the X Y plot. Fig. 15(a) shows examples of the X Y plot. The whisker draws an elliptical deflection curve and the curves in the inner part (which are represented as thick lines in the figure) indicate the surface slope of the test bar (the inner and outer parts indicate the protracting and retracting phase in a whisking period, respectively). Each slope angle was measured over 10 sweeping cycles, and a set of averaged slope angles is shown in Fig. 15(b). The estimate of the slope angle from the X Y plot quite well agrees with the real slope of the bar. As an extension of the experiments, we tested objects with conical shapes. Fig. 16 shows the deflection signals of two types of cones for 3 5 sweeps of a whisker, and the inner part (protraction phase) of the curve for each whisking cycle shows a steady slope, which represents the surface slope of a cone. To see the effect of the contact protraction angle 4 on the slope estimation, we measured the two channel sensor signals over several contact protraction angles (4 16 ). Regardless of the 4 The robot can adjust the maximal protraction angle (sweeping angle) and we measured the protraction angle after the onset of contact, corresponding to the sweeping angle during the contact duration. This is called the contact protraction angle. protraction angles, the slope information can be obtained from the X Y plot. In the retraction period, they have a different course of whisker movement, depending on the curvature of the object, the protraction angle, surface friction and the elastic property of the whisker material. Presumably, the shape of the X Y plot may reveal the curvature of an object under certain conditions Vertical shape by multiple whiskers In the case of ragged surfaces or strong friction, or for steep vertical slopes, the artificial whisker will not be able to slide along the object s surface. To overcome this limitation of a single whisker, we use multiple vertically stacked whiskers to recognize the vertical shape of a target object. We built a conically-shaped object consisting of circular disks with different diameters (each disc has a height of 10 mm), and we stacked two magnetic whiskers vertically. In this case, we do not observe any whisker slip in vertical direction. The onset time of deflection can determine the angular position of an object as well as the vertical shape. For example, a conically shaped object produces a time-shift of the deflection signal between the upper whisker and the lower whisker. Fig. 17 shows the time course of deflection signals for a conically-shaped object. In Fig. 17(a), the upper whisker has a relatively small amplitude, since its contact duration with the object is short. We can first estimate the distance information by the magnitude of deflection amplitude or deflection velocity. Then we calculate the difference between the peak amplitude in the upper whisker signal and that in the lower. The sign of the difference will determine which part, the upper or the lower, is contacted first. The difference between the two amplitudes at a given contact distance will provide information about the size difference between the upper and the lower part. We can also estimate the vertical shape by a time shift in deflection velocity signals see Fig. 17(b). However, fast sweeping produces a small time shift, which may result in a precision problem when estimating the relative size difference. It seems that the difference between the deflection amplitudes can be a more prominent cue to recognize the vertical shape. To analyze the time shift of deflection between the upper and the lower whisker, two types of circular discs were stacked to build a test object, which has a distance 180 mm for the initial contact position. The lower part has a disc whose diameter was

12 240 D. Kim, R. Möller / Robotics and Autonomous Systems 55 (2007) Fig. 15. X Y plot of deflection signals (a) examples of X Y plot over two channels with several slope tests (thick lines: estimated slope) (b) estimation of slope with X Y channels ( : real data, solid line: theory) Fig. 16. X Y plot of deflection signals for two types of conically-shaped objects (dotted: 16, dotdashed: 12, dashed: 8, solid: 4 of contact protraction angle). Fig. 17. Time course of deflection signals for a conically-shaped object (a) deflection amplitude (b) deflection velocity (the upper and lower whiskers contact circular disks with 105 mm and 70 mm diameters, respectively).

13 D. Kim, R. Möller / Robotics and Autonomous Systems 55 (2007) Fig. 18. Time difference and amplitude difference of two whiskers stacked vertically for relative size difference (sweeping frequency 0.77 Hz is used and the error bar indicates 95% confidence interval by t statistic over five whisking cycles) (a) time difference (b) difference of deflection amplitude at the end of protraction (lower whisker upper whisker). 110 mm and the upper one has varying sizes from 25 mm to 115 mm. The discs are positioned such that both vertically stacked whiskers can touch the discs. Thus, the maximum size difference will be 85 mm. Fig. 18 shows the averaged time difference and amplitude difference over five whisking cycles for varying size differences; low-pass filtered signals are used as in the above experiments. The time difference was calculated by measuring the instantaneous times at the peaks of deflection velocities, where the time for the maximal deflection velocity is close to the contact onset 5 time. The amplitude difference was calculated by measuring deflection amplitudes at the end of protraction. The time difference or amplitude difference is almost linearly proportional to the size difference of circular discs. As shown in Fig. 18, the amplitude difference can predict the relative size difference more exactly, since the time shift of deflection between the two whiskers tends to be influenced by noise. As an alternative sensor technology, an array of piezoelectric sensors directly produces the deflection velocity instead of the deflection amplitude. A stack of piezo-type whiskers provide the time-shift in the velocity signals depending on the vertical shape of an object. Thus, the shape of an object can be determined from the temporal whisker signal obtained for a given sweeping speed. The pattern of the velocity signals from piezoelectric sensors is quite close to that from magnetic sensors (not shown in this paper). The velocity signal of piezoelectric sensors can be a dominant feature to detect the vertical shape of an object, as is the deflection amplitude measured with magnetic sensors. Unlike the above results with magnetic sensors for the vertical shape, it seems that real rodents might have a preference to use deflection velocities rather than deflection amplitudes, because neurons in barrel cortex are more responsive to the deflection velocity 5 Actually, the time for the maximal velocity does not exactly match the onset time of contact, since we use low-pass filtered outputs over noisy sensor signals. or vibration speed of rodent whiskers than to deflection amplitude [23,2,3]. 4. Discussion We suggested that localization information of a target object should be the basis of shape recognition. Active whisking allows one to determine the distance and angular position of an object only by measuring the time course of deflection amplitude. The method is not affected by the elastic property of the whisker material or by the diameter of the whisker. The deflection amplitude and deflection velocity signals shown in the experiments correspond to the activation of slowly adapting (SA) and rapidly adapting (RA) mechanoreceptors at the follicle, respectively. The two types of neurons are very common in biological tactile sensors such as the skin or the whisker follicle of animals [12,24]. The former type of neurons can signal stimulus magnitude for a given pressure, while the latter type of neurons cease firing in response to sustained stimulation but is sensitive only when the stimulus amplitude changes. Recent physiological experiments show that SA and RA neurons in the trigeminal ganglion distinctively respond to several modes of active whisking such as touching an object, whisker bending, and detaching from an object [24]. Moreover, it was reported that neurons in the barrel cortex encode the deflection velocity of rodent whiskers [23,2,3], which is closely related with the property of RA cells. In contrast, the SA cells at the follicle can differentiate the deflection level of the whisker shaft [9] and achieve directional selectivity which seems to be passed to the thalamic neurons in the brain [17]. Thus, our active whisker system is closely in line with the biological process to use deflection amplitude and velocity as sensory afferents. It was also shown that modeling the mechanoreceptors as strain sensors for deflection can produce similar patterns of ganglion neuron firings [19]. Thus, the suggested method of tactile perception appears to be a plausible mechanism for rodent whisking behaviour. The deflection

14 242 D. Kim, R. Möller / Robotics and Autonomous Systems 55 (2007) Table 1 Methods with artificial whiskers ; each method is described in the following sections, s1: Section 3.1, s2, s3: Section 3.2.1, s4: Section 3.3, m1, m2: Section 3.2.2, m3: Sections , m4: Section 3.2.5, m5, m6: Section 3.4 Single whisker No. Method Task s1 Deflection amplitude/velocity Object localization a s2 Deflection signals at various protractions Point and surface contact a s3 Deflection signals at various protractions Lateral shape s4 Two orthogonal channels Slope of objects (smooth surface) a Multiple whiskers No. Method Task m1 Time-shift of contact onsets Object localization b m2 Deflection at the contact onset of a delayed whisker Object localization m3 X Y plot over whisker-2 vs. whisker-1 Lateral shape m4 Collection of deflection signals at various robot positions Lateral shape a m5 Time-shift of deflection velocity Vertical shape b m6 Difference between deflections of multiple whiskers Vertical shape (ragged surface) a a Indicates high precision results. b Indicates low precision results. signals of whiskers can be one of the main sources of shape information. Shape information of an object is divided into two parts, lateral and vertical shape information with respect to the whisking direction. Multiple whiskers provide more reliable information of shape or distance of a target object than a single whisker. In addition, the shape recognition may be involved with the integration of motion direction across multiple vibrissae [1]. Brecht et al. [5] assumed that the whiskers of rodents may be binary sensors which only tell if touched or not, and that the clusters of whiskers at each side of face play a major role in detecting the distance of an object. Our experiments support that each whisker has the capability to provide continuous distance information rather than just binary information as well as to track directional cues. The shape can be derived from a combination of distance, the two channel information of a whisker, or the time difference of contact onsets retrieved using multiple whiskers. It seems that head movement is involved with shape recognition or texture discrimination of rodents [5,11]. In our experiments, precise recognition of shape, for example, to distinguish a curved surface from an edged surface, would need a collection of whisking signals at different robot positions, which may resemble the behaviour pattern that rodents use to refine tactile sensory information. We summarize the methods used for distance estimation and shape recognition with our artificial whiskers in Table 1. In principle, the time course of deflection signals with a single whisker can be a direct source of lateral shape information, if high precision of sensor signals are available. Our experiments show that a single whisker alone can distinguish point contact and surface contact, but it may have difficulty in discriminating a curved surface and an edged surface, since the precision of the deflection signal is reduced by noise and natural frequencies. So far there is no physiological or behavioral data available on the precision of tactile localization or shape detection in rodents. It is still an open question whether rodents can discriminate complex shapes with a single whisker, or whether they interpret each whisker sensor as a simplified binary sensor or low-precision sensor and use multiple whiskers for shape recognition processing. 5. Conclusion In this paper, we suggest a new approach to recognizing the shape of a target object with a mechanical probe. The method uses biomimetic whiskers to model the design of the follicle of rodent whiskers. Deflection amplitude or deflection velocity signals provide localization information of a target object, that is, contact distance and angular position of an object. Also they show the potential of detecting lateral shape and the vertical shape of objects. For lateral shape recognition, we found the deflection amplitude changes depending on whether a sweep of whiskers experiences point contact or surface contact along the surface of a target object. This effect is caused by the dependency of deflection on the contact distance. The deflection signal helps discriminate round objects with a varying curvature or objects with different lateral shapes, like round and square objects. We present several methods to estimate the contact distance and to detect the lateral shape of a target object with a single whisker or with multiple whiskers. Multiple whiskers can provide more precise information of shape and contact distance rather than a single whisker. A single whisker with two channels arranged orthogonally to each other can recognize the slope of the vertical shape. Alternatively, the vertical shape can be estimated by two or more whiskers stacked vertically. For instance, a conically shaped object will produce a time-shift in whisker signals between the upper whisker and the lower whisker. The time-shift of contact can be measured by observing the deflection velocity. Another approach is to use the time course of deflection amplitudes, since the difference of deflection amplitudes also provides the vertical shape information.

15 D. Kim, R. Möller / Robotics and Autonomous Systems 55 (2007) The suggested methods for shape recognition may provide a clue to how real rodents process tactile signals of their whiskers for shape recognition, and can be applied to provide autonomous robots with the same capabilities. Acknowledgement This work has been supported by the EU in the project AMOUSE (IST ). References [1] M.L. Andermann, C.I. Moore, A somatotopic map of vibrissa motion direction within a barrel column, Nature Neuroscience 9 (2006) [2] E. Arabzadeh, R.S. Petersen, M.E. Diamond, Encoding of whisker vibration by rat barrel cortex neurons: Implications for texture discrimination, Journal of Neuroscience 27 (2003) [3] E. Arabzadeh, E. Zorzin, M.E. Diamond, Neuronal encoding of texture in the whisker sensory pathway, PLOS Biology 3 (1) (2005) [4] S. Bovet, R. Pfeifer, Emergence of delayed reward learning from sensorimotor coordination, in: Proceedings of the Int. Conf. on Intelligent Robots and Systems, IEEE, 2005, pp [5] M. Brecht, B. Preilowski, M.M. Merzenich, Functional architecture of the mystacial vibrissae, Behavioural Brain Research 84 (1 2) (1997) [6] G.E. Carvell, D.J. Simons, Biometric analyses of vibrissal tactile discrimination in the rat, The Journal of Neuroscience 10 (8) (1990) [7] M. Fend, Whisker-based texture discrimination on a mobile robot, in: Proceedings of the European Conf. on Artificial Life, Springer Verlag, 2005, pp [8] M. Fend, S. Bovet, H. Yokoi, R. Pfeifer, An active artificial whisker array for texture discrimination, in: IEEE Conf. on Int. Robots & Systems, IEEE, 2003, pp [9] K.-M. Gottschaldt, A. Iggo, D.W. Young, Functional characteristics of mechanoreceptors in sinus hair follicles of the cat, Journal of Physiology 235 (1973) [10] J.A. Hartings, S. Temereanca, D.J. Simons, High responsiveness and direction sensitivity of neurons in the rat thalamic reticular nucleus to vibrissa deflection, Journal of Neurophysiology 83 (5) (2000) [11] M.J. Hartmann, Active sensing capabilities of the rat whisker system, Autonomous Robots 11 (2001) [12] E.R. Kandel, J.H. Schwartz, T.M. Jessell, Principles of Neural Science, fourth ed., McGraw-Hill, New York, [13] M. Kaneko, N. Kanayama, T. Tsuji, Vision based active antenna, in: International Conference on Robotics and Automation, IEEE, 1996, pp [14] M. Kaneko, N. Kanayama, T. Tsuji, Active antenna for contact sensing, IEEE Transactions on Robotics and Automation 14 (2) (1998) [15] A.G. Kelly, Fundamentals of Mechanical Vibration, second ed., Mc-Graw Hill, [16] D. Kim, R. Möller, A biomimetic whisker for texture discrimination and distance estimation, in: S. Schaal, et al. (Eds.), From Animals to Animats 8, Proceedings of the International Conference on the Simulation of Adaptive Behavior, MIT Press, 2004, pp [17] S.H. Lichtenstein, G.E. Carvell, D.J. Simons, Responses of rat trigeminal ganglion neurons to movement of vibrissae in different directions, Somatosensory and Motor Research 7 (1) (1990) [18] M. Lungarella, V.V. Hafner, R. Pfeifer, H. Yokoi, An artificial whisker sensor for robotics, in: IEEE Int. Conference on Intelligent Robots and Systems, IEEE, 2002, pp [19] B. Mitchinson, K. Gurney, P. Redgrave, C. Melhuish, M. Pearson, I. Gilhespy, T. Prescott, Empirically inspired simulated electromechanical model of the rat mystacial follicle-sinus-complex, Proceedings of Royal Society B: Biological Sciences 271 (1556) (2004) [20] D.J. Pinto, J.C. Brumberg, D.J. Simons, Circuit dynamics and coding strategies in rodent somatosensory cortex, Journal of Neurophysiology 83 (3) (2000) [21] R.A. Russell, Using tactile whiskers to measure surface contours, in: International Conference on Robotics and Automation, IEEE, 1992, pp [22] A. Seth, J. McKinstry, G. Edelman, J. Krichmar, Spatiotemporal processing of whisker input supports texture discrimination by a brainbased device, in: S. Schaal, A. Ijspeert, A. Billard, S. Vijayakumar, J. Hallam, J.-A. Meyer (Eds.), From Animals to Animats 8, Proceedings of the International Conference on the Simulation of Adaptive Behavior, MIT Press, 2004, pp [23] M. Shoykhet, D. Doherty, D.J. Simons, Coding of deflection velocity and amplitude by whisker primary afferent neurons: Implications for higher level processing, Somatosensory and Motor Research 17 (2) (2000) [24] M. Szwed, K. Bagdasarian, E. Ahissar, Encoding of vibrissal active touch, Neuron 40 (2003) [25] T. Tsujimura, T. Yabuta, Object detection by tactile sensing method employing force/torque information, IEEE Transactions on Robotics and Automation 5 (4) (1989) [26] N. Ueno, M. Kaneko, Dynamic active antenna, in: Proc. of Int. Conf. on Robotics and Automation, IEEE, 1994, pp [27] J.F. Wilson, A. Chen, A whisker probe system for shape perception of solids, ASME Journal of Dynamic Systems, Measurement and Control 117 (1995) DaeEun Kim received his B.E. and MSc(Eng) in the department of computer science and engineering from Seoul National University and the University of Michigan at Ann Arbor, respectively. He was a lecturer in Korea Air Force Academy from 1994 to 1997 and then joined a research team involving with the bus-underground smartcard ticketing system in Seoul. He received Ph.D. degree from the University of Edinburgh in From 2002 to 2006, he was a research scientist in Max Planck Institute for Human Cognitive and Brain Sciences. Currently he is a senior researcher in University of Leicester. His research interests are in the area of biorobotics, autonomous robots, artificial life, neural networks and neurobiology. Ralf Möller received a Ph.D. in electrical engineering from the Technical University of Ilmenau, Germany, and a Venia legendi in computer science from the University of Zurich, Switzerland. He is heading the Computer Engineering Group at the Faculty of Technology of Bielefeld University. His research interests include neural networks, models of visual cognition, visual robot navigation, and biorobotics.