Measurement of Force Vector Field of Robotic Finger using Vision-based Haptic Sensor

Transcription

1 2008 IEEE/RSJ Iteratioal Coerece o Itelliget Robots ad Sstems Acropolis Covetio Ceter Nice, Frace, Sept, 22-26, 2008 Measuremet o Force Vector Field o Robotic Figer usig Visio-based Haptic Sesor Katsuari Sato, Kauto Kamiama, Hideaki Nii, Naoki Kawakami, ad Susumu Tachi, The Uiversit o Toko, Japa Abstract It is epected that the use o haptic sesors to measure the magitude, directio, ad distributio o a orce will eable a robotic had to perorm deterous maipulatios. Thereore, we have developed a ew tpe o iger-shaped haptic sesor that ca measure a three-dimesioal orce vector ield over a cotact surace. The sesor cosists o a trasparet elastic bod, two laers o iteral blue ad red markers, ad a CCD camera to capture the movemets o the markers. Usig the elastic theor, we ca calculate the orce vector ield rom the captured movemets o the markers. However, the elastic theor caot be applied to the iger-shaped sesor because o its complicated shape. Thereore, we use actual measuremets or the calibratio i order to develop a prototpe o the sesor. The, we evaluate its basic perormace. The result shows that the sesor perormace ca be improved urther, ad the sesor ca be successull used i a robotic had. the magitude, directio, ad distributio o a applied orce. Covetioal sesors mail measure ol oe or two o these parameters. However, all three parameters are essetial or stable graspig ad deterous maipulatio. Secod, the sie o the sesor should be small. I order to set ma haptic sesors i a humaoid robot had, it is preerable to use compact sesors. Fiall, the sesor should be a high-perormace device. I order to obtai accurate haptic iormatio, high magitude, spatial, ad temporal resolutios are better. I. INTRODUCTION HAPTIC iormatio acquired through the igertips is importat or huma sice it allows them to maipulate a object deterousl. The object ca be grasped i a stable maer usig haptic iormatio such as the relective orce, rictioal orce, ad poit o cotact with the object. Robotic sstems use a similar methodolog. I order to pick up, move, or la dow a object, a robotic had requires haptic iormatio. Thereore, robotic hads require haptic sesors to maipulate objects deterousl. Recetl, two pricipal tpes o haptic sesors have bee iveted or robotic hads [1] [2]: a orce vector sesor that measures three- or si-dimesioal orce at a poit ad a orce ield sesor that measures the distributio o a oe-dimesioal orce. The objective o our research is to achieve deterous maipulatio i various activities such as rollig cla betwee the igertips b usig a robotic had with a haptic sesor (Fig. 1). We have laid dow three requiremets or the haptic sesor o the robotic had. First, the sesor should be able to measure Katsuari Sato, Hideaki Nii, Naoki Kawakami, ad Susumu Tachi are ailiated with the Graduate School o Iormatio ad Techolog, The Uiversit o Toko, Japa;; (Katsuari_Sato,Hideaki_Nii)@ipc.i.utoko.ac.jp,(kawakami, tachi)@star.t.u-toko.ac.jp. Kauto Kamiama is ailiated with the Research Ceter or Advaced Sciece ad Techolog, The Uiversit o Toko, Japa; kauto@hal. rcast.u-toko.ac.jp. This research was iaced b Nitta Corporatio. Fig. 1. Eample o maipulatio b robotic had usig our igertip sesor. The blue ad gree suraces show i the laptop displa represet the iormatio measured b the sesors mouted o the ide iger ad the thumb o the robotic had, respectivel. The irst requiremet ca be satisied b itegratig ma orce vector sesor elemets [3] [4]. However, the use o ma sesors ca complicate the sesor structure ad decrease the sesor perormace. Ferrier et. al. have proposed aother approach to satis the irst requiremet [5]. The have developed a small tactile sesor to measure the orce vector ield. However, this sesor requires a comple processig method to acquire the directio o orce, leadig to iaccurac ad istabilit. I order to satis all the abovemetioed requiremets, we have created a iger-shaped haptic sesor that ca measure a three-dimesioal orce vector ield applied o its surace. The techolog used b this sesor is the same as that used i the visio-based haptic sesor developed b us [6]. This /08/$ IEEE. 488

2 sesor has a simple structure; hece, it satisies the secod ad third requiremets, i.e., havig small sie ad high resolutio. I this paper, we preset the measuremet theor, coiguratio, ad calibratio method o the iger-shaped haptic sesor. I additio, we evaluate its accurac whe it is used as a orce vector sesor ad a orce ield sesor. I this equatio, u idicates the two-dimesioal movemet vectors o the red ad blue markers captured b the camera is the three-dimesioal orce vector, ad H is the coversio matri. The elemets o H are calculated b usig the elastic theor. We obtai rom u b solvig the iverse problem [8] o equatio (1). II. FINGER-SHAPED HAPTIC SENSOR T 1 T ( H H ) H u. (2) A. Theor or measuremet o orce vector ield I order to eable the iger-shaped haptic sesor to measure the orce vector ield, we have applied the techolog used i the visio-based haptic sesor. This sesor cosists o a trasparet elastic bod with two laers o a marker matri ad a color CCD camera. Whe a orce is applied to the surace, the markers move as show i Fig. 2. The movemets o the markers are captured b the camera. We calculate the ceter o the markers beore ad ater applicatio o the orce. Thereore, we ca record the movemets o markers ad calculate the applied orce usig the elastic theor [7]. side view top view deormatio orce Fig. 2. Movemets o markers i elastic sesor bod caused b orce applied to sesor surace [6]. We use two suppositios to calculate the orce vector ield rom the movemets o the markers. The irst suppositio is that the sesor is a semi-iiite elastic bod. The secod suppositio is that the deormatio o the elastic bod is liear. O the basis o these suppositios, the relatioship betwee the movemet o marker ad orce is give b equatio (1) as ollows: u H. (1) u hr, hr, h R R u hr, hr, h R R u, H u B hb, hb, hb ub h B, hb, hb It should be oted that the visio-based sesor uses a sigle camera to capture the movemets o the two laers o markers. I order to calculate the three-dimesioal orce vector, it is ecessar to measure the movemets o the markers i a three-dimesioal maer. A stereo camera is suitable or capturig the movemets o the markers. However, the use o a stereo camera icreases the sie o the sesor ad the diicult i calibratig the camera. Hece, to overcome these problems, we use two laers o markers ad oe camera. This visio-based haptic sesor does ot require a electroic compoets such as resistors or capacitors. Hece, it is eas to adapt the shape ad sie o the sesor to that o the igertip. I additiourther developmets i camera techolog will result i improved spatial ad temporal resolutio o the sesor. The elastic bod o the sesor provides a added advatage. The elasticit eables the sesor to appropriatel measure the haptic iormatio b reproducig the phsical iteractio ivolved i the sese o touch. Furthermore, the sesor is highl durable. B. Desig o sesor We costruct the igertip sesor or the robotic had b applig visio-based techolog. The coiguratio o the sesor is show i Fig. 3. The elastic bod ad CCD camera are attached to the plastic base o the robotic had. The elastic bod is made rom trasparet silico rubber. The shape o the elastic bod is partl spherical ad partl clidrical; hece, it resembles a huma igertip. There are blue ad red markers o the surace o the elastic bod. Because the sesor uses visio-based techolog, light rom the eviromet produces oise. Thereore, the surace o the sesor is covered b a black sheet made rom the silico rubber. I additio, to capture the marker movemets clearl, the are illumiated b a LED mouted ear the camera. We use a camera with a wide-agle les so that the camera ca capture all the markers. C. Calibratio method I covetioal visio-based haptic sesors, the applied orce is calculated rom the movemet o the markers usig equatio (1). However, equatio (1) has bee derived o the basis o two suppositios. These suppositios are ot applicable to the iger-shaped haptic sesor. Hece, we estimate the elemets o the coversio matri H rom actual 489

3 measuremets. Figure 4 shows the algorithm or the costructio o H. First, a orce is applied o the surace o the sesor alog three directios; the compoets o the orce alog the three directios are deoted b, ad. The, we measure the orce elemets caused b applig the orce as ollows: where m idicates the m-directioal elemet o orce whe -directioal orce is applied. Net, we capture the movemets o the markers u. The -directioal marker movemets o a marker are give b equatio (4). [ u, u u ] (3) u, (4) where um idicates the m-directioal movemet o marker whe -directio orce is applied. Usig ad u, we ca calculate the elemets o matri H as ollows: 1 ( h [ h, h h ]) h u, (5) CCD camera base red ad blue trasparet elastic bod marker matri Fig. 3. Coiguratio diagram o iger-shaped visio-based haptic sesor elastic bod H h1 h2 hn 3. Costructio o matri H LED 1. Applicatio o kow orce to surace N 2. Measuremet o marker movemet u Fig. 4. Algorithm or costructio o coversio matri H. The elemets o H are calculated rom the measured orce ad the movemets o the markers deoted b u. Figure 5 shows the apparatus used or the costructio o matri H. We push the sesor surace b usig a probe. The probe edge is spherical ad the diameter is 3.0 mm. The probe is mouted o a si-ais orce sesor to measure the magitude o the applied orce. Further, we use a stage to cotrol the applied orce. We irst appl a orce alog the -directio ad the appl suitable orces alog the - ad -directios. si-ais orce sesor probe iger-shaped haptic sesor jig or -stage Fig. 5. Image o apparatus used or costructio o coversio matri H. D. Costructio o prototpe o iger-shaped haptic sesor We have costructed a prototpe o the iger-shaped haptic sesor. The dimesios o the trasparet elastic bod o the prototpe are mm³. The elastic bod is made rom silico rubber (Shi-Etsu Chemical Co., KE-109). The camera is a custom-made oe comprisig a video compressio uit ad our image capturig uits (ViewPLUS Ic.). The resolutio o each image is piels. The ceters o the markers are tracked with sub-piel accurac. The rame rate o the camera is 60 ps. Figure 6 shows the captured images o the markers. To capture the markers clearl, their diameters are set to 0.5 mm (approimatel 15 piels). Moreover, i order to prevet the overlappig o the red ad blue markers, the are separated b a distace o 2.5 mm (Fig. 6, let). The red ad blue markers are placed at depth o 0.5 ad 2.0 mm, respectivelrom the surace o the sesor. We prit the markers usig the silk-scree pritmakig techique. The markers are made rom silico rubber ad a colorat. The calibrated positios o the orces are show i Fig. 6 (right). I the prototpe, the markers are separated b a distace o 2.5 mm. Thereore, we appl a orce at 25 poits o the surace o the sesor. The poits are also separated b a distace o 2.5 mm. The magitude o the applied orce is approimatel 100 g. Figure 7 shows the costructed prototpe ad the calculated orce vector ield. Whe the sesor surace comes i cotact with a object, it measures the correspodig orce vector ields. 490

4 2.5 mm 2.5 mm samplig poit 2.5 mm 2.5 mm magitude alog the three directios:,, ad. Each directioal orce, ad, is represeted b a oe-dimesioal orce vector ad the summatios o the orce deoted b F, F, ad F, respectivel, are calculated as show i equatio (6). Fig. 6. Images o markers captured b camera (let) ad poits used or samplig orce (right). F F F k 1 k 1 k 1 ( k) ( k) ( k) (6) Fig. 7. Images o costructed sesor (let) ad captured orce vector ield (right). III. EVALUATION The iger-shaped haptic sesor ca be used both as a orce vector sesor ad as a orce ield sesor. Thereore, we evaluate the basic perormace o the costructed prototpe usig the ollowig our criteria: the accurac o measuremet, the resolutio o the orce magitude whe it is used as a orce vector sesor, the spatial resolutio whe it is used as a orce ield sesor, ad the time required or calculatio. The apparatus used i the evaluatio o the eviromet is the same as that used or calibratio (Fig. 6). A. Measuremet accurac The orce vector ield is calculated rom the movemets o the ceters o the markers captured b the camera. Thereore, a error that occurs while capturig the movemets o the markers ma produce oise i the measuremet o the orce. I this sectio, we evaluate the accurac o the measuremet o the movemets o the markers whe o orce is applied to the surace o the sesor. We capture two images o the markers whe o orce is applied. The, we displace the markers i oe o the images b 5 piels alog the -directio. The displacemet vectors o the markers are calculated usig these two images. Table I shows the average ad the stadard deviatio o the - ad -displacemets o the red ad blue markers. The result shows that the movemets o the markers are measured with subpiel accurac. B. Resolutio o magitude The iger-shaped haptic sesor ca be used as a orce vector sesor. We evaluate the resolutio o the orce We eamie the resolutio ad liearit o the measured orce vectors b comparig the measured orce F with the applied orce. The applied orce is measured b usig the si-ais orce sesor ad the cotrolled stage. I this evaluatio, we appl a orce usig a clidrical probe with a diameter o 10 mm. To ivestigate the resolutio o magitude i the - ad -directio, we irst applied a orce o 200 g alog the -directio. We the applied the orce alog the - or -directio with magitudes i the rage o g b icreasig i steps o 20 g. To ivestigate the resolutio alog the -directio, we applied a orce alog the -directio with magitudes i the rage o g b icreasig i steps o 20 g. The values obtaied rom the sesor are recorded. The measuremet procedure is repeated 5 times or each directio. The graphs show i Fig. 8 preset the results o the evaluatio. The horiotal ad vertical aes represet the applied orce ad the measured orce, respectivel. The dots show the results o each trial. From these results, it is observed that the maimum resolutio o magitude is 30 g i each directio. Thus, the liearit o the sesor is coirmed. TABLE I MEASUREMENT OF MARKER MOVEMENT Marker Directio Average (piel) Std. Deviatio (piel) Red Blue C. Spatial resolutio The iger-shaped haptic sesor ca be used to measure a orce ield. I this sectio, we evaluate the spatial resolutio o the sesor. For the evaluatio, we obtai the magitude o each orce vector whe a orce is applied at a certai poit o the surace o the elastic bod. The magitude o the orce vector is calculated as show i equatio (7) (7) 491

5 Measured orce F [g] Measured orce F [g] Measured orce F [g] Applied orce F [g] Applied orce F [g] Applied orce F [g] Fig. 8. Relatio betwee applied orce ad measured orce i -directio (top), -directio (middle), ad -directio (bottom). It should be oted that the samplig iterval is ot suicietl small to evaluate the spatial resolutio. Thereore, we adopt the ollowig procedure to evaluate the spatial resolutio. Two adjacet samplig poits o the sesor surace are selected. The distace betwee these two poits is subdivided ito poits, ad a kow orce is applied i the -directio at each poit. The, the orce vectors at each poit are calculated b cosiderig each poit as the origi. A graph is costructed b plottig the distace rom each origi ad the calculated orce magitude o the horiotal ad vertical aes, respectivel. B usig the method eplaied above, the desit o the poits i the samplig regio is icreased. The spatial resolutio o the orce distributio ca be obtaied rom the slope o the graph. The magitude o the orce applied i the -directio is ied at 200 g. The diameter o the probe used to appl the orce is 3.0 mm. The adjacet samplig poits are located at the ceter ad the edge o the area i which the orce ield is measured. The actual samplig iterval is 2.5 mm, ad the distace betwee adjacet poits is subdivided ito itervals o 0.5 mm. We evaluate the spatial resolutio at both the ceter ad the edge o the sesor surace ad i both the - ad -directios. Figure 9 shows the results o the eperimet. The irst graph shows the results obtaied b applig a orce at the ceter o the sesor. The secod graph shows the results obtaied b applig a orce at the edge o the sesor. The values plotted o the vertical ais are ormalied usig the maimum value o the orce magitude. The dots i the graphs represet the poits at which the orce is applied ( [mm], [mm]). From the graphs i Fig. 9, it ca be observed that the ull width hal maimum is approimatel 5 mm. Thereore, the spatial resolutio o the orce is estimated to be approimatel 5.0 mm. These results are idetical to those obtaied i the evaluatios at other positios ad directios. Thereore, the complicated shape o the sesor does ot aect the spatial resolutio. The discotiuous result aroud the cetral positio is caused b a error i the calculatio o the pseudo-iverse matri i equatio (2). D. Calculatio time For deterous maipulatio, we eed to obtai haptic iormatio rom multiple sesors. Thereore, we must process a large amout o iormatio rom dieret sesors simultaeousl. I this sectio, we evaluate the calculatio times or the iger-shaped haptic sesor ad our sesors. The calculatio time is obtaied rom the requec o the CPU. The speed o the processor used i the PC i this eperimet is 2.8 GH (Itel Petium D processor). The program rus o Widows XP SP2. The accurac o measuremet is 1 µs. The calculatio time is measured rom 100 trials, ad the average ad stadard deviatio o the calculatio time are show i Table II. I this prototpe, the camera uit captures our images at 67 ps. Thereore, the PC must calculate the orce vector ields o the our sesors withi 16 ms. The result satisies this requiremet eve whe we process the iormatio rom the our sesors simultaeousl. 492

6 Normalied orce Normalied orce Distace rom orce applied poit [mm] Distace rom orce applied poit [mm] (5.0, 4.0) (5.0, 4.5) (5.0, 5.0) (5.0, 5.5) (5.0, 6.0) (4.0, 0.0) (4.5, 0.0) (5.0, 0.0) (5.5, 0.0) (6.0, 0.0) Fig. 9. Force measured b high-desit samplig. The irst ad secod graphs show the results o the applicatio o orce at the ceter ad the edge o the sesor, respectivel. TABLE II CALCULATION TIME Number o sesors Average [µs] Std.Deviatio [µs] IV. DISSCUSSION The iger-shaped haptic sesor ehibits liear respose ad its orce magitude resolutio is 30 g i all the three directios. Moreover, the spatial resolutio o the sesor is 5.0 mm i the - ad -directios at a poit. These results show that the iger-shaped haptic sesor ca be used as a both orce vector sesor ad a orce ield sesor. The results also idicate a possibilit or urther improvemet i the perormace o the sesor. The spatial resolutio o the sesor is idetical to that obtaied rom the samplig theor. Thereore, the spatial resolutio ca be improved b reducig the diameter o the markers ad the distace betwee them. For eample, to obtai a spatial resolutio similar to that o the huma igertip (approimatel 2.0mm [9]) or the sesor, the distace betwee the markers ad the diameter o each marker should be 0.2 ad 1 mm, respectivel. I the prototpe, the marker diameter is 0.5 mm or piels, ad the error i the measuremet o the marker movemet is 0.15 piel. Thereore, we ca reduce the sie o the markers ad the distace betwee them i order to realie the desired speciicatios. The time resolutio o the sesor ca also be improved. I the prototpe sesor, the calculatio time is less tha 2.5 ms eve ater usig our sesors. Thereore, a time resolutio o 400 H ca be realied b improvig o the rame rate o the camera. The iger-shaped haptic sesor is capable o perormig basic uctios ad ca be improved urther. Thereore, we believe that the implemetatio o this haptic sesor will eable robotic hads to perorm deterous maipulatios. V. CONCLUSION I order to eable a robotic had to perorm deterous maipulatios, we developed a iger-shaped haptic sesor that ca measure a three-dimesioal orce vector ield. We applied the techolog used i the visio-based haptic sesor developed b us to the iger-shaped haptic sesor. Because o the complicated shape o the iger-shaped sesor, we used actual measuremets or the calibratio i order to develop a prototpe o the sesor. We the evaluated the basic perormace o the sesor. The results o the evaluatio impl that urther improvemet i the resolutio o the sesor is possible. Hece, we will urther modi the iger-shaped sesor to eable a robotic had to maipulate objects ad realie deterous maipulatio b usig the iormatio o the orce vector ield. REFERENCES [1] H.R. Nicholls, M. H. Lee, A Surve o Robot Tactile Sesig Techolog, It. J. Robotics Research, Vol. 8, pp. 3-30, [2] M. H. Lee, H. R. Nicholls, Tactile Sesig or Mechatroics A State o the Art Surve, Mechatroics, Vol. 9, pp. 1-31, [3] J. G. Silva, A. A. Carvalho, D. D. Silva, A Strai Gauge Tactile Sesor or Figer-mouted Applicatios, o IEEE Tras. O Istructio ad Measuremet, Vol. 51, No. 1, pp [4] M. Ohoka, Y. Mitua, K. Hattori, I. Higashioka, Data Coversio Capabilit o Optical Tactile Sesor Featurig a Arra o Pramidal Projectios, IEEE Iteratioal Coerece o Multisesor Fusio ad Itegratio or Itelliget Sstems, pp , [5] N. J. Ferrier, R. W. Brockett, Recostructig the Shape o a Deormable Membrae rom ImageData, The Iteratioal Joural o Robotics Research, Vol. 19, No. 9, pp , [6] K. Kamiama, H. Kajimoto, N. Kawakami, S.Tachi, Evaluatio o a Visio-based Tactile Sesor, I Proc. o Iteratioal Coerece o Robotics ad Automatio (ICRA2004), 2004, pp [7] L. D. Ladau, E.M.Lishit, Theor o Elasticit, ButterworthHeiema, [8] W. Meke, Geophsical Data Aalsis: Discrete Iverse Theor, Academic Press Ic., [9] E. R. Kadel, J. H. Schwart, T. M. Jessel, Priciples o Neural Sciece, 4th Editio, McGraw-Hill,