MULTI-CHANNEL VIBROTACTILE DISPLAY FOR TELEOPERATED ASSEMBLY

Transcription

1 2002 IEEE Internatonal Conference on Robotcs and Automaton, Washngton D.C., May MULTI-CHANNEL VIBROTACTILE DISPLAY FOR TELEOPERATED ASSEMBLY Thomas Debus Tae-Jeong Jang Perre Dupont Robert Howe A.M.E Boston Unversty Boston, MA E.C.E Kangwon Natonal Unversty Chuncheon, Korea A.M.E Boston Unversty Boston, MA D.E.A.S Harvard Unversty Cambrdge, MA Abstract Ths paper presents the desgn and testng of a mult-channel vbrotactle dsplay. It s composed of a cylndrcal handle wth four embedded vbratng elements drven by pezoelectrc beams. Vbratons are transmtted to the hands through arrays of pns. The devce was tested n sensory substtuton for conveyng nformaton durng a teleoperated peg nserton. Results show that the devce s effectve n reducng peak s durng the nserton task. Index Terms- teleoperaton, vbraton, peg-n-hole nserton. I. INTRODUCTION Augmentng percepton n man-machne systems conssts of two parts. Frst, machne sensor data must be nterpreted n a way approprate to the task. Second, the task-specfc nformaton must be communcated to the operator n a format that addresses the lmtatons of human sensory nformaton processng. Ths step nvolves tradeoffs on the number of bts to nclude n a sensory communcaton channel (e.g., how many possble drectons and magntudes need be represented), the number of channels drected to a specfc sense (e.g., how many dfferent vsual dsplays) and the number of senses to whch nformaton channels are drected (e.g., vsual, audtory, tactle). For a partcular task, the utlty of a sensory dsplay s determned by how well t conveys task-specfc nformaton to the operator. On frst glance, the utlty of vbrotactle dsplays can appear modest. They have been shown nferor to audtory dsplays n terms of communcaton channel wdth [6]. Furthermore, vsual dsplays can be more natural due to the apparent domnance of vson over other senses [7]. Wthn the framework of human sensory processng, however, a large channel bandwdth requres more human processng to nterpret and can, under certan condtons, actually ncrease task executon tme [6]. In addton, the use of multple vsual dsplays that requre the swtchng of attenton can also slow task performance [1]. Addtonal factors can make the use of a dsplay modalty neffectve or costly. The operator s envronment can contan nose sources that nterfere wth nformaton transmsson on a sensory channel. For example, durng teleoperated surgery, vocal communcaton between nonoperator team members can nterfere wth an audtory dsplay. Smlarly, overlayng vsual dsplays (e.g., conveyng nformaton) wth an endoscope vew durng surgery can be costly and non-ntutve due to scene complexty and the wde range of vewponts encountered. To maxmze effectveness and mnmze cost, a general desgn rule for sensory dsplays s to use the mnmum possble nformaton channel wdth on a sensory channel wth low nose and whch s not already recevng a sgnal requrng complex processng,.e., avod attenton swtchng. Wthn ths context, vbrotactle dsplays are promsng for tasks that by default are rch n audtory and vsual stmul and for whch feedback s ether mprecse (e.g., mantanng constant s), too fral or costly (e.g., undersea ol operatons), or s destablzng (e.g., teleoperaton wth tme delay). Ths paper presents our frst work on mult-channel vbrotactle dsplays for such applcatons. Much of the effort was focused on desgn ssues. The next secton presents a bref revew of pror work on vbrotactle dsplays. The followng secton presents the mechancal and electrcal desgns of the mult-channel vbrotactle devce. The subsequent secton descrbes an expermental evaluaton of the tactle devce durng a peg-n-hole nserton task performed usng a teleoperaton system composed of a master PHANToM robot and a remote W.A.M robot. Conclusons are presented n the fnal secton of the paper. II. BACKGROUND Force substtuton va vbrotactle dsplay s not a new concept. It has been an ongong research topc for the past

2 forty years. Durng ths perod, many vbrotactle systems have been desgned, utlzng a broad range of actuators and spannng a wde range of applcatons, from undersea teleoperaton [5] to adng the blnd [2]. These devces can be roughly grouped accordng to the number of vbratng elements and, f more than one, whether the elements are wdely separated or placed n a closely packed array. The latter case s exemplfed by [2] n whch a seres of 0.1mm-dameter pns are arranged n a matrx confguraton wth sx rows of 24 vbratng pns. Drven by pezoelectrc crystals, the devce s used to convert optcal sgnals to vbrotactle nformaton. Called Optacon, t s used by blnd people to feel the mage vewed by an assocated handheld camera. In [9], a shape dsplay consstng of a regular array of 10 pns actuated usng SMA wre was used to convey tactle nformaton. The devce had a -3dB pont of 40 Hz and detectable output that could stll be felt at frequences approachng 150 Hz. The devce was used to transmt nformaton about the texture and contact state. The shape dsplay was fast enough to keep up wth fnger speed when scannng over small object features; however, t was much more complex and expensve that tradtonal vbraton dsplays that use pezoelectrc crystals or small motors. Devces exemplfyng wdely spaced vbratng elements nclude those descrbed n [8] and [6]. In [8], a system composed of three contact pont actuators was desgned. Each actuator used a dsk-drve head postonng motor controlled va a dgtal angular poston feedback controller to produce vbratons at a frequency range from DC to above 300Hz. The system was attached to the thumb, ndex, and mddle fnger and was used for a varety of tactual perceptual studes. In [6], an expermental study of the use of audtory and vbrotactle dsplay as sensory substtuton for feedback was conducted. Specfcally, a vbrotactle dsplay consstng of vbratng voce cols placed at the fngertps and palm of the domnant hand was used to perform a peg-n-hole task. Each vbrator had a resonant frequency of 250 Hz, wth ntensty proportonal to the magntude of the assocated measured usng sensng resstors. Results showed that the vbrotactle or audtory dsplays dd not speed up the task n the case of clear vsual feedback. However, they dd help n the cases of an obstructed vew or tme delay. Few vbrotactle devces are used n real ndustral tasks. One notable excepton can be found n [5], where a vbrotactle system desgned for the harsh condton of undersea hydraulc connector matng was presented. Frst a sensor usng pezoelectrc contact sensor was bult and packed to ft nto the manpulator grpper. Second, a smple voce col motor mounted on an alumnum base was clamped on the unmodfed master controller and was used as the vbrotactle dsplay. The prelmnary result showed that the devce consstently eased the undersea connectng task. Fnally, some commercal systems have also been developed. One s The Cybertouch from Immerson. Ths devce s composed of sx vbrotactle dsplays attached to the palm and fngertps of the Cyberglove, a fully nstrumented glove that provdes up to 22 jont-angle measurements. Each actuator provdes sustan vbratons at a frequency rangng form 0 to 125Hz, wth a 1.2N peak to peak ampltude at 125Hz. The system s used prmary to nteract wth vrtual envronments. III. DISPLAY DESIGN The motvatng desgn premse was that the best approach for teleoperaton would be to mount vbrotactle modules n the handle or stylus of the master devce. The goal was to provde the most straghtforward mappng between vbratons on the hand and coordnate drectons on the master and remote manpulators. For ease of constructon, t was decded to equally space four vbrotactle modules around the crcumference of a cylndrcal handle. As shown n Fgure 1, the four resonators can be used to represent postve and negatve s (or torques) wth respect to two drectons n the tool frame. The vbrotactle modules were desgned accordng to the followng goals: They should not nterfere wth the natural grp of the handle. They should permt a varety of grasp locatons on the handle. The vbraton ampltude should be unform over the surface of each module. The vbraton ampltude should be ndependent of the grasp. FIG.1. VIEW ALONG CYLINDER AXIS SHOWING VIBRATION CONDITIONS OF FOUR RESONATORS. These goals led to a desgn n whch the vbrotactle modules were embedded flush wth the cylndrcal handle. Fgure 2 shows the devce wth one module dsassembled. The module s about 1 by 2 allowng for a varety of grasps and fnger szes. To acheve a unform level of vbraton over the module s surface ndependent of grasp, each module conssts of a pezoelectrc bmorph beam whch sts freely n a cavty under 11 rows of 4 pns, spaced 0.1 nch apart. Vbratons are transmtted to

3 the hand through the pns, whch pass through holes n the module s cover. The pns n each row vary n length so that they conform to the radus of the module s cover plate. By allowng free-free moton of the beam wthn the cavty, nodes of vbraton (dead spots) are precluded such that the vbrotactle response s perceved to be unform over the surface of the module. Furthermore, the depth of the cavty s such the beam can stll vbrate freely under the pns regardless of grasp. Whle a large grasp does affect hand mpedance and thus energy transfer from the pns to the hand, the beam and pns contnue to vbrate aganst the skn. IV. EXPERIMENTAL SETUP AND PROTOCOL A. Target Applcaton It has been shown that wth a clear real-tme vsual dsplay, addng nformaton through sensory substtuton does not speed up a task [4],[6]. For connector nserton, nserton tme can measure the dffculty of the task; however, t does not gve any ndcaton about the qualty of task performance. Consder, for example, an operator tryng to hs/her way through the socket whle focused only on reducng tme. Even f ths operator successfully nserts the peg n record tme, ths strategy can stll fal f the connector, socket or remote manpulator are damaged due to excessve. The followng experments nvestgate how nformaton, conveyed through two sensory channels (vbraton and vsual), affects peak s. B. Expermental Setup FIG.2. DISASSEMBLED VIBROTACTILE DISPLAY REVEALING PIEZOELECTRIC BEAM, CAVITY IN WHICH IT VIBRATES AND FLOATING ARRAY OF PINS. The crcut drvng the four modules s capable of delverng a varety of nput voltages to the resonators (up to 80V; Hz; snusodal, trangular or square waves). The nput voltage to each resonator s ampltude modulated at a frequency of ~300 Hz, whch s the most senstve frequency of human fngers [3] and s also the resonant frequency of the pezoelectrc beams. As currently desgned, the two nput channel voltages (x,y) are mapped to four outputs by dvdng them nto ther postve and negatve parts (x +,x -,y +,y - ), each of whch s used as the nput to a resonator crcut. Assumng a nput, the output s as shown n Fgure 1. The fnal devce, as shown n Fgure 2, s a 4-nch long cylnder of outsde dameter 1 1/4 nch and nsde dameter 3/8 nch. The devce has a total mass of 130 grams. Whle ntally ntended for mountng on the master handle, the fnal sze of the devce made t approprate to hold n a power grasp (e.g., lke holdng a hammer). As a result the palm, thumbs, ndex and mddle fnger are n contact wth one or more vbrotactle modules durng ths grasp. It was found n prelmnary experments that operators had more success grppng the master wth a dextrous grasp (e.g., lke holdng a pen). When gven the choce, operators preferred a drect and dextrous grasp of the master wth the domnant hand whle power graspng the vbrotactle devce n the other hand. The laboratory teleoperator testbed used n these experments conssts of a PHANToM haptc nterface as the master controller and a Barrett Whole Arm Manpulator (WAM) as the remote robot (Fgure 3). The PHANToM (Model 1.5, Sensable Technologes, Cambrdge, Mass., USA) s used as a passve 6 degree of freedom nput devce, and the motors are not actvated. The WAM (Barrett Technologes, Cambrdge, Mass., USA) s a redundant arm wth 7 degrees of freedom. Only 5 axes are requred for a peg-n-hole nserton task, and therefore the upper arm roll and fnal wrst roll axes are locked. Optcal encoders measure the jont poston on both robots, and veloctes are computed usng fltered backward dfferences. The workspace s roughly 0.2 m n dameter for the master robot and 1.0 m n dameter for the remote robot. The WAM robot s controlled by a dedcated RISC processor (Model DS1103, dspace GmbH, Paderborn, Germany) runnng at a 10 khz servo rate. The PHANToM jont data s read by a PC at a rate of 1 khz and wrtten nto memory shared by the PC and the RISC processor every 200 Hz. FIG.3. (A) PHANTOM MASTER ARM. (B) WAM REMOTE ROBOT ARM WITH CONNECTOR MATING APPARATUS.

4 Teleoperaton s accomplshed wth a smple proportonal-dervatve controller wth feedforward gravty and motor torque rpple compensaton. Usng ether jont or Cartesan poston, two control modes are avalable to the operator. Frst, an ncremental jont-to-jont mappng between the PHANToM and the WAM can be used to roughly poston and orent the peg. Second, an ncremental Cartesan poston mode, n whch velocty and orentaton of the master robot are mapped to the remote workspace, can be used to precsely nsert the peg. In both cases, the master robot jonts are converted to remote robot jont postons and velocty and then to torque commands by the followng control laws. remote K ( q q ) K ( q q ) ( 1) p master remote v master remote gravty rpple gravty rpple Here q, q, τ, τ and τ are the th components of jont poston, velocty, torque, gravty compensaton and motor torque rpple compensaton. The connector-socket apparatus was smulated by a par of PVC plastc tubes. The connector was 52.9 mm n dameter and 100 mm long. The socket had an nsde dameter of 53 mm and was mounted perpendcularly to a planar surface that could be pvoted to a range of nclnaton angles between 0 and 57 degrees from the horzontal. The operator can montor the remote envronment va three fxed vdeo cameras. Fgure 4 shows the two closeup vews and the dstant vew of the hole that are avalable to the operator. The two close-up vews are angled to allow the user a stereoscopc vew of the hole. The remanng slot of the quad dsplay s used to show whch control mode s actve and other graphcal nformaton when needed (.e., graphcal dsplay mode). The vbrotactle devce uses estmates as nputs and then outputs vbratons proportonal to the ntensty of these s. The cylndrcal shape of the dsplay makes t well desgned for nserton tasks. In effect, the four resonators make a natural mappng to the sdes of the hole, ndcatng to the user when and where the manpulated peg s n contact wth the socket. It s assumed that no sensor s avalable; nstead the complance of the remote robot s used to estmate. Ths technque only works n the quas-statc part of the nserton, when the operator s n contact wth the envronment. In the other phases of the nserton, the quasstatc assumpton does not hold, and n ths case, a smple dead zone s mplemented to zero any transent caused by the unmodeled dynamcs of the system. As a result, any estmated s below 20N are gnored. Ths reducton of the bandwdth does not reduce the performance of the devce snce s are well above ths threshold durng the nserton phase. To estmate s, poston errors between the master and the remote robot are computed and multpled by a stffness constant as shown n the followng equaton. master remote F k( x x ) (2) The stffness gans k are drectly related to the controller gans and the mechancal complance of the remote manpulator. To dentfy the stffness gans, calbrated masses have been placed on the remote manpulator end effector, and poston dsplacements from the set ponts have been measured. Table 1 summarzes these results. TABLE 1 REMOTE MANIPULATOR STIFFNESS GAINS. X-drecton Y-drecton Z-drecton k (N/m) FIG.4. QUAD VIEW USED DURING THE TELEOPERATED INSERTION TASK. In addton to vsual feedback, vbrotactle dsplay and graphcal representaton of the estmates are used to convey nformaton to the operator. Lookng at ths table, one can see that the system s farly complant. For example, consder an nserton where the peak nserton s around 100N. To produce ths a deflecton of 25 mm s necessary. Such complance facltates the task; however, t also reduces the need for substtuton devces. Ths complance was found to be manly caused by tendon drve elastcty of the wrst. Fnally, these estmates are mapped to 10V usng a 24 bts D/A converter. The graphcal dsplay, as shown n Fgure 5, provdes nformaton smlar to that of the vbrotactle devce. The ntensty and drecton of the estmates are mapped nto a sphere of varyng dameter and locaton. For example, f the peg s n contact wth the bottom sde of the hole, the sphere moves toward the bottom of the graphcal representaton of the hole. The dstance from the center of the hole and the sze of the sphere, are drectly propor-

5 tonal to the ntensty. As wth the vbrotactle dsplay, f the s below the 20N threshold, no contact nformaton s gven. Compared to the vbrotactle devce, the graphcal dsplay provdes rcher nformaton. In effect, there s no error n drecton mappng as opposed to the vbrotactle devce where ampltude modulaton of four modules must ndcate all possble drectons. FIG.5. THE GRAPHICAL DISPLAY. C. Experment Protocol Estmated peak and nserton tme are measured to assess task performance. Pror to data collecton, the users are traned untl they become fully accustomed to the system. Operator tranng for a partcular dsplay mode s termnated when the standard errors assocated wth ther nserton tme and peak estmate plateau. Dependng on the user, ths can take between one and two hours. Once fully traned, each operator completes two sets of experments. For each set, three consecutve trals usng each dsplay are performed. A Latn square technque s used to determne the order n whch the dsplays are tested. A total of nne subjects voluntarly partcpated n the experment (7 males, 2 females, all wth engneerng backgrounds, ages 19-35). In the end, each subject performed a total of 18 nsertons. V. EXPERIMENTAL RESULTS The results for the peg-n-hole nserton task are summarzed n Table 2. Three feedback condtons were tested and compared: 1) vsual feedback 2) graphcal dsplay combned wth vsual feedback and, 3) vbrotactle dsplay combned wth vsual feedback. Mean peak and nserton tme were computed for each dsplay. The table s separated nto two groups of operators because three subjects were unable to decrease the peak usng ether the graphcal or the vbrotactle dsplay. Although there s no statstcal reason to dscard data from these subjects, they are clearly not people one would hre for teleoperaton nvolvng anythng expensve, fragle or dangerous. Statstcal analyss for the group composed of sx subjects showed that vsual nformaton alone provded greater mean peak than when combned wth the graphcal dsplay vsual =86.3 graph. =51.3, p<0.001 or wth the vbrotactle dsplay vsual =86.3 vb. =63.8, p<0.011.the graphcal dsplay, however, provded sgnfcantly lower mean peak than the vbrotactle devce graph. =51.3 vb. =63.8, p< In Fgure 6, these results are normalzed usng the vsual dsplay as a baselne. As shown, the peak estmates are reduced by 40 percent wth the graphcal dsplay and 26 percent wth the vbrotactle dsplay. When all 9 subjects are ncluded, peak reductons are stll statstcally sgnfcant; however, the percentage reducton s reduced to 20 percent and 9 percent for the graphcal and vbrotactle dsplays, respectvely. TABLE 2 EXPERIMENTAL RESULTS Mean Peak Force (N) Mean Inserton Tme (s) V.D G.D Vb.D V.D G.D Vb.D 9 subjects subjects Where V.D, G.D, and Vb.D stand for vsual dsplay, graphcal dsplay, and vbrotactle dsplay, respectvely. FIG.6. NORMALIZED PEAK FORCE ESTIMATE AS A FUNCTION OF DISPLAY MODES. Fgure 7 focuses on nserton tme and shows that on average the nserton tme s 31 percent and 24 percent longer usng the graphcal dsplay and vbrotactle devce, respectvely. Detaled analyss showed that vsual feedback alone provded a lower mean nserton tme than when combned wth the graphcal dsplay, tme tme vsual =5.28 graph. =6.92, p<0.001 or the vbrotactle devce tme tme vsual =5.28 vb. =6.55, p< However, the vbrotactle devce provded lower mean nserton tme than the tme tme graphcal dsplay vb. =6.55 graph. =6.92, p<0.095.

6 VII. CONCLUSIONS FIG.7. NORMALIZED INSERTION TIME AS A FUNCTION OF DISPLAY MODES. VI. DISCUSSION These results demonstrate that basc estmaton together wth sensory substtuton can be effectve n conveyng useful task nformaton durng teleoperaton. In partcular, both the graphcal and vbrotactle dsplays led to a sgnfcant decrease n peak nserton, and thus, reduced the rsk of damage to the connector, socket and remote manpulator. The superorty of the graphcal dsplay for reducton s expected snce mappng of drecton s transparent to the operator n comparson wth the coarse drectonalty provded by the four-element vbrotactle devce. In agreement wth pror results [6], use of the dsplays n concert wth a clear vdeo sgnal ncreased nserton tme. Whle the vbrotactle dsplay seems to be more effcent than the graphcal dsplay, the level of sgnfcance s only 9.5 percent; more tests are needed to confrm ths hypothess. A varety of factors can be used to explan the ncreased nserton tme. Frst, as more nformaton s made avalable to the operator, more cogntve processng s also requred, nducng delay n the operator motor response. In ths stuaton, the vbrotactle dsplay presents an ndependent channel of communcaton that may facltate the nformaton processng. A second factor s that torque, rather than, provdes better nformaton for nserton tasks. Whle helps by warnng the user of msalgnment; t does not drectly ndcate the correctve moton. Torque nformaton s obvously more approprate for ndcatng both msalgnment and correctve acton. Its effect on nserton tme, however, s an open queston. Fnally, robot complance facltates the nserton, reducng the need for graphcal or vbrotactle dsplay of nformaton. Although ths s a prelmnary study that requres further valdaton, these results ndcate that the vbrotactle devce provdes a clear beneft n the ntended applcaton n terms of peak reducton. The proposed desgn presents a low cost and flexble devce that can be use n a wde range of applcatons. In teleoperated assembly the devce can be used wthout nterferng wth normal task executon and wthout sgnfcant modfcatons to the exstng manpulator. Consequently, f the dsplay fals; t has no mpact on the manpulator s capabltes, and operatons can contnue wthout delay. In addton, avodng the use of a vsual dsplay may prove advantageous for applcatons whch already nvolve multple or complex vsual stmul. As for future work, a new vbrotactle dsplay has been desgned and bult for use wth a dextrous grasp. Ths cylndrcal devce weghs 40 grams, has a 0.6 dameter and 4.75 length. Three vbrotactle arrays around ts permeter are mounted to contact the fngertps when the devce s grpped lke a pen. The effect of mountng ths devce as the master manpulator handle wll be studed along wth a comparson of sensory substtuton usng and torque nformaton. VIII. REFERENCES [1] Bejczy, A., Dotson, R., Brown, J., and Lews, J. (1981). Manual Control of Manpulator Forces and Torques usng Graphc Dsplay. Proceedng of IEEE Internatonal Conference on Cybernetcs and Socety, Seattle, Washngton, October, [2] Blss, J., Katcher, M., Rogers, C., and Shepard, R. (1970). Optcal-to-Tactle Image Converson for the blnd. IEEE Transactons on Man-Machne Systems, (11)1, March, [3] Boff, K., and Lncoln, J. (1988). Engneerng Data Compendum: Human Percepton and Performance. Oho: H.G. Anderson Aerospace Medcal Research Laboratory. [4] Debus, T., Stoll, J., Howe, R., and Dupont, P. (2000). Cooperatve Human and Machne Percepton n Teleoperated Assembly. D. Rus and S. Sngh, eds., Expermental Robotcs VII. The Ffth Internatonal Symposum, Honolulu, HI, December, Lecture notes n control and nformaton scences, Sprnger-Verlag, Berln, n press. [5] Dennerlen, J., Mllman, P., and Howe, R (1997). Vbrotactle Feedback for Industral Telemanpulators,. Sxth Annual Symposum on Haptc Interfaces for Vrtual Envronment and Teleoperator Systems, ASME Internatonal Mechancal Engneerng Congress and Exposton, Vol. 61, Dallas, November, [6] Massmno, M. (1992). Sensory Substtuton for Force Feedback n Space Teleoperaton. MIT Ph.D. Thess, Department of Mechancal Engneerng, June. [7] Posner, M., Nssen, M. and Klen, R. (1976). Vsual Domnance: An Informaton-Processng Account of ts Orgns and Sgnfcance. Psychologcal Revew, (82) 2, [8] Tan, H., and Rabnowtz, W. (1996). A New mult-fnger tactual dsplay. Proceedngs of the ASME Dynamc Systems and Control Dvson, Atlanta, [9] Wellman, P, Pene, W., Favalora, G., and Howe, R. (1998). Mechancal Desgn and Control of a Hgh-Bandwdth Shape Memory Alloy Tactle Dsplay. A. Casals and A.T. de Almeda, eds., Expermental Robotcs V. The Ffth Internatonal Symposum, Berln, Lecture notes n control and nformaton scences, vol. 232, Sprnger-Verlag,