Cartesian Task Allocation for Cooperative, Multilateral Teleoperation under Time Delay

Save this PDF as:

Size: px
Start display at page:

Download "Cartesian Task Allocation for Cooperative, Multilateral Teleoperation under Time Delay"


1 Cartesia Task Allocatio for Cooperative, Multilateral Teleoperatio uder Time Delay Michael Pazirsch 1, Rib Balachadra 1 ad Jordi Artigas 1 Abstract Field robots robots used ustructured ad dyamic eviromets ad teleoperatio have shifted to the focus of a variety of dustrial braches the past few years. The lack of space the atomic dustry, o oil platforms ad space applicatios demads additioal adaptatios to curret robotic setups. I this paper a MMSS (MultiMasterSgle Slave) haptic teleoperatio system is proposed through which oe operator usg two master arms ca maipulate objects a cooperative way via oe slave robot ad a virtual grippg pot. To ease the executio of a peghole task of big objects, a task allocatio the cartesia frame of the virtual grippg pot is troduced additioally. The stability of this multilateral system with time delay is guarateed by the Time Doma Passivity Approach. Therefore the system is divided to several modular subsystems which reders the system easily adaptable to other scearios. Idex Terms cooperatio, multilateral teleoperatio, task allocatio, peghole, MOMR, MMMS, TDPA I. INTRODUCTION With the progress of robotic techology, the areas of its applicatio emerged immesely the past. It became feasible to use robots for plat mateace or complex costructioal tasks hazardous eviromets (see [1]). Still, efficiecy ad ergoomics are key factors for success, especially whe robots ustructured, dyamic ad arrow eviromets eed to be teleoperated from distace. Research o MultiMasterMultiSlave systems (MMMS) has maly focused o teractios via objects (evirometal impedaces more geerally, [2],[3]), modelmediated cooperative teleoperatio ad collisio avoidace ([4],[5]). Those challeges are crucial e.g. mimally vasive surgery, whereat efficiecy ad safety ca be improved through the use of MMMS systems (compare [6]). The above metioed MMMS systems are mostly based o a bilateral cotrol ratioale, which the haptic formatio is oly exchaged betwee a sgle master ad a sgle slave system ([7]). Other works propose the couplg of two master devices to oe slave robot, rederg a quasitrilateral system ([8], [3]). [9],[1] propose true trilateral systems, where the volved master devices are also coupled to each other. Those systems are so far preferetially used for trag ad rehabilitatio aspects rather tha for cooperatio. I those multilateral systems, a variety of task allocatio types were proposed. I [11], oe master cotrolled the edeffector of the slave robot, whereas a secod master was used to chage the cofiguratio of the redudat slave robot 1 Michael Pazirsch, Rib Balachadra ad Jordi Artigas are with the departmet for Aalysis ad Cotrol of Advaced Robotic Systems, Germa Aerospace Ceter, Oberpfaffehofe, Germay a way that the edeffector pose is ot affected. I [12], projective force mappgs were troduced to impose specific boudaries oto the slave robot s motio. As adaptive cotrol has bee applied, models of hardware, operator ad eviromet were required. I [13], two masters with differet degrees of freedom (DoF) were used to cotrol a slave robot with three degrees of freedom. Haptic feedback could costitutioally oly be displayed o the available DoFs of each master device. Lee et al. showed [1] that through the Time Doma Passivity Approach (TDPA), a disjot axis cotrol with time delay is feasible, such that two subsets of DoFs of a redudat slave robot cosistg of a mobile platform movg a serial robot could be depedetly cotrolled by two master devices. A similar setup has bee applied [14]. Also the approaches [1] ad [15] o multilateral systems are based o the TDPA. Time delay MMMS systems is hadled [9] ad [16] by the wave variable method. I [17], the effect of time delay o differet types of cooperative cotrol methods MMMS systems is aalyzed more closely. [18] ad [19] developed approaches for SMMS (Sgle MasterMultiSlave) systems able to hadle big objects despite delayed commuicatio. I both papers a local graspg cotroller orgaized the grippg of a big object by the slave robots. I [18], the scatterg trasform was used to cosider the delay. [19] proposed a modelbased adaptive sychroizg cotroller with dampg jectio term. Clearly, the dexterity levels of a operator who is performg a telemaipulatio is flueced by the telerobotic platform beg used. The methods preseted this paper aim at creasg the skillfuless of a operator maipulatg large objects through the distace ustructured ad especially arrow eviromets allowg oly compact robotic systems. The addressed sceario cosists of a teleoperatio system with two masters ad a sgle slave ad a costraed commuicatio chael with time delay. To demostrate the efficiecy of the proposed method, a peghole task of a log pipe has bee used. The system is desiged a multilateral fashio, that is, all the volved robots are coupled with each other, allowg a accurate ad tuitive teractio. A task allocatio the Cartesia space is furthermore troduced to ease the peghole process. This paper is structured as follows: Chapter II troduces the multilateral structure ad the cosidered cotrol approach. I chapter III, the sceario of cooperatio is explaed more closely ad the required extesios of the system are implemeted. The coducted experimets are preseted chapter IV ad the results are summed up chapter V.

2 II. MULTILATERAL CONTROL The system desig is based o the multilateral cotrol approach proposed [1]. A geeric socalled trackpopc system based o the TDPA has bee troduced which allows a variable ad guarateed stable haptic teractio of a arbitrary umber of agets (see Fig. 1). Agets ca be huma operators with master devices, slaves their eviromet or autoomous telliget uits. Each track represets the bilateral haptic coectio betwee two agets. Fig. 2 depicts H.M. PCU v 3 v 5a F 3 PI1 F 5a F 5b T DPN2 T DPN1 F 6b PI2 v 6a F 6a F 8 S.E. PCU v 5b v 6b v 8 Aget Λ1 Track Γ2 Track Γ1 Aget Λ3 Track Γ3 Aget 1 Aget Λ2 AgetΛ1 Track AgetΛ2 Fig. 3. Network represetatio of a teleoperatio system with positiopositio architecture out v 1 F 1 2Port F 2 E R,2P out E R,2P Fig. 1. Exemplary assembly of Tracks to a multilateral system the sigal flow diagram of a PositioPositioarchitecture (PP) with time delay. O both sides of that commuicatio chael, a PIcotroller is located which acts like a virtual sprgdamper system o the master ad slave positios ad velocities respectively. I order to use the passivitybased TDPA, the eergy behavior of the system has to be evaluated. Therefore the bilateral sigal flow diagram has to be trasduced to the etwork represetatio (Fig. 3) the electrical aalog of the sigal flow diagram such that eergies ca be measured at the subsystem s ports. This trasformatio is explaed [2]. The time delays T 1 ad T 2 the commuicatio chaels are represeted by the Time Delay Power Networks TDPN (proposed [21]). The agets cota besides huma, eviromet ad the hardware devices also the power distributg subsystems PCU that have bee troduced [1]. The eergy that ca be measured at,del T 1,del T 2 Huma Master PI1 PI2 Slave Ev F 1 F 3 F 8 F 1 Fig. 2. Sigal Flow Diagram of a teleoperatio system with positiopositio architecture the ports of the etwork subsystems is cosidered to check the passivity of e.g. a 2port subsystem (see Fig. 4): (t)e R,2P (t) (1) where E R/L,2P are the eergies flowg o the right/left side of the 2port (2P) which ca be computed based o cojugate power pairs at each port. The power P L/R,2P flowg at the Fig. 4. E R,2P I ad out eergies of a 2port etwork left/right port of a 2port ca be calculated as follows: P L,2P (t)=v 1 (t)f 1 (t), P R,2P (t)= (t)f 2 (t). Aalyzg the sig of P L/R,2P, oe ca split up the power cocerg the flow directios: P L/R,2P /out. For stace: { (t) = { P L,2P P L,2P out (t) =, if P L,2P (t)< P L,2P (t), if P L,2P (t)>, if P L,2P (t)> P L,2P (t), if P L,2P (t)<. ad The subdices ad out dicate the power (or eergy) flowg to ad out o f the etwork o the left (L) or the right (R) side respectively. The the /out flowg eergies E L/R,2P /out o the left/right side ca be calculated: /out (t)= t P L,2P /out (τ)dτ, ER,2P /out (t)= t P R,2P /out (τ)dτ. As these eergies ca oly be observed the two directios of a track separately, the passivity coditio (1) has to be reformulated: (t)e R,2P (t)= (t T 1 ) Eout R,2P (t)e R,2P (t T 2 ) (t). Sce the eergies are purely creasg the approach meets with equatio (2) ad the coditios (3) the passivity criterio (1). (t)eout R,2P (t), E R,2P (t)eout L,2P (t) (3) I a trackpopc system two Passivity Cotrollers (PC) are troduced the tracks which dissipate the eergy geerated the respective PI ad TDPN. I [1], it was show out (2)

3 that through the use of trackpopcs ad uder the wellaccepted assumptio that the agets behave passively their teractio the whole system is passive ad thus stable. Thus, ay combatio of tracks ad agets is prove to be stable. No models of the hardware ad o further aalytical stability proof is ecessary. This structure ad the stability cotrol approach respectively ca be easily exteded to the 6DoF case if the positio cotroller is desiged the cartesia space such that all DoFs ca be hadled separately. Still, as we focus this paper o the maipulatio of big objects ad o the task allocatio the multidof case, ew modules have to be developed ad vestigated cocerg their eergetic behavior. Fig. 6. Bilateral telemaipulatio of a pipe v 3 v 5a v 6a H.M. PCU F 2 PC1 F 3 PI1 F 5a T DPN1 F 6a F 5b T DPN2 F 6b PI2 F 8 PC2 F 9 S.E. PCU v 5b v 6b v 8 AgetΛ1 Track AgetΛ2 Fig. 7. Couplg of master devices through virtual pipe Fig. 5. Network represetatio of a trackpopc system with positiopositio architecture III. MULTILATERAL COOPERATION The sceario of terest is a MMSS system with task allocatio which should simplify the assembly of a log pipe to a plug fixed o the wall. I a geeral bilateral teleoperatio setup, it is tough to fulfill this task especially if the grippg positio is deviatg from the pipe ed which has to be plugged. This complexity is visualized Fig. 6 where the tool frames I of the slave ad II of the master device are coupled by track Γ1, visualized by a sprg damper system. Whe the operator rotates the pipe aroud frame I to the right orietatio (perpedicular to the wall) the pipe ed will move dowward ad away from the plug positio. It is difficult to keep the pipe ed at the plug s positio sce the orietatios ad traslatios therefore eed to be adapted a iterative maer. For such types of tasks it is helpful to troduce aother master through which the same operator ca cotrol the positio of a additioal pot o the object. I Fig. 7, master 2 is grippg at the virtual grippg pot PE the pipe ed. The tool ceter pot II of master 1 determes the positio of PE through a projectio of II via T oto II PR. Through Track Γ2 the two devices are thus coupled a virtual pipe like behavior. Fig. 8 depicts the aspired multilateral cooperatio setup. The virtual grippg pot at the pipe ed PE ca be cosidered as to be maipulated by a virtual slave. The tool frames I ad II are projected to the respective pipe ed frames I PR ad II PR. I PR represets the pose of the real pipe ed. The devices are coected to each other by the tracks Γi the pipe ed, i.e. the track Γ1 of Fig. 6 is projected to the pipe ed. It is importat Fig. 8. Multilateral Cooperatio with spatial sprg the pipe s ed that the cotrollers are desiged the pipe ed as the task allocatio will have to be implemeted this coordate frame later o. Durg the pluggg of the pipe the master arm cotrollg the pipe ed should have the whole might o the pipe ed s traslatio. Thus, the operator s right arm (master 2) ca easily fix the pipe ed s positio close to the plug positio. The task of the left operator arm (master 1) is to brg the pipe to the correct orietatio perpedicular to the wall. Therefore the Master 1 receives up to the whole authority o the rotatio of the pipe ed.

4 A. Virtual Grippg Pot The projectio of the coordate frames I ad II to I PR ad II PR ca be computed as follows: H I PR/II PR = H I/II T, W I/II = W T T W I PR/II PR, with [ ] [ ] T= TR T p ad W T= TR 1 TR where H f r is the homogeous trasform from base frame to frame f r. W I PR ad W II PR are the force F ad torque M outputs of the PI cotrollers which are set to the master ad slave devices (W I to slave, W II to master 1 ad W III to master 2). The projectios of the coordate frame I to I PR ad vice versa are exemplary depicted Fig. 9. The trasformatio matrix T from tool frame to pipe ed frame is assumed to be kow. The passivity of the projectio ca be easily prove aalytically. To guaratee the eergy preservatio of the fixed couplg i=1 (Wi A q A i )= i=1 (Wi B q B i ), (4) with the geeralized velocities q, has to hold. As velocities ad forces/torques have to be vestigated, the traslatioal deviatio T p does t eed to be cosidered. With W A i = j=1 ( W T ji W B j ) ad q B i = j=1 it ca be show that (4) is always guarateed: ( i=1 j=1 B. Task Allocatio ( W T ji W B j ) q A i )= i=1 (W B i j=1 ( W T i j q A j) ( W T i j q A j)). Apart from the projectio of positios ad forces/torques, the task allocatio has to be vestigated. The task allocatio factor α Γi,L2R/R2L Rot/Tras scale the forces (Tras) ad the torques (Rot) set from the PIcotroller PI2 from left to right (L2R) or PI1 from right to left (R2L) track Γi (compare Fig.2). The fuctioality of the task allocatio is described by the tesity of the coordate frames Fig. 1. The arrows dicate the forces ad torques set from oe device to the two others. The coordate frame of the slave is dark as its feedback to the masters is ot varied by the allocatio factor α Γ1,R2L Tras/Rot ad α Γ3,R2L Tras/Rot (see TABLE I). The traslatioal feedback of master 2 is ot altered by the task allocatio (α Γ2,R2L/Γ3,L2R Tras = 1). The traslatioal DoFs of master 1 ad the rotatioal DoFs of master 2 are lighter as the feedback of those DoFs is scaled dow by the task allocatio (α Γ1,L2R/Γ2,R2L Tras < 1 ad α Γ2,R2L/Γ3,L2R Rot < 1). The traslatios of master 2 ca be affected by cross couplgs if the device s orietatios would be absolutely domated by master 1. Therefore the scalgs α Rot of master 1 ad 2 should both be set to.5. Fig. 11 depicts the resultg trackpopc for delayed cooperative teleoperatio with virtual grippg pot projectios ad task allocatio. As the projectio blocks PRi ad the PCcotrolled parts of the track are prove to be passive, the whole system will always be passive ad thus stable. Fig. 9. Trasformatio of forces/torques ad poses Fig. 1. Cartesia Task Allocatio the pipe ed Device α Value Master 1 α Γ1,L2R Tras α Γ2,L2R Tras Master 1 α Γ1,L2R Rot α Γ2,L2R Rot.5 Master 2 α Γ2,R2L Tras α Γ3,L2R Tras 1 Master 2 α Γ2,R2L Rot α Γ3,L2R Rot.5 Slave α Γ1,R2L Tras α Γ3,R2L Tras 1 Slave α Γ1,R2L Rot α Γ3,R2L Rot 1 TABLE I TASK ALLOCATION SETTINGS IV. EXPERIMENT The followg experimets have bee performed with the DLR HMI a bimaual haptic device ad the humaoid robot SpaceJust (see Fig. 12 ad Fig. 13). The first two experimets focus the couplg of the master devices by the virtual pipe ad the task allocatio performace. The third experimet vestigates the whole multilateral cooperatio volvg the slave robot. I all experimets, the time delay has bee set to zero for simplificatio. The trackpopc s robustess agast delay has already bee show previous publicatios. I the first experimet, the task allocatio is disabled such that master 1 ad master 2 have the same authority o the pipe ed. Fig. 14 depicts the motio of the pipe. Both devices (P m2,p m1 ) ad thus the virtual pipe are at first moved upwards (4.5s to 6s, see Fig. 15, the positio trackg plots depict positio deviatios from the itial positios such that the pipe legth offset ydirectio is H.M. PCU AgetΛ1 v 1b F 1b F 5b v 5b PR1 F 2 T DPN2 PC1 F 3 αti L2R F 6b PI2 v 6b v 3 Track i αti R2L PI1 F 8 v 8 v 5a F 5a PC2 T DPN1 F 9 PR2 v 6a F 6a F 1b v 1b S.E. PCU AgetΛ2 Fig. 11. Network represetatio of a trackpopc system for cooperative teleoperatio with task allocatio

5 Pipe Fig. 12. DLR HMI Fig. 13. DLR SpaceJust Pipe Fig. 16. Pipe motio with task allocatio P m1 Pm1 PE P m2 Fig. 14. Pipe motio without task allocatio ot cosidered). The master 1 reoriets the pipe (6s to 7s) through rotatio aroud P m2 ad the virtual grippg pot Pm1 PE, which depeds o the master 1 pose P m1, respectively. Durg the rotatio aroud the pipe ed the positio of this virtual grippg pot is ot absolutely fixed, as ca be see Fig. 14. This happes due to cross couplgs of rotatios ad traslatios the robot arms. From secod 7s to 1s both devices are moved horizotally. Durg the whole procedure the master 2 positio P m2 ad the virtual grippg pot Pm1 PE match very well. I the secod experimet the task allocatio scalgs have bee chose correspodg to TABLE I. At first (Fig. 17, 8s to 9.5s), master 1 rotates the pipe aroud P m2. Thaks to the task allocatio, the operator therefore oly eeds to keep the rotatioal DoFs rather loose o both devices ad commad the rotatio aroud the xaxis comfortably ad accurately through a force alog the zaxis. The master 2 commads a traslatioal motio of the virtual pipe the xyplae (see 1s to 12s). Fally ( the time betwee Fig. 17. time[s] Positio trackg with task allocatio 12.5s to 13s) master 1 is pushg the virtual pipe dow (see Fig. 16), but due to the task allocatio, the force part of W m2 actg o master 2 is always zero (compare Fig. 18). The third experimet volves the whole multilateral cooperatio (depicted Fig. 12 ad Fig. 13) with the task allocatio values of TABLE I. The task is the sertio of a pipe as depicted Fig. 19. At first, the real ad the virtual pipe are rotated aroud the ed of the real pipe Ps PE ad the virtual pipe Pm1 PE respectively to horizotal orietatio. The the pipe ed is pushed to the xpositio of the plug. I order to plug the real pipe the devices are afterwards pushed horizotally alog the pipe axis (yaxis) P m1 P.1 m1 PE P.2 m time [s] Fx [N] Fy [N] Fz [N] time[s] Mx [Nm] My [Nm] Mz [Nm] time[s] W m1 Wm1 PE W m2 Fig. 15. Positio trackg without task allocatio Fig. 18. Forces ad torques durg maipulatio with task allocatio

6 .5.1 Fig Pipe.5.1 Pipe motio multilateral system with task allocatio P PE m1 P m2 P PE s Fig. 2. Trackg of virtual grippg pot o real ad virtual pipe ed ad master P m1 P s Fig. 21. Trackg of slave ad master 1 devices to the plug.the positio followg (see Fig. 2 ad Fig. 21) of the three devices is absolutely satisfactory though the cross couplgs ad the costrats caused by sgularities all three robots fluece the trackg performace slightly. V. CONCLUSION The multilateral trackpopc approach for delayed teleoperatio has bee exteded to the multidof case a MMSS system. A virtual grippg pot has bee implemeted via passive projectios to ease the maipulatio of a log object by a sgle slave robot. Through the bimaual operatio of the pipe, its rotatioal DoFs ca be more accurately affected through forces tha they could be a uimaual system purely through torques. The cartesia task allocatio helped to mata the cogruece of pipe ed ad plug positios. The positio followg of the three devices, the virtual ad real grippg pots was satisfactory. ACKNOWLEDGMENT This paper was supported by the project Developmet of a TeleService Ege ad TeleRobot Systems with Multi Lateral Feedback fuded by the Mistry of Trade, Idustry & Eergy of Korea. REFERENCES [1] D. G. Lee, G. R. Cho, M. S. Lee, B.S. Kom, S. Oh, ad H. I. So, Humacetered evaluatio of multiuser teleoperatio for mobile maipulator umaed offshore plats, IEEE Iteratioal Coferece o Itelliget Robots Ad Systems, pp , [2] A. Peer, S. Hirche, C. Weber, I. Krause, ad M. Buss, Itercotetal multimodal telecooperatio usg a humaoid robot, IEEE Iteratioal Coferece o Itelliget Robots ad Systems, pp , 28. [3] S. Sirouspour, Modelg ad cotrol of cooperative teleoperatio systems, IEEE Trasactios o Robotics, vol. 21, pp , 25. [4] C. Passeberg, A. Peer, ad M. Buss, Modelmediated teleoperatio for multioperator multirobot systems, IEEE Iteratioal Coferece o Itelliget Robots ad Systems, pp , 21. [5] P. Malysz ad S. Sirouspour, Dualmaster teleoperatio cotrol of kematically redudat robotic slave maipulators, IEEE Iteratioal Coferece o Itelliget Robots Ad Systems, pp , 29. [6] M. Pazirsch, J. Artigas, A. Tobergte, P. Kotyczka, C. Preusche, A. AlbuSchaeffer, ad G. Hirzger, A peertopeer trilateral passivity cotrol for delayed collaborative teleoperatio, EuroHaptics, vol. 12, pp , 212. [7] D. Feth, B. A Tra, R. Grote, A. Peer, ad M. Buss, Sharedcotrol paradigms multioperatorsglerobot teleoperatio, Huma Cetered Robot Systems. Sprger Berl Heidelberg, 29, pp [8] S. Sirouspour ad P. Setoodeh, Adaptive olear teleoperatio cotrol multimaster/multislave eviromets, IEEE Iteratioal Coferece o Cotrol Applicatios, pp , 25. [9] T. Kao ad Y. Yokokohji, Multilateral teleoperatio cotrol over timedelayed computer etworks usg wave variables, Haptics Symposium, pp , 212. [1] M. Pazirsch, J. Artigas, J.H. Ryu, ad M. Ferre, Multilateral cotrol for delayed teleoperatio, IEEE Iteratioal Coferece o Advaced Robotics, pp. 1 6, 213. [11] P. Malysz ad S. Sirouspour, Trilateral teleoperatio cotrol of kematically redudat robotic maipulators, The Iteratioal Joural of Robotics Research, vol. 3, pp , 211. [12] P. Malysz ad S. Sirouspour, Cooperative teleoperatio cotrol with projective force mappgs, IEEE Haptics Symposium, pp , 21. [13] S. Katsura, T. Suzuyama, ad K. Ohishi, A realizatio of multilateral force feedback cotrol for cooperative motio, IEEE Trasactios o Idustrial Electroics, vol. 54, pp , 27. [14] P. Malysz ad S. Sirouspour, Task performace evaluatio of asymmetric semiautoomous teleoperatio of mobile twarm robotic maipulators, IEEE Trasactios o Haptics, vol. 6, pp , 213. [15] H. V. Quag ad J.H. Ryu, Stable multilateral teleoperatio with time doma passivity approach, IEEE Iteratioal Coferece o Itelliget Robots Ad Systems, pp , 213. [16] H. LeBlac, E. Eyisi, N. Kottestette, X. Koutsoukos, ad J. Sztipaovits, A passivitybased approach to deploymet multiaget etworks, Iformatics Cotrol, Automatio ad Robotics. Sprger Berl Heidelberg, 211, pp [17] N. Y. Chog, T. Kotoku, K. Ohba, ad K. Komoriya, Remote coordated cotrols multiple telerobot cooperatio, IEEE Iteratioal Coferece o Robotics Ad Automatio, pp , 2. [18] D. Lee ad M. W. Spog, Bilateral teleoperatio of multiple cooperative robots over delayed commuicatio etwork: Theory, IEEE Iteratioal Coferece o Robotics Ad Automatio, pp , 25. [19] R. Mohajerpoor, I. Sharifi, H. A. Talebi, ad S. M. Rezaei, Adaptive bilateral teleoperatio of a ukow object hadled by multiple robots uder ukow commuicatio delay, IEEE Iteratioal Coferece o Advaced Itelliget Mechatroics, pp , 213. [2] J. Artigas, J.H. Ryu, ad C. Preusche, Time doma passivity cotrol for positiopositio teleoperatio architectures, Presece: Teleoperators ad Virtual Eviromets, vol. 19, o. 5, pp , 21. [21] J. Artigas, J.H. Ryu, C. Preusche, ad G. Hirzger, Network represetatio ad passivity of delayed teleoperatio systems, IEEE Iteratioal Coferece o Itelliget Robots Ad Systems, pp , 211.