Dynamic Model Displacement for Model-mediated Teleoperation

Transcription

1 Dynaic Model Displaceent for Model-ediated Teleoperation Xiao Xu Giulia Paggetti Eckehard Steinbach Institute for Media Technology, Technische Universität München, Munich, Gerany ABSTRACT In this paper, we study and extend the concept of odel-ediated teleoperation (MMT) for teleaction systes which provide live video feedback fro the reote side with a tie delay. In MMT, the haptic feedback is rendered locally on the operator side using a siple object surface odel in order to keep the haptic control loop stable in the presence of counication delays. Because the live video fro the reote side is received with delay, this results in a visual-haptic asynchrony for the displayed interaction events. In addition, sudden odel paraeter updates can lead to odel-jup effects for the displayed haptic feedback. Both effects degrade the user experience and syste perforance. To address these issues, we propose an extension of MMT which we call odel-displaced teleoperation (MDT) in this paper. In MDT, we adaptively shift the position of the local surface odel to delay the haptic contact with the environent, thus copensating the visual-haptic asynchrony and avoiding the odel-jup effect. As the haptic feedback is still rendered locally, the advantages of the MMT approach are retained and instabilities in the haptic interaction are avoided. In our experients, we deterine the optial displaceent coproise between visual-haptic asynchrony, the odel-jup effect and perceived distance errors. Moreover, the subjective experience and objective task perforance of the proposed MDT and the original MMT for a teleoperation setup with soft objects are evaluated. Our results show that the users prefer the MDT ethod copared to MMT once the counication delay between the teleoperator and the operator exceeds 50s. In addition, the task error rate is reduced by about 50% and the subjects are better able to control their contact force for syste delays larger than 50s if the MDT ethod is eployed. Index Ters: odel-ediated teleoperation, asynchrony copensation, odel-jup effect, odel displaceent 1 INTRODUCTION A telepresence and teleaction (TPTA) syste consists of three ain parts: the huan operator (OP)/aster syste, the teleoperator (TOP)/slave syste, and the counication link/network in between [1]. The TOP is typically controlled by position or velocity coands generated by the OP and returns the haptic and visual signals sensed during its interaction with the reote environent back to the OP. The coands and the visual-haptic data are exchanged over a counication network. On the operator side the haptic and visual feedback signals are displayed to the user, which allows the users to haptically and visually ierse theselves in the reote environent. For teleoperation systes with geographically separated operators and teleoperators, tie delay introduced by the counication network always exists. It is well known that even a sall tie de- e-ail: xiao.xu@tu.de e-ail:giulia.paggetti@tu.de e-ail:eckehard.steinbach@tu.de lay in the haptic channel jeopardizes the syste stability and perforance [2]. Several control architectures have been developed to enable a stable TPTA syste in the presence of counication delays. In [3, 4], the concept of odel-ediated teleoperation (MMT) is proposed, which guarantees a stable TPTA syste by local rendering of the haptic feedback using a siple object surface (e.g. a plane). Sudden odel updates occur when new environent paraeters (position and ipedance) are detected at the TOP and signaled to the OP, which leads to disturbing odel-jup effects [5]. In [5], a haptic rendering ethod is proposed to attenuate the odel-jup effect for position change of rigid objects. However, for non-rigid environents, the odel-jup effect for ipedance change (stiffness, friction, etc.) persists. In addition, due to the negligible delay of the locally rendered haptic feedback, the asynchrony between the displayed haptic signals and the video feedback received fro the reote side grows with increasing counication delay [3]. The effect of delay on huan task perforance has been investigated in any studies [6-8]. For exaple, in [6], the effect of the visual-feedback delay on the user s task copletion tie is exained. It is shown that the perforance is affected for delays exceeding 75 s. In [7], the consequence of tie delay on visual-haptic asynchrony judgents is investigated. It is found that subjects could easily detect the visual-haptic asynchrony when the stiulus onset exceeds 50 s. Siilarly, in [8] it is shown that during a haptic tapping task with delayed visual feedback, the task perforance decreases and the subjective task difficulty increases with growing visual delay. This asynchrony disturbs the user perception and thus needs to be copensated for, or at least reduced. The state-of-the-art to copensate for the video delay in real TPTA systes is to augent the operator with a so-called predictive display. In [9, 10], a virtual odel of the reote environent is overlayed on the real video iages and the geoetric odeling errors are also dealt with using augented reality concepts. However, when using graphic prediction only, the predictive contact force is lost. The cobination of a predictive display and predictive force feedback is successfully addressed in [11], where the predictive haptic and visual signals are synchronized. The predictive display approach, however, asks for a precise geoetric odel of the reote environent. If the TPTA syste works in a coplex and visually rich environent (coplicated geoetric features, textures, dynaic environent), it is difficult to extract such inforation. In this case, rather than applying the predictive display ethod, it is preferable to watch the delayed real video directly, with no loss of precision and detail. Thus, a better ethod is needed to copensate for the asynchrony between the visual and haptic feedback signals. In this paper we describe a novel approach that copensates for the asynchrony between the haptic and visual feedback and reduces the odel-jup effect for MMT. Siilar to [3, 4], siple surface odels of the reote environent are built using distance sensors to generate haptic signals locally on the OP side. These surface odels are invisible to the user. To copensate for the visualhaptic asynchrony and reduce the odel-jup effect, we change the position of the virtual surface odel on the OP side by shifting it along the otion direction of the OP. Thus, the odel rendering is locally delayed and the visual-haptic asynchrony as well as the odel-jup effect are copensated for (or at least reduced). As

2 Video Model Paraeters Video Delayed video feedback displayed at the OP Haptic signals at the OP at the sae tie instant Operator /Master Force Position /Force Local Model Network Force Local Model Sensor data Position /Force Teleoperator / x s Actual floor (reote) Visual-haptic asynchrony s x obj = x obj Virtual odel of floor (local) Master x Figure 1: Overview of a MMT syste (adopted fro [3]). the haptic signals are still rendered locally without delay, the syste reains stable. We refer to the proposed extension of MMT as odel-displaced teleoperation (MDT). Two experients in this paper are conducted to find out the optial displaceent coproise and show the iproveent both in subjective quality and task perforance for MDT. The reainder of this paper is organized as follows. Sec. 2 briefly reviews the concept of MMT. Sec. 3 explains the proposed approach. Sec. 4 describes the design of the experiental evaluation and suarizes the results. We conclude this paper in Sec MODEL-MEDIATED TELEOPERATION MMT [3] uses the position of the slave robot and the force/torque feedback signals to build a virtual odel of the reote environent. The odel paraeters are transitted to the OP side where a local virtual odel is constructed accordingly. While the user interacts with the reote environent, the haptic feedback is generated locally without any delay based on this virtual odel. On the TOP side, the slave is position-force controlled [3, 4] (Fig. 1). The control schees for the slave and aster side in [3] and [4] are as follows. The force to be displayed by the aster is: F = k p (x proxy x ) (1) where k p, x proxy and x are the environent stiffness (received fro the TOP), the proxy (for haptic rendering) and aster positions, respectively. The slave is position controlled in the direction tangent to the planar surface and force controlled in the direction perpendicular to the planar surface: F s = F pd F with F pd = ax(f pd,f thresh ). (2) where F s is the slave force, F pd is the proportional-derivative (PD) control force coand on the slave side and F thresh is set as the axiu PD force [3]. In the following, siilar to [3, 4], the environent is locally approxiated by a frictionless planar surface. Distance sensors on the TOP side are eployed to estiate the environent geoetry. Different fro [4], the object surface in the reote environent is considered to be non-rigid and its stiffness varies spatially. As stated in Sec.1, the visual-haptic asynchrony grows with increasing counication delay. Additionally, odel updates lead to the odeljup effect which jeopardizes the syste perforance [5]. In the following subsections we discuss the visual-haptic asynchrony and odel-jup issues of the original MMT in detail. 2.1 Asynchrony in Visual-haptic Feedback The left-hand side of Fig. 2 illustrates the delayed video feedback of the slave position on the TOP side as it is displayed to the operator. The right-hand side shows for the sae tie instant the estiated local virtual odel and the current position of the aster on the OP side. The haptic feedback is rendered locally without delay which results in a visual-haptic asynchrony. The aster gets in contact with the local virtual odel and generates a haptic Figure 2: Visual-haptic asynchrony for 1-DoF MMT with counication delay. The user experiences the collision haptically T d (round-trip delay) before seeing it visually. x s ob j and x ob j are the real floor position on the TOP side and the virtual floor position on the OP side, respectively. contact signal before the contact actually occurs in the reote environent. This iplies that the displayed haptic signal is always ahead of the displayed visual feedback signal in a TPTA syste with counication delay. This can also lead to a haptically unexpected collision, which refers to the situation where the users watch the feedback video and see there is still a distance between the slave and environent, while the haptic contact signal is already locally generated and displayed through the haptic interface to the users. Therefore, our hypothesis is that, due to the unexpected collision, the contact force at the collision tie instant is beyond the control of the user and the slave as well as the environent ay experience undesired collision forces. 2.2 Model Jup Effect For reote object surfaces with spatially varying stiffness, the virtual plane position and orientation can be estiated on the TOP side before the slave gets in contact with the environent, while the plane stiffness can be only estiated after that [4]. As the haptic force is rendered locally without delay based on the virtual odel and the aster position, an initial estiation of the plane stiffness on the OP side is necessary. After the actual stiffness value fro the TOP side arrives at the OP side, the stiffness of the local virtual plane is updated. If the initially estiated stiffness k is softer p(init) (under estiation) or harder (over estiation) than the easured stiffness k s p, the user experiences the so-called odel-jup effect [5]. Fig. 3 illustrates the odel-jup effect for an under estiated object stiffness. After the OP receives the easured environent stiffness k p fro the TOP side, the stiffness of the local virtual plane on the OP side is updated fro k p(init) to k p. As a result, the corresponding aster force fro Eq. (1) increases during a short tie. For both under and over estiation, the fast and unexpected force change on the OP side degrades the user experience [5]. In addition, on the TOP side the behavior of the slave ay becoe uncontrollable and dangerous if it receives and applies the changed force fro the OP side (slave in force controlled ode, Eq. (2)). In general, with the described issues of the MMT ethod the users experience difficulties in controlling the applied force and estiating the environent properties. Therefore, the degradation of the syste perforance caused by the visual-haptic asynchrony ([6] and Sec. 2.1) and the odel-jup effect needs to be copensated for. 3 MODEL-DISPLACED TELEOPERATION In this section, we describe the details of our proposed MDT approach for copensating the visual-haptic asynchrony and avoiding the odel-jup effect. As discussed in Sec. 2, the reote environent is locally approxiated by a frictionless planar surface with spatially varying stiffness located in 3D space. The position and orientation of the plane can be deterined using distance sensors

3 on the TOP side, while its stiffness is estiated during contact between the slave and the environent [4]. 3.1 Model Displaceent Delaying the haptic feedback signals on the OP to copensate for the asynchrony between the locally rendered haptic feedback and the video feedback leads to unstable teleoperation [2]. Rather than delaying the haptic feedback signals directly, we propose to displace the local virtual plane. As illustrated in Fig. 4, the local plane is shifted by the distance x shi ft fro the original position x ob j = xs ob j in the direction of the aster s otion. In Fig. 4, x d is the distance between the slave and the plane surface detected by the distance sensors. To fully copensate for the visual-haptic asynchrony the new plane position on the OP side ust be x ob j(new) = x + x d (x is the aster position). Therefore, the plane displaceent x shi ft at the current tie t is: x shi ft (t) = xob j(new) (t) x ob j (t) = x (t) + x d (t) xob s j (t) (3) Fro Fig. 4 we know x s ob j = x s + x d (x s is the slave position). Assue the round-trip delay is T d, cobined with Eq.(3) we have: t x shi ft (t) = x (t) x s (t) = x (t) x (t T d ) = ẋ (τ)dτ (4) t T d The displaceent x shi ft (t) ust be calculated in real tie in order to deal with arbitrary aster oveent and round-trip delay. Particularly, for constant exploration velocity ẋ and roundtrip delay T d, a siplified equation to calculate x shi ft fro (4) is x shi ft = ẋ T d, which eans the x shi ft is now constant and can be directly calculated once velocity and delay are known. According to our MDT, the force rendering on the OP side is now synchronized with the video and the stiffness easureent. Hence, the initially estiated plane stiffness k p(init) on the OP side is no longer necessary. The plane stiffness is directly set to k p right after the OP receives it and the haptic feedback signals are rendered based on the shifted plane odel with the stiffness k p. Therefore, the sudden updating of the odel paraeters and hence the odel-jup effect are avoided. Actually, to easure the object stiffness k p on the TOP side, a certain tie period t is necessary. Thus, there is an asynchrony t between the received video and stiffness signal, which ust be taken into account when the odeljup effect needs to be copletely avoided. In general, the ain difference between our MDT ethod and the original MMT ethod is that our MDT ethod doesn t render the local object odel at the absolute location (xob j = xs ob j ) after the first contact ([3, 5]), but with an adaptive shift x shi ft. Delayed video feedback displayed at the OP k p s (a) Haptic signals at the OP at the sae tie instant Master k p(init) Delayed video feedback displayed at the OP k p s (b) Haptic signals at the OP at the sae tie instant F Master k p(init) Figure 3: Illustration of the odel-jup effect. (a) Before the real environent stiffness is known, the OP uses an initial stiffness k p(init) for local force rendering. (b) After the correct stiffness is available on the OP side (k p ), the stiffness of the local odel is updated to k p. In this exaple k p(init) < ks p, thus, a sudden force change is applied to the user as illustrated by the vertical force vectors F and F. k p F ' Delayed video feedback displayed at the OP s x obj x s Actual floor x d Fro distance sensors Haptic signals at OP at the sae tie instant x obj Master x x d x shift xobj (new ) Shifted virtual odel of floor Figure 4: Model displaceent ethod with full asynchrony copensation. After the OP receives x d and xob s j fro the TOP side, the required displaceent x shi ft and the new (shifted) plane position xob j(new) for fully copensating the asynchrony can be estiated. 3.2 Distance Errors As discussed in (4), both increasing aster velocity ẋ and increasing round-trip delay T d cause larger required plane displaceents to fully copensate for the visual-haptic asynchrony and to avoid the odel-jup effect. The plane displaceent x shi ft requires an extra oveent on the OP side to get in contact with the object surface, which, if too big, provides an unrealistic experience to the user of interacting with the reote environent. We consider this unrealistic experience as distance errors. In general, if x shi ft is too large, although the asynchrony is fully copensated for, it gives an unrealistic kinesthetic experience of distance. On the other hand, a reduction of x shi ft (under-copensation) could cause perceivable visual-haptic asynchrony and distortions caused by the odel-jup effect. Therefore, we design an experient to find the optial (in ters of overall subjective experience) displaceent coproise between visual-haptic asynchrony and perceived distance errors (Sec. 4.1). 4 EXPERIMENTS In our experient, a displaceent threshold x liit is given to liit the axiu plane shift value in order to find out the optial displaceent coproise, which eans the actual plane shift is xshi actual ft = in(x shi ft,x liit ). Based on the results collected, a second experient is perfored, in the context of a coplicated task, to quantify the iproveents for our MDT approach in subjective experience and task perforance. Our MDT approach is copared and contrasted with the original MMT ethod. 4.1 Experient A As described in Eq.(4), both the user behavior (exploration velocitiy ẋ ) and the round-trip delay T d influence the optial displaceent coproise. In this experient, we deterine this coproise as a function of both factors (ẋ and T d ). For fixed exploration velocity and round-trip delay, a plane displaceent of x shi ft = ẋ T d can fully copensate for the visual-haptic asynchrony and avoid the odel-jup effect Experiental Set-up Participants A total of 17 subjects (12 ales) participated in this experient, ranging in age fro All of the were righthanded. 4 of the had never used a haptic device before, 6 of the had soe prior experience and the reaining use such a device on a regular basis. Materials and Design The test software is based on the CHAI3D library ( The SensAble PHANTOM Oni haptic device is used for the experient. Delayed video fro the reote environent is displayed on the coputer onitor.

4 Figure 5: The virtual environent in the first experient. The orange ball oves vertically with the target velocity as a reference. The big white ball is the HIP of the operator and the white point on the plane is the projection of the HIP. Task Description and Experiental Procedure Subjects are asked to control a haptic interaction point (HIP) with constant velocity to touch a 3D plane in the virtual environent several ties (Fig. 5). As the velocity ẋ is the key paraeter in this experient, we provide visual and nuerical instructions for the subjects as a reference. By following the otion of a reference ball and watching the nuerical instructions, the subjects can adjust their oveent velocity closely to the target velocity (see Fig. 5). The starting position of the HIP is about 10c (non-scaled) above the 3D plane. The visual feedback on the onitor is delayed according to the assued round-trip delay T d. The target exploration velocities are 5c/s, 10c/s, 20c/s and 35c/s and the delays are 50s (sall), 200s (ediu) and 400s (large). The tested displaceent liits are x liit ={0c, 2c, 3c, 5c, 8c, 12c} (0c-shifting corresponds to the original MMT approach). The experient has 3 sessions. Each session has a fixed roundtrip delay (50s, 200s and 400s) and 24 rounds (4 target velocities 6 plane displaceent liits). For each round the target velocity and displaceent liits are chosen randoly. Before the experient, a zero-delay case is shown as a reference to the subjects. After each round the user is asked to give a subjective rating for the naturalness of the visual-haptic experience copared to the reference based on a rating scale (1 - copletely distorted (unnatural), 2 - disturbing, 3 - slightly disturbing, 4 - perceptible degradation, 5 - no difference with the reference case (natural)). Then, according to the result of the subjective rating vs. the liits of plane displaceent we can find out the perceptually optial coproise (liit) of the plane displaceent value for different exploration velocities and round-trip delays Results and Discussion As shown in Tab. 1 at a velocity of less than 35c/s all subjects are able to adjust their exploration velocity closely to the target. This result suggests that the user s ability to adapt the velocity to the target is sufficient for target velocities below 35c/s. As shown in Fig. 6(a) for 50s round-trip delay the subjective rating is between 4 (perceptible degradation) and 5 (no difference with the reference case). This result suggests that the subjects are hardly able to perceive a visual-haptic asynchrony of 50s. This result is not surprising if we consider that previous studies have shown that huans are able to perceive visual-haptic asynchrony only if it is larger than 50s (e.g. [7]). Considering the nearly flat subjective rating in Fig. 6(a), it is hard to find the optial displaceent coproise. Therefore, we choose the plane shift of 2c as the optial coproise, which is enough to fully copensate for the asynchrony for all the investigated velocities for the delay of 50s (x shi ft = 35c/s 50s = 1.75c < 2c). For a round-trip delay of 200s and 400s the iproveent shown by using our MDT ethod becoes clear. As shown in Fig. 6(b), for an asynchrony of 200s a higher subjective rating is found for MDT. The perceptually optial coproise is found, for each velocity except for 5c/s, at a plane shift liit of 3c. At a Table 1: Mean and standard deviation of the easured velocities as a function of the round-trip tie target Delay 50s Delay 200s Delay 400s velocity 5c/s 5.2±1.2c/s 5.4±1.6 c/s 5.2±2.1 c/s 10c/s 10.2±2.3c/s 10.4±2.8c/s 11.0±2.4c/s 20c/s 21.8±2.7c/s 22.2±3.2c/s 22.1±4.3c/s 35c/s 33.8±5.6c/s 32.4±7.1c/s 30.9±9.3c/s velocity of 5c/s, the subjective rating is alost flat (around 4) for a plane shift liit of 0c to 12c. The sae tendency can be observed for a delay of 400s as shown in Fig. 6(c). At a velocity of 5c/s, except for 0c-plane shift, the subjective rating is always around 3.5. For higher velocities ( 10 c/s) the turning point is at a plane shift liit of 5c. Note that the actual plane shift xshi actual ft = in(x shi ft,x liit ) is not always identical to the x liit. For exaple, for the target velocity of 20c/s with a delay of 200s in Fig. 6(b), a axiu plane shift x shi ft = 20c/s 200s = 4c is enough to fully copensate for the asynchrony. Thus, for the tested shift liits 5c, 8c and 12c, the actual plane shift xshi actual ft is always 4c, which leads to a nearly flat subjective rating curve for the range of shift liits fro 5c to 12c. The sae trend can be also observed for other target velocities in Fig. 6(b) and 6(c). In suary, the experient shows that for a delay of 200s and 400s our MDT approach is able to iprove the subjective experience for all the velocities considered. Moreover, for a delay of 50s, 200s and 400s, the optial coproise of the plane displaceent liits is 2c, 3c and 5c, respectively. These values are adopted in our second experient. 4.2 Experient B As discussed in Sec. 2.1 and 2.2, unexpected collisions caused by visual-haptic asynchrony as well as the odel-jup effect lead to unstable aster force at the instant of collision. In addition, a growing visual-haptic asynchrony increases the task copletion tie [6]. Moreover, sudden stiffness updates (odel jups) give the user tie-variant stiffness inforation and ay lead to wrong decisions when the users are estiating the environent stiffness. Therefore, in the second experient we consider the error of the applied force, the error of the stiffness estiation and the copletion tie as the three ain factors to evaluate the task perforance. Meanwhile, the axiu contact force is also investigated. In this experient we apply the previously found optial displaceent coproise on the designed task experient. The easureent tie of object stiffness t (see Sec. 3) is neglected Experiental Set-up Participants 17 subjects, new to the study, are selected following the sae criteria as in the first experient. Materials and Design A Force Diension Siga.7 device (aster) is operated by the users on the OP side. The TOP consists of a KUKA LWR ar (slave), a JR3 force sensor and a Phidgets distance sensor (range 4-30c) (Fig. 7). The software environent is based on ROS ( and the H3D library ( The round-trip delays T d in this experient are set to be 50s, 200s, 400s and 1000s (included to investigate larger delay). The previously deterined displaceent liits 2c, 3c and 5c are adopted for the delay of 50s, 200s and 400/1000s, respectively. Task Description and Experiental Procedure The experiental task consists of oving a robotic ar to explore an artificial

5 Subjective evaluation [1 5] c/s 2 10 c/s 20 c/s Delay 50s 35 c/s plane shift liits (c) (a) Subjective evaluation [1 5] c/s 2 10 c/s 20 c/s Delay 200s 35 c/s plane shift liits (c) (b) Subjective evaluation [1 5] c/s 10 c/s 20 c/s 35 c/s Delay 400s plane shift liits (c) (c) Figure 6: Results of Experient A. Mean and standard deviation of the subjective ratings vs. the tested plane shift liits for delays of 50s, 200s and 400s and different exploration velocities. A plane shift liit of 0c corresponds to the original MMT approach in [3] Table 2: Preference between the MMT and MDT ethods. Syste Delay 50s 200s 400s 1000s MMT 41% 29% 3% 9% MDT 59% 71% 97% 91% huan liver and to identify the perceived stiffness (Fig. 7). Both subjective evaluation and task perforance are investigated. The artificial liver is divided into three areas of the sae diension. A total of three different stiffnesses are siulated, soft (1N/c), ediu (5N/c) and hard (25N/c), and randoly presented during the experient. In each area, the initial stiffness k is randoly generated fro the three stiffness values at the p(init) beginning of each round. The k is necessary for the original p(init) MMT and our MDT with displaceent coproise. Subjects are instructed to ove the robotic ar to touch, one at a tie, all the three areas and to identify the corresponding stiffness. Subjects are also instructed to control the applied force to the aster (less than 4N) and the tie used to coplete the task (less than 15s). The application of a force bigger than 4N, a task-tie longer than 15s and erroneous stiffness identification of the touched area are error conditions of which the subjects are infored in the training session. Before the start of the experient, subjects are trained for both force and tie control and stiffness detection, ensuring that all participants are approxiatively at the sae level. Four sessions of two rounds each are perfored twice. During each session a specific tie delay (50s, 200s, 400s, 1000s) and two different approaches, one for each round (MDT and original MMT), are applied and randoly presented ((4 sessions 2 rounds) 2 = 16 rounds). After each round, the subjective evaluation is collected by a questionnaire with 5-point Liker-scales in it. The questionnaire consists of the following stateents: S1 I copleted the task required without any error and S2 I had the feeling/sensation that the teporal synchrony between haptic and visual rendering was unnatural. Point 1 on the scale corresponds to strongly disagree and point 5 to strongly agree. After each session the subjects are asked to pick-out which round of the corresponding session they prefer (considering naturalness and cofort). Statistical analysis is applied for the perforance data. Two saple T-test and Wilcoxon rank su test are eployed for the paraetric and nonparaetric data, respectively Results and Discussion As reported in Table 2, a rate of preference (preferred round) bigger than 59% is obtained for our MDT ethod for all delay values. With a growing round-trip delay, a stronger user preference can be Figure 7: The setup of the second experient. observed. For exaple, for the delay of 400s, the preference is as high as 97%, which shows a great iproveent of user experience for our MDT ethod. Considering the results of the questionnaire about S2, for all the four delays considered, subjects report a bigger sense of unnaturalness for MMT copared to our MDT ethod (Fig. 8). Both these results suggest that, copared to the original MMT ethod, our MDT approach is able to iprove the sense of naturalness and cofort due to the reduced sensation of asynchrony and odel-jup effect during the collision tie. A different consideration is needed for the results of the questionnaire about S1. For delays of 50s and 200s a very close subjective evaluation rating is found, but with a different nuber of errors (7% and 15% for the MMT and 5% and 8% for our MDT ethod). These results suggest that when our MDT ethod is applied subjects are better able to realize if errors are ade. Further studies are required to clarify this hypothesis. As shown in Table 3, ore stiffness identification errors are ade for the MMT ethod than for the MDT ethod and about twice the error rate can be observed when the delay is larger than 50s. Indeed, with a delay of 200s, 15% errors are ade for MMT, copared to 8% errors for the MDT ethod. The sae trend is also found for a delay of 400 s (30% copared to 11% errors) and 1000 s (38% copared to 18% errors). This result gives a clear hint of the task perforance iproveent obtained, already for delays of 50s, by using the MDT copared to the MMT ethod. The odel-jup effect could be the reason for the higher error rate for the original MMT as discussed in Sec 2.2. The suddenly updated environent stiffness disturbs the users perception. The users perceive two stiffness values, k p(init) and k p, which could cause an estiation error if k p(init) k p. Siilar to the stiffness errors, a saller error rate of force control is found for the MDT ethod copared to MMT (Table 3). For the MMT ethod, for delays of 200s, 400s and 1000s, an

6 Table 3: Coparison of the task perforance between MMT and MDT (total rate, ean value and standard deviation). Investigated Factors Delay 50s Delay 200s Delay 400s Delay 1000s MMT MDT MMT MDT MMT MDT MMT MDT total error rate (stiffness) 7% 5% 15% 8% 30% 11% 38% 18% total error rate (applied force) 5% 3% 16% 7% 29% 14% 45% 21% ean ax. contact force (N) 3.2± ± ± ± ± ± ± ±1.3 ean tie consuption (s) 9.8± ± ± ± ± ± ± ±1.3 Figure 8: Results of syste delay vs. ean and standard deviation of the subjective rating. error rate about twice the error rate for the MDT ethod is found. Meanwhile, the ean axiu contact force is ore than 0.6N lower if the delay is ore than 200s and the MDT ethod is eployed, which verifies our hypothesis in Sec This result suggests that when the delay is bigger than 50 s, if our MDT ethod is eployed, the subjects are better able to control the applied force, which is andatory in several contexts like surgical environents. In Tab. 3, the task copletion tie is reported. The results suggest that, apart fro 1000s delay, subjects are able to coplete the task in tie (less than 15s) for both approaches. A shorter ean copletion tie is found for the MDT ethod. As suggested by previous studies [6]-[8], visual-haptic asynchrony increases the tie needed to coplete the task. In addition, after perceiving the odel-jup effect, the users need extra tie to re-estiate the environent stiffness. Thus, the results found by our experient suggest that the MDT ethod is able to reduce the copletion tie due to a reduced visual-haptic asynchrony and odel-jup effect. Moreover, according to the statistical analysis, a significant difference between the two approaches for all three factors (stiffness errors, force errors and copletion tie) is found for the round-trip delay of 400s and 1000s (p < 0.05), which confirs that our MDT ethod iproves the task perforance significantly for large delays. In suary, these results show that an iproved sense of cofort and naturalness is found for the odel displaceent ethod. In addition, a higher level of task perforance is reached. 5 CONCLUSION AND FUTURE WORK In this paper, we present a novel approach to copensate for the asynchrony between the haptic and visual feedback signals and reduce the odel-jup effect for odel-ediated teleoperation with real video feedback. For a coplex dynaic environent, using a predictive display is not an option. Instead, real video captured on the reote side is displayed on the operator side. However, this leads to a round-trip tie dependent asynchrony between haptically and visually displayed events as well as perceivable distortions due to the odel-jup effect. A odel displaceent algorith is applied to copensate for this asynchrony and to reduce the odeljup effect copared to the original MMT approach. As the haptic signals are still rendered locally, the syste reains stable even for significant counication delays. Copared to the original MMT approach, the subjects report an iproved quality of experience for our MDT ethod. In addition, they are also better able to control their applied force ore accurately if our MDT approach is eployed. Moreover, our MDT approach shows a reduction of the task error rate by ore than 50% and the task perforance is significantly iproved for large counication delays. In future work, we plan to evaluate our MDT approach for ore coplex environents, including dynaic, deforable objects. In addition, the jitter of the counication delay will be also considered. Moreover, a depth caera will be used to build ore precise virtual odels of the object surfaces in a real reote environent. ACKNOWLEDGEMENTS This work has been supported by the European Research Council under the European Unions Seventh Fraework Prograe (FP7/ ) / ERC Grant agreeent no The authors would also like to thank Nicolas Alt and Burak Cizeci of the Institute for Media Technology at Technical University Munich for their technical support on the KUKA ar. REFERENCES [1] W. Ferrell, and T. Sheridan. Supervisory control of reote anipulation. Spectru, IEEE, vol. 4, no. 10, pp , [2] D. Lawrence. Stability and transparency in bilateral teleoperation. IEEE Transactions on Robotics and Autoation, vol. 9, no. 5, pp , [3] P. Mitra, and G. Nieeyer. Model ediated teleanipulation. International Journal of Robotics Research, vol. 27, no. 2, pp , [4] B. Willaert, J. Bohg, H. Brussel, and G. Nieeyer. Towards ulti-dof odel ediated teleoperation: using vision to augent feedback. Haptic Audio Visual Environents and Gaes (HAVE), 2012 IEEE International Workshop on (2012). [5] B. Willaert, H. Brussel, and G. Nieeyer. Stability of Model-Mediated Teleoperation: Discussion and Experients. Eurohaptics, 2012, Tapere, Finland. [6] I. MacKenzie, and C. Ware. Lag as a deterinant of huan perforance in interactive systes. Proceedings of the INTERACT 93 and CHI 93 conference on Huan factors in coputing systes, pp , [7] I. Vogels. Detection of teporal delays in visual-haptic interfaces. Huan factors: The journal of the Huan Factors and Ergonoics Society, vol. 46, no. 1, pp , [8] C. Jay, and R. Hubbold. Delayed visual and haptic feedback in a reciprocal tapping task. Eurohaptics Conference, 2005 and Syposiu on Haptic Interfaces for Virtual Environent and Teleoperator Systes, World Haptics First Joint, pp , [9] K. Yerex, D. Cobzas, and M. Jagersand. Predictive display odels for tele-anipulation fro uncalibrated caera-capture of scene geoetry and appearance. in Proc. IEEE Int. Conf. Robot. and Auto., vol. 3, pp , [10] S. Katsura, and K. Ohnishi. A realization of haptic training syste by ultilateral control. IEEE Trans. Ind. Electron., vol. 53, no. 6, pp , [11] H. Li, and A. Song. Virtual-Environent Modeling and Correction for Force-Reflecting Teleoperation With Tie Delay. Industrial Electronics, IEEE Transactions on, vol. 54, no. 2, pp , 2007.