Effect of Force Feedback on Performance of Robotics-Assisted Suturing

Similar documents
Methods for Haptic Feedback in Teleoperated Robotic Surgery

Haptic Feedback in Robot Assisted Minimal Invasive Surgery

Wearable Haptic Feedback Actuators for Training in Robotic Surgery

A Big Challenge of Surgical Robot Haptic Feedback

Haptic Feedback in Laparoscopic and Robotic Surgery

Cutaneous Feedback of Fingertip Deformation and Vibration for Palpation in Robotic Surgery

Application of Force Feedback in Robot Assisted Minimally Invasive Surgery

Surgical robot simulation with BBZ console

Towards robotic heart surgery: Introduction of autonomous procedures into an experimental surgical telemanipulator system

Teleoperation with Sensor/Actuator Asymmetry: Task Performance with Partial Force Feedback

Small Occupancy Robotic Mechanisms for Endoscopic Surgery

Medical Robotics. Part II: SURGICAL ROBOTICS

INDIRECT FEEDBACK OF HAPTIC INFORMATION FOR ROBOT-ASSISTED TELEMANIPULATION. by Masaya Kitagawa. Baltimore, Maryland September, 2003

Computer Assisted Medical Interventions

Using Simulation to Design Control Strategies for Robotic No-Scar Surgery

VerroTouch: High-Frequency Acceleration Feedback for Telerobotic Surgery

Wireless Master-Slave Embedded Controller for a Teleoperated Anthropomorphic Robotic Arm with Gripping Force Sensing

AC : MEDICAL ROBOTICS LABORATORY FOR BIOMEDICAL ENGINEERS

Enhanced Transparency in Haptics-Based Master-Slave Systems

On Application of Virtual Fixtures as an Aid for Telemanipulation and Training

Performance Issues in Collaborative Haptic Training

Introduction to robotics. Md. Ferdous Alam, Lecturer, MEE, SUST

Chapter 2 Introduction to Haptics 2.1 Definition of Haptics

Methods and mechanisms for contact feedback in a robot-assisted minimally invasive environment

Research article Methods for haptic feedback in teleoperated robot-assisted surgery

Control design issues for a microinvasive neurosurgery teleoperator system

Novel machine interface for scaled telesurgery

An Inexpensive Experimental Setup for Teaching The Concepts of Da Vinci Surgical Robot

Differences in Fitts Law Task Performance Based on Environment Scaling

Fuzzy Logic Based Force-Feedback for Obstacle Collision Avoidance of Robot Manipulators

Elements of Haptic Interfaces

Virtual Reality Based Training to resolve Visio-motor Conflicts in Surgical Environments

Medical robotics and Image Guided Therapy (IGT) Bogdan M. Maris, PhD Temporary Assistant Professor

The Haptic Impendance Control through Virtual Environment Force Compensation

Evaluation of Haptic Virtual Fixtures in Psychomotor Skill Development for Robotic Surgical Training

HUMAN Robot Cooperation Techniques in Surgery

Modeling and Experimental Studies of a Novel 6DOF Haptic Device

FALL 2014, Issue No. 32 ROBOTICS AT OUR FINGERTIPS

Design and Control of the BUAA Four-Fingered Hand

Università di Roma La Sapienza. Medical Robotics. A Teleoperation System for Research in MIRS. Marilena Vendittelli

Peter Berkelman. ACHI/DigitalWorld

Los Alamos. DOE Office of Scientific and Technical Information LA-U R-9&%

Touch Feedback in a Head-Mounted Display Virtual Reality through a Kinesthetic Haptic Device

Tactile Interactions During Robot Assisted Surgical Interventions. Lakmal Seneviratne

Haptic Tele-Assembly over the Internet

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING,

Effects of Geared Motor Characteristics on Tactile Perception of Tissue Stiffness

Jane Li. Assistant Professor Mechanical Engineering Department, Robotic Engineering Program Worcester Polytechnic Institute

Analysis of Suture Manipulation Forces for Teleoperation with Force Feedback

Maneuverability Evaluation of a Surgical Robot for Single-Port Surgery

Curriculum Vitae IMAN KHALAJI

University of Alabama at Birmingham. ObGyn Residency. Laparoscopy Training Lab PGY 1-4. Individual Pelvic Trainer Tasks

Development of a Master Slave Combined Manipulator for Laparoscopic Surgery

Image Guided Robotic Assisted Surgical Training System using LabVIEW and CompactRIO

Acquisition of Multi-Modal Expression of Slip through Pick-Up Experiences

Chapter 1. Introduction

Intracorporeal Knot-Tying and Suturing Techniques in Laparoscopic Surgery: Technical Details

Techniques of the hand tie and instrument tie

MEAM 520. Haptic Rendering and Teleoperation

Haptic control in a virtual environment

Computer Haptics and Applications

MEAM 520. Haptic Rendering and Teleoperation

Surgeon s Third Hand: An Assistive Robot Endoscopic System with Intuitive Maneuverability for Laparoscopic Surgery

Simendo laparoscopy. product information

The Effect of Haptic Degrees of Freedom on Task Performance in Virtual Surgical Environments

Evaluation of robotically controlled advanced endoscopic instruments

Autonomous Surgical Robotics

Haptic Virtual Fixtures for Robot-Assisted Manipulation

Haptic Camera Manipulation: Extending the Camera In Hand Metaphor

Bibliography. Conclusion

Discrimination of Virtual Haptic Textures Rendered with Different Update Rates

Multichannel vibrotactile display for sensory substitution during teleoperation

Parallel Robot Projects at Ohio University

HAND-SHAPED INTERFACE FOR INTUITIVE HUMAN- ROBOT COMMUNICATION THROUGH HAPTIC MEDIA

DEVELOPING SENSORS FOR SURGERY SUPPORT ROBOTS Mona Kudo

Force Feedback Benefit Depends on Experience in Multiple Degree of Freedom Robotic Surgery Task Abstract

SMart wearable Robotic Teleoperated surgery

Experimental evaluation of magnified haptic feedback for robot-assisted needle insertion and palpation

Surgeon-Tool Force/Torque Signatures - Evaluation of Surgical Skills in Minimally Invasive Surgery

Force feedback interfaces & applications

Laser-Assisted Telerobotic Control for Enhancing Manipulation Capabilities of Persons with Disabilities

Haptic interaction. Ruth Aylett

DEVELOPMENT AND IMPLEMENTATION OF A TELEROBOTIC SYSTEM WITH VISUAL AND HAPTIC FEEDBACK: CURRENT PROGRESS

Wednesday, October 29, :00-04:00pm EB: 3546D. TELEOPERATION OF MOBILE MANIPULATORS By Yunyi Jia Advisor: Prof.

Haptics CS327A

Haptic presentation of 3D objects in virtual reality for the visually disabled

Title: A Comparison of Different Tactile Output Devices In An Aviation Application

Assembly Guide Robokits India

A Tactile Magnification Instrument for Minimally Invasive Surgery

PRODUCTS AND LAB SOLUTIONS

Integrating PhysX and OpenHaptics: Efficient Force Feedback Generation Using Physics Engine and Haptic Devices

Transforming Surgical Robotics. 34 th Annual J.P. Morgan Healthcare Conference January 14, 2016

Design of an Intermediate Layer to Enhance Operator Awareness and Safety in Telesurgical Systems

Introduction To Robotics (Kinematics, Dynamics, and Design)

1 May Telesurgery with haptic sensation: The future of surgery. Michael Stark The New European Surgical Academy (NESA)

Integrating Tactile and Force Feedback with Finite Element Models

Measurements of the Level of Surgical Expertise Using Flight Path Analysis from da Vinci Robotic Surgical System

Development of Flexible Pneumatic Cylinder with Backdrivability and Its Application

Product Information. Force/torque sensor FT

Actual trajectory. Desired trajectory. Actual trajectory z [m] 0.1. z [m] 0.1. Desired trajectory 0.

Transcription:

The Fourth IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics Roma, Italy. June 24-27, 2012 Effect of Force Feedback on Performance of Robotics-Assisted Suturing Ali Talasaz, Student Member, IEEE, Ana Luisa Trejos, Student Member, IEEE, and Rajni V. Patel, Fellow, IEEE. Abstract This paper is aimed at exploring the effect of force feedback on the performance of a knot-tightening task in robotics-assisted minimally invasive surgery (RAMIS). In this work, we evaluate performance during the knot-tightening task in three scenarios: without force feedback, with visual force feedback and with direct force reflection on the subject s hand. Different performance measures have been implemented: quality of the knot, amount and consistency of the tightening force applied on the suture, user s control of the instrument, tissue damage, and task completion time. Seven subjects participated in this study and were asked to tighten the second throw of surgical knots using a dual arm teleoperation system that is capable of force reflection in 7 Degrees of Freedom (DOFs), 6-DOF rigid body motion plus the gripper. The results show that visual force feedback allows superior performance in the quality of the suture knots with high consistency in the tightening force, while direct force feedback can significantly improve the user s control of the instrument. I. INTRODUCTION Robotics-assisted minimally invasive surgery (RAMIS) can provide many advantages to patients and to the health care system by increasing dexterity in manipulating laparoscopic instruments, significantly reducing hand tremor, and increasing precision during a minimally invasive surgery (MIS) task [1]. A well-known and widely used RAMIS system is the da Vinci system from Intuitive Surgical Inc. [2]. While this system offers superior dexterity and position control, it has the drawback of not providing haptics (the sense of touch) to the surgeon. Incorporating haptics in a surgical teleoperation system is a critical issue as it would provide the surgeon with the feel of interaction forces between the instrument and tissue during MIS. Many researchers have explored the effect of haptic feedback in RAMIS [3] [7]. Force feedback has been integrated with tactile feedback to improve the performance of tactile sensing for tumor localization in [8] [10]. Wagner, et al. [11] also explored the effects of force feedback on a blunt dissection task for different force feedback gains. One task Manuscript received January 31, 2012. This research was supported by the Ontario Centers of Excellence under grant IC50272; by the Natural Sciences and Engineering Research Council (NSERC) of Canada under Grant CRDPJ349675-06; and by infrastructure grants from the Canada Foundation for Innovation awarded to the London Health Sciences Centre (Canadian Surgical Technologies & Advanced Robotics (CSTAR)) and to The University of Western Ontario (UWO) (R.V. Patel). Financial support for Ms. Trejos has been provided by an NSERC Alexander Graham Bell Canada Graduate Scholarship. A. Talasaz and A.L. Trejos are with CSTAR, Lawson Health Research Institute, 339 Windermere Road, London, ON, Canada and with the Department of Electrical and Computer Engineering, UWO, London, ON, Canada (phone: 519-685-8500 ext. 36443, email: atalasaz@uwo.ca; analuisa.trejos@lhsc.on.ca). R.V. Patel is with CSTAR, the Department of Electrical and Computer Engineering and the Department of Surgery, UWO (phone: 519-685-8500 ext. 36618, email: rvpatel@uwo.ca). frequently used in minimally invasive surgery is suturing - a complex task requiring precise and dexterous movements, tissue puncturing, and thread stretching [12]. Although 3D visual clues offered by the da Vinci surgical system might partly compensate for the loss of force feedback for this application, relying on visual 3D images based on the deformation of tissue means that the surgeon may have already damaged the tissue. For tasks involving inelastic tissue or an obstructed view, considerable damage to the tissue might result without the surgeon s knowledge [13]. Some researchers looked into the differences between manual and robotics-assisted suturing: the surgical performance of a set of basic endoscopic movements and intracorporeal suturing were compared in [14] using manual and robotically assisted laparoscopic instruments. Kitagawa et al. [15] studied the difference between applied suture forces in three knot-tying exercises: hand ties, instrument ties using needle drivers, and robot ties using the da Vinci surgical system. Given the lack of a master-slave teleoperated sytem that is capable of force reflection in 7-DOFs, most research that has investigated the effects of haptic feedback on the performance of robotics-assisted suturing has explored the effect of sensory substitution. Sensory substitution has been proposed in the literature as an alternative to kinesthetic haptic feedback to the surgeon through other sensory cues such as vibro-tactile feedback [16], visual [17], [18], or audio feedback [3]. The effect of visual and auditory force feedback was studied in [19] to determine whether it could improve the performance of a knot-tightening task in RAMIS. An augmented reality system was proposed in [13] and [20] to present visual force feedback overlaid on top of the moving instrument tips. These studies also aimed to evaluate the performance of visual force feedback in robotics-assisted knot-tightening. However, the sensorized instruments in these studies were only capable of measuring instrument bending forces. Zhou, et al. in [21] conducted some experiments to investigate the effect of haptic feedback on the learning curve of a knot-tightening task. They showed that the novice surgeons who had some training with haptics were more consistent in their task performance than that of those who trained without haptics. They also had a shorter learning curve [22]. In this work we explore the effect of force feedback in both sensory substitution (visual presentation of the interaction forces) and haptic feedback (direct force reflection to the operator s hand) when performing a robotics-assisted knottightening task. One important feature of the setup presented herein is the capability of force reflection in all 7-DOFs. 978-1-4577-1198-5/12/$26.00 2012 IEEE 823

To the best of our knowledge, this is the first work that explores the effect of direct force feedback in 7 DOF for a suture-manipulation task in a robotics-assisted masterslave teleoperated system for MIS. In this work, sensory substitution was also provided by a bar indicator; the height of the bar-graph varies with the magnitude of the interaction forces between the instrument and its environment. This bar-graph was added to the camera vision overlooking the surgical field. Three scenarios of operation will be considered in this work; no force feedback, visual force feedback and direct reflection of the force feedback. The remainder of this paper is organized as follows: Section II describes our dual arm teleoperation setup. The experiments designed for this paper are described in Section III. The results are presented in Section IV and discussed in Section V, and finally, concluding remarks are given in Section VI. II. 7-DOF DUAL ARM TELEOPERATION SETUP Fig. 1 shows the dual-arm teleoperation setup used in these experiments, which consists of two Mitsubishi PA10-7C robots as the slave system controlled remotely over a dedicated network by two customized Quanser Haptic Wands TABLE I: 7-DOF Haptic Wand Characteristics Fig. 1: Haptics-enabled master-slave teleoperation setup doing suturing. [23] as the master interface. The Haptic Wand used in our test-bed is a 7-DOF haptic device that is capable of position and force reflection in three translational DOFs, three rotational DOFs, in addition to grasping motion. This device had originally 5-DOFs [23] and was modified at CSTAR to add the yaw and grasp motion [24]. The device consists of two identical 5-bar linkages that are driven by two identical motors creating 2-DOF motion in a horizontal plane. Each 5-bar linkage also uses one actuator to create the roll orientation of the plane. In the 5-DOF Haptic Wand, the upper and lower arm structures are connected by a handle through two Universal joints that restrict the mechanism to five DOFs instead of six. In the modified Haptic Wand, the handle has two split sections, each attached to a corresponding grasp lever (end-effector). Two handle drive motors are included in the mechanism, each independently controlling the corresponding handle and thereby providing two more DOFs for yaw and grasp motion. The Haptic Wand endeffector is defined as the point in the middle of the handle. The haptic device workspace and the maximum continuous force/torques along the translational and orientational directions at operating position are summarized in Table I. At the other end, two 7-DOF Mitsubishi PA10-7C robots 7-DOF Haptic Wand Workspace Translation (mm) 480W x 450H x 250D Rotation (deg) ±85 (roll) ±65 (pitch) ±160 (yaw) 90 (grasp) Maximum Continuous/Peak Force (N) 2.3/7.7 (X) 2.1/7.0 (Y) 3.0/9.0 (Z) Torque (N.mm) 230/750 (roll) 250/810 (pitch) 113/368 (yaw) 113/368 (grasp) Fig. 2: Sensorized instruments showing various tips and a closeup of the strain gauges applied to the cable shafts. 824

were employed as the slaves in the teleoperation test-bed. The four-layer control architecture for each 7-DOF Mitsubishi PA10-7C robot consists of the host control computer, a motion control card, a servo controller and the robotic arm. The host computer communicates with the PA10-7C arm at a sampling rate of 1 khz. The host computer controls the robot and sends data packets via the ARCNET protocol to the servo controller. Two da Vinci tools have been sensorized and mounted on the 7-DOF Mitsubishi PA10-7C robots as the end effectors. Originally, the motion of the da Vinci tool is controlled through 4 pairs of cable-shaftcable assemblies for each of the 4 DOFs (roll (θ) about the instrument axis, pitch (φ), yaw of gripper 1 (ψ 1 ) and yaw of gripper 2 (ψ 2 )). In order to measure the forces acting at the tip of these instruments, strain gauges were added to three of the pairs of cable shafts (see Fig. 2). The roll about the instrument axis would cause the wires from the strain gauges to tangle and so the cable assemblies controlling this extra DOF were eliminated from the design. Then, six EA-09-015DJ-120 strain gauges (Vishay Micro- Measurements) were mounted and rigidly glued to stainless steel shafts (1.1 mm in diameter) belonging to the six remaining cable assemblies. The gauges on each cable pair were connected in a Type II Half Bridge configuration using Quanser strain gauge amplifiers. A Quanser Q8 Hardwarein-the-Loop board is responsible for capturing the signals from the amplifiers. Using the embedded strain gauges, we can capture the interaction torque/torsions acting at the tip in 3DOFs. An ATI Gamma 6-DOF force/torque sensor [25] is also attached to the robot wrist to measure the translational forces exerted by the end effector on the tissue. In order to assess the amount of damage to the tissue that is being sutured, another ATI Gamma 6-DOF force sensor was placed underneath the tissue to measure the forces being applied on the tissue when performing the MIS task. All the readings from the force sensors used in the setup were passed through a low-pass filter with a cutoff frequency of 50 Hz in order to reduce measurement noise and to apply smooth forces to the operator. This setup is a research platform for studying the effect of haptics in MIS. For clinical use, the externally mounted ATI force sensor should be replaced with a force sensor capable of being inserted into the patient s body and capable of providing translational force measurement without tool-trocar friction interference. Each single arm slave, the Mitsubishi PA10-7C robot with the da Vinci tool attached, has 10-DOFs for position control of which 3-DOFs are redundant and are used for control purposes. The implementation of the controllers was done on two Windows-based systems, one for the master and the other for the slave. The communication between the two computers was done using the User Datagram Protocol (UDP). All control algorithms were implemented on the QuaRC Real- Time software which automatically generates real-time code directly from Simulink designed controllers targeting Windows [23]. All of the controllers for the master and slave manipulators were implemented at a sampling frequency of 1 khz. The communication between the master and the slave Fig. 3: Experimental testbed for tightening knots. PCs and transmission of the force and position data were also made at the same rate. III. EXPERIMENT DESIGN To explore the effect of haptics in a RAMIS suturing task, seven participants (4 male, 3 female) were asked to secure the second throw of surgical knots using the dual arm teleoperation system. The subject population included one medical professional with haptics experience, three subjects with some experience and three subjects with no experience. Each subject was asked to tighten five knots with three different scenarios of sensory feedback in a randomized order: No Force Feedback (NFF): the subject only used camera vision to tighten the knots and did not receive any feedback of the contact between the instrument and the environment. Visual Force Feedback (VFF): the subject was given visual feedback about the level of interaction between the instrument and the environment in addition to the camera vision. Direct Force Feedback (DFF): force feedback of the interaction between the instrument and the environment was directly reflected to the subject s hand in addition to the camera vision. The subjects could approach the knots from any angle they chose. They could feel the interaction forces in all 7-DOFs through the Haptic Wands in the DFF scenario, and see the magnitude of the forces visually in the VFF scenario. Three colors corresponding to three levels of applied instrument force magnitude were visually presented to the user. The main objective of showing force in different levels was to assure the user that the force being applied on the suture was sufficient to secure the knot. The first level colored in green was the range of force/tension that is below the threshold required for a tight knot, the second level shown in yellow is the range sufficient for a tight knot and the last level (red) denotes the danger zone for forces that may damage the tissue or break the suture if the applied force is within or beyond that range. Preliminary experiments showed that if the visualization of the force feedback was provided for each of the 7 DOFs individually, the subject would not be 825

able to correlate the given visual feedback of the force with the motion of the haptic wand. However, visually providing the total magnitude of the force for each instrument was sufficient to guide the subject on the pulling force applied by each instrument on the suture. The subjects were allowed to practice securing the knots using the dual arm teleoperation system with the different control modes until they felt comfortable using the setup. Since the setup was sensorized in all 7-DOFs, they were able to approach the two ends of the knots from any desired position and with any desired orientation, grab them tightly and pull on the suture. However, they were asked to apply forces symmetrically to ensure that the knot was secure and tight. In the experiments, each subject performed 15 knottightening trials under each of the three feedback scenarios; NFF, VFF, and DFF. The first trial for each scenario was considered as a practice trial and was not included in the analysis. Therefore, 12 trials for each participant were considered for performance analysis (96 trials in total). Since the tightening force varies based on the material of the suture, some preliminary experiments were performed to determine the level of the forces that would be sufficient to have a secure knot. The sutures used in the experiments were Ethicon 3-0 silk. Artificial skins made from silicone rubber were used as tissue. For the suture used, 4 N was sufficient to secure the knot. If a subject applied forces greater than 6 N, he/she could damage the tissue or break the suture (this was set as the threshold of the red zone). In order to prevent sliding between the grippers and the suture when the subject tightens the knot, the grippers were commanded to maintain a 2 N grasping force when the grippers were closed. The aims of this study were: what force feedback scenario is more effective when tightening knots and is more comfortable for the operator? Which scenario is better at improving the consistency of the applied forces in a RAMIS endoscopic knot-tightening task? Does haptic feedback or visual sensory substitution improve task completion time? Which scenario allows users to cause the least damage to the tissue? And does haptic feedback help the operator better control the laparoscopic instrument when performing the task? To answer these questions, the performance of the participants was assessed for each mode of control according to the following criteria: The quality of the suture. Three outcomes can result from the performance of a knot-tightening task, including a loose knot, a broken knot or a tight knot. The tightened sutures were collected for assessment later by a medical professional. The amount of force applied on the suture when the knot is tightened. The force measurements along with the position profile of the instrument end effector were recorded to be used later for measuring the pulling force applied by each participant in the different control modes. The consistency of pulling forces. The subjects were asked to apply forces on the sutures consistently in each feedback scenario. The consistency of the force applied by the participants for different scenarios was compared later using a one-way ANOVA. The collision factor. This is a useful parameter to determine how much pressure the tissue was under when the laparoscopic instrument made contact with the tissue when performing a knot-tightening task. The collisions can be measured by integrating the negative force applied to the tissue in a direction perpendicular to the tissue surface (this force should always be upward if the instrument never touches the tissue). This force was measured by the ATI force sensor that was placed underneath the tissue. The number of hits between the instrument and the tissue. This parameter along with the collision factor are the ones that evaluate the performance of the participants in controlling the instrument when performing a knot-tightening task. A contact force more than 1 N, acting perpendicularly to the tissue, was considered as a hit. Task completion time: the time required for completing the task by each participant was also recorded for each scenario. For further analysis, a one-way ANOVA with the Tukey test was conducted to determine if there is a significant difference for the assessment criteria among the trials performed with NFF, VFF, and DFF. A significance level of 0.05 was used for the one-way ANOVA. IV. RESULTS Figs. 4 8 show the results of the knot-tightening task for the three feedback scenarios: no force feedback (NFF), visual force feedback (VFF), and direct force feedback (DFF). The quality of the knots tightened by the subjects is presented in Fig. 4. The results show that without any force feedback, seven knots were loose and one suture was broken. Receiving force feedback on the subject s hand when performing the task, resulted in two loose knots, while with visual force feedback all suture knots were sufficiently tight without any loose or broken sutures. Fig. 5 shows the average tightening force applied on the suture and its standard deviation for the different scenarios. The force applied using visual force feedback was the most consistent one among those three scenarios. The statistical analysis shows a significant difference for the VFF scenario compared to the NFF and DFF scenarios (p < 0.001 for both VFF-NFF, and VFF-DFF). Fig. 6 presents the amount of pressure applied to the tissue due to collisions between the instrument and the tissue. The statistical analysis reveals that a significant difference occurred in the collisions with the DFF compared to NFF and VFF (p = 0.021 for DFF-NFF and p = 0.001 for DFF-VFF). Fig. 7 also shows that the DFF scenario had the minimum number of hits with the tissue (p = 0.002 for DFF-NFF and p < 0.001 for DFF-VFF). Fig. 8 shows the time required to accomplish the task for each scenario. The statistical analysis shows a significant difference in the VFF scenario compared to the NFF and DFF methods (p = 0.030 for VFF-NFF and p = 0.049 for 826

Fig. 4: Quality of the knots. Fig. 6: Collision factor in the three scenarios. Fig. 5: Pulling force applied on the sutures in the three scenarios. Fig. 7: Number of hits on the tissue in the three scenarios. VFF-DFF). No significant difference in task completion time was seen for the DFF scenario compared to the NFF scenario (p = 0.978). V. DISCUSSION As mentioned earlier, the performance evaluation criteria included the quality of the knot, the amount and variability of the tightening force applied on the suture, the user controllability over the instrument, tissue damage, and task completion time. The results confirm that if the amount of tightening force is given to the subject in a visual form, he/she can ensure that sufficient force is applied in order to secure the knot. Having haptic feedback reflected to the subject s hand can also decrease the number of loose or broken sutures compared to when the subject has no feedback from the interaction of the instrument with the suture. However, as the results showed, since the subjects did not know how much force was sufficient to tighten the knots, each applied a different amount of force that ended up with two loose knots. Fig. 5 also confirms that the consistency of applied forces during the robotics-assisted knot-tightening task with sensory substitution was significantly greater than that achieved with no force feedback or direct force feedback. Although both DFF and NFF showed significant variation in the applied tightening force by different subjects, the main advantage of the DFF scenario over the NFF scenario is that it allows forces to be felt directly on the hand, which decreases the number of loose or broken sutures. Despite the advantages of sensory substitution (VFF) over haptics (DFF) from the point of view of suture knot quality and tightening force consistency, Figs. 6 and 7 demonstrate worse performance in the visual force feedback scenario and superior performance in the direct force feedback scenario with regards to the user s control of the instrument. The Fig. 8: Task completion time in the three scenarios. results show that both the NFF and VFF were unable to help the participants control the instrument effectively and caused the instrument to hit the tissue several times when performing the knot-tightening task. The statistical analysis shows that collision factor using DFF was significantly lower than that using NFF and VFF. These phenomena can be justified by the 7-DOF force reflection capability provided through the master-slave teleoperation system to the participant s hand which provides more intuitive control when performing a RAMIS task. Visually representing the magnitude of the interaction force between the instrument and the environment cannot give subjects an intuitive feeling about the direction in which this force is being applied. On the other hand, providing force visualization for each DOF is also distracting to the user and difficult for him/her to control the forces applied to the tissue. The other source of error that caused poor performance in the VFF scenario was that when the operator attempted to grab one end of the suture using one instrument, he/she lost focus of what the other instrument was doing, which caused the instrument to collide with the tissue. With respect to task completion time, the results show that not only no improvement was attained using either haptics or sensory substitution, but also the latter caused the task 827

to take longer. This is reasonable because of the additional time needed by the participants to ensure that the tightening force was in the yellow zone. After finishing the experiments, each participant was asked to choose the scenario that he/she was more comfortable with. Almost all participants agreed that both visual and haptic feedback are needed. However, the novice participants preferred to have visual feedback from the interaction forces since they could get a better measure of how much force they were applying on the suture. Those who had some experience with haptics preferred direct force reflection on their hands and found it difficult to control the instrument without haptic feedback. All the aforementioned results show that neither sensory substitution alone nor haptics alone can significantly improve the performance of the knot-tightening task in RAMIS. Sensory substitution has superior performance in the quality of suture knots with high consistency in the tightening force because the user knows how much force he/she needs to apply to secure the knots. Haptics can significantly improve the performance of suture-manipulation because of the intuitive feeling provided to the user through the haptic interface. This study shows that visual presentation of the magnitude of the interaction force needs to be incorporated with a hapticsenabled teleoperated system to let the user know how much force is required to secure the knot. This would be the focus of future work in the context of a clinical application. VI. CONCLUSION The problem of incorporating haptic feedback in a robotics-assisted suture-manipulation task was explored in this paper. The experiments were performed in three scenarios: without force feedback, with visually presented force feedback and with direct force reflection to the user. The main focus on this work was to explore which way of presenting force feedback could be more effectively used, and how each modality could help the user to increase the performance. Statistical analysis showed that the consistency of applied forces during a robotics-assisted knot-tightening task with sensory substitution was significantly greater than that achieved with no force feedback or direct force feedback. However, using direct force feedback, the user was able to control the instrument more effectively. VII. ACKNOWLEDGEMENT The authors would like to thank Dr. Jacob Apkarian, and Paul Karam of Quanser Consulting Inc. for their help with the QuaRC software and the Haptic Wand. REFERENCES [1] S. Hirche, M. Ferre, J. Barrio, and C. Melchiorri, Advances in Telerobotics. Springer Berlin / Heidelberg, 2007. [2] http://www.intuitivesurgical.com. [3] A. M. Okamura, Haptic feedback in robot-assisted minimally invasive surgery, Current opinion in urology, vol. 19, no. 1, pp. 102 107, 2009. [4] E. P. Westebring, V. D. Putten, R. H. M. Goossens, J. J. Jakimowicz, and J. Dankelman, Haptics in minimally invasive surgery - a review, Minimally Invasive Therapy, vol. 17, no. 1, pp. 3 16, 2008. [5] M. V. Ottermo, M. Vstedal, T. Lang, Stavdahl, Y. Yavuz, T. Johansen, and R. Marvik, The role of tactile feedback in laparoscopic surgery, Surgical Laparoscopy Endoscopy and Percutaneous Techniques, vol. 16, no. 6, pp. 390 400, 2006. [6] G. Tholey, J. P. Desai, and A. E. Castellanos, Force feedback plays a significant role in minimally invasive surgery: results and analysis, Annals of Surgery, vol. 241, no. 1, pp. 102 109, 2005. [7] M. Mahvash, J. Gwilliam, R. Agarwal, B. Vagvolgi, L. M. Su, D. D. Yuh, and A. M. Okamura, Force-feedback surgical teleoperator: Controller design and palpation experiments, in 16th Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems, 2008, pp. 465 471. [8] A. Talasaz, R. V. Patel, and M. D. Naish, Haptics-enabled teleoperation for robot-assisted tumor localization, in IEEE International Conference on Robotics and Automation, 2010, pp. 5340 5345. [9] R. L. Feller, C. K. L. Lau, C. R. Wagner, D. P. Perrin, and R. D. Howe, The effect of force feedback on remote palpation, in IEEE International Conference on Robotics and Automation, 2004, pp. 782 788. [10] C. R. Wagner, D. P. Perrin, R. L. Feller, R. D. Howe, O. Clatz, H. Delingette, and N. Ayache, Integrating tactile and force feedback with finite element models, in IEEE International Conference on Robotics and Automation, 2005, pp. 1 6. [11] C. R. Wagner, N. Stylopoulos, and R. D. Howe, The role of force feedback in surgery: Analysis of blunt dissection, in 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2002, pp. 68 74. [12] J. P. Ruurda, I. A. M. J. Broeders, B. Pulles, F. M. Kappelhof, and C. van der Werken, Manual robot assisted endoscopic suturing: timeaction analysis in an experimental model, Surgical endoscopy, vol. 18, no. 8, pp. 1249 1252, 2004. [13] T. Akinbiyi et. al., Dynamic augmented reality for sensory substitution in robot-assisted surgical systems, in IEEE Engineering in Medicine and Biology Society, 2006, pp. 567 570. [14] A. Garcia-Ruiz, M. Gagner, J. H. Miller, C. P. Steiner, and J. F. Hahn, Manual vs robotically assisted laparoscopic surgery in the performance of basic manipulation and suturing tasks, Archives of Surgery, vol. 133, no. 9, pp. 957 961, 1998. [15] M. Kitagawa, A. Okamura, B. Bethea, V. Gott, and W. Baumgartner, Analysis of suture manipulation forces for teleoperation with force feedback, in International Conference on Medical Image Computing and Computer Assisted Intervention, 2002, p. 155162. [16] S. Peddamatham, W. Peine, and H. Z. Tan, Assessment of vibrotactile feedback in a needle-insertion task using a surgical robot, in Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2008, pp. 93 99. [17] J. Dargahi, S. Najarian, and R. Ramezanifard, Graphical display of tactile sensing data with application in minimally invasive surgery, Canadian Journal of Electrical and Computer Engineering, vol. 32, pp. 151 155, 2007. [18] A. P. Miller, Z. Hammoud, J. S. Son, and W. J. Peine, Tactile imaging system for localizing lung nodules during video assisted thoracoscopic surgery, in IEEE International Conference on Robotics and Automation, 2007, pp. 2996 3001. [19] M. Kitagawa, D. Dokko, A. M. Okamura, B. T. Bethea, and D. D. Yuh, Effect of sensory substitution on suture manipulation forces for surgical teleoperation, Studies in health technology and informatics, vol. 98, pp. 157 163, 2004. [20] C. E. Reiley, T. Akinbiyi, D. Burschka, D. C. Chang, A. M. Okamura, and D. D. Yuh, Effects of visual force feedback on robot-assisted surgical task performance, J. Thorac Cardiovasc Surg., vol. 135, no. 1, pp. 196 202, 2008. [21] M. Zhou, S. Tse, A. Derevianko, D. B. Jones, S. D. Schwaitzberg, and C. G. L. Cao, The effect of haptic feedback on laparoscopic suturing and knot-tying: A learning curve study, in Proceeding of the Human Factors and Ergonomics Society, 2008, pp. 880 884. [22] N. Oshima, J. Solis, H. Ishii, N. Matsuoka, K. Hatake, and A. Takanishi, Integration of an evaluation function into the suture/ligature training system WKS-2R, in IEEE International Conference on Robotics and Automation, 2008, p. 10941099. [23] Quanser. [Online]. Available: http://www.quanser.com [24] H. Bassan, A. Talasaz, and R. V. Patel, Design and characterization of a 7-dof haptic interface for minimally invasive surgery test-bed, in IEEE/RSJ Conference on Intelligent Robots and Systems, 2009. [25] ATI. [Online]. Available: http://www.ati-ia.com 828