Cutaneous Feedback of Fingertip Deformation and Vibration for Palpation in Robotic Surgery Claudio Pacchierotti Domenico Prattichizzo Katherine J. Kuchenbecker
Motivation Despite its expected clinical benefits, current teleoperated surgical robots do not provide the surgeon with haptic feedback, largely because grounded forces can destabilize the system s closed-loop controller. 3
Objectives In this work we present a feedback approach that enables the surgeon to feel (1) contact forces and (2) vibrations, while guaranteeing the stability of the control loop. Delivering cutaneous cues to the surgeon s skin conveys rich information and does not affect the stability of the teleoperation system. 4
Haptic feedback system The haptic feedback system is composed of (1) a BioTac tactile sensor, in charge of registering contact forces and vibrations at the operating table, and (2) a cutaneous feedback device, in charge of applying those forces and vibrations to the surgeon. 6
Cutaneous feedback device Link to video 7
Mapping tactile sensations from the BioTac to the cutaneous device Contact forces and vibrations sensed by the BioTac are directly mapped to input commands for the cutaneous device s motors using a model-free datadriven algorithm based on look-up tables. 8
Mapping tactile sensations from the BioTac to the cutaneous device BioTac R 2211 hydro-acoustic pressure sensor (DC and AC pressure) 19 electrodes? 3 servos vibrotactile motor Cutaneous device R 44 9
Data collection (servo motors) mobile platform Platform configuration (input for the servo motors) Tactile sensation on the BioTac (19 electrodes + DC pressure) We can define µ d ( ), that maps a given BioTac sensation to the corresponding platform configuration. 10
Data collection (vibrotactile motor) mobile platform 2-seconds-long sweep sine Platform configuration (input for the servo motors) Transfer function between the vibrations sensed by the BioTac sensor (AC pressure) and the ones played by the vibrotactile motor We can define µ v ( ), that maps a given platform configuration to the corresponding transfer function coefficients. 12
Mapping (simplified) Search for the closest sensations experienced by the BioTac during the data collection Map those sensations to the corresponding platform configuration Map that platform configuration to the corresponding filter coefficients Filter AC pressure signal C. Pacchierotti, D. Prattichizzo, K. J. Kuchenbecker. Cutaneous feedback of fingertip deformation and vibration for robotic surgery. Submitted to IEEE Transactions on Biomedical Engineering, 2015. C. Pacchierotti, D. Prattichizzo, K. J. Kuchenbecker. Displaying Sensed Tactile Cues with a Fingertip Haptic Device. Submitted to IEEE Transactions on Haptics, 2015. C. Pacchierotti, D. Prattichizzo, K. J. Kuchenbecker. A data-driven approach to remote tactile interaction: from a BioTac sensor to any fingetip cutaneous device. In Proc. EuroHaptics, Pages 418-424, Versailles, France, 2014. 13
Experimental evaluation We evaluated the proposed cutaneous-only approach by carrying out a palpation task using a da Vinci Surgical System. The remote environment is composed of a tissue phantom heart model. A plastic stick is embedded into the tissue model at 1.5 mm from the surface, and it is not visible from the outside. The plastic stick simulates the presence of a calcified artery. soft tissue phantom rigid hidden stick 15
Experimental evaluation Task: explore the tissue model to try to detect the orientation of a hidden plastic stick. Each participant made 4 repetitions for each feedback condition proposed: (1) cutaneous feedback provided by the servo motors (condition S), (2) cutaneous feedback provided by the servo and vibrotactile motors (condition SV), (3) no force feedback (condition N). 16
Experimental evaluation We evaluated the proposed cutaneous-only approach by carrying out a palpation task using a da Vinci Surgical System. The remote environment is composed of a tissue phantom heart model. A plastic stick is embedded into the tissue model at 1.5 mm from the surface, and it is not visible from the outside. The plastic stick simulates the presence of a calcified artery. soft tissue phantom rigid hidden stick 17
Experimental evaluation Task: explore the tissue model to try to detect the orientation of a hidden plastic stick. Each participant made 4 repetitions for each feedback condition proposed: (1) cutaneous feedback provided by the servo motors (condition S), (2) cutaneous feedback provided by the servo and vibrotactile motors (condition SV), (3) no force feedback (condition N). 18
Palpation experiment Link to video C. Pacchierotti, D. Prattichizzo, K. J. Kuchenbecker. Cutaneous feedback of fingertip deformation and vibration for robotic surgery. Submitted to IEEE Transactions on Biomedical Engineering, 2015. 19
Results No force feedback Cutaneous feedback (contact forces only) Cutaneous feedback (contact forces + vibrations) Eleven subjects chose condition SV as the most effective feedback condition, six subjects chose condition S, and only one chose condition N. 20
Results: dragging strategy only All the seven subjects who used the dragging strategy found condition SV to be the most effective at letting them detect the orientation of the plastic stick. 24
Conclusions Providing cutaneous feedback significantly improved the task performance in all the considered metrics; subjects who used a dragging strategy achieved even better results with cutaneous feedback of fingertip vibrations; subjects highly preferred conditions providing cutaneous feedback over the one without any haptic feedback. 25
Future work In the next future we plan to carry out a new human subject study enrolling both novices and experienced da Vinci surgeons; extend this cutaneous-only approach to pinch grasping; modify the cutaneous feedback device to use continuous rotation servos and add a force sensor on the platform; investigate the practical translational aspects of the proposed cutaneous system. 26
Thank you! 27
Questions? 28