Robotic Applications in BioMedicine: From Molecular Imaging to Heart Surgery Medical Robotics Laboratory, PGH 315 Department of Computer Science http://mrl.cs.uh.edu/home.html Nikolaos V. Tsekos Junmo An, Nicholas von Sternberg, Giulia Toti (*), Habib Zaid, Mahmut Unan PhD: Nikhil Navkar, Ahmet Sonmez, Erol Yeniaras, Fabien Benoteau, MS: Johann Lamaury, Yousef Hedayati, Daniele Faris BSc (BME): Habib Zaid
What do we do? Develop computational methods and hardware, and integrate them for performing image-guided and robot-assisted interventions and surgeries From Concepts to Actual Working Prototypes! MRI robot control MRI Surgical Robots Haptics Endoscopic Sensors 1 mm MRI Robot Tracking
MIROS: A Cyber-Physical System Built from the Ground Up in Our Department! We bridge and link the physical and the cyber world
MIROS Multimodal Image-guided RObot-assisted Surgeries MR Scanner ScannerState(t) MRData(t) AoP(t) GPU Visualization Module Visual interface MRData(t) Sensor-Control Module Motion-Filter Module GPGPU FPGA-2 Data Processing Module {TP J (r, t)} {GC J (r, t)} AoP(t) AoP(t): {Encoders ROBOT (t)} MARK/TUBE Collision Detection Collision Response OP IN (t) OP GUIDE (t) Force- Feedback Module {Encoders FFI (t)} {DoF FFI (t)} ROBOT FPGA-1 Robot Control Module RobotState(t) Force-feedback Interface {DoF ROBOT (t)}
Computational Thread: Processing Dr. N. Navkar Real-time MRI feed Slices #1 #2 #3 Three oblique-to-each-other slices: 49 ms/slice (20 Hz) Projection Bands Slice #1 Slices #2 & #3 Control Curves Diastole Systole Diastole Systole Force Feedback Interface Dynamic Virtual Fixtures Visual Interface
Cardiac Surgeries (Fast-fast-fast!!!) Virtual Fixtures Generated On-the-Fly from the Continuous Feed of the Real-time MRI Guidance Virtual Fixtures Forbidden Region Virtual Fixtures Maneuver on the curve Maneuver inside the corridor NO NEED for Models, Assumptions and unnecessary computational layers!
Translation rods dspace Unit Knob A Yaw motor (b) capstan wire Tip Angulation Motor UNIT A (a) UNIT B Knob B (c) Pitch Motor Translation Motor 1 wires (c) Motor 2 Roll Motor Knob A Tip Angulation Motor Knob B (d) Linear bearings (e) (f) wire Capstan
Enhanced Reality and Haptics Dr. N. Navkar & Dr. E. Yeniaras The forces exerted on the operator direct the actual robot (gray) to match its proxy (green) The proxy is calculated to stay always inside the forbidden region corridor Force-Feedback Interface (In-house developed 5 DoF) Visualization Interface (High Definition LCD)
In-house Developed MR-compatible Robot N. Sternberg (PhD); H. Zaid, BSBME; J. An (PhD) Upper stage 1 DOF Middle stage 2 DOF
A B
3mm Tracking A Robot With MRI Junmo An (PhD) L C R 3mm Container with MR signal source Tuning Capacitor Photo-resistor Robot Controls which markers are MR visible! -Faster - Less Computing! Light protective sleeve Lens cap Optical fiber Cable (15m)
The MRI+Optical Needle Probe Dr. Ahmet Sonmez MR Spectroscopy Sensitive Area of the RF Coil Light Induced Fluorescence SI 1500 (a.u.) 1000 500 0 1.8 mm comp-3 comp-2 comp-1 10000 300 um co-axial cable 5000 to RF coil 0 SI (a.u.) comp-3 comp-2 comp-1 RF Coil Needle Wall Optical Sensor Laser emission (a) Light detection 1 Z (cm) 0.5 0-0.5-1.5-1 3 x 300 um 2 optical fibers 3 to optical sensor (b) 4 1 Chemical shift (ppm) 0 1 Z (cm) 0.5 0-0.5-1 -1.5 500 550 RF Coil Wavelength (nm) The first ever endoscopic work on concurrent MRI and optical sensing! 600
Thank you! E-mail: ntsekos@cs.uh.edu Visit us at PGH 315