ience e Schoo ol of Com mpute er Sc Visual Computing in Medicine The Bangor Perspective School of Computer Science Bangor University
Pryn hwn da Croeso y RIVIC am Prifysgol Abertawe Siarad Cymraeg? Schoo ol of Computer Sciencee
Visualization and Medical Graphics Group Established Sept 2003 Five academics Nigel John Ik Soo Lim Jonathan Roberts Bill Teahan Rafal Mantiuk Five RAs Chris Hughes Serban Pop Rick Walker Meirion Hughes Llyr ap Cynedd Five PhD Students Highlights of first 5 years: Satava Award 2006 50+ Publications 1.8M Research Grants Hosted International Conferences
Growing trend to use Medical Virtual Environments and Augmented Reality Educational Tools Diagnostic Aid Patient specific data 3D visualizaton of medical image modalities Virtual Endoscopy Planning Aid Surgical planning Interventional radiology Radiotherapy treatment Rehearsal Aid Guiding Aid Training Aid
Multidisciplinary i li Team is Needed d Surgeons and clinicians the end users Clinical engineers Data measurements Build hardware Computer Scientists Write the software Integrate and maybe build the hardware system Psychologists Task Analysis Validation Studies School of Computer Science
Task Analysis Design of a medical simulator must begin with a detailed d task analysis Indentify all steps of the procedure What happens when things go wrong? Identify critical steps for safe completion of task; these can be used for metrics
Documented Task Analysis 7
Documented Task Analysis 8
Components of a Medical Simulator Input DICOM Data; 3D Geometry, e.g. surgical tools, needles Computer Segmentation 3D Reconstruction Collision detection Soft tissue deformation Force models Respiration Image generation Interaction Stereoscopy Haptics Physical Model
Interventional ti Radiology (IR) IR procedures use medical imaging to guide instruments to treat t specific pathology minimally invasively Vascular IR techniques involve blood vessels (arteries, veins) Visceral IR is concerned with organ systems (liver, bile ducts, urinary tract, gut) We have been building several simulators for (IR) that incorporate haptics...
Simulated Ultrasound Guidance (HTD funded) d)
Ultrasound-like like images are generated by: First extracting MPR image from the voxel data, or segmented mesh data, based on the position and orientation of the transducer Acoustic shadowing effects and noise are added using multitexturing. Ultrasound-like lik Images on the GPU 12
Simulation of X-ray Attenuation ti on the GPU With volume data, X-ray attenuation using the Beer Lambert Law can be computed on the GPU using Ray Tracing By considering the thickness penetrated by a ray into the object Usually have data from only one time step See Poster at EG2010! Works in conjunction with simulation of movement due to respiration Based on a surface model
ImaGINe-S Prototype t Validated for Biopsy and Nephrostomy procedures Second Place in 2009 Eurographics Medical Prize 14
Training i in Vascular IR Procedures 15
Replicating Real World Feel Needle force / motion tracking during arterial puncture 80 60 40 20 0-20 -40-60 20 22 24 26 28 30 32 34 Time (s) J Zhai, T How, Liverpool Clinical Engineering
Palpation Simulator Palpation can be used to identify the site of a needle puncture Requires both tactile and force feedback Prototype Haptics device for palpation NovINT Falcon provides force feedback Experimenting with tactile feedback interface - piezoelectric materials, speakers, pin arrays, hydraulics
Palpation Simulator Also using chromakey technology so that real hands can interact with CGI objects Novel for medical VE Collaboration with Italian Institute of Technology School of Computer Science
Fast Blood Flow Model with Particle Systems Flow in a horizontal channel with a bifurcation: As the vessel narrows the flow velocity increases:
Charged Particles for Soft Tissue Deformation with Haptics
Cost Effective Basic Ultrasound Training i Cost Effective Training for Ultrasonography Using Wii Remote with MotionPlus Aim is to deploy in developing countries Remote tutoring with Skype Touch cues Vibrate Wii Remote Shadow of transducer on patient s body 21
Bangor Augmented Reality Education for Teaching Anatomy (BARETA) RP Model is tracked using an Ascension minibird Software supports slab rendering and virtual resection We are part of a Marie Curie Initial Training Networks (ITN) proposalp Surgical Training with Augmented Reality Techniques, START School of Computer Science
Coloured Brain (EPSRC) Feasibility study with Bradford University and Leeds General Infirmary Create BRDF for a photorealistic surface model of a living human brain School of Computer Science
Web 3D Consortium (TATRC funding) Co-chair Medical Working Group Developed medx3d: Standards Enabled Desktop Medical 3D DICOM support Volume rendering in X3D Import/export library Ontology support Actively engaged with DICOM committees X3D could become DICOM s 3D standard 24
Low Fidelity Simulators These training tools were built for Web-based delivery Used keyboard and mouse No haptics Validation study proved that they aid the training process Moorthy, K., Mansoori, M., Bello, F., et al. (2004). Evaluation of the benefit of VR simulation in a multi-media web-based educational tool. In Medicine meets virtual reality 12. pp. 247 252 Match the fidelity to the cues and actions required for task training
Mentoring in Second Life Create a secure teaching environment for remote mentoring of medical students Collaboration with Luton Hospital Uses Second Life and secure web server running SQL Booking and search facilities School of Computer Science
Research Challenges Strive for realism with real time interaction How high a fidelity simulation can be achieved? What fidelity of simulation is actually needed? Computer graphics research Photo realistic rendering, using the GPU, soft tissue deformation, blood flow, modelling tools, etc. Haptics research Holding virtual objects with your fingers, rather than via a tool Realistic force and tactile models School of Computer Science
Contact t Details Visualization and Medical Graphics Research Group, School of Computer Science, Bangor University http://www.vmg.cs.bangor.ac.uk/ Email: n.w.john@bangor.ac.uk