Supported by grant number R18HS020460 from the Agency for Healthcare Research and Quality (AHRQ). The context is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality Development of an Infusion Pump Prototype and Evaluation Environment to Improve the Safety of Medical Devices through Simulation AHRQ HS20460 A. Ravitz, MS, PE J. Benson, MS, PE G. Tran, MS M. Ruffolo N. Shrestha, MS J. Pham, MD, PhD P. Doyle, PhD K. Carson, Sc.M. M. Ijagbemi, MPH R. Wyskiel, RN 2
Project Evolution
Workshop Overview (AIM 1) 4
Framework Takes too much time to read pump status; Rate is displayed rather than dose. Banner display limits ability to determine which channel is running drug; leads to workaround. Delay in monitoring infusion status; Interpret rate as dose. Matter of convenience; Over or under infusion; Affects rate. Quickly identify medication being infused; Display both rate and dose information. 5
Themes from Workshop Findings Systems Integration Programming Navigation Information Prioritization & Presentation Control Standardization Context Awareness Analysis and synthesis of themes revealed prototype features 6
Prototype Development (AIM 2) High-level requirements captured in Concept of Operations to guide the design & implementation of prototype. Infusion display and control requirements. Addressed study task objectives & human factors design objectives 7
Prototype Development (AIM 2) Iterative design and development Test and training scenarios to define set of infusion tasks using range of medications. Feedback from pilot study helped improve user interface, user testing protocol, moderator scripts, and moderator interface. 8
MIP Architecture POE EMR PHARMACY DERS Wireless Access Point MySQL Database Tablet (Prototype Interface) Laptop Server Order Entry (Generate QR Codes) Smartphone (QR/Barcode Scanning) 9
MIP Prototype Setup of MIP Prototype and Scanner 10
MIP User Interface Setup of New Infusion - Scan Channel 11
MIP User Interface Setup of New Infusion - RN Verification (2 nd Check) 12
MIP User Interface Setup of New Infusion Verification Page 13
MIP User interface Infusion Pump Monitor Status 14
MIP User interface Infusion Pump Status - Heparin 15
MIP User Interface Change Dose 16
Evaluation (AIM 3) 5 Days of Testing 41 ICU Nurses Methods One scenario involving Lactated Ringers, Norepinephrine, Heparin, Zosyn & Vancomycin Auto-programming vs. Manual programming 17
Selected Results * Level of significance = p < 0.05 18
Selected Results * Level of significance = p < 0.05 19
NASA-TLX Scores by Method NASA Task Load Index Manual Programming (n=40) Automated Programming (n=41) P value* Composite (weighted), mean (SD) 31.1 (15.4) 28.3 (15.7) 0.08 Mental Demand 39.4 (21.2) 33.8 (18.6) 0.02 Physical Demand 21.8 (18.1) 23.8 (20.1) 0.47 Temporal Demand 31.4 (19.8) 31.0 (21.8) 0.76 Performance 28.9 (19.7) 20.7 (15.9) <0.01 Effort 35.3 (18.3) 32.6 (22.7) 0.24 Frustration 25.5 (19.1) 30.9 (27.5) 0.17 *P value from paired t-test for n=40 with both types of entry Composite: weighted summation of mental, physical, temporal, performance, effort, and frustration. Performance is reverse coded where 0 = Perfect Performance; and 100 = Failure * Level of significance = p < 0.05 20
Study Findings Tablet-based interface design was easy to navigate, user-friendly and intuitive to use (Debrief) Poor sensitivity of touch screen in both auto and manual mode (Post-Test Survey). Perceived accuracy of entering physician order was higher in auto-programming mode (Post-Task Survey). Risk of not catching potential errors originating from order in manual and auto modes (Debrief). Auto-programming can lead to a false sense of security when error exists in actual order (Debrief). 21
Conclusions Task completion was similar in both modes. Manual had greater number of steps Faster completion times in manual programming likely related to familiarity with manual programming. Did auto-programming reduce errors? High risk errors potentially greater in manual Auto-programming is potentially safer. Less mentally demanding (NASA-TLX) Higher performance (NASA-TLX/Debrief) 22
Final Products Two Reports AHRQ Final Report Comprehensive Report (for public release) MIP Prototype (available through no cost or fee licensing agreements) MIP Design Documentation Graphical user interface recommendations based on user feedback and HCI guidelines hlp://www.aami.org/htsi/infusion/materials/medicapon%20infusion_pump_workshop_johnshopkinst_092812.pdf hlp://www.hopkinsmedicine.org/armstrong_insptute/improvement_projects/infusion_pump.html 23
24
Back up slides 25
User Interface Design Process AIM 1 AIM 2 AIM 3 26
Outcome Measures Task complepon & Pme NASA- TLX ParPcipant perceppons Structured Post- Task survey QualitaPve Feedback 27
Selected Problem Statements Addressed by Design of the Prototype Interface 28
Problem Statements 1 of 2 29
Problem Statements 2 of 2 30
Best Practices Items above threshold 31
Best Practices Items below threshold 32