Monday, June 28, Joseph Cafazzo, PhD PEng

Transcription

1 Joseph Cafazzo, PhD PEng

2 Modern Times, United Artists 1936

3 Science Finds, Industry Applies, Man Conforms Slogan from the 1933 Chicago World s Fair

4 "People Propose, Science Studies, Technology Conforms" Don Norman s person-centered motto for the 21st century. "Things that make us smart." Addison-Wesley, 1993.

5

6

7

8 Poor Design Images - c prompt ballot friendly fire

9 We have a serious training problem that needs to be corrected We need to know how our equipment works: when the battery is changed, it defaults to its own location. We ve got to make sure our people understand this Senior Air Force Official

10

11 Recount, HBO 2008

12

13

14 Radiation Oncology and Patient Safety You are not alone

15 How do we prevent error? Hierarchy of Effectiveness 1. Forcing functions & constraints 2. Automation/computerization 3. Simplification/standardization 4. Reminders, checklists, double checks 5. Rules & policies 6. Training & education tech focus people focus

16 Human Factors in Healthcare IOM 1999: 44, 000 to 98, 000 US deaths annually due to medical error Baker, Norton 2004: 9, 000 to 23, 000 deaths annually due to preventable adverse events

17 User Centered Design Analyze user requirements Evaluate Design & Prototype

18 Design & Prototype

19 Evaluate Expert/Heuristic Analysis Visibility Consistency Efficiency Flexibility Autonomy

20 Evaluation Criteria Efficiency & Speed Task completion time Task accuracy & error frequency Number of requests for help Number of attempts to correct errors Impact & Severity of error (patient safety) Workload/Difficulty scale (1-7) User feedback & preferences

21 Evaluate Usability Testing Representative users Realistic scenarios Think-aloud protocol Record video & audio Qualitative & quantitative

22

23 Evaluation of a Functional Prototype

24 Improving Patient Safety during Radiation Therapy through Human Factors Alvita Chan Principle Investigators: Dr. Mohammad Islam, Dr. Joe Cafazzo Co-Investigators: Dr. Tony Easty, Dr. David Jaffray

25 Systematic Literature Review A systematic review on existing literature was conducted to understand sources of errors 7 distinctive studies were found 40-50% errors occur during treatment delivery Most frequent errors: Incorrect patient position (e.g. wrong site, wrong SSD, wrong couch height) Incorrect treatment accessories (i.e. Shielding blocks, bolus) Yeung TK et al. Radiotherapy and Oncology. 2005, 74: Marks LB et al. International Journal of Radiation Oncology, Biology and Physics. 2007, 69:

26 40.4% Yeung TK et al. Radiotherapy and Oncology. 2005, 74:

27 Work Environment Analysis Identified issues and developed recommendations regarding treatment/ control room setup Ambient Noise/Interruptions Furniture/computer features and placement Storage Lighting Temperature and humidity

28

29 Workflow Analysis Mapped out entire workflow of the treatment delivery process using the Unified Modeling Language (UML) 2.0 Activity Diagrams * From 24 observations, 6 for each treatment team

30

31 Heuristic Evaluation Systematic inspection of current user interfaces according to a list of guidelines/heuristics E.g. Visibility of system state, minimize memory load, informative feedback, flexibility and efficiency, prevent errors Identified 75 usability problems Number of Problems Identified Usability Problems by Severity Ratings High Medium Low Positive Severity Ratings Zhang et al. Journal of Biomedical Informatics. 2003, 36: 23-30

32 Workflow Analysis Identified areas that can be improved: Heavy reliance on policy Checking process during patient setup Least effective Hierarchy of Effectiveness 1. Forcing functions and constraints 2. Automation / Computerization 3. Simplification / Standardization 4. Checks, double checks, checklists 5. Policy and training Hierarchy of effectiveness in preventing errors Institute for Safe Medication Practices. ISMP Medication Safety Alert. 1999, 4 (11)

33 How can the process be improved? Focused on checking process during patient setup Redesigned components of the user interface: To lessen the reliance on policy and procedure To improve efficiency of the checking process To display information based on the users needs Design process: Expert input + 2 focus groups 12

34

35

36 DRR is displayed visibly to improve checking compliance Redesigned Interface Patient picture is displayed visibly for easy identification

37 Redesigned Interface Notes that required certain actions to be performed are highlighted New messages and outstanding items are highlighted Statuses of approval dates and images are clearly displayed DOB given by patient is checked by system Both SSD and depths are displayed Forcing function/checklist is used to improve checking compliance

38 Redesign of Existing System Current interface: 14 mouse clicks Redesigned interface: 4 mouse clicks Improved efficiency 17

39 Experimental Evaluation - Usability Testing 16 radiation therapy students from PMH were recruited to participate An actor played the role of a second therapist Performed patient setup tasks using both the current and redesigned interfaces 18

40 Experimental Design 2 interfaces x 4 scenarios 3 out of the 4 scenarios were designed with a high potential for common errors to occur: Overlooking an important note Shifting couch incorrectly Overlooking a change in approval dates 19

41 Rate of Errors P<0.04 P<

42 Task Time P<

43 User Satisfaction Post-test Questionnaire: 22

44 User Satisfaction Better at showing the information needed Attributes Current Redesigned Significance Better at drawing attention to important items Better for catching an important message, a change in approval date, or an unapproved image Enable treatment to be delivered more safely 0% 94% Yes (P<0.01) 0% 100% Yes (P<0.01) 0% 100% Yes (P<0.01) 13% 81% Yes (P<0.01) Easier to use 25% 56% No More efficient 19% 63% No Preferred system 25% 56% No 23

45 Limitations Students vs. experienced therapists Mannequins vs. real patients Redesigned interface was only a prototype, not all features were functional Only the first iteration of the design cycle 24

46 Acknowledgements Project team: - Dr. Joe Cafazzo - Dr. Mohammad Islam - Dr. Tony Easty - Dr. David Jaffray - Tara Rosewall Funding agencies: - NSERC, NPSF, AAPM, CPSI Other: - Catherine Dupuis - Mary Stewart-Hardy - Varuna Prakash - Mark Fan - Radiation therapists at PMH - Radiation therapist students 26

47 Summary Humans are fallible. We will err. We can reduce frequency and consequences of errors by designing systems within our limitations Solutions should aim towards the top of the Hierarchy of Effectiveness

48

49 healthcarehumanfactors.com Joseph Cafazzo, PhD PEng