A Process Assessment Model for Assessing the Risk Associated with placing a Medical Device on a Medical IT Network Silvana Togneri MacMahon, Fergal Mc Caffery, Frank Keenan Regulated Software Research Group & Lero Dundalk Institute of Technology Dundalk THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide Lero 2013.
Presentation Overview Introduction Problem Background Overview of IEC 80001-1 Approach to the Development of the PRM and PAM Overview of the PRM and PAM Overview of the Assessment Method Progress to Date and Future Work Conclusions THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 2 2
Introduction Problem: While medical devices are stringently regulated prior to marketing, placing a device onto an IT network may result in the device not behaving as intended. Solution: IEC 80001-1: 2010 Application of risk management for ITnetworks incorporating medical devices - Part 1: Roles, responsibilities and activities was developed to address these risks. To avoid any unintended consequences of placing a device on an IT network, a high level of co-operation among risk management stakeholders, including Medical Device Manufacturers (MDM) and Responsible Organisations (RO), is required Our Contribution: Our research focuses on providing an assessment framework to allow Responsible Organisations and Medical Device Manufacturers to understand the requirements of the standard and assess themselves against these requirements. THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 3 3
Problem Background: Medical IT Networks Increasingly, medical devices are being designed to exchange electronic information with other devices, including medical devices. Placing a device on an IT network can introduce risks that may not have been considered during the design and manufacture of the device. As hospitals become more reliant on networks, placing more devices on the network, any network failure compromises hospital operations and patient care is impacted. Network down time in safety critical systems is not acceptable. * *Bavesh Patel, Director of Biomedical Engineering, Washington Hospital Centre Why Clinical Networks need maintenance and an overview of IEC 80001-1. THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 4 4
Networked Medical Devices: Benefits and Risks Benefits: o Increased Exchange of Data o Streamlined work processes save 4 to 36 minutes, prevent 24 data errors daily, save 100 hours daily in a typical hospital~. Better, cheaper patient care. Risks*: o Limitation or error within any of the networked devices o operational inefficiencies o unauthorized access to information, or delayed, lost, or corrupted data Threat to patient safety ~ Quantifying The Value Of Medical Device Connectivity - Martin Poppelaars *Installation issues or operational activities such as software upgrades, cyber-security efforts, or remote servicing of medical or IT system components. THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 5 5
IEC 80001-1 Overview FDA Cluster of Reports of Cyber Attacks on Hospitals in 2003/2004 WHY Key Properties: Safety Effectiveness Security IEC 80001-1 WHY Cyber security Guidance Network Integration of Medical Devices WHO Responsible Organisations Medical Device Manufacturers Other IT Providers THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 6 6
IEC 80001-1: Roles and Responsibilities Responsible Organisation: o o o o o Risk Management policy & Risk Management Process Medical IT Network Risk Manager &Risk Management File Risk Analysis, Evaluation, Control and evaluation of residual risk Life cycle approach to Risk Management Establishment of a network; addition of a device to a network; modification, maintenance or removal of a device from a network Medical Device Manufacturer & Providers of Other Information Technology: o Provide documentation to Responsible Organisations to allow them to safely place devices on the network o the intended use of the medical device and the network, required characteristics and configurations of the network, technical specifications, and security requirements THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 7 7
Assessment against IEC 80001-1? No method of assessment against IEC 80001-1 is currently available. To assess against IEC 80001-1 a Process Reference Model (PRM), Process Assessment Model (PAM) and Assessment method are required. In order to develop a process assessment model, we investigated the following: Review of Process Assessment Standards ISO/IEC 15504-2 requirements for PRMs and PAMs. What standards are similar to IEC 80001-1? How are assessments performed against these standards? How were these assessment methods developed? THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 8 8
Approach to Development of PRM and PAM - Standards IEC 80001-1 Application of Risk Management for IT-Networks Incorporating Medical Devices Review of PAMs for standards similar to IEC 80001-1 provides requirements Template for IEC 80001-1 Process Reference Model (PRM) provides description of processes assessed by ISO/IEC 15504-2 Performing an assessment ISO/IEC TR 24774 Guidelines for process definition provides requirements for assessment IEC 80001-1 Process Assessment Model (PAM) THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 9 9
Assessment against similar standards: TIPA Assessment Framework Focus on ISO/IEC 20000 Similar standard to IEC 80001-1 Service Management Standard Design, transition, delivery and improvement of services Lifecycle Approach -Plan, Do, Check, Act Similar roles and similar processes TIPA was developed by CRP Henri Tudor, Luxembourg. Can be used to assess against ISO/IEC 20000 or Information Technology Infrastructure Library (ITIL). Developed using the TIPA transformation process. THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 10 10
TIPA Transformation Process - IEC 80001-1 Collection of Requirements for IEC 80001-1 Requirement Trees Goal Trees ISO/IEC 15504-2 Requirements ISO/IEC TR 24774 Guidance Process Reference Model for IEC 80001-1 Process Assessment Model for IEC 80001-1 THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 11 11
Responsible Organisation PRM and PAM Process Overview Risk Management Processes: PLAN Risk Management Policy Processes Risk Management Policy Medical IT Network Risk Management Planning Processes Medical IT Network Planning Responsibility Agreements Medical IT Network Documentation Organisational Risk Management Medical Device Manufacturer DO Medical IT Network Risk Management Processes Medical IT Network Risk Management Risk Analysis & Evaluation M Risk Control Residual Risk Change Release Management & Configuration Management Change Release & Configuration Management Decision on how to apply Risk Management Go Live Live Network Risk Management Processes Monitoring Event Management CHECK ACT Providers of Other Information Technology THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 12
IEC 80001-1 PRM Sample Process : RO Name: Risk Analysis & Evaluation Context: This process allows the Responsible Organisation to identify risks related to the incorporation of medical devices into an IT network. Once these risks have been identified, the process allows the Responsible Organisation to analyse & evaluate the risks throughout the life cycle. The risk evaluation process will be based on the risks identified during the risk analysis phase. Purpose: The purpose of the Risk Analysis process is to identify, analyse & evaluate risk related to the incorporation of Medical Device into IT Networks. Outcomes: As a result of the successful implementation of the Risk Analysis process : 1. Hazards that are likely to arise from the medical IT network are identified. 2. For each identified hazard, the associated risks are estimated using available information or data throughout the lifecycle. 3. Possible consequences of harm (where probability of occurrence cannot be estimated) are listed for use in risk control. 4. The results of these activities are recorded in the medical IT-network risk management file. 5. Where the estimated risk(s) is so low that risk reduction need not to be pursued, the rationale for this decision is documented in the medical IT-network risk management file. 6. Where the estimated risk(s) are not acceptable, risk control measures are implemented according to the Risk Control Process. Requirements traceability: IEC 80001-1, 4.4.2. Risk Analysis [1,2,3,4] IEC 80001-1, 4.4.3, (a). Risk Evaluation [5] IEC 80001-1, 4.4.3, (b). Risk Evaluation [6] THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide
IEC 80001-1 PAM Sample Process: RO Base Practices: MRM.1.1.BP1: Identify likely hazards. Identify hazards that are likely to arise from the Medical IT Network. [IEC 80001-1, 4.4.2] [Expected Result: 1]. MRM.1.1.BP2: Estimate associated risks. Estimate associated risks using available information or data throughout the lifecycle for each identified hazard. [IEC 80001-1, 4.4.2] [Expected Result: 2]. MRM.1.1.BP3: List possible consequences of harm. List possible consequences of harm (where probability of occurrence cannot be estimated) for use in risk control. [IEC 80001-1, 4.4.2] [ Expected Result : 3 ] MRM.1.1.BP4: Record results of Risk Analysis and Evaluation activities. Record the results of these activities in the medical IT Network Risk Management file. Record instances where the estimated risk is so low that risk reduction need not be pursued (as per RM plan) in the medical IT Network Risk Management File. [IEC 80001-1, 4.4.2] [IEC 80001-1, 4.4.3, (a)] [Expected Result 4, 5]. MRM.1.1.BP5: Implement Risk Control Measures. Implement Risk control measures according to the Risk Control Process, where estimated risk(s) are not acceptable. [IEC 80001-1, 4.4.3, (b)] [Expected Result : 6] Inputs: 08-03 Risk Management plan [MRM1.1, BP.4] [Expected Result 4,5] 08-04 Risk Mitigation plan [MRM1.1, BP.5] [Expected Result 6] Outputs: 03-02Risk log [MRM1.1, BP.1] [Expected Result 1] 15-01 Risk analysis report [MRM1.1, BP.2] [Expected Result 2] 15-02 Risks status report [MRM1.1, BP.2] [Expected Result 2] 07-01 Risk Measure [MRM1.1, BP.2, 3] [Expected Result 2, 3] 03-03 Hazard log [MRM1.1, BP.1] [Expected Result 1] 03-04 Consequences log [MRM1.1, BP.3] [Expected Result 3] 13-02 Risk action request [MRM1.1, BP.5] [Expected Result 6] 16-02 Medical IT network Risk Management file [MRM1.1, BP.4] [Expected Result 4,5] THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide
IEC 80001-1 PRM Sample Process: MDM Name: Context: Purpose: Outcomes: Responsibility Agreements In order to establish the responsibilities of Medical Device Manufacturers and Other IT providers, Responsibility Agreements are drafted. The purpose of the process is to establish the responsibilities of Medical Device Manufacturers and Other IT providers in regard to risk management responsibilities. As a result of the successful implementation of the Responsibility Agreements Process: 1. The need for one or more documented responsibility agreements is determined whenever a medical device is incorporated into an IT network or the configuration of such a connection is changed. 2. A responsibility agreement defines the responsibilities of all relevant stakeholders throughout the lifecycle. 3. A responsibility agreement covers one or more projects or the maintenance of one or more medical IT-networks. Requirements traceability: IEC 80001-1, 4.3.4. Responsibility Agreements [1, 3] IEC 80001-1, 4.3.4 (a) to (h). Responsibility Agreements [2] THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide
Assessment Method In order to assess against the IEC 80001 PAM, an assessment method is required. Ensures that the assessment is consistent and that evidence is obtained to substantiate any ratings which are given. Ensures that the assessment scope is clear and that the documentation process is sufficient. An assessment method provides details on the organizations performance through using a set of questions (related to each process) to enable the assessor to determine the capability level at which the process is being performed. Currently being developed - 1 process from the 14 processes THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 16 16
Assessment Method Goals & Concerns Addressing the perspectives of various risk management stakeholders (within the RO and external stakeholders). Variation in scale among ROs. Experience of IT Medical IT Network Risk Manager Lightweight Assessment Method. THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 17 17
Assessment Method - Overview Assessment Briefing Site Briefing( RO or MDM) Conduct Interviews Generate Assessment Results Strengths Weaknesses Prepare Finding Report Deliver Finding Report Recommendations Implement Recommendations THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 18 18
Assessment Questions Assessment method will contain question(s) related to each of the base practices Based on the interviews a rating will be given for each of the questions Ratings will be Fully, Largely, Partially or Not Complete Findings Report will be drafted identifying strengths and weaknesses related to current RM processes and if applicable make recommendations to improve current RM processes. Follow up Assessment may be performed at a later date following the implementation of improvement actions. THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 19 19
IEC 80001-1 Assessment Method THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide
Assessment Method: St James s Hospital Working with Clinical Engineering team Provide an understanding of the assessment process PRM, PAM and Assessment Method Development of the assessment questions Working with cross disciplinary team within the hospital also IT department, management and clinicians Assessment method is updated on the basis of feedback Performing mock assessments Improvements to RM process have been made based on the mock assessments THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 21 21
Progress to Date and Future Work PRM and PAM approved as NP for inclusion in IEC 80001 family of standards comment resolution. IEC 62A JWG7 Validation by developers of IEC 80001-1 standard. Validation of AM in HDO setting St James s Hospital, Dublin. Validation of PRM and PAM Models by the SPICE community Trial Assessment and Follow-up Assessment. THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 22 22
Conclusion IEC 80001-1 has been developed to address these risks related to placing a medical device onto an IT network by setting out the roles, responsibilities and activities during the risk management process. In order to fully realise the benefits that networked medical devices can provide and ensure quality patient care, ROs and MDMs need to co-operate in the management of risk thoughout the life cycle of the medical device. The development of a PRM, PAM and Assessment method will allow ROs and MDMs to assess themselves against the requirements of the standard and can be used as a baseline for the improvement of risk management processes. THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. Slide 23 23
Thank You - Any Questions? Silvana Togneri MacMahon Silvana.MacMahon@dkit.ie This research is supported by the Science Foundation Ireland (SFI) Stokes Lectureship Programme, grant number 07/SK/I1299, the SFI Principal Investigator Programme, grant number 08/IN.1/I2030 (the funding of this project was awarded by Science Foundation Ireland under a co-funding initiative by the Irish Government and European Regional Development Fund), and supported in part by Lero - the Irish Software Engineering Research Centre (http://www.lero.ie) grant 10/CE/I1855 THE Lero IRISH 2012 SOFTWARE ENGINEERING RESEARCH CENTRE Lero 2013. 2012. Slide 24