A Process Assessment Model for Assessing the Risk Associated with placing a Medical Device on a Medical IT Network

Transcription

1 A Process Assessment Model for Assessing the Risk Associated with placing a Medical Device on a Medical IT Network Silvana Togneri MacMahon, Fergal Mc Caffery, Frank Keenan Regulated Software Research Group & Lero Dundalk Institute of Technology Dundalk THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide Lero 2013.

2 Presentation Overview Introduction Problem Background Overview of IEC Approach to the Development of the PRM and PAM Overview of the PRM and PAM Overview of the Assessment Method Progress to Date and Future Work Conclusions THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 2 2

3 Introduction Problem: While medical devices are stringently regulated prior to marketing, placing a device onto an IT network may result in the device not behaving as intended. Solution: IEC : 2010 Application of risk management for ITnetworks incorporating medical devices - Part 1: Roles, responsibilities and activities was developed to address these risks. To avoid any unintended consequences of placing a device on an IT network, a high level of co-operation among risk management stakeholders, including Medical Device Manufacturers (MDM) and Responsible Organisations (RO), is required Our Contribution: Our research focuses on providing an assessment framework to allow Responsible Organisations and Medical Device Manufacturers to understand the requirements of the standard and assess themselves against these requirements. THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 3 3

4 Problem Background: Medical IT Networks Increasingly, medical devices are being designed to exchange electronic information with other devices, including medical devices. Placing a device on an IT network can introduce risks that may not have been considered during the design and manufacture of the device. As hospitals become more reliant on networks, placing more devices on the network, any network failure compromises hospital operations and patient care is impacted. Network down time in safety critical systems is not acceptable. * *Bavesh Patel, Director of Biomedical Engineering, Washington Hospital Centre Why Clinical Networks need maintenance and an overview of IEC THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 4 4

5 Networked Medical Devices: Benefits and Risks Benefits: o Increased Exchange of Data o Streamlined work processes save 4 to 36 minutes, prevent 24 data errors daily, save 100 hours daily in a typical hospital~. Better, cheaper patient care. Risks*: o Limitation or error within any of the networked devices o operational inefficiencies o unauthorized access to information, or delayed, lost, or corrupted data Threat to patient safety ~ Quantifying The Value Of Medical Device Connectivity - Martin Poppelaars *Installation issues or operational activities such as software upgrades, cyber-security efforts, or remote servicing of medical or IT system components. THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 5 5

6 IEC Overview FDA Cluster of Reports of Cyber Attacks on Hospitals in 2003/2004 WHY Key Properties: Safety Effectiveness Security IEC WHY Cyber security Guidance Network Integration of Medical Devices WHO Responsible Organisations Medical Device Manufacturers Other IT Providers THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 6 6

7 IEC : Roles and Responsibilities Responsible Organisation: o o o o o Risk Management policy & Risk Management Process Medical IT Network Risk Manager &Risk Management File Risk Analysis, Evaluation, Control and evaluation of residual risk Life cycle approach to Risk Management Establishment of a network; addition of a device to a network; modification, maintenance or removal of a device from a network Medical Device Manufacturer & Providers of Other Information Technology: o Provide documentation to Responsible Organisations to allow them to safely place devices on the network o the intended use of the medical device and the network, required characteristics and configurations of the network, technical specifications, and security requirements THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 7 7

8 Assessment against IEC ? No method of assessment against IEC is currently available. To assess against IEC a Process Reference Model (PRM), Process Assessment Model (PAM) and Assessment method are required. In order to develop a process assessment model, we investigated the following: Review of Process Assessment Standards ISO/IEC requirements for PRMs and PAMs. What standards are similar to IEC ? How are assessments performed against these standards? How were these assessment methods developed? THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 8 8

9 Approach to Development of PRM and PAM - Standards IEC Application of Risk Management for IT-Networks Incorporating Medical Devices Review of PAMs for standards similar to IEC provides requirements Template for IEC Process Reference Model (PRM) provides description of processes assessed by ISO/IEC Performing an assessment ISO/IEC TR Guidelines for process definition provides requirements for assessment IEC Process Assessment Model (PAM) THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 9 9

10 Assessment against similar standards: TIPA Assessment Framework Focus on ISO/IEC Similar standard to IEC Service Management Standard Design, transition, delivery and improvement of services Lifecycle Approach -Plan, Do, Check, Act Similar roles and similar processes TIPA was developed by CRP Henri Tudor, Luxembourg. Can be used to assess against ISO/IEC or Information Technology Infrastructure Library (ITIL). Developed using the TIPA transformation process. THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 10 10

11 TIPA Transformation Process - IEC Collection of Requirements for IEC Requirement Trees Goal Trees ISO/IEC Requirements ISO/IEC TR Guidance Process Reference Model for IEC Process Assessment Model for IEC THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 11 11

12 Responsible Organisation PRM and PAM Process Overview Risk Management Processes: PLAN Risk Management Policy Processes Risk Management Policy Medical IT Network Risk Management Planning Processes Medical IT Network Planning Responsibility Agreements Medical IT Network Documentation Organisational Risk Management Medical Device Manufacturer DO Medical IT Network Risk Management Processes Medical IT Network Risk Management Risk Analysis & Evaluation M Risk Control Residual Risk Change Release Management & Configuration Management Change Release & Configuration Management Decision on how to apply Risk Management Go Live Live Network Risk Management Processes Monitoring Event Management CHECK ACT Providers of Other Information Technology THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 12

13 IEC PRM Sample Process : RO Name: Risk Analysis & Evaluation Context: This process allows the Responsible Organisation to identify risks related to the incorporation of medical devices into an IT network. Once these risks have been identified, the process allows the Responsible Organisation to analyse & evaluate the risks throughout the life cycle. The risk evaluation process will be based on the risks identified during the risk analysis phase. Purpose: The purpose of the Risk Analysis process is to identify, analyse & evaluate risk related to the incorporation of Medical Device into IT Networks. Outcomes: As a result of the successful implementation of the Risk Analysis process : 1. Hazards that are likely to arise from the medical IT network are identified. 2. For each identified hazard, the associated risks are estimated using available information or data throughout the lifecycle. 3. Possible consequences of harm (where probability of occurrence cannot be estimated) are listed for use in risk control. 4. The results of these activities are recorded in the medical IT-network risk management file. 5. Where the estimated risk(s) is so low that risk reduction need not to be pursued, the rationale for this decision is documented in the medical IT-network risk management file. 6. Where the estimated risk(s) are not acceptable, risk control measures are implemented according to the Risk Control Process. Requirements traceability: IEC , Risk Analysis [1,2,3,4] IEC , 4.4.3, (a). Risk Evaluation [5] IEC , 4.4.3, (b). Risk Evaluation [6] THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide

14 IEC PAM Sample Process: RO Base Practices: MRM.1.1.BP1: Identify likely hazards. Identify hazards that are likely to arise from the Medical IT Network. [IEC , 4.4.2] [Expected Result: 1]. MRM.1.1.BP2: Estimate associated risks. Estimate associated risks using available information or data throughout the lifecycle for each identified hazard. [IEC , 4.4.2] [Expected Result: 2]. MRM.1.1.BP3: List possible consequences of harm. List possible consequences of harm (where probability of occurrence cannot be estimated) for use in risk control. [IEC , 4.4.2] [ Expected Result : 3 ] MRM.1.1.BP4: Record results of Risk Analysis and Evaluation activities. Record the results of these activities in the medical IT Network Risk Management file. Record instances where the estimated risk is so low that risk reduction need not be pursued (as per RM plan) in the medical IT Network Risk Management File. [IEC , 4.4.2] [IEC , 4.4.3, (a)] [Expected Result 4, 5]. MRM.1.1.BP5: Implement Risk Control Measures. Implement Risk control measures according to the Risk Control Process, where estimated risk(s) are not acceptable. [IEC , 4.4.3, (b)] [Expected Result : 6] Inputs: Risk Management plan [MRM1.1, BP.4] [Expected Result 4,5] Risk Mitigation plan [MRM1.1, BP.5] [Expected Result 6] Outputs: 03-02Risk log [MRM1.1, BP.1] [Expected Result 1] Risk analysis report [MRM1.1, BP.2] [Expected Result 2] Risks status report [MRM1.1, BP.2] [Expected Result 2] Risk Measure [MRM1.1, BP.2, 3] [Expected Result 2, 3] Hazard log [MRM1.1, BP.1] [Expected Result 1] Consequences log [MRM1.1, BP.3] [Expected Result 3] Risk action request [MRM1.1, BP.5] [Expected Result 6] Medical IT network Risk Management file [MRM1.1, BP.4] [Expected Result 4,5] THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide

15 IEC PRM Sample Process: MDM Name: Context: Purpose: Outcomes: Responsibility Agreements In order to establish the responsibilities of Medical Device Manufacturers and Other IT providers, Responsibility Agreements are drafted. The purpose of the process is to establish the responsibilities of Medical Device Manufacturers and Other IT providers in regard to risk management responsibilities. As a result of the successful implementation of the Responsibility Agreements Process: 1. The need for one or more documented responsibility agreements is determined whenever a medical device is incorporated into an IT network or the configuration of such a connection is changed. 2. A responsibility agreement defines the responsibilities of all relevant stakeholders throughout the lifecycle. 3. A responsibility agreement covers one or more projects or the maintenance of one or more medical IT-networks. Requirements traceability: IEC , Responsibility Agreements [1, 3] IEC , (a) to (h). Responsibility Agreements [2] THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide

16 Assessment Method In order to assess against the IEC PAM, an assessment method is required. Ensures that the assessment is consistent and that evidence is obtained to substantiate any ratings which are given. Ensures that the assessment scope is clear and that the documentation process is sufficient. An assessment method provides details on the organizations performance through using a set of questions (related to each process) to enable the assessor to determine the capability level at which the process is being performed. Currently being developed - 1 process from the 14 processes THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 16 16

17 Assessment Method Goals & Concerns Addressing the perspectives of various risk management stakeholders (within the RO and external stakeholders). Variation in scale among ROs. Experience of IT Medical IT Network Risk Manager Lightweight Assessment Method. THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 17 17

18 Assessment Method - Overview Assessment Briefing Site Briefing( RO or MDM) Conduct Interviews Generate Assessment Results Strengths Weaknesses Prepare Finding Report Deliver Finding Report Recommendations Implement Recommendations THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 18 18

19 Assessment Questions Assessment method will contain question(s) related to each of the base practices Based on the interviews a rating will be given for each of the questions Ratings will be Fully, Largely, Partially or Not Complete Findings Report will be drafted identifying strengths and weaknesses related to current RM processes and if applicable make recommendations to improve current RM processes. Follow up Assessment may be performed at a later date following the implementation of improvement actions. THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 19 19

20 IEC Assessment Method THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide

21 Assessment Method: St James s Hospital Working with Clinical Engineering team Provide an understanding of the assessment process PRM, PAM and Assessment Method Development of the assessment questions Working with cross disciplinary team within the hospital also IT department, management and clinicians Assessment method is updated on the basis of feedback Performing mock assessments Improvements to RM process have been made based on the mock assessments THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 21 21

22 Progress to Date and Future Work PRM and PAM approved as NP for inclusion in IEC family of standards comment resolution. IEC 62A JWG7 Validation by developers of IEC standard. Validation of AM in HDO setting St James s Hospital, Dublin. Validation of PRM and PAM Models by the SPICE community Trial Assessment and Follow-up Assessment. THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 22 22

23 Conclusion IEC has been developed to address these risks related to placing a medical device onto an IT network by setting out the roles, responsibilities and activities during the risk management process. In order to fully realise the benefits that networked medical devices can provide and ensure quality patient care, ROs and MDMs need to co-operate in the management of risk thoughout the life cycle of the medical device. The development of a PRM, PAM and Assessment method will allow ROs and MDMs to assess themselves against the requirements of the standard and can be used as a baseline for the improvement of risk management processes. THE IRISH SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 23 23

24 Thank You - Any Questions? Silvana Togneri MacMahon Silvana.MacMahon@dkit.ie This research is supported by the Science Foundation Ireland (SFI) Stokes Lectureship Programme, grant number 07/SK/I1299, the SFI Principal Investigator Programme, grant number 08/IN.1/I2030 (the funding of this project was awarded by Science Foundation Ireland under a co-funding initiative by the Irish Government and European Regional Development Fund), and supported in part by Lero - the Irish Software Engineering Research Centre ( grant 10/CE/I1855 THE Lero IRISH 2012 SOFTWARE ENGINEERING RESEARCH CENTRE Lero Slide 24