Robert A. Martin 19 March 2018

Transcription

1 Robert A. Martin 19 March 2018

2

3

4

5

6

7

8 Students helped assemble a collection of commercial IoT devices and record their RF emissions

9 369 Requests for Information 299 Requests to Register 131 Teams entered the challenge 50% International (35 countries) 8 Teams submitted answers Team 0xDEADBEEF Wins Runners up: Team Pulzze Systems and Team Tietronixs

10 B 0x9 Preamble Payload C 0xd B 0x9 Clear cluster distinctiveness C 0xd

11 Expanding Attack Surfaces of Transportation Systems

12

13

14

15

16

17 The Key System Characteristics of Trustworthiness as a Quality Measure Industrial IoT Quality is a continuum of system characteristics OT Security (IEC 62443*) meets IT Security (ISO 27000*) Privacy (GDPR*), Resilience (ISO*, IEC*), Reliability (NIS*) are quality features in both OT and IT Determine and ensure quality measures per vertical, e.g. audit, certification

18 Claims of Trustworthiness Gathering Evidence for Assurance Cases

19 Claims of Trustworthiness Gathering Evidence for Assurance Cases

20 Claims of Trustworthiness Gathering Evidence for Assurance Cases

21 The Key System Characteristic: Safe

22 But if every IIoT System has a unique array of requirements how do we manage that? Possible IIoT System Trustworthiness Requirements

23 Group Requirements around families of IIoT Systems that similar functions, environment, and other context?

24 Infusion Pumps Total Product Life Cycle Guidance for Industry and FDA Staff Support for Safety Case Generation via Model Transformation Chung-Ling Lin, Wuwei Shen Department of Computer Science Western Michigan University Kalamazoo, MI, USA {chung-ling.lin, Richard Hawkins Department of Computer Science The University of York York, UK Document issued on: December 2, 2014 The draft of this document was issued on April 23, This document supersedes the Guidance on the Content of Premarket Notification [510(k)] Submissions for External Infusion Pumps, issued March, OMB Control Number: Expiration Date: 5/31/2017 For questions regarding this document, please contact Alan Stevens, General Hospital Devices Branch, Office of Device Evaluation at or via at The technological features of the devices. For questions regarding safety assurance cases, please contact Richard Chapman, General Hospital Devices Branch, Office of Device Evaluation at You or should via describe at how any differences in technology may affect the comparative safety and performance of your device. For questions regarding pre-clearance inspections, please contact Francisco Vicenty, Respiratory, Ear/Nose/Throat, General Hospital, Infectious Control, and Ophthalmic 5. Safety Devices Assurance Branch, Case Office of Compliance at or via at Infusion pump 510(k) submissions typically include changes or modifications to software, materials, design, performance, or other features compared to the predicate. Accordingly, FDA expects that most new devices (as well as most changed or modified devices For questions pertaining to manufacturer reporting requirements, please contact Sharon Kapsch at 5 ) will have differences in technological characteristics from the legally marketed predicate device even if or via at sharon.kapsch@fda.hhs.gov. sharing the same intended use. Under section 513(i) of the Federal Food, Drug, and Cosmetic Act (the FD&C Act), determinations of substantial equivalence will rely on whether the information submitted, including appropriate clinical or scientific data, demonstrate that the new or modified device is as safe and effective as the legally marketed predicate device and does not raise different U.S. Department of Health and Human Services questions of safety and effectiveness in comparison to the predicate device. Food and Drug Administration Center In determining for Devices whether and your new, Radiological changed, or Health modified infusion pump is substantially equivalent, FDA recommends that you submit your information through a framework known as a safety assurance case. 6 Office of Device Evaluation Division of Anesthesiology, General Hospital, The safety assurance case (or safety case) consists of a structured argument, supported by a body of valid scientific Respiratory, evidence Infection that provides Control, an organized and case that the infusion pump adequately addresses hazards associated with its Dental intended Devices use within its environment of use. The argument should be commensurate General Hospital with the potential Devices risk Branch posed by the infusion pump, the complexity of the infusion pump, and the familiarity with the identified risks and mitigation measures. ABSTRACT Assessing the safety of complex safety- or mission-critical systems under ever tightening time constraints with any degree of confidence is a growing challenge for industry and regulators alike. One method of helping to address this situation is through the use of assurance cases. Challenges abound here as well; too little or too much abstraction or poorly constructed arguments can affect confidence that a system will perform as intended. The automatic generation of a (safety) assurance case not only can expedite a development process but also leverage the ability to perform compliance checking. In this paper, we propose a novel framework which weaves a safety case pattern, guidance metamodel, and a development process metamodel together to generate a safety assurance case, which facilitates checking the conformance of the system to the guidance. As a case study, we use the GPCA infusion pump project as a subject to illustrate how this framework can aid in compliance checking using the infusion pump guidance published by FDA as a reference oracle. Keywords Compliance checking; model transformation; safety-critical systems; safety case. 1. INTRODUCTION Assessing the safety of complex safety- and mission-critical systems, such as medical devices, under ever tightening time constraints with an acceptable level of confidence is a growing challenge for industry and regulators alike. One method of helping to address this is through the use of safety assurance cases (or safety case in short) [1]. For instance, the U.S. Food and Drug Administration (FDA) recently released an infusion pump a guidance document on the total product lifecycle for infusion pumps [2], which recommends infusion pump manufacturers to use safety assurance case ( safety case ) as a structured means to organize and present to FDA the information supporting the safety claims of their infusion pump devices. In this paper, we take the infusion pump guidance as an example to discuss how to automatically construct a safety case in safety critical domains. The construction and review of a safety case for an infusion pump system are a daunting task for various stakeholders such as Copyright retained by the authors. manufacturers and FDA regulators due to the following reasons. Firstly, the guidance provides general requirements on what types of safety properties that a safety case should argue about and what kind of evidence it should collect from development artifacts. But, it leaves it up to device manufacturers to decide the ways of constructing a safety case in terms of using the collected evidence to support the specific safety claims articulated for their devices. This however creates a gap between the guidance s requirements and the device development process for the device that needs the manufacturers to properly bridge when constructing their safety cases. This gap also makes it challenging for regulators to review the safety cases, because they need to first understand how guidance requirements are mapped to the safety claims in the safety cases and then evaluate the trustworthiness and qualification of the collected evidence in supporting these claims. Exacerbating the problem is the poor quality of evidence and arguments assembled in the safety cases: many safety cases suffer from the structural problems, such as too little or too much abstraction and poorly constructed arguments. Secondly, like many other guidance documents or standards across the safety critical industries, the guidance intends to be generic to ensure its applicability to as many infusion pump devices as possible. Consequently, it creates a space for different stakeholders, such as suppliers, clients, and certifiers, to come up with different understanding/interpretation of the guidance s requirements. For example, the guidance recommends manufacturers to conduct hazard analysis to identify the risks associated with their devices and use the results to define the safety claims to be included in the safety cases. However, it leaves it up to manufacturers to decide the specific hazard analysis techniques to use and the process of using such techniques. The difference among stakeholders in interpreting the guidance creates a communication gap between them. Safety cases need to be constructed properly to help to remediate the difference, rather than making it worse. To address the above challenges, we propose a novel model-based framework, called SPIRIT, that applies the notions of safety case patterns and model weaving to support the mechanical generation and validation of safety cases. Central to SPIRIT is to utilize safety case patterns [3] to enable the mechanized and consistent generation of safety cases for the same type of systems. In this way, the cost of constructing safety cases can be reduced and the confidence of such safety cases can be improved, by reusing the safety case patterns that have been proven as successful in past practices to promote the communication among stakeholders. Beside the safety case pattern, SPIRIT requires two additional inputs: a guidance metamodel, in the format of a UML class diagram, to denote the guidance and remediate the stakeholders difference in interpreting the guidance; and a development process metamodel that defines how a manufacturer designs their infusion 5 Based on FDA s analysis of these devices, FDA expects that most changes or modifications to infusion pumps could significantly affect the safety or effectiveness of the devices and would therefore require submission of a new 510(k). See 21 CFR (a)(3). Note that a change to the intended use or technology of a 510(k)-cleared device may render the device not substantially equivalent (NSE) to a legally marketed predicate. For detailed information about substantial equivalence and 510(k) submissions, refer to the FDA guidance entitled, The 510(k) Program: Evaluating Substantial Equivalence in Premarket Notifications [510(k)] ( Any such device may thus be a class III device and require a premarket approval application (PMA), unless the device is reclassified under section 513 of the Federal Food, Drug, and Cosmetic Act. 6 For more information about assurance case reports, see, for example: Graydon, P., J. Knight, and E. Strunk, Assurance Based Development of Critical Systems, Proc. of 37 th Annual International Conference on Dependable Systems and Networks, Edinburgh, U.K., 2007; Kelly, T., Arguing Safety A Systematic Approach to Managing Safety Cases, Ph.D. Dissertation, University of York, U.K., 1998; Kelly, T., Reviewing Assurance Arguments - A Step-by-Step Approach, Proc. of Workshop on Assurance Cases for Security - The Metrics Challenge, Dependable Systems and Networks, July 2007; Kelly, Tim, and J. McDermid, Safety Case Patterns Reusing Successful Arguments, Proc. of IEE Colloquium on Understanding Patterns and Their Application to System Engineering, London, Apr. 1998; Weinstock, Charles B. and Goodenough, John B., Towards an Assurance Case Practice for Medical Devices, Carnegie Mellon Software Engineering Institute, October 2009; Hawkins, Richard, et. al., A New Approach to Creating Clear Safety Arguments, Safety-critical Systems Symposium, Southampton, UK, February 2011; UK Ministry of Defence, Defence Standard 00-56, Safety Management Requirements for Defence Systems Part 1 and Part 2, June Figure 9 Safety case model of GPCA system 9

25 NASA System Safety Framework (cont.) Assuring Safety Ensuring Safety 21

26

27

28

29 ad/ RFP Template: ab/ Object Management Group ad/ RFP Template: ab/ Highland Avenue Needham, MA USA Telephone: Facsimile: Safety and Reliability for UML Request For Proposal OMG Document: ad/ Letters of Intent due: 15 June 2017 Submissions due: 28 August 2017 Objective of this RFP The correct management and use of information concerning the safety and reliability of a safety-critical system is vital to that system s development costs and its eventual safety. The application of model-based approaches and tools can improve these tasks by automating manual tasks, adding clarity, and improving communication of complex ideas and concepts. This RFP looks to provide a standard profile for the OMG Unified Modeling Language TM (UML ) that works with the OMG Systems Modeling Language (SysML ) to allow the integration of safety and reliability information directly in a system model, where it can be modeled and processed directly with other system information. This RFP solicits proposals for a profile and/or optionally a model library for the Unified Modeling Language that enables the following: Capturing safety and reliability information in a system model. Reasoning on the safety and reliability information, both directly on the model and indirectly via model transformations. OMG RFP 23 June Visualising safety and reliability information. Exchanging safety and reliability information between a system model and external tools. Tracing between safety information, reliability information, and other information stored in a system model. For further details, see Section 6 of this document. 1 Introduction 1.1 Goals of OMG The Object Management Group (OMG) is a software consortium with an international membership of vendors, developers, and end users. Established in 1989, its mission is to help computer users solve enterprise integration problems by supplying open, vendor-neutral portability, interoperability and reusability specifications based on Model Driven Architecture (MDA). MDA defines an approach to IT system specification that separates the specification of system functionality from the specification of the implementation of that functionality on a specific technology platform, and provides a set of guidelines for structuring specifications expressed as models. OMG has published many widely-used specifications such as UML [UML], BPMN [BPMN], MOF [MOF], XMI [XMI], DDS [DDS] and CORBA [CORBA], to name but a few significant ones. 1.2 Organization of this document The remainder of this document is organized as follows: Section 2 Architectural Context. Background information on OMG s Model Driven Architecture. Section 3 Adoption Process. Background information on the OMG specification adoption process. Section 4 Instructions for Submitters. Explanation of how to make a submission to this RFP. Section 5 General Requirements on Proposals. Requirements and evaluation criteria that apply to all proposals submitted to OMG. OMG RFP 23 June

30

31

32 Utilizing Appropriate Detection Methods to Collect Evidence to Gain Assurance Design Review Code Review Attack Surface Analysis Static Analysis Tool A Static Analysis Tool B Dynamic Analysis Tool C Fuzz Testing Pen Testing Blue Teaming Red Teaming

33

34

35 The Assurance Case for a System Builder using Assured Components

36

37