A New Safety Theory: Concept, Methodology, and Application M.Y. Cai, C.J. Liu Complex and Intelligent System Research Center East China University of Science and Technology Shanghai, China Email: caimengya88@163.com, cjliu@ecust.edu.cn C.A. van Luttervelt Faculty of Mechanical Engineering Delft University of Technology Delft, Netherlands Email: c.a.vanluttervelt@wbmt.tudelft.nl J.W. Wang Industrial and Manufacturing Systems Engineering University of Hong Kong Hong Kong, China Email: wangjw1979@gmail.com Y. Lin, Member, IEEE Department of Mechanical and Industrial Engineering Northeastern University Boston, USA Email: yilin@coe.neu.edu W.J. Zhang*, Senior Member, IEEE Complex and Intelligent System Research Center East China University of Science and Technology Shanghai, China And Department of Mechanical Engineering University of Saskatchewan Saskatoon, Canada * Corresponding author: chris.zhang@usask.ca Abstract Critical systems are systems that have direct effects on human health and life. Large production systems are an example of the critical system. Safety is a concept that is ubiquitous in our life but has never been clearly defined. A common confusion with safety is that safety is viewed to be the same as reliability. In this paper, we revisit the concept of safety and propose a new approach for design and operation management of critical systems for safety. The new approach puts its foot on the threats to human health and life and an integrated consideration of reliability, robustness and resilience for safety. The new approach for safety is thus called 3R for safety. Keywords system; safety; reliability; robustness; resilience. I. INTRODUCTION Is it safe is a common question to humans. The word It here refers to a product or a system in general. As a common sense, this question is concerned with any possible threat to the human health and life or more generally to the human freedom to live in connection with a system. Therefore, the concept of safety is related to both the system and human. A system may be completely collapsed, which indicates the poor reliability and/or robustness of the system, but the collapsed system may not affect the human life per se and thus the system is not unsafe. Note that the foregoing system can be a natural system or an engineering system. Human beings have long been battling with natural systems for safety, primarily concerning the environment and resource. Modern industrialization has provided technologies and tools for human beings to create artifact systems (or engineering systems), which are generally of four kinds: agricultural systems, manufacturing systems, product systems, and service systems [19]. These systems have been foundations for the modern society but they have also created some sources of safety problems. It is obvious that these systems may fail and thus may create threats to the human life and health. Safety engineering is a relatively young discipline, and it includes safety science, safety technology, and design and operation of systems for safety. Safety engineering is an interdisciplinary discipline because it makes sense when associated with the foregoing general types of systems. However, not all systems have the sense of safety concerns. Critical systems are the systems that can create a high degree of safety concerns. Examples of critical systems are financial systems, telecommunication systems, manufacturing systems and so on, which produce necessary products for the modern society. The financial crisis in 2008 has given modern human beings a fresh impression of how failures of critical systems could create threats to human life and health at a large scale. This paper concerns the safety of critical systems. The primary objective of this paper is to propose a new paradigm for safety engineering. This is motivated by the critical review of the existing knowledge on safety engineering and by the recent emerging of resilience engineering along with the proposal by Hollnagel et al. [3] that resilience The authors want to thank a financial support to this work by NSERC through a strategic project grant on resilience engineering to the corresponding author.
engineering should be considered as an ultimate paradigm of safety engineering. The remaining part of this paper is organized as follows. Section 2 summarizes the state of knowledge on safety engineering. Section 3 presents a new paradigm for safety engineering. Section 4 outlines theories and methodologies for safety engineering based on the new paradigm. Section 5 gives examples to illustrate the application of the theories and methodologies for enhancing the safety of artifact systems. Section 6 concludes this paper. II. THE STATE OF KNOWLEDGE ON SAFETY ENGINEERING 2.1 The Existing Theory for Safety In literature, safety is defined as freedom from those conditions that can cause death, injury, occupational illness, or damage to or loss of equipment or property, or damage to the environment [2]. This definition, however, misses an important element, that is the system element; merely conditions do not allow us to think of means to enhance safety. This definition also misses the human factor which is part of the system. A new definition of safety will be presented later in Section 3 of the present paper. For the time being, the missing elements regarding the safety concept are assumed, and the existing approaches to design and manage artifact systems for safety are then summarized and commented. In the period from the 1950 s to 1990 s, safety is related to a system s reliability and robustness, which further leads to the disciplines of reliability engineering and robust engineering. The representative school of thinking in that period of time is normal accident theory (NAT) [13] and high reliability organizations (HRO) [8]. Perrow s NAT theory considers that accidents of technological systems are inevitable and normal [13]. He further considered two related dimensions interactive complexity and coupling complexity as two sources of the system vulnerability susceptible to accidents. HRO theory examines the system safety from the organizational perspective, which is in essence similar to NAT. In fact, Perrow s first dimension is related to robustness and his second dimension is related to reliability, which will be elaborated later in this paper. Reliability is defined as an ability of a system or component to perform its required functions under stated conditions for a specified period of time [16, 20]. In particular, reliability specifies the probability that no operational interruptions will occur during a stated time interval. It is noted that one of the important means to improve reliability is to design redundancy into a system. Perrow s coupling complexity implies a network topology of an artifact system. One of the tasks in reliability engineering is to analyze the reliability of a network system [1], so Perrow s second dimension is related to reliability. Robustness is a property of a system related with noise; in particular, robustness is a property that allows a system to maintain its functions against internal and external perturbations or noises [7, 20]. In other words, the robustness of a system focuses on how the system is insensitive to noises. Perrow s interactive complexity implies that a kind of uncertainty in a network system can more likely exist in the interface between any two components. Uncertainties are in a broad sense noises, and the two are used interchangeably in the present paper. In short, Perrow s first dimension is related to robust engineering. Remark 1: The relevance of safety to reliability and robustness is such that when a system has failures, the system may become unsafe. However, failures may not treat the human health and life. It is clear that the safety theory of Perrow has only considered the failure of a system but not the treat of the failure to the human health and life, and therefore, the theory of Perrow is not a complete one to safety. Remark 2: From the 1990 s to 2000 s, the artifact system includes humans as a part of the system. This is due to the technological advancement on software and automation, which changes human roles as supervisors. Human reliability and robustness are brought into the domain of system failures [9, 14]. However, inclusion of humans in this manner does not imply that the threat to the human health and life is considered; in fact, this manner merely considers that the human is a part of the system. One method for safety engineering at that time is STAMP (System-Theoretic Accident Model and Processes) proposed by Leveson [9]. The hypothesis of this model is that accidents should be viewed from a systems perspective. In this conception, accidents occur when external disturbances, component failures, or dysfunctional interactions among system components are not adequately handled by the control system; that is, they result from inadequate control or enforcement of safety-related constraints on the development, design, and operation of the system. This approach to safety still misses the human side especially threats to the human health and life. The approach merely puts emphasis on the error coming from a system that includes human operators. Recently, the concept of resilience engineering emerges to relate it to safety engineering. Resilience in the engineering context is defined as a system s ability to recover its function after a partial damage of the system [20, 21, 23]. This definition of resilience is called Resilience I in the present paper. It is noted that the definition of resilience undergoes an evolution. Hollnagel et al. [3] defined resilience with two key phrases, namely (i) function recovery and (ii) disturbance. This definition stays between the definition of robustness (in light of disturbances on the substance flowing in the system and/or on the parameter that describes the structure of a system) and the definition of Resilience I (in light of damages on the structure of a system). Let us call this definition of resilience, Resilience II. Hollnagel et al. [3] further suggested that resilience (Resilience II) be a destiny of safety. However, in fact, Hollnagel et al. s safety theory as such has not considered the threat to the human health and life either. 2.2 The Existing Practice for Safety The existing safety engineering program and curriculum in institutions seems to be a relatively new major in North America and perhaps other places in the world. In general, focuses in the safety engineering program and curriculum in
North America are accident analysis (due to both machine and human errors) and software reliability. In the past, several unsafe disasters happened around the world. The first one may refer to Japanese Fukushima Daiichi (FD) nuclear disaster [23]. According to the report of Masayuki Nakao on the 2011 CIRP general assembly, several measures designed for accidents mitigation did not work in that event. The second example is the accident of chemical explosion and fire at Bartlo Packaging Incorporated (BPI) facility located in Arkansas in the United States in 1995 [5]. The fire caused toxic smoke, which further resulted in mass evacuation of residents a couple of miles away from the accident site. The third one is the loss of the Mars Polar Lander (MPL) and the two Deep Space 2 (DS2) probes in the United States [6]. The accident was such that the MPL and DS2 probes were launched and arrived at Mars. Communications ended according to plan. However, since then, communication could never be resumed. 2.3 Critique The existing safety science or theory evolves from reliability, to robustness, and to resilience, but none of them explicitly considers the factor of threats to the human life and health. All the paradigms for safety seem to assume: whenever a system fails, there will be a threat to humans anyway. This assumption is not true. In life cycle engineering, there is a phase called recycling, where a failed product is thrown away and recycled; clearly no unsafe event would happen in this case. The paradigm to think resilience as a destiny of safety in [3] is not adequate, nor is the Perrow s safety theory (which only considers reliability and robustness). In fact, safety is related to system failures as well as threats to the human health and life, and system failures can be caused by the fact that a system is not reliable, not robust, and/or not resilient. The first example of unsafe system (Japanese Fukushima Daiichi (FD) nuclear disaster) can be considered as a failure related to resilience, as it is a post-accident error. The second example can be considered as a failure related to reliability, as it is a pre-accident error. The third example can be considered as a failure related to robustness, as the failure occurs in operation, especially an error in the interface between two components or subsystems. As a final note, the Leveson s theory with STAMP technique [9] may look like it considers all the three, i.e., reliability, robustness, and resilience. However, the approach is much rooted in control engineering, excluding system design. It is perhaps true that the traditional thinking of design is much related to reliability; that is design for reliability. However, design for robustness and design for resilience are meaningful theories for systems [21, 23]. The next section presents a new paradigm for safety engineering. III. THE NEW PARADIGM FOR SAFETY ENGINEERING The new paradigm for safety engineering is composed of two assertions. Assertion I (definition): Safety considers both an artifact which in general includes a human operator (i.e., the artifact is a human-machine system) and a human who receives the effects from the artifact or human-machine system (Fig. 1). Let us call that human human effector or human factor. Safety is about threats to both the human operator and the human effector while the three parties (machine, human operator, and human effector) are interacting (Fig. 1). Fig. 1. Human-in-the-loop safety engineering. Machine Human Operator Threats Human Effector With the above assertion, failures of a system, say a car loses its breaking function, do not necessarily imply an unsafe situation, as there may be no driver in that car or if the car runs in a sufficiently wide field and there is no human effector in that field. Therefore, assessment of an artifact system for safety cannot be made with the artifact system alone. In other words, the assessment for safety must put human in the loop. Assertion II (3R-safety): Assessment of an artifact for safety must consider reliability, robustness and resilience of the artifact, as all of them are relevant to the failure of the artifact. The 3R refers to reliability, robustness, and resilience. Remark 3: Recently, Hollnagel et al. [4] give a new definition to resilience (let us call it Resilience III) as: Resilience is the intrinsic ability of a system to adjust its functioning prior to, during, or following changes and disturbances so that the system can sustain required operations under both expected and unexpected conditions. This definition in fact makes the resilience cover the reliability, robustness, and resilience (Resilience I), which is questionable. This is because each of them, reliability, robustness, and Resilience I, makes sense for their own distinct contribution to the system behavior, and sometimes there may be conflict in design and operation of the system for reliability, robustness, and Resilience I, and as such not separating the three may actually create an integrity problem with the system behavior. For instance, a system may be designed to be more modularized in order that the system is more resilient according to Resilience I [23], but the modularization tends to increase the number of interfaces among components and increase in the number of interfaces may further degrade the system s robustness as well as reliability. IV. THE NEW DISCIPLINE OF SAFETY ENGINEERING Any engineering discipline consists of definition of the discipline such as safety. Then the knowledge of discipline has the three categories: measurement, analysis, synthesis, and
operation. In this section, the three categories of knowledge on safety engineering are outlined. 4.1 Safety Assessment and Analysis Assessment or measurement for safety is the foremost important step to consider in the discipline of safety engineering. According to the aforementioned definition of safety, the safety assessment must consider (i) the state of the machine system based on the 3R-safety paradigm (i.e., reliability [1], robustness, and resilience [17]), (ii) the state of the human who receives the effect of the system in particular with respect to the human health and life [21], and (iii) the impact of the machine state on the human state. Several remarks can be made in the following. Remark 4: It is noted that various methods and techniques for assessment for reliability, robustness, and resilience available in the literature can be taken for this activity. It is further noted that the assessment may be subjective in nature and thus should follow the statistics approach in human factors engineering or the expert-opinion approach [11]. Remark 5: The safety measurement as described above differs from the existing one in that the existing safety measurement only considers (1) the reliability but neither the robustness nor resilience and (2) the machine side rather than both the machine and the human who receives the effect of the machine operation. Remark 6: Safety analysis refers to generation of the assessment scores for a given artifact system and human who receives the effect of the system operation. 4.2 Safety Synthesis and Operation Management Safety synthesis refers to the determination of the structure of a system (including the human operator) to achieve both the system function and safety requirement. According to the 3Rsafety paradigm, the operation of a system is also related to the safety. Therefore, the operation management is to determine the procedure of operation or process to meet both the system function and safety requirement for a given system. A couple of remarks are made in the following. Remark 7: The structure of a system here includes the machine (including the human operator) and the human who receives the effect of the machine operation. Remark 8: The procedure of the system operation includes the planning, scheduling and operation controller. The planning and scheduling activities determine the resources allocation and distribution temporarily and spatially. The control is similar to the conventional machine control. The inclusion of the planning, schedule and operation control is due to the shift of the reliability-based safety engineering to the 3R-safety engineering. In essence, the 3R-safety engineering includes not only the design (i.e., determination of the structure of an artifact system) but also the operation management (i.e., determination of the plan and schedule for the operation of an artifact system). An example of the design and operation management for safety based on the 3R-safety approach can be found in the emergency evacuation process [17]. 4.3 Integrated Design and Operation Management The new paradigm actually implies that safety is not only the business of design but also that of operation management. Indeed, resilience is closely related to operation management, planning and scheduling in this case [17]. Let us consider an operation of a system as a process system namely a plan or a schedule as a system that has the structure, behavior, function and so on [10, 22]. The structure of an operation or an operational system makes sense in that for example, a schedule includes the elements such as the time or timing information for a particular vehicle which carries victims or medical resources to leave the manufacturing site to the customer site [17]. It thus naturally comes to a methodology that the structure of an artefact system and the structure of an operational system should be integrated. This is especially true for a production system, as a production system has two subsystems: infrastructure subsystem and substance subsystem [23]. The performance of the production system is therefore an aggregated result of the performances of both subsystems. While the infrastructure subsystem has much to do with design of the artifact system in a conventional sense, the substance system is the business of planning and scheduling [23]. Therefore, integrated design and operation is expected to produce a further optimal solution to a system for a better performance and a better safety [17]. V. APPLICATION OF THE NEW SAFETY THEORY This section will demonstrate the usefulness of the new safety theory outlined in Section 4, in particular to the three accidents mentioned in Section 2.2. The goal is to illustrate how the artifact systems in these accidents can be redesigned to enhance their safety. With respect to the Japanese nuclear disaster, the reactor system has the function to shut down the reactor. There are three redundant approaches for this shut-down function, say A, B, and C. However, A, B, and C are decoupled, i.e., C B A, where means dependent on in this case. This dependency implies that three redundant approaches are in fact not completely independent. In particular, when A is unabled, B and C are unabled too, which is unfortunately the case in that event. With respect to the PBI disaster, a root cause analysis [5] shows that (1) an explosive chemical product was placed near to a heat source and (2) two chemical products (A, B) have opposite properties with respect to water (A is water reactive but B not). In a normal operation, the temperature of that heat source should be T but in that particular event, the real temperature was T+ΔT (ΔT is a tolerance due to some uncertain factors). Unfortunately, in that event, T+ΔT was over the threshold temperature, which caused product A to decompose and become explosive. Since A and B were at the same site, the approach to extinguish the fire with water was then not possible. With respect to Loss of MPL, one of the possible causes in the loss of MPL event is the premature shutdown of the decent engine which is further attributed to inadequate interface between two components [9]. This is perhaps further related to
both component physical degradation and unexpected operation condition drifts. Now let us see how the new safety theory, as outlined in Section 3 and Section 4, may improve the design and operation management to avoid or mitigate the foregoing disaster events. For the Japanese nuclear disaster, according to the new safety paradigm, resilience needs to be included in the safety measure for critical systems. Further, design for resilience, in particular, design of redundancy into the post-accident safety measure needs to follow Axiom 1 of the axiomatic design theory [15] that is to make redundant designs uncoupled. For the PBI disaster, it is related to all the R s (i.e., reliability, robustness, and resilience). For reliability, the spatial relation between the product and heat source should rationally be decided. For robustness, insensitivity to an unexpected increase of temperature of the heat source should be considered in the operation of the system that includes A, B, and heat source. For instance, there should be a facility to realtime monitor the temperature of the heat source and to realtime monitor the surrounding materials of the heat resource. Finally, a correct post-accident operation must be such that first of all, no water is used to extinguish the fire (again based on the uncoupling design principle) and second, isolate the heat source from A and B and use a substance C (if any) that does not create any conflicting effect to A and B. For the loss of MPL, according to the new theory for safety, design for robustness needs to be considered. This is helpful to make the MPL more robust with respect to the interface uncertainty. In short, the safety is not only about the design and reliability but the operation and robustness and resilience. VI. CONCLUSION This paper proposed a new paradigm for safety engineering along with an outline of the theory for design and operation of artifact systems for safety. There are two assertions in this new paradigm: (1) putting the human in the safety assessment loop and (2) integrating reliability, robustness and resilience (3R) in design and operation of artifacts for safety. Under the two assertions, safety is connected with reliability, robustness, and resilience. The discussions of the nature of safety and of the application of the proposed safety theory to the three accidents help to conclude that the proposed safety paradigm along with its theory is promising. Another contribution of this paper is to clarify the definition of resilience. According to the present paper, there are three definitions of resilience, namely Resilience I, II, and III. Particularly, Resilience III covers reliability, robustness, and resilience I, and the present paper argues that Resilience III is problematic, as it fails to recognize potential conflicts in systems in terms of reliability, robustness, and resilience. REFERENCES [1] R. Billinton, and R. N. Allan, Reliability Evaluation of Power Systems, 2nd ed., New York: Plenum Press, 1996. [2] Department of defense of USA, Standard Practice for System Safety, 2000. [3] E. Hollnagel, D. D. Woods, and N. Leveson (Eds.), Resilience Engineering: Concepts and Precepts, Hampshire: Ashgate Publishing, Ltd., 2007. [4] E. Hollnagel, J. Paries, D. Woods, and J. Wreathall, Resilience Engineering in Practice: A Guide Book, Ashgate Publishing, Ltd., 2010. [5] IEEE, Explainer: What Went Wrong in Japan's Nuclear... - IEEE Spectrum, spectrum.ieee.org/.../nuclear/, 2011. [6] JPL special review board, Report on the Loss of the Mars Polar Lander and Deep Space 2 Missions, JPO-D-18709, 2000. [7] H. Kitano, Biological robustness, Nat Rev Genet, vol. 5, no. 11, pp. 826 837, 2004. [8] T. R. La Porte, High reliability organizations: unlikely, demanding, and at risk, J Conting Crisis Man, vol. 4, no. 2, pp. 60-71, 1996. [9] N. Leveson, A new accident model for engineering safer systems, Safety Sci, vol. 42, no. 4, pp. 237 270, 2004. [10] Y. Lin, and W. J. Zhang, Towards a novel interface design framework: function-behavior-state paradigm, Int J Hum Comput Stud, vol. 61, no. 3, pp. 259-297, 2004. [11] X. Liu, A. Ghorpade, Y. L. Tu, and W. J. Zhang, A novel approach to probability distribution aggregation, Inf Sci, vol. 188, pp. 269-275, 2012. [12] PBS, EPA/OSHA Joint Chemical Accident Investigation Report, 1999. [13] C. Perrow, Normal Accidents: Living with High-Risk Technologies, Princeton University Press, 1999. [14] J. Rasmussen, Risk management in a dynamic society: a modelling problem, Safety Sci, vol. 27, no. 2, pp.183-213, 1997. [15] N. P. Suh, Axiomatic design theory for systems, Res Eng Des, vol. 10, no. 4, pp.189-209, 1998. [16] A. K. Verma, S. Ajit, and D. R. Karanki, Reliability and Safety Engineering, London: Springer, 2010. [17] J. W. Wang, F. Gao, and W. H. IP, Measurement of resilience and its application to enterprise information systems, Enterp Inf Syst, vol. 4, no. 2, pp.215-223, 2010. [18] J. W. Wang, W. J. Zhang, W. H. Ip, An integrated road construction and resource planning approach to the evacuation of victims from single source to multiple destinations, IEEE trans Intell Transp Syst, vol. 11, no. 2, pp.277-289, 2010. [19] J. W. Wang, H. F. Wang, W. J. Zhang, W. H. Ip, and K. Furuta, On a unified definition of the service system: What is its identity? Systems Journal, IEEE, vol. 8, no. 3, pp.821-826, 2014. [20] W. J. Zhang, Is resilience the destiny for safety management paradigm? Presentation at the Northeastern University of China, 2007. [21] W. J. Zhang, and Y. Lin, Principles of design of resilient systems and its application to enterprise information systems, Enterp Inf Syst, vol. 4, no. 2, pp.99-110, 2010. [22] W. J. Zhang, Y. Lin, and N. Sinha, On the function-behavior-structure model for design, Proceedings of the Canadian Engineering Education Association, 2011. [23] W. J. Zhang, and C. A.van Luttervelt, Towards a resilient manufacturing system, CIRP Ann Manuf TECHN, vol. 60, no. 1, pp.469-472, 2011.