Structured Natural Language Requirements in Nuclear Energy Domain

Transcription

1 Structured Natural Language Requirements in Nuclear Energy Domain Towards Improving Regulatory Guidelines Eero Uusitalo, Mikko Raatikainen, Tomi Männistö Department of Computer Science and Engineering Aalto University Helsinki, Finland Teemu Tommila VTT Technical Research Centre of Finland Abstract Requirements of a system are gathered from various stakeholders, but especially in safety critical application domains, such as the nuclear energy domain, public authorities also impose requirements. Major parts of requirements are often written in natural language. Despite being widely applied and a convenient means, natural language requirements have deficiencies such as impreciseness and vagueness. One approach to improve especially existing requirements is to rewrite the requirements applying structured natural language templates such as Easy Approach to Requirements Syntax (EARS). In this paper, we describe results of an initial quasi-experimental study of applying EARS to nuclear energy domain requirements. The initial results were assessed by stakeholders from public authorities and power companies, and show improvement and utility of applying EARS. Finally, we describe our planned future work to apply EARS to parts of Finnish public authority guidelines for nuclear safety (YVL), which are currently undergoing major rework. Keywords-Requirements engineering; authority requirements; regulatory compliance; nuclear energy; structured natural language requirements I. INTRODUCTION A variety of engineering domains that are seen as important considering, e.g., privacy, public safety and environmental aspects, are constrained by regulatory requirements on systems. However, ensuring compliance of a system with regulatory requirements is an issue that has been identified in requirements engineering [1]. One instance of a regulated domain is nuclear energy, which is strictly controlled by legislation and governing bodies due to the intrinsic risks associated with nuclear fission. There is currently a resurgence of activity in the nuclear energy field in Finland. Permits have been given for construction of new reactors resulting in approximately tripling the existing amount of nuclear power production, and modernization of existing reactors is ongoing. The increased activity on the field has resulted in a greater need of regulatory activities and guidance. The currently effective regulatory guides on nuclear safety have been found to be open to interpretation due to vagueness and ambiguity. As a consequence, the power companies are consuming disproportionate amounts of time and effort in ensuring regulatory compliance of the design of new and modernized reactors. Partly in response to these issues, STUK, the radiation and nuclear safety authority, is aiming to improve its regulatory guides on nuclear safety called YVL [2]. One goal of the improvement work is to express better safety requirements in YVL by reducing e.g. ambiguity, conflicts and vagueness. Our research project, which involves STUK and power companies that are responsible for the operation of the nuclear power plants (NPPs), contributes towards this goal. An aspect of the improvement is investigating the feasibility of applying structured natural language requirement format to YVL. The generic research problem is to improve RE in NPP taking into account the entire field. That is, on the one hand, this concerns both public authorities and power companies including their business partners. On the other hand, this concerns both activities that stakeholders carry out as well as the resulting RE statements. Thus, the objectives are to provide improvements that either public authorities or power companies, or both, have value or satisfactions of. The research method we apply is design science approach [3, 4]. This paper provides first steps towards this direction by introducing idea and initial results of structured NL requirements to requirements in nuclear energy domain. On the basis of the results we outline future research challenges and the solutions approach for the challenges. More specifically, this paper presents preliminary results as well as future plans on applying the Easy Approach to Requirements Syntax (EARS) [5] formatted natural language requirements approach into parts of the YVL. The preliminary results suggest application of EARS is feasible to some areas in the problem domain, the approach improves the safety requirements presented in YVL and that there is utility in using EARS Fourth International Workshop on Requirements Engineering and Law (RELAW 2011) /$ IEEE 67

2 TABLE I. EXAMPLES OF CURRENTLY EFFECTIVE YVL GUIDELINES [2]. YVL number Guide title Number of Last revision pages date YVL 1.0 Safety criteria for design of nuclear power plants YVL 2.0 Systems design for nuclear power plants YVL 2.1 Nuclear power plant systems, structures and components and their safety classification YVL 3.0 Pressure equipment of nuclear facilities YVL 5.5 Instrumentation systems and components at nuclear facilities This paper is organized as follows. Section II presents background to the subject matter. Section III describes the research method used. Results are presented in section IV and discussion in section V. Finally, future work is presented in section VI. II. BACKGROUND A. Nuclear power in Finland and the role of YVL After more than two decades of relatively static operation, the nuclear energy domain is currently undergoing a renaissance with several nations planning to increase production of nuclear energy. In Finland, the construction of one new reactor is underway, two permits for additional reactors are granted, modernization of automation systems is planned or ongoing in the four existing reactors, and several operators are expressing interest in additional reactors. Nuclear power in Finland is controlled by law and decrees. For the purposes of practical interpretation, STUK has developed the YVL guides [2] on the basis of law and decrees, which specify mainly safety requirements that public authority requires. The power companies are one key stakeholder group for the YVL guides, as the plants they own and operate must comply with safety requirements presented in YVL. Additional stakeholder groups include constructors of the plants, who are responsible for carrying out the construction work, and subcontractors used by the constructors. Finally, STUK uses YVL to verify the plant complies with the law and decrees throughout the life cycle of the plant. The YVL guides currently in effect date from 1986 to 2008 in their most recent revision, and have been subject to an evolutionary process where additions and modifications have been made as necessary. 74 different YVL guides are currently in effect. YVL covers all aspects of NPP construction, operation and life-cycle, including technologies, development processes and human aspects. Some examples of the currently effective YVL guides are shown in Table I. Some examples of YVL statements will be presented in Section 4 of this paper. Currently, a major reorganization and rewrite of YVL is taking place - a revolution rather than evolution. This work aims to, e.g., increase clarity and eliminate vagueness and ambiguity from YVL, thus resulting in better safety requirements for the various stakeholder groups. As a regulatory guide, the YVL has some characteristics that require attention. It cannot be in conflict with the law and decrees. Furthermore, the YVL cannot extend beyond the requirements of the law. The role of YVL is to provide detail and guidance on how to interpret the law in practice. At the same time, YVL must remain neutral to different kinds of design solutions so any particular provider s solution to an issue is not favored over any other solution that is compliant with the law, though in practice YVL often reflects existing and proven solutions. These characteristics mean that from a theoretical standpoint YVL could be seen thoroughly as a set of requirements documents, as it specifies how a system should behave without implementation detail or design decisions. In practice, due to partial roots in existing design solutions, YVL assumes the presence of certain design elements such as specific instrumentation. As a result of the aforementioned factors, the YVL contains both purposeful vagueness due to generic nature of the documents; and design and implementation details that are due to the specific nature of the application domain. One class of methods, namely formal requirements notations are claimed to be effective in specifying requirements for safety-critical systems due to their strengths such as facilitating unambiguous, rigorously analyzed requirements. One example of such an approach for the nuclear energy domain is the NuSCR notation [6]. However, formal methods are not seen as suitable for application in YVL due to issues discussed below. The needs of the various stakeholder groups and the characteristics of the guide mean YVL needs to be understandable to professionals that are involved in the nuclear energy domain. Furthermore, regulatory guides are practically always in natural language. Thus, the use of natural language for requirements is prevalent in the current guide and an apparent necessity in future revisions. B. Structured natural language requirements One way of bringing clarity and rigor into natural languages is using structured natural language. Examples of such approaches in requirements engineering include requirement boilerplates [7] and Easy Approach to Requirements Syntax [5]. 68

3 The basic idea behind the above examples to using structured natural language in requirements is the same. Various skeletons of sentences are provided, from which the most appropriate is selected for a given requirement. An example of the most basic form of skeleton is given below, in which a characteristic of a system is expressed: The <system name> shall <capability>. Following the selection of the appropriate skeleton, the statements or values are filled in to the appropriate places. As a result of this translation, the original requirements are modified to comply with the syntactic rules, sometimes producing more than one requirement from the original one. Finally, it is important to note that these approaches do not specify document structure as templates, but only syntax of individual requirement sentences. C. Easy approach to requirements syntax The structured natural language approach used in this work is the Easy Approach to Requirements Syntax (EARS), explained briefly in this section based on the work of Mavin et al [5]. EARS has been developed at Rolls- Royce Control Systems, and subsequently successfully applied to a variety of requirement documents related to complex, safety-critical systems, including regulatory safety requirement documents [8]. The method has been developed primarily for stakeholder requirements, as opposed to e.g. technical requirements. In EARS, the requirement skeletons are based on five different sentence templates and their combinations. The five templates are as follows: Ubiquitous requirements, which are always active Event-driven requirements, which are triggered by an event on the system boundary State-driven requirements, where the system response depends on the current state of the system Unwanted behaviors, which are used to cover undesirable situations Optional features, which are applicable only when a system includes a particular feature. An example of an event-driven requirement in EARS is given in the following, both as the skeleton and as the instanced requirement: When <optional preconditions> <trigger>, the <system name> shall <system response>. When a step counter changes, the rod control system shall provide audible feedback to the operator. combinatorial explosion can occur from a particularly complex original requirement. Additionally, inherently complex requirements will remain complex. The application of EARS properly often requires multiple iterations of processing the original and translated requirements to ensure consistency of the translation with the original requirements. There is no single correct way to apply the EARS skeletons to express a requirement often there are several approaches available, sometimes with subtly different implications with regards to the meaning. EARS does not impose a strict structure to any of the elements that are filled in during the translation. As an example, in the above event-driven requirement example, the system response could include any number of responses, in sparse or great detail. This means that throughout the translation work, characteristics of good requirements such as testability and unambiguity [9] should be considered. III. RESEARCH METHOD This research adheres to design science approach [3, 4]. The general overview and background of the application domain was captured as a descriptive account of the state of the practice in a pre-study [10] using a case study method [11]. On the basis of the background and findings, we initiated a new research effort adhering to design science approach to requirements engineering practices in nuclear energy domain which will be detailed below. The objectives of the research are to construct prescriptions that improve requirements engineering in NPP domain. Prescriptions adhere to a causal structure having the form of If B is desired goal in situation A, do X. The prescriptions are assessed in terms of truth and utility. Truth means that the desired goal is achieved and the intervention actually results in the goal. Utility means the stakeholders have value or satisfaction of achievement of the goal. Utility therefore includes assessing the goal and the intervention. In the case of goal, utility can be assessed as desired improvement or new situation. In the case of intervention, utility includes, but is not limited to, the effort spent in the intervention, and the knowledge or learning needed for carrying out the intervention. A. Research questions This paper studies three research questions, which are as follows: 1. Can the EARS approach be applied to the NPP domain? 2. Does application of EARS improve the subject documents? 3. Does application of EARS into the subject documents have utility? In EARS, all requirements are atomic, which means that translation of an original requirement can produce multiple EARS statements, and as a worst case scenario a 69

4 TABLE II. TECHNICAL DOCUMENTS THAT WERE UNITS OF RESEARCH AND THE TREATMENT THEY RECEIVED. Unit YVL 2.1 Safety classification of systems [2] YVL B.1 Safety Design of Nuclear Power Plant[12] Requirements Specification for Rod Control System Upgrade: A Generic Specification for Westinghouse Pressurized Water Reactors (RS-RCSU)[13] Processed pages / total Treatment Status pages 11/11 Single iteration of EARS Effective guideline 2/48 Single iteration of EARS Draft guideline 74/138 Two iterations of EARS White paper from Electric Power Research Institute B. Study design We carried out the initial study adhering to quasiexperimental study design using the one-group pretestposttest design [14]. The units, treatments, outcomes and roles are detailed below as well as the collected data. Additionally, the units and the treatment they received are listed in Table II. The units in question in the study were as follows. The first unit was YVL 2.1 Safety Classification of Systems [2], which is currently in effect. The second unit was limited parts of YVL B.1 Safety Design of a Nuclear Power Plant [12], which is in draft status and not in effect yet. The third unit was Requirements Specification for Rod Control System Upgrade: A Generic Specification for Westinghouse Pressurized Water Reactors [13] (hereafter referred to as RS-RCSU). This document is freely available as a generic specification for NPP control rod automation system. The treatment applied in this study was the application of the EARS approach as described in section IIC. In each application of the treatment, issues such as vagueness, interpretation and ambiguity were noted for each analyzed requirement and interpreted using supporting material where possible. A note of required interpretation was recorded in the cases where exact information was not found in the supporting material. For YVL 2.1 and YVL B.1, a single iteration of EARS was applied by identifying fragments of text containing requirement sentences and then translating their meaning into EARS format. For RS-RCSU, another researcher processed the document, identifying functional and safety requirements and collecting them in a table with reference to the original section. EARS was then applied into these requirements, using the original document as reference for clarification. The resulting EARS statement set was then compared to the original RS-RCSU, and differences in meaning were noted. The EARS sentences were then reprocessed into a set of EARS sentences that matched with the original intent of RS-RCSU. The outcomes that were examined in the study are whether the EARS can be applied to the selected units at all, whether the application of EARS improves the requirements, and whether the application of EARS has utility. The latter two outcomes are dependent on the first outcome, as improvement and utility cannot be examined if the treatment is not feasible. The roles in this study are as follows: The researcher, the first author of this paper, administered the treatment. The NPP domain was previously unfamiliar to the researcher, and while RE was familiar, the EARS method was not. The evaluation of the outcome of research question one, feasibility of treatment, was carried out by the researcher. The evaluation of the result of research questions two and three, improvement and utility, was carried out by an expert panel, assembled in a workshop. The panel consisted of 3 STUK representatives and 2 power company representatives, who all had strong foundation and prior knowledge in the problem domain, but little previous knowledge of approaches to structured natural language requirements. Additionally, 5 researchers with varying background in the NPP domain were present in the panel. In the workshop, the generic idea of structured NL requirements was presented to the panel, followed by more detailed presentations on the requirement boilerplate approach and EARS. Then, the research method, treatment, results of treatment and feasibility of treatment were presented to the panel. The collected data were opinions of the expert panel on both the treatment and the outcome. Responses of panel participants on treatment, feasibility, outcomes and utility were recorded as researcher notes. Logs of researcher hours spent in applying the treatment were also recorded. In addition, researcher notes on needed interpretation of requirements as well as subjective experiences of the researcher on both the feasibility of the treatment as well as carrying out the treatment were captured. IV. RESULTS The panel approved of the basic concept of structured natural language requirements, and the EARS approach was seen as promising. This was due to two factors: Encouraging results reported from application to authority requirements in another safety-critical application domain [8] and the relative simplicity of the approach, which was seen to promote the possible introduction of the approach within various stakeholders. 70

5 The feasibility, improvement and utility of the application of the treatment to the units of this research is described in the following sections. Table III gives an overview of the results for each unit. Table IV presents examples of text extracted from the original documents and respective EARS translations. A. Feasibility The feasibility of the treatment is described in this section with regards to the units receiving the treatment. The application of EARS into YVL 2.1 Safety Classification of Systems was not successful. The main reason for the failure was that the currently effective YVL 2.1 does not appear to represent accurate requirements as such. Instead, it describes various safety classifications that should exist in a NPP design, and presents various criteria which affect the classification of a system, but rarely states explicit cases when a system should belong in a classification. An example of such a statement is shown on the first row of Table IV. Consequently, the document does not appear to be a requirements document per se, as it does not specify what should be implemented [15] but rather what could be implemented. This is the likely reason for failure in the application of the treatment. The opinion of the expert panel on the result was, to quote one participant: <I am not> surprised at all that the method did not work on the YVL 2.1. This opinion was followed by discussion on the shortcomings on the current YVL. The application of EARS into the selected parts of YVL B.1 Safety Design of a Nuclear Power Plant was successful. The selected statements of the YVL B.1 were decomposed into atomic statements and translated into the format specified by EARS in a single-pass manner, that is, there was one iteration of EARS application. Thus, the treatment was feasible. The difference of YVL 2.1 and YVL B.1 is TABLE III. RESULTS OF FEASIBILITY, IMPROVEMENT AND UTILITY FOR EACH UNIT. Unit Feasibility Improvement Utility YVL 2.1 Fail N/A N/A YVL B.1 Success Yes Yes RS-RCSU Success Yes Unsure noticeable, where the latter is constructed more carefully into single statements of more rigorous expression. This is one key factor that positively affected the feasibility of treatment into YVL B.1. The application of EARS into functional requirements from RS-RCSU was successful. The original document contains properly expressed requirements sentences and thus it was relatively easy to apply the treatment into this document. B. Improvement of outcome The application of EARS to YVL B.1 was greeted with positive results by the panel. The presentation of the EARSformat requirements revealed that although the resulted requirements were a natural interpretation of the original requirement, the results had explicated some assumptions from the original document. The assumptions made by the researcher were not correct, and this was seen to highlight the ambiguity of the sentences of YVL B.1 that required domain knowledge to understand correctly. In practice, the treatment brought out unknown weaknesses and highlighted improvement needs in YVL B.1. An example is shown on the second row of Table IV. The noteworthy difference is that the original statement does not specify what system the statement pertains to. The researcher s translation assumes the requirement applied to the control system only. This assumption was false and noticed immediately by the panel, who agreed the original statement was lacking as it did not specify the target of the statement. As EARS forces identification of system name, these omissions become obvious. TABLE IV. EXAMPLES OF SENTENCES EXTRACTED FROM ORIGINAL DOCUMENTS AND THEIR EARS TRANSLATIONS. Source document Original statement EARS statement YVL 2.1 YVL B.1 RS-RCSU When a structure or component essentially affects a system s safety significance, or when the structure or component is needed to accomplish the system s safety function, the structure or component is assigned to the same safety class as the system itself. Less important system parts may be in a lower safety class or in Class EYT. On the other hand, individual components may be assigned to a safety class higher than the system itself, for example at points where the system connects to a system in a higher safety class. It must be possible for the operators to manually actuate the safety functions required in response to any specific event if they deem it necessary in order to ensure safety. Loss of function of the logic module must be detected and result in an Urgent Alarm on the operator s annunciator panel. N/A The control system shall allow the operators to manually actuate any safety functions. The rod control system shall detect loss of function of the logic module. When loss of function of logic module is detected, the rod control system shall create an urgent alarm on the operator s annunciator panel. 71

6 In general it was considered important that the guidelines of YVLs should be correctly interpreted even by a relatively novice person who does not yet possess strong domain knowledge. The above example demonstrated this is currently not the case. The application of EARS to RS-RCSU was seen to somewhat clarify the original expressions in the document, but the reaction of the panel was mild, probably because the document was not important to them. In addition, the panel was not familiar with RS-RCSU. Nevertheless, in the course of administering the treatment, the researcher identified hidden assumptions, missing and conflicting requirements as well as ambiguity in the original document. The panel agreed on the findings. C. Utility of application The panel s opinion was that there was utility in applying EARS into YVL B.1. The method appeared relatively lightweight in terms of effort and learning required as compared to the benefits of its application. Further trials of the method were hoped for in order to determine whether YVL B.1 could be improved in its entirety by the use of the method. Ideas for further work were brought up even to the extent that personnel of STUK should be trained in use of EARS. Nevertheless, more thorough evaluation of EARS in terms of feasibility and required effort was still hoped in order to receive further assurance of utility. Furthermore, STUK representatives expressed interest in receiving guidelines and training on EARS with the intent of in-house trials of the method. The utility of applying EARS into RS-RCSU remained more uncertain. Even though the results attained in improving the content of the document were promising, the panel was unsure whether these results could be generalized to their own requirements specifications. This uncertainty was due to the panel members not being familiar with the structure and content of RS-RCSU. Therefore, although the panel appreciated the results and believed that there were significant improvements, application of EARS to NPP requirements was not conclusively proven to have utility. Nevertheless, further examination of utility of EARS in the actual requirement specifications used by power companies was seen as relevant and remained a potential future research item after more thorough application to YVL B.1. V. DISCUSSION The results that have been achieved so far in this research work are promising; however it is important to remember that the results are currently very preliminary. Further research is necessary to gain more extensive results that can be evaluated more thoroughly. The application of structured natural language, and more specifically, the EARS approach, seems to have merit in authority requirements currently under investigation. Firstly, the accessibility of natural language to all stakeholders strongly favors its use in authority requirements over specialized notations. Secondly, using structured natural language bring some rigor and clarity into natural language sentences. Thirdly, EARS consists of a manageable set of five sentence templates and does not have further constraints on expression besides the required basic structures. This was seen to promote possible introduction of the method within various stakeholders. The application of the EARS method requires extensive analysis of the original requirements, which is reflected in improved quality of requirements. At this point it is unclear whether the actual analysis of the requirements is more important than their eventual expression in structured format. However, it should be noted that the current draft YVL guides have already undergone several iterations of improvement work. Despite the previous work, application of EARS highlighted ambiguities and assumptions from the draft guide very quickly. Consequently it appears that the method provides an approach that can be applied with relative ease. The method can provide guidelines that help analyze the requirements further. There are certain risks that need to be taken into account when considering the application of a requirements documentation approach based on templates. Firstly, EARS does not provide guidance on its own on how to write good requirements that are, e.g., testable, unambiguous and necessary. Thus practices that support writing good requirements are necessary regardless of adoption of EARS. Furthermore, Komssi et al. [16] have found there is a possibility that the use of specification templates can over a period of time lead to work that produces superficially attractive requirements that are low in content. The use of practices such as document inspections and walkthroughs can help mitigate these risks. Generalizability of structured natural language requirements in general and EARS in particular should also be studied more thoroughly. There are certain contextual variables about our setting that need to be taken into account such as that there already existed a relatively polished draft text that had undergone several iterations; and persons who had written the text are not, to the best of our understanding, requirements engineering experts. For example, applying EARS during initial documentation of newly elicitated requirements, and its application in new product development is still questionable and needs to be studied. It appears unlikely that the improvement of YVL will completely eliminate the need of communication between STUK, power companies and other various stakeholders with regards to safety requirements. Research performed in the software industry suggests communication links between the requirement owners and other stakeholders helps overcome deficiencies in requirements documentation, but better documentation also helps reduce the need of excessive communication into a more manageable level [17]. VI. FUTURE WORK As concrete next steps, we have undertaken the application of EARS to YVL B.1 draft in its entirety. The result of the work will be examined in both improvement and utility with the expert panel of key stakeholders in the NPP domain. We are also examining more objective measurements to assess the resulting EARS formatted requirements in addition to subjective expert opinions. 72

7 The application of EARS or other structured language requirements approaches in stakeholder requirements of Finnish NPPs is under consideration as well. More specifically, we are interested in examining requirements of control rod automation and control systems of Finnish NPPs and whether correspondence of the system with authority requirements can be improved by application of the structured natural language approach. Control rod automation and control systems are key systems in a NPP, as they provide control for the fission process in the reactor core by adjusting the position of the control rods through automatic or manual operation. Finally, we are also interested in examining whether EARS needs to be tailored or extended for the NPP domain and what kinds of supporting information, document templates and requirements metadata could be used in the NPP domain. ACKNOWLEDGMENT The work presented in this paper has been conducted in the SAREMAN research project, which is a part of the SAFIR2014 research program. REFERENCES [1] T. Breaux, M. Vail, and A. Anton, Towards regulatorycompliance: Extracting rights and obligations to align requirements with regulations, in IEEE International Conference on Requirements Engineering, 2006, pp [2] List of regulatory guides on nuclear safety (YVL). STUK. Available at Edilex Legal Information Service, (Accessed ) [3] S. Gregor and D. Jones, "The anatomy of a design theory," Journal of the Association for Information Systems, vol. 8, pp , [4] J. E. Van Aken, "Management research as a design science: Articulating the research products of mode 2 knowledge production in management," Br. J. Manage., vol. 16, pp , [5] A. Mavin, P. Wilkinson, A. Harwood and M. Novak, "Easy approach to requirements syntax (EARS)," in Proceedings of 17 th IEEE International Requirements Engineering Conference (RE'09), pp , [6] J. Yoo, S. Cha, C. H. Kim and D. Y. Song, "Synthesis of FBD-based PLC design from NuSCR formal specification," Reliab. Eng. Syst. Saf., vol. 87, pp , [7] E. Hull, K. Jackson and J. Dick, Requirements Engineering. Springer- Verlag New York Inc, [8] A. Mavin and P. Wilkinson, "Big ears (the return of easy approach to requirements engineering)," in Proceedings of the 18 th IEEE International Requirements Engineering Conference (RE 10), pp , [9] A. M. Davis, Software Requirements: Objects, Functions, and States. Prentice-Hall, Inc. Upper Saddle River, NJ, USA, [10] M. Raatikainen, T. Männistö, T. Tommila and J. Valkonen, "Challenges of Requirements Engineering: A Case Study in Nuclear Energy Domain," To be Published in the Proceedings of the 19th IEEE International Requirements Engineering Conference, [11] R. K. Yin, Case Study Research: Design and Methods. Sage Publications, Inc, [12] List of regulatory guides on nuclear safety (YVL) in draft status. STUK. (Accessed ) [13] Requirement Specification for Rod Control System Upgrade: A Generic Specification for Westinghouse Pressurized Water Reactors. EPRI, Palo Alto, CA, [14] W. R. Shadish, T. D. Cook and D. T. Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton, Mifflin and Company, [15] I. Sommerville and P. Sawyer, Requirements engineering: a good practice guide, John Wiley & Sons, [16] M. Komssi, M. Kauppinen, K. Toro, R. Soikkeli and E. Uusitalo, "Lessons Learned from Integrating Specification Templates, Collaborative Workshops, and Peer Reviews," Requirements Engineering: Foundation for Software Quality, pp , [17] E. J. Uusitalo, M. Komssi, M. Kauppinen and A. M. Davis, "Linking requirements and testing in practice," in Proceedings of the 16 th IEEE International Requirements Engineering Conference ( RE'08), pp ,