COMPLIANCE WITH THIS PUBLICATION IS MANDATORY

Transcription

1 BY ORDER OF THE COMMANDER AFRL INSTRUCTION AIR FORCE RESEARCH LABORATORY (AFRL) 17 MARCH 2008 Scientific/Research and Development SCIENCE AND TECHNOLOGY (S&T) SYSTEMS ENGINEERING (SE) COMPLIANCE WITH THIS PUBLICATION IS MANDATORY ACCESSIBILITY: Publications and forms are available at RELEASABILITY: There are no releasability restrictions on this publication. OPR: AFRL/XP (Mr. William L. Nolte) Certified by: AFRL/XP (Dr. Kenneth W. Barker) Supersedes AFRLI , 20 April 2005 Pages: 25 This instruction implements Air Force Policy Directive (AFPD) 61-1, Management of Science and Technology, and is to be used in conjunction with Air Force Materiel Command Instruction (AFMCI) , Advanced Technology Demonstration Technology Transition Planning, Air Force Research Laboratory Instruction (AFRLI) , Program Baseline Development, and AFRLI , AFRL Laboratory Management Review (LMR) Process. This instruction provides direction to ensure that systems engineering (SE) is understood and implemented at every level. Both AFRL's program management and technology transition process is significantly enhanced with the application of SE. This instruction applies across the entire portfolio of AFRL Science and Technology (S&T) programs. Application of this instruction to basic research programs is optional to the director of each technology directorate and AFOSR. The level of SE effort is to be tailored to the needs of the individual S&T activity and customer expectations. Ensure that all records created as a result of processes prescribed in this publication are maintained in accordance with (IAW) AFMAN (will convert to AFMAN ), Management of Records, and disposed of IAW the Air Force Records Disposition Schedule (RDS) located at Refer recommended changes and questions about this publication to the office of primary responsibility (OPR) using the AF IMT 847, Recommendation for Change of Publication; route AF IMT 847s from the field through the appropriate functional chain of command. SUMMARY OF CHANGES This version deletes references to the dormant SE Steering Group and introduces the SE Council. It assigns overall SE responsibility in AFRL to the AFRL Chief Engineer (AFRL/XP, Deputy for Program Management and Systems Engineering). The relationship between the AFRL SE

2 2 AFRLI MARCH 2008 process and standard acquisition practices within the Department of Defense (DoD) is explained in Attachment 1. The column labeled What the Program Manager should know about his or her program of Attachment 2 includes the Supplemental Information for each applicable maturity level as recommended by the DoD TRA Deskbook available on the Defense Acquisition University web site: Background. AFRL S&T SE evolved from the AFRL Affordability Initiative. Under the AFRL Affordability Initiative, AFRL piloted approaches on Advanced Technology Demonstration (ATD) programs to fully implement a sound program management process that embodied SE principles. S&T SE builds on the lessons learned from the affordability pilots and applies both SE principles and affordability best practices to the entire AFRL portfolio of technology programs: Basic Research 6.1 (Figure A2.2.), Applied Research 6.2 (Figure A2.3.), Advanced Technology Development 6.3 (Figure A2.4.), to Advanced Technology Demonstration 6.3 (Figure A2.5.), Manufacturing Technology 7.8 (Figure A2.6.) efforts, and externally funded programs. Essential elements of S&T SE include early and continuous customer involvement, robust requirements development and coordination, selection of technology approaches that represent the best value solutions for customers, upfront technology transition planning, and development of program/management structures that focus on achieving required results (exit criteria) while accounting for and managing program risk. The SE activities in this instruction enhance the application of these lessons learned Objective. The objectives of the application of SE to AFRL S&T programs are: 2.1. Stronger AFRL program management and decision-making processes and competencies Alignment and integration of the AFRL technology transition process with those used by customer organizations Consistent delivery of technology products that represent best value solutions to needed warfighting capabilities. Best value is defined as the optimum balance of technology solutions that meet customer requirements and the transition risk associated with successful acquisition of those technology solutions Improved rate of successful technology transition to our customers SE Responsibilities AFRL Commander (AFRL/CC). Provides policy guidance and resources for the SE Initiative Commissions a standing corporate SE Council and a corporate SE working group (WG). The SE WG reports to the SE Council and assists the SE Council in developing and implementing the SE process within AFRL.

3 AFRLI MARCH In coordination with the Corporate Board, promotes specific SE Initiative procedures for AFRL-wide implementation Assesses SE implementation during program baseline reviews and recommends improvements Directors of Technology Directorates (TDs). Responsible for TD application of SE Appoint a member of the AFRL SE Council who assists in the development of corporate SE policy and the implementation of SE across AFRL and who has the authority to commit directorate resources to SE Appoint a directorate member to the AFRL SE WG Ensure that managers, at every level, fully incorporate SE evaluation criteria required by this instruction for program review and oversight activities to assess the level and quality of SE implementation Ensure a high level of competence in the application and execution of SE principles and practices across the directorate s work force Assess the success of directorate SE implementation through the SE evaluation criteria of Attachment 2, take corrective action when required, and identify potential SE procedure improvements to AFRL Plans and Programs (AFRL/XP). Use of the Attachment 2 criteria for Basic Research is optional Provide directorate support to SE tasks in support of technology development efforts AFRL Chief Engineer. Responsible for corporate-level direction, content, and status of the SE Initiative In coordination with the Corporate Board, directs the development of documents and champions the vision, policy, goals, objectives, strategies, plans, and activities of the SE Initiative Serves as the functional OPR for SE within AFRL Serves as chair of the SE Council Champions the use of SE principles and procedures throughout the AFRL work force Communicates with senior leadership in customer organizations to ensure customer awareness of the SE Initiative and verify that customer needs are being met.

4 4 AFRLI MARCH Recommends AFRL S&T SE improvements to the Corporate Board as required Appoints a secretariat to the SE Council Appoints an executive secretary for administering the activities of the Corporate SE WG AFRL SE Council. The SE Council is the AFRL corporate body responsible for improving and strengthening the culture and discipline of SE in all S&T programs within AFRL. The SE Council is comprised of the AFRL Chief Engineer; one senior SE officer from each TD, including Air Force Office of Scientific Research (AFOSR); and a secretariat appointed by the AFRL Chief Engineer. The SE Council will: Develop and recommend implementation actions for the policies, processes, and directives necessary to integrate SE principles into all AFRL S&T programs as appropriate Promote the adoption of tailored SE best practices, tools, and metrics across the AFRL S&T portfolio Define SE training, certification requirements, and position coding for AFRL program managers, scientists, and engineers Coordinate and integrate the AFRL SE process with AFMC product centers, sustainment and logistics centers, and other customers capability planning, road mapping, and platform-specific SE plans and processes Coordinate AFRL SE policies and processes with the Air Force Center for Systems Engineering Champion the use of SE to bolster Corporate Investment Strategy decisions based on a better understanding of customer requirements and an improved definition of technology challenges and options AFRL Corporate SE WG. An AFRL forum to support, sustain, and enhance the practice of SE. The SE WG is the working arm of the SE Council. Comprised of one SE officer from each TD including AFOSR and AFRL/XP, the SE WG carries out tasks assigned by the SE Council working through ad hoc sub-panels drawn from its membership. Its activities are administered through an executive secretary appointed by the AFRL Chief Engineer Assists the SE Council in developing and coordinating corporate SE Initiative policies, vision, goals, objectives, strategies, plans, and activities Fosters the implementation and institutionalization of SE principles/practices.

5 AFRLI MARCH Identifies SE best practices, procedures and tools, and makes them available for use across AFRL Documents SE accomplishments for communication to customers and stakeholders Provides networking/support for directorate SE efforts Develops and assesses SE metrics Represents AFRL at SE-related meetings and conferences Reports to the SE Council and assists the SE Council in developing and implementing the SE process within AFRL SE Best Practices. The DoD acquisition community recognizes 16 SE processes whose implementation represents SE best practice. AFRL S&Es are expected to understand the principles and concepts underlying these processes and appropriately size how they will be implemented on their S&T efforts. Attachment 1 overviews what these processes normally involve in acquisition efforts, what they are likely to involve within AFRL S&T programs, and why they are important to AFRL researchers. Additionally, each description refers to one or more key questions from paragraph These questions are the typical questions asked during a laboratory management review (LMR) to assess the sufficiency of the SE effort. Throughout this section, and especially in the key questions, the term customer requirements is meant in a broader than normal sense. As the expression is used here, requirements includes the evolving customer needs as they grow in definition and specificity while the technology matures. Early in technology development, during basic research, requirements are given as one or more research hypotheses. Later in development, requirements may be given as mission needs, desired capabilities, or a requirements document Key Questions. The key questions are the same across the entire AFRL S&T portfolio; however, the level of knowledge available to a program manager will change as a technology develops and matures. While every S&T program manager is expected to know the answers to these questions, the amount of information needed to document the answer to a particular question satisfactorily, i.e., during program baseline reviews, LMRs, and/or technical reviews, will change as a program progresses from a 6.1 effort through 6.2 to 6.3. Use of the key questions during reviews of basic research programs is optional to the director of each technology directorate and AFOSR. The SE key questions are: Who is your customer? What are the customer s requirements? How will you demonstrate you have met the requirements?

6 6 AFRLI MARCH What are the technology options? Which is the best approach? What are the risks to developing the selected technology? How will you structure your program to meet requirements and mitigate risk? What is your business-based transition plan that meets customer approval? 4.3. SE Assessment. SE Assessment standards (Attachment 2) contain detailed assessment standards for each of the SE key questions for AFRL S&T work units throughout all phases of development. When applied to a work unit during LMRs, the tool at Attachment 2 will designate the overall score for the work unit against SE evaluation criteria. Each TD shall evaluate SE performance as specified by paragraph Forms Prescribed Forms. None Adopted Forms. None. MICHAEL L. CARLSON Colonel, USAF Chief of Staff

7 AFRLI MARCH Attachment 1 TRANSLATING THE DEFENSE ACQUISITION GUIDEBOOK SYSTEMS ENGINEERING PROCESSES TO TERMINOLOGY REFLECTING AFRL PRACTICES A1.l. This attachment illustrates how the eight Defense Acquisition Guidebook (DAG) Systems Engineering (SE) technical management processes and the eight DAG SE technical processes are carried out in the AFRL S&T environment. The eight technical management processes are performed continuously and concurrently throughout the S&T effort, while the eight technical processes, which make up the SE Vee, are performed sequentially, although with considerable iteration up and down the Vee and feedback checking across the Vee. Figure A1.1. The Systems Engineering Vee Technical Processes Transition Requirement Development Validation Logical Analysis Verification Design Solution Integration Implementation Technical Management Processes Decision Analysis Tech Planning Tech Assessment Req ts Mgt Risk Mgt Config/Data Mgt Interface Mgt - Show Process - Assess Metrics - Trace - Plan - Plan - Show Approach - Review Tech - Document - Assess - Identify - Review - Rationale - Mitigate - Control - Integrate (IPT) - Monitor - Account -Verify/Audit A1.2. For each process, the process name as found in the DAG is given first, followed by a list of the AFRL SE Key Questions that apply to this SE process. Next is the process description from the DAG followed by a translation of the DAG verbiage to fit into an S&T context. Each process section concludes with a discussion of why this particular process is important to the AFRL researcher.

8 8 AFRLI MARCH 2008 A1.3. The eight SE technical management processes are as illustrated in Figure A1.2. The eight SE technical processes are as illustrated in Figure A1.3. Tailoring the AFRL Key Questions to fit the maturity of the technology and type of program will be covered in Attachment 2.

9 AFRLI MARCH Attachment 1 TRANSLATING THE DEFENSE ACQUISITION GUIDEBOOK SYSTEMS ENGINEERING PROCESSES TO TERMINOLOGY REFLECTING AFRL PRACTICES Figure A1.2. Defense Acquisition Guidebook s 8 Technical Management Processes Defense Acquisition Guidebook s 8 Technical Management Processes Technical management processes manage the technical development of the system increments, including the supporting or enabling systems. These processes are applied continuously at every step of the SE Vee and its Technical Processes. Note: Process is defined here simply as an organized set of activities and an identified set of individuals who carry them out Decision Analysis Key Questions 3, 5 DAG: Decision analysis activities provide the basis for evaluating and selecting alternatives when decisions need to be made. Decision analysis involves selecting the criteria for the decision and the methods to be used in conducting the analysis. Analyses include, but are not limited to, trade studies, models and simulation, supportability analysis, level of repair analysis, post fielding support analysis, repair versus discard, and cost analysis augmented with virtual and/or physical prototypes. Decision criteria will be influenced by interoperability constraints; size; transportability requirements; maintenance concept; affordability; reliability, availability, and maintainability goals; and schedule. AFRL: Decision analysis involves selecting the criteria for the decision and the methods to be used in conducting the analysis. Analysis must be conducted to help choose among alternatives to achieve a balanced, supportable, robust, and cost effective program. Analysis methods include trade studies, modeling and simulation, cost/benefit analysis, and the analytic hierarchy process (AHP). These studies should be augmented with virtual and/or physical prototypes, where applicable, prior to making decisions on best alternative. Decision criteria will be influenced by the level of technology maturity, the stability of requirements and goals, available funding, and the growth potential of the technology under review. Importance: R&D almost always involves making choices (e.g., determining which technology offers the best performance) or performing assessments for other people to make choices (e.g., recommending whether other researchers should bother with a certain line of inquiry). Viewing decision analysis in terms of a process prompts the researcher to consider which decisions are critical for the work to be successful and how to best make them such as by bringing in other experts, creating a decision table with weights and factors, using modeling and simulation, or any other method that is most appropriate. The more critical the decision the more rigor and resources that ought to be applied.

10 10 AFRLI MARCH 2008 Figure A1.2. Defense Acquisition Guidebook s 8 Technical Management Processes Technical Planning Key Questions 3, 4, 6, 7 DAG: Technical planning activities ensure that the systems engineering processes are applied properly throughout a system s life cycle. Technical planning, as opposed to program planning, addresses the scope of the technical effort required to develop the system. A mandated tool for this activity is the Systems Engineering Plan. Each of the technical processes requires technical planning. Technical planning for implementation, integration, verification, validation, and transition processes and their accompanying systems can reveal constraints and interfaces that will result in derived technical requirements. AFRL: Technical planning activities ensure that the technical activities are conducted properly throughout a system s life cycle. Technical planning, as opposed to program planning, addresses the scope of the technical effort required to achieve program technical goals. Key here is to have defined exit criteria, products, or deliverables that can be tracked with progress measured. Otherwise, the work is finished when the manager so states, but nothing of significance may have been accomplished. Technical planning can reveal constraints and interfaces that will result in derived technical requirements. Depending on the maturity of the technology under development, the researcher may contribute to the Systems Engineering Plan, which is owned and maintained by the acquisition activity. Importance: If one fails to plan, one plans to fail. Anon. Keeping this saying in mind prompts the researcher to consider all the activities needed, how long they take, how to best order them, what resources are needed for them and who is best suited to accomplish them. If it is complex, and more often than not, AFRL s research is technically complex, then it makes sense to write it down and get the input of others on it (in particular those who ve done similar activities before and those who will be doing it this time.) Technical Assessment Key Questions 3, 5 DAG: Technical assessment activities measure technical progress, technology maturity, and the effectiveness of plans and requirements. Activities include Technical Performance Measurement and the conduct of technical reviews. A structured review process should demonstrate and confirm completion of required accomplishments and exit criteria. Technical assessment activities discover deficiencies or anomalies that often result in the application of corrective action. AFRL: Technical assessment activities measure technical progress, technology maturity, and the effectiveness of plans and requirements. Activities include Technical Performance Measurement, Technology Readiness Assessment, and the conduct of technical reviews. The review process demonstrates and confirms completion of required accomplishments and S&T exit criteria. Technical assessment activities discover deficiencies or anomalies that may result in the application of corrective action. Importance: Too often, R&D efforts are not clear up front on what they must accomplish or they just continue despite not making any progress. If the technical planning was effective, technical assessment can also be effective. If not, technical assessment is not of much value. Using a technical assessment process gives the researcher a sound way to determine how to know what good means technically for the work at hand and, thereby, structure the research towards achieving it. Furthermore, it provides a means to assess the progress being made. Without thinking through what is good the researcher may miss what is important to the end customer or become fixated on less important factors. Similarly, without thinking about how to measure what is good one s research can potentially provide the wrong data for making an assessment on whether it has been successful. This does not mean the immediate cancellation of any work not meeting a metric, as even that is potentially informative. On the other hand, it does indicate the need to find out why results are not meeting projections, as it may mean it is time to write-up what has been learned to date and move on to something more productive.

11 AFRLI MARCH Figure A1.2. Defense Acquisition Guidebook s 8 Technical Management Processes Requirements Management Key Questions 1, 2, 3 DAG: Requirements management provides traceability back to user-defined capabilities as documented through the Joint Capabilities Integration and Development System. Requirements management maintains the traceability of all requirements from capabilities needs, documents all changes to those requirements, and records the rationale for those changes. AFRL: Requirements management provides traceability back to defined needs and goals. Requirements management maintains the traceability of all requirements from needs, documents all changes to those requirements, and records the rationale for those changes. Importance: We work too hard to be working on the wrong things. Using requirements management ensures research comes from a pedigreed source and any changes in direction given to the researcher are; first, recognized as being changes (versus slid by as clarification or a suggestion on what would be better ), second, the ramifications are understood, and, third, there is agreement to the changes before they are pursued. The greater the number of customers having requirements, the total number of requirements, the level of interdependence between the requirements, and the number of researchers involved the more important it is to have a structured means to manage research requirements so that it is clear to all who should be working on what. Risk Management Key Questions 6, 7 DAG: Risk management encompasses program cost, schedule, and performance risk identification, analysis, mitigation planning, mitigation plan implementation, and tracking. Risk management should begin at the earliest stages of program planning and continue throughout the total life cycle of the program. AFRL: Risk management encompasses program cost, schedule, and performance risk identification, analysis, mitigation planning, mitigation plan implementation, and tracking. Risk management should begin at the earliest stages of program planning and continue for the total duration of the R&D program. Importance: Look before you Leap Aesop s Fable about the Frog and the Water Well (if you have not heard it it does not end well for the frog). It is easier and often less of an impact to an R&D effort for a researcher to avoid a potential problem than deal with the impact should that problem comes to pass. Consequently, it makes sense to identify and address risks that through a combination of their consequence and likelihood pose the most serious potential problems. Simply thinking through how an effort is likely to encounter problems and accounting for it in an experiment plan may suffice for a well-experienced person working a familiar activity. However, as an effort becomes more complex, different from experience, or important the more sense it makes to take a structured approach. The effort to avoid potential problems will itself take valuable time and resources but that is why a structured approach is valuable as it allows the researcher to focus on those risks most in need of attention.

12 12 AFRLI MARCH 2008 Figure A1.2. Defense Acquisition Guidebook s 8 Technical Management Processes Configuration Management Key Questions 2, 3, 8 DAG: Configuration management (CM) is the application of sound business practices to establish and maintain consistency of a product s attributes with its requirements and product configuration information. Configuration management includes system hardware, software, and documentation (data). Activities result in a complete audit trail of decisions and design modifications, and facilitate the development of open systems. AFRL: CM is the application of sound business practices to establish and maintain consistency of a product s attributes with its requirements and product configuration information to the degree reasonable compared with the maturity/size/complexity of the program. CM is done primarily by the contractor for contracted efforts but needs to be in place and reviewed by the government program manager. For in-house research, CM is the responsibility of the government program manager. CM includes keeping a record of laboratory/experiment hardware configuration when measurements are gathered, software version and modifications used, and documentation (data) resulting from the experiment/demonstration. Activities result in a complete audit trail of decisions affecting laboratory equipment/software design modifications. Importance: When complexity, quantities, or changes cloud understanding, CM helps provide clarity to the researcher. With proactive consideration of CM, the researcher identifies what is important to know if it changes, or how it got changed, or what the state is at the moment, and then take steps to provide for that insight. For example when used in testing, it can help ensure a variance in results is only a response to the change variable wanted and not an unintended change in the test article. Data Management Key Questions 2, 7, 8 DAG: Data management (DM) consists of the disciplined processes and systems used to plan for, acquire access, manage, protect, and use data of a technical nature to support the total life cycle of the system. DM applies policies, systems, and procedures to identify and control data requirements; to responsively and economically acquire, access, and distribute data; and to analyze data use, enabling the sharing, integration, and management of data by government and industry. AFRL: In many cases of R&D, data is the only product generated by the program, and thus, it should be rigorously managed. Within AFRL, the R&D Case File process, as managed through the Defense Technical Information Center (DTIC), is the prescribed data management system. Importance: Research is about obtaining knowledge, which comes from gaining a new level of understanding from the available information, which itself comes from analyzing data. Consequently, thinking through what data is needed, how to obtain it, use it, and take care of it is fundamental to R&D. Actively thinking it through is the first step and may suffice for simple endeavors. Of course if the data issues involved are complicated due to the amount of data, the various types of data, their various locations, their difficulty to access or their difficulty to understand, then writing down the approach and getting it reviewed provides for the best chance of success.

13 AFRLI MARCH Figure A1.2. Defense Acquisition Guidebook s 8 Technical Management Processes Interface Management Key Questions 2, 3, 8 DAG: The interface management process ensures interface definition and compliance among the elements that compose the system, as well as with other systems with which the system or system elements must interoperate. Interface management control measures ensure that all internal and external interface requirement changes are properly documented. External interfaces are identified through the Joint Capabilities Integration and Development System process and its accompanying documents and architectures. Interface control functions enable integration of system and sub-systems; support system maintenance, future enhancement, and upgrades; and provide input data for continuous risk management efforts. AFRL: Interface management ensures interface definition and compliance among the elements that compose the laboratory system (internal interfaces), as well as with other systems with which the operational system or system elements might interact (external interfaces). Interface management control measures ensure that all internal and external interface requirement changes are properly documented in accordance with the configuration management plan and communicated to all affected elements of the program. It is a key enabler of open systems and interoperability. Interface definition is usually done by the contractor and overseen by the government. Interfaces must be defined in sufficient detail to facilitate necessary communication/interaction among systems, subsystems, and components. Importance: If a researcher has not contemplated what interfaces there are that must come together, the likelihood of success is very low. Keeping track of how elements come together is at best a task that grows linearly with the number of elements involved. In reality, it is worse than linear because one must often be concerned with several aspects of how two things must come together. For example, two sides of a connector could have mechanical, thermal, electrical, and logical features that must be appropriately considered if they are to mate and work together to transmit data. Adding to the difficulty is that one person or organization may not own both sides of the interface and the owning organizations may not be collocated for easy collaboration. Consequently, applying some organization to the tasks aids in identifying key interfaces, obtaining concurrence from both sides on the adequacy of the design, and ensuring there is coordination and consensus before either side changes. If a technology will be part of a system of systems or family of systems, Interface Management becomes simultaneously more difficult and more important than for stand-alone systems.

14 14 AFRLI MARCH 2008 Figure A1.3. Defense Acquisition Guidebook s 8 Technical Processes, aka SE V Defense Acquisition Guidebook s 8 Technical Processes, aka SE V Technical processes are used to design the program products, including the supporting or enabling elements. Eight technical processes make up the SE Vee. These processes are sequential, but iterative. The general flow is down the left side of the Vee and up the right side with continuous feedback across the Vee. Requirements Development Key Questions 1, 2, 7 DAG: The requirements development process takes all inputs from relevant stakeholders and translates the inputs into technical requirements. Together with the user, establish and refine operational needs, attributes, performance parameters, and constraints. Translate customer needs into program and system requirements such as performance parameter objectives and thresholds, along with affordability, schedule, and technical constraints. Provide technical support to market research. AFRL: The requirements development process takes all inputs from relevant stakeholders and translates the inputs into technical requirements. Good requirements are quantifiable, have unique definitions, and specified thresholds and objectives. Researchers work with the user to establish and refine goals, attributes, performance parameters, and financial and schedule constraints, and then ensure that all relevant requirements are addressed during the science and technology effort. Requirements development translates customer needs into S&T program and system requirements. Requirements provide technical support to the technology search at the beginning of the S&T program. Importance: Working with the user in the translation from user requirements into technical requirements is key if the eventual solution is to truly meet their needs. For example, what does fast mean? Fast as now? Faster than now? 20 percent faster than now? Or, 20X as fast as now? For researchers it is important to understand the user s requirements because technology may come from the other direction. In other words, if a technology would enable a 20X increase in speed, is that of value to the user?

15 AFRLI MARCH Figure A1.3. Defense Acquisition Guidebook s 8 Technical Processes, aka SE V Logical Analysis Key Questions 3, 4, 7 DAG: Logical analysis is the process of obtaining sets of logical solutions to improve understanding of the defined requirements and the relationships among the requirements (e.g., functional, behavioral, temporal). Once the logical solution sets are formed, performance parameters and constraints can be allocated and derived technical requirements defined. The Department of Defense uses functional analysis/allocation. The output of this process is the functional architecture that puts all of the functions in order, thereby, sequencing all of the system tasks that should occur. The function architecture provides the complete set of functions to be performed along with the relationships among the functions. AFRL: Logical analysis is the process of obtaining sets of logical solutions to improve understanding of the defined requirements and the relationships among the requirements (e.g. functional, behavioral, temporal). Once the logical solution sets are formed, performance parameters and constraints can be allocated and derived technical requirements defined. The Department of Defense uses functional analysis/allocation although behavioral analysis, timeline analysis, object-oriented analysis, data-flow analysis, a structured analysis may also be used. Logical analysis partitions a technical problem into self-contained, cohesive, logical groupings of elements and, where appropriate, defines the key interfaces. Importance: Logical analysis is a critical step between capturing what the customer needs technically and starting to design. Even the most basic item has some design elements and a holistic look at the full set of technical requirements enables overarching strategies for allocating them to the design elements. The more complex the item, the more considerations that come into play. These may include the value of a conventional or creative configuration, how long does the item perform various tasks, whether interfaces could be simplified and thereby reduce risk, or how industry is structured to provide items like engines and tires such that trying to lump the functionality of the engine and the tire together may create unwanted challenges (or represent new opportunities). Users want results, so, theoretically, how the effect is achieved is inconsequential; but in reality, it matters greatly to them how it impacts upon real world matters like doctrine, operations, training, manpower, leadership, personnel, facilities let alone whether it is affordable. Importantly for researchers, this same need for logically looking at the total problem also translates over to the researcher s own need to logically formulate the best project for the research at hand. Design Solution Key Questions 5, 6, 7 DAG: The design solution process translates the outputs of the requirements development and logical analysis processes into alternative design solutions and selects a final design solution. The process iterates with requirements development and logical analysis and integrates with the technical management processes to identify and select the best solution. The output of this process is the design or physical architecture sufficiently detailed to allow upward and downward traceability of requirements. AFRL: The design solution process translates the outputs of the requirements development and logical analysis processes into alternative technical solutions and selects a final technical path to explore. The process iterates with requirements development and logical analysis and integrates with the technical management processes to identify and select the best solution. The output of this process is the design or physical architecture sufficiently detailed to allow upward and downward traceability of requirements. Importance: With the elements identified, they need to be designed in detail to have the characteristics necessary. For the researcher the elements of the project also need to be detailed out. For example, the data collection method must be thought through and outlined so that it provides data that is useful for the planned method of data analysis.

16 16 AFRLI MARCH 2008 Figure A1.3. Defense Acquisition Guidebook s 8 Technical Processes, aka SE V Implementation Key Questions 4, 5, 7 DAG: Implementation is the process that actually yields the lowest level system elements in the system hierarchy. Depending on the technologies and systems chosen, the implementation process imposes constraints on the design solution. Implementation may also involve packaging, handling, and storage, depending on where or when the system element needs to be integrated into a higher-level assembly. Developing the supporting documentation for the system element such as the manuals for operations, maintenance, and/or installation are also a part of the implementation process. AFRL: Implementation is the process that yields the fundamental capability of the program. It can include some testing of the individual elements before they pass to integration. Developing supporting documentation for the system, such as the as-built configuration, or discovered limitations of the concept, is also a part of the implementation process. Importance: In the end, the elements identified must be brought into existence. How this is going to occur is an important consideration for the researcher. These choices must be considered in the earlier phases. If the researcher designed a test article did (s)he do so with the necessary detail (specifications and drawings and material selections) that someone else could fabricate it? If a piece of software is central to a researcher s test effort is it better to outsource it to experts or create it internally to allow quick modification? These are all questions that show it is important for the researcher to give due consideration to this process phase. Integration Key Questions 3, 5, 7 DAG: Integration is the process of incorporating the lower-level system elements into a higher-level system element in the physical architecture. The plan or strategy for the Integration process, including the assembly sequence, may impose constraints on the design solution. Integration also refers to the incorporation of the final system into its operational environment defined external interfaces. AFRL: Integration is the process of incorporating the lower-level system elements into a higher-level system element in the physical architecture. The plan or strategy for the integration process, including the assembly sequence, may impose constraints on the design solution Importance: Major issues can emerge if prior consideration is not given to how the parts come together as a whole or the activity is not managed well. This applies in research as much as in building operational systems. As a real case in point, a key component within a laser system was required to always be maintained within a very narrow temperature band. As a consequence, it was nearly impossible to transport and assemble, let alone later operate, thereby, making the whole R&D system ineffective as a solution. As another example, too often when classified data is needed to be used in research, it is not fully considered ahead of time, resulting in delays in gaining access for project personnel, limits on who can participate in the research, or an unintended rendering of the end research project classified, thus reducing its distribution.

17 AFRLI MARCH Figure A1.3. Defense Acquisition Guidebook s 8 Technical Processes, aka SE V Verification Key Questions 3, 7 DAG: The verification process confirms that the system element meets the design-to or build-to specifications. It tests the system elements against their defined requirements ( build-to specifications). The nature of verification activities changes throughout the system s life cycle; however, design solutions at all levels of the physical architecture are verified through a cost-effective combination of analysis, examination, demonstration, and testing, all of which can be aided by modeling and simulation. AFRL: The verification process confirms that the laboratory/experimental system element meets design specifications. It tests the system elements against their defined requirements (predicted versus experimental results). The nature of verification activities changes throughout the system s life cycle; however, design solutions at all levels of the physical architecture are verified through a cost-effective combination of analysis, examination, demonstration, and testing, all of which can be aided by modeling and simulation. Importance: Just as it is critical for the developer to determine if the system built performs as expected, it is important for researchers to assess whether the technology they developed performs as expected. However, if it does not work out in research despite best efforts, then it is a learning point. In science terms it is hypothesis testing. A strong verification effort enables the researcher or others to build upon the effort if done poorly then the contribution of the research is limited. It need not hinder exploration, but it does need to confirm what has indeed been learned. Validation Key Questions 2, 3, 7 DAG: The validation process tests the performance of systems within their intended operational environment with anticipated operators and users. In the early stages of the system s life cycle, validation may involve prototypes, simulations or mock-ups of the system, and a model or simulation of the system s intended operational environment. AFRL: Validation tests the performance of the technology against the original program goals. In the early stages of technology development, validation may involve simulations of a larger system that the program supports with test data in a simulated operational environment. Any testing results/data need to be captured so that they are available for further development/research/maturation efforts. Importance:.The researcher is not the end consumer and validation provides for an assessment of whether the work met the actual needs of that consumer whether it is an AFOSR sponsor or a product office needing a technology maturation effort. It is not uncommon for success to be claimed with some particular technology when its use would create a greater burden than the original problem trying to be solved. It should be an early consideration since it may need to shape the rest of the effort. For example, the results of an S&T effort may need to be validated in a proven model, and in that case, the data should be collected in a manner to enable its use within that model. It very well may be that once the user sees the results they determine that the technology is not what they should have requested.

18 18 AFRLI MARCH 2008 Figure A1.3. Defense Acquisition Guidebook s 8 Technical Processes, aka SE V Transition Key Questions 3,7, 8 DAG: Transition is the process applied to move the system element to the next level in the physical architecture or, for the end-item system, to the user. This process may include installation at the operator or user site. AFRL: Transition is the process that culminates with a supportable technology in the hands of the warfighter. The transition process is applied in a step-by-step manner to move the technology to the next level in the developmental cycle. The needs of follow-on phases should be considered early in the program and be included in all of the technical management processes. Importance: The effort is not complete until the product intended is provided to the intended user; therefore, this should be an important consideration when establishing the effort. A research effort may be supplying articles for other researchers to test, it may be design knowledge, or algorithms provided to a contractor to employ, it could be a prototype to support limited operations. Whether or not items are produced, all efforts represent an obtainment of knowledge that needs to be captured in a technical report and transitioned via DTIC for others to learn from

19 AFRLI MARCH Attachment 2 TAILORING THE SYSTEMS ENGINEERING PROCESS BY USING THE AFRL KEY QUESTIONS A2.1. This attachment demonstrates the tailoring of the AFRL SE process by considering the maturity of a technology under development during program baseline reviews and LMRs. As shown below in Figure A2.1., the amount of knowledge available to a researcher/program manager increases as the technology matures. This means that the depth of detail required to answer the key questions changes as a technology matures. A2.2. The tool that these images are taken from is available in Microsoft Excel format from any member of the SE Council or SE WG. Figure A2.1. AFRL Key Question

20 20 AFRLI MARCH 2008 A2.3. The remaining pages of this attachment show specifically the information needed to satisfactorily answer each of the key questions during or in preparation for baseline or LMRs for each type of R&D program, from Basic Research 6.1 (Figure A2.2.), Applied Research 6.2 (Figure A2.3.), Advanced Technology Development 6.3 (Figure A2.4.), to Advanced Technology Demonstration 6.3 (Figure A2.5.), Manufacturing Technology 7.8 (Figure A2.6.) efforts. Use of the key questions during reviews of basic research programs is optional to the director of each technology directorate and AFOSR. The supplemental information mandated by the DoD TRA Deskbook is included under What the program manager should know about his or her program on each worksheet.

21 AFRLI MARCH Figure A2.2. Basic Research (6.1) Question and Answer Matrix Use of the key questions during reviews of basic research programs is optional.

22 22 AFRLI MARCH 2008 Figure A2.3. Applied Research (6.2) Question and Answer Matrix

23 AFRLI MARCH Figure A2.4. Advanced Technology Development (6.3) Question and Answer Matrix

24 24 AFRLI MARCH 2008 Figure A2.5. ATD Question and Answer Matrix

25 AFRLI MARCH Figure A2.6. Manufacturing Technology (7.8) Question and Answer Matrix