bu`bomq=colj=qeb mol`bbafkdp== lc=qeb== pbsbkqe=^kkr^i=^`nrfpfqflk== obpb^o`e=pvjmlpfrj== tbakbpa^v=pbppflkp== slirjb=f=

Transcription

1 NPS-AM bu`bomq=colj=qeb mol`bbafkdp== lc=qeb== pbsbkqe=^kkr^i=^`nrfpfqflk== obpb^o`e=pvjmlpfrj== tbakbpa^v=pbppflkp== slirjb=f= Acquisition Research Creating Synergy for Informed Change May 12-13, 2010 Published: 30 April 2010 Approved for public release, distribution unlimited. Prepared for: Naval Postgraduate School, Monterey, California = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 1=

2 Report Documentation Page Form Approved OMB No Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE MAY REPORT TYPE 3. DATES COVERED to TITLE AND SUBTITLE Illustrating the Concept of Operations (CONOPs) Continuum and Its Relationship to the Acquisition Lifecycle 5a. CONTRACT NUMBER 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Analytic Services Inc,2900 South Quincy St. Suite 800,Arlington,VA, PERFORMING ORGANIZATION REPORT NUMBER 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR S ACRONYM(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited 11. SPONSOR/MONITOR S REPORT NUMBER(S) 13. SUPPLEMENTARY NOTES 7th Annual Acquisition Research Symposium to be held May 12-13, 2010 in Monterey, California. U.S. Government or Federal Rights License 14. ABSTRACT Though consistently noted as critical to successful system design and implementation, the Concept of Operations (CONOPs) artifact appears to be underutilized. This report demystifies the CONOPs artifact. It delves into the barriers that prevent optimal use of CONOPs and presents a framework for incorporating an?integrated? CONOPs into the Defense Acquisition Lifecycle. 15. SUBJECT TERMS 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT a. REPORT unclassified b. ABSTRACT unclassified c. THIS PAGE unclassified Same as Report (SAR) 18. NUMBER OF PAGES 48 19a. NAME OF RESPONSIBLE PERSON Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18

3 The research presented at the symposium was supported by the Acquisition Chair of the Graduate School of Business & Public Policy at the Naval Postgraduate School. To request Defense Acquisition Research or to become a research sponsor, please contact: NPS Acquisition Research Program Attn: James B. Greene, RADM, USN, (Ret.) Acquisition Chair Graduate School of Business and Public Policy Naval Postgraduate School 555 Dyer Road, Room 332 Monterey, CA Tel: (831) Fax: (831) jbgreene@nps.edu Copies of the Acquisition Sponsored Research Reports may be printed from our website = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 2=

4 Illustrating the Concept of Operations (CONOPs) Continuum and Its Relationship to the Acquisition Lifecycle Robert Edson Robert Edson is Vice President for Enterprise Development at Analytic Services Inc., and Director of the Applied Systems Thinking Institute (ASysT). He is responsible for strategic and transformational programs within the corporation. In his role as Director of ASysT, he leads an institute whose mission is to advance the application of systems-thinking principles in the fields of national security and homeland security. Robert is an Adjunct Professor at Stevens Institute of Technology and has a BS in Biology from George Mason University and a MS in Physical Oceanography and Meteorology from the Naval Postgraduate School. Robert Edson Analytic Services Inc, (ANSER.org) The Applied Systems Thinking Institute (ASysTi.org) 2900 South Quincy St. Suite 800 Arlington VA, Phone Robert Edson@anser.org Jaime Frittman Jaime is a Senior Analyst with Analytic Services Inc. Jaime began her career in the United States Air Force working in the field of Readiness and Emergency Management. Jaime applied her knowledge of Chemical, Biological, Radiological, Nuclear (CBRN) Warfare Defense and Response to the development and acquisition of CBRN Defense Weapon Systems. Jaime has a Master s of Engineering in Systems Engineering from Stevens Institute of Technology. Jaime Frittman Analytic Services Inc, (ANSER.org) The Applied Systems Thinking Institute (ASysTi.org) 2900 South Quincy St. Suite 800 Arlington VA, Phone Jaime.Frittman@anser.org Abstract Though consistently noted as critical to successful system design and implementation, the Concept of Operations (CONOPs) artifact appears to be underutilized. This report demystifies the CONOPs artifact. It delves into the barriers that prevent optimal use of CONOPs and presents a framework for incorporating an integrated CONOPs into the Defense Acquisition Lifecycle. Introduction The ability of development programs to avoid challenges associated with schedule, budget, and technical performance has been consistently poor (Turner, Verma & Weitekamp, 2009, p. 7). A recent FAA sponsored study noted that in order to avoid these pitfalls, one of the most significant artifacts is the creation of a CONOPs (Turner, et. al., 2009, p. 27). The report further noted the need to have alignment between the evolving = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 238=

5 CONOPs, the [Enterprise Architecture], 1 and the governance system (Turner et al., 2009, p. 32). The Manual for the Operation of The Joint Capabilities Integration and Development System (JCIDS, 2009) provides an illustration of the alignment of the enterprise architecture and the governance system by connecting JCIDS activities with milestone decisions. While important, this illustration is missing the alignment of a critical system success component, the CONOPs document. In order to encourage successful system development and acquisition we must understand the context of the CONOPs as it relates to the larger total acquisition lifecycle. Research Goals and Objectives This research informs the acquisition development lifecycle process by articulating the importance of the CONOPs Acquisition relationship and by illustrating how various CONOPs documents are introduced at critical points in the JCIDS development timeline to create a more robust and integrated concept of operations. Goals of the research include: Define the various CONOP types Explain the relationship of system-level CONOPs to acquisition activities Assess the current alignment of CONOPs and CONOPs-related documents with DoD acquisition governance and enterprise architecture processes Explore the maturity phases of CONOPs documents Document the relationship of each instantiation of the CONOPs to acquisitionrelated activities Assess the use of CONOPs and the disconnect, if any, between the perceived importance of CONOPs and the actual utilization of CONOPs. Methodology This research was conducted by combining traditional research methods with systems thinking tools and practices. Traditional analysis included literature review, data analysis, and comparative analysis. The Conceptagon 2 framework for systems thinking was applied to the research data. This framework encourages holistic system analysis by providing a series of seven triplets related to specific system characteristics. Use of the Conceptagon provided insight into interior and exterior boundaries, information flows, hierarchies, and other relevant system characteristics. Though the individual sets of triplets are not explicitly discussed in this paper, each of the seven triplets served as a cornerstone for consideration of system characteristics throughout this research. 1 An enterprise architecture (EA) describes the fundamental organization of a complex program as a minimum, the EA relates the requirements, resourcing (funding), acquisition system and the program office within an agency and the overall business framework of key stakeholders (Turner et al., 2009, pp ). 2 The Conceptagon is a systems thinking framework introduced by Boardman and Sauser (2008). For additional information on the Conceptagon as a systems thinking tool, reference Systems Thinking: A Primer (Edson, 2008) available at = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 239=

6 Literature Review. A literature review of documents related to the role of CONOPs was conducted. This review included documents published in industry, in professional journals, acquisition journals, and in Department of Defense (DoD) regulations, instructions, and publications. The literature review also included the Defense Acquisition University Website which provided access to publications, communities of interest, and ask a professor question and answer forums. In addition to existing literature, a questionnaire related to the use and usefulness of CONOPs was developed and distributed (see Appendix A). The pool of survey respondents was too small to enable the extraction of valid conclusions. To overcome the lack of respondents, results of the survey were compared to a similar survey 3 on the same subject. Data Analysis. Information collected during the literature review was assessed for: Terms used Purpose Relationship to acquisition activities Relationship to integrated CONOPs This assessment was instrumental in establishing a baseline for the CONOPS artifact and its use within the development and larger acquisition process. Terms Used and Purpose of the Document. For the first assessment, a broad search of terms used synonymously with CONOPs was conducted. The initial assessment covered an array of CONOPs documents, looking at CONOPs that describe the actions of a military force or organization as well as CONOPs that detail characteristics of a system from an operator s point of view. The intent of this assessment was to determine consistency of the meaning and purpose of the term CONOPs and to identify terms used in place of CONOPs. Once a set of recurring terminology was identified, the intended purpose of each document was recorded. This allowed us to assess similarities and variances associated with each of the terms. This assessment also gave us insight into role of CONOPs, if any, in acquisition activities as well as any barriers to the use of CONOPs. Relationship to Acquisition Activities. Variances among purpose and meaning were detected in the initial assessment. To account for the variance, each CONOPs-related document was plotted on a JCIDS-Acquisition relationship diagram. This enabled us to visualize different points of input and influence of each of the identified CONOPs-related documents. Using this assessment we further identified three distinct phases of CONOPs development that directly correspond to acquisition related activities. Relationship to Integrated CONOPs. Appearance of CONOPs-related documents in the JCIDS-Acquisition timeline revealed that CONOPs, either in an integrated form or in several smaller instantiations, occurred across the entire acquisition lifecycle. These documents (some termed CONOPs, others operating under a different name) were then 3 The Roberts survey, conducted in 2008, inquired about the use, usefulness and upkeep of CONOPs. Roberts survey had a larger pool of respondents numbering 108 responses from 18 different companies. This pool significantly outnumbered the 6 responses gained from our own survey. Unlike the Roberts survey which was sent to engineers, and was composed of system engineers, lead system engineers, test engineers, design engineers, and project managers (Roberts, 2008), our pool of respondents included members of the user community, which offered an additional perspective to data gained from the Roberts survey. = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 240=

7 assessed for their similarity to an integrated ConOps document spanning the full acquisition lifecycle. 4 Systems Thinking. Analyzing system characteristics by use of the Conceptagon provided a comprehensive view of the acquisition lifecycle. Each set of triplets was considered as we looked at each aspect of the project. To illustrate, as we looked at the landscape of the system (i.e. governance, enterprise architecture, and CONOPs) we considered the triplet of wholes, parts, relationships. The larger acquisition system which included all three primary elements of the landscape was the whole, individual processes and inputs to the processes were considered parts, and the purpose of each input, and its effect on the whole constituted the relationships. The Value of a CONOPs to System Development. The value of a CONOPs to system development is multi-faceted wherein the CONOPs plays a role across the entire life-cycle: from need identification, to system inception and development, to system disposition and disposal. Our research of literature, standards, and instructions indicates a number of ways in which the CONOPs adds value to acquisition and system development processes. Some of the key ways in which a CONOPs adds value are provided in Table 1. Table 1. Value of the CONOPs Under-Utilization of the CONOPs CONOPs Value Helps scope the problem & solution Bridges where we are and want to be Illustrates how a system will function Facilitates communications among stakeholders Provides a logic trail of capability Provides baseline for measuring system efficacy Provides basis for requirements Despite its value, the CONOPs, at least in its full form, is not consistently used in system development. In fact, a recent survey showed that 1/3 of all programs queried did not have a CONOPs (Roberts, 2008, p. 39). Similarly, in a series of interviews and surveys conducted for this research, the majority of respondents indicated that a CONOPs was critical to the system s success but was under-utilized. Comparable studies on CONOPs have pointed out that even when a CONOPs is written it is often after the system is developed and done so in an effort to satisfy a Milestone Decision requirement; this boxchecking activity strips the CONOPs of its intended role in the creative process (Nelson, 2007, pp. 5-6). Our survey results appear to support this, with our respondents indicating that a concept for how the system will be employed is usually written, but it is written after the system is developed. This means the CONOPs is based on the requirements as opposed to the requirements being based on the CONOPs. Similarly, in the Roberts survey, 4 For the purpose of this paper, the IEEE format for ConOps (IEEE, 1998) is representative of an integrated CONOPs. The IEEE nomenclature for a concept of operations is ConOps as opposed to CONOPs. = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 241=

8 18% of respondents said that CONOPs on programs they worked were not completed until after the requirements were complete (Roberts, 2008, p. 28). With the CONOPs document seen as critical to defining and employing a proposed system, why is it that the CONOPs is often missing or developed as an afterthought? Barriers to Effective CONOPs Use Our research indicates that there are four barriers to the use of CONOPs throughout the acquisition lifecycle. These barriers include: 1. Definition and Purpose. There is variance in the term used to describe a CONOPs document, as well as an inconsistent application of the term. Often, this results in misunderstanding of what a CONOPs is, how it is used, and what type of information it should contain. 1. Targeted Audience. Closely tied to the variance in definition and use, the intended audience of the CONOPs document is unclear. 2. Timing and Placement in the Acquisition Development Lifecycle. There is confusion regarding to what phase of development a CONOPs applies. 3. Comprehensive View and Consistent Involvement by Stakeholders. Many forms of the CONOPs document are just a small subset of what system development really needs these subsets do not incorporate a complete view of the system. Additionally, many of these CONOPs are by various authors using different stakeholder sets. Definition and Purpose. Our study detected considerable variance in the application of the term CONOPs. As a result of the variance, misunderstanding and purpose are major barriers to the use of CONOPs. This variance makes it unclear what a CONOPs is, how it should be used, by whom it should be used, and when it should be used. Military Concepts and System-Level CONOPs. Within the Department of Defense (DoD) there are higher-order and lower-order CONOPs. Higher-order CONOPs, include Capstone, Institutional, Operating, Functional, and Integrating concepts, which, in descending order, become more narrow in scope and more detailed by applying to a smaller mission set (for clarity we will term higher-order CONOPs military concepts for the remainder of this document). These concepts describe the conduct of military action at the operational level of war (Schmitt, 2006, p.1). Military concepts are derived directly from military strategy and provide a premise for the future capabilities the military will need. According to Joint Publication 1-02, these CONOPs are a verbal or graphic statement that clearly and concisely expresses what the joint force commander intends to accomplish and how it will be done using available resources (JP1-02, p. 114). The materiel capabilities needed to achieve the goals of the military concepts are described in lower-order, system-level CONOPs. The system-level CONOPs band includes capability- specific CONOPs. According to the Institute for Electrical and Electronics Engineers (IEEE), these CONOPs are a user-oriented document that describes system characteristics for a proposed system from the user s viewpoint (IEEE, 1998, p. i). = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 242=

9 Additional CONOPS Variance. Within both military concepts and system-level CONOPs there are several types of documents all of which are called either CONOPs or some variation of the term CONOPs. Joint Service and Component Service publications 5 have already defined and documented the different types of military concepts (i.e., institutional, operating, etc). However, the different types of system-level CONOPs are less well defined and vary from publication to publication. Adding to the confusion is the fact that each CONOPs document may include similar or dissimilar information. 6 The Perceived Purpose of the CONOPs Document. As discussed above, the purpose of a CONOPs can range from describing aspects of a military operation (military concept) to describing characteristics of a specific system (system-level CONOPs). But even within system-level CONOPs, the purpose can range from describing all system attributes, system stakeholders, and system tasks, to focusing solely on the employment of the system. Examples of different CONOPs document names and associated purposes are provided in Table 2. 5 Types of military concepts are defined in publications such as the Chairman of the Joint Chiefs of Staff Instruction Joint Operations Concept Development System ; the Air Force Policy Directive Air Force Concept Development ; CONOPs TO DOCTRINE: Shaping the Force From Idea Through Implementation (Fleet Forces, 2004). 6 Daniels and Bahill (2004) point out that CONOPs documents are rarely consistent in content, detail, and format (para. 4, p. = 307). do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 243=

10 Table 2. Perceived Purposes of CONOPs Term Purpose Reference User CONOPs System CONOPs ConOps CONOPs ConOps Concept of Employment CONOPs Use-Case Scenarios [Defines] basic system requirements. [It] describes what the user wants the system to achieve and the context in which the system will be utilized [Defines] how the system will actually be used and provides insight into the total system solution for both short-term and long-term requirements. Similar to ANSI/AIAA 7 OCD Provides the user community a vehicle for describing their operational needs that must be satisfied by the system under development Provides a capability description (what an operational unit does) or a prescription of what should be done. [Transforms] the allocated what to the how and so completes a chain all the way to an instantiation of the system that enables capabilities. Description of how a weapon system will be [used] in a combat environment [Provides] the vision and intent for how the system should work within an operational environment in an easy to read format. Similar to a CONOPs (see preceding CONOPs definition) but less ambiguous and therefore can be used directly for extracting requirements in an unambiguous way. Companion & Mortimer, n.d. Companion & Mortimer, n.d. Jost, 2007, p. 14 Nelson, 2007, p. 2 Nelson, 2007, p. 2 Ask a Professor (AAP, 2009 Daniels & Bahill, 2004, p. 306, sec 4.1 Daniels & Bahill, 2004, p. 307, sec 4.1 Targeted Audience. Just as the name for and content of a CONOPs document varies, so may the intended audience. Currently the audience is dependent on who is writing the CONOPs, what type of CONOPs is being written, and its intended placement within the acquisition timeline. The CONOPs can be written to speak to all communities or can focus on individual communities, such as engineers, customers, test agencies, or decision authorities. A CONOPs that only speaks to a specific community may be problematic in its potential to lack sufficient detail required by the unaddressed audience. Timing and Placement in the Acquisition Lifecycle. The placement of the CONOPs refers to the insertion point of the CONOPs into the larger acquisition activity. The 7 ANSI/AIAA is an abbreviation for the American National Standards Institute/American Institute of Aeronautics and Astronautics. ANSI/AIAA standard G , Guide for the Preparation of Operational Concept Documents, discusses information that relates to system operational concepts and describes which types of information are most relevant, their purpose, and who should participate in the effort (ANSI/AIAA, 1993, abstract). = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 244=

11 input of the CONOPs into the acquisition process is dependent upon the identified purpose of the CONOPs document. For instance, a military concept will occur earlier in the acquisition lifecycle than a system-level CONOPs. With the relative importance of the CONOPs widely understood (see Table 1), it is difficult to envision proceeding through the acquisition lifecycle without some form of the CONOPs document. To that end, we believe that although not necessarily called a CONOPs, and not in a neat and integrated package, critical elements of the CONOPs are occurring in an ad-hoc manner across the acquisition timeline. Nelson echoes this belief, stating that, [the] main reason we overlook the central role and scaling of the ConOps is that we give different names to the same thing at different scales (Nelson, 2007, p. 9). CONOPs Placement According to Official Literature. In DoD acquisition documents, such as the DoD 5000 series and JCIDS documents, CONOPs are identified as an input to a larger acquisition process. In these documents the term CONOPs on its own usually refers to a military concept. As such, it is placed as a precursor to system concept selection. Figure 1 provides an organizational construct for the position of the military concept to the JCIDS timeline. This construct depicts the hierarchy of military concepts as a linear progression from broad to most narrow focus (left to right). These military concepts then feed into the JCIDS-Acquisition process as a basis for the Capabilities Based Assessment (CBA). Figure 1. Relationship of the Military Concept to JCIDS-Acquisition (Modified from JCIDS, 2009) The Manual for the Operation of JCIDS references at least two more types of CONOPs. The first of these is found in Enclosure B of the JCIDS Manual. Here, the reference is to a sustainment CONOPs (JCIDS, 2009, p. B-B-5). The manual states that as a sustainment key performance parameter (KPP) metric, the sustainment CONOPs should be traceable to the system s Initial Capability Document (ICD) and Capability = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 245=

12 Development Document (CDD). This implies that the sustainment CONOPs is an input to the acquisition lifecycle process after the ICD and CDD have been written 8. A second reference to CONOPs is made later in Enclosure B. This time, the reference is to a CONOPs for the system. Contextually, this CONOPs for the system, or System CONOPs, provides documentation of a comprehensive analysis of the system and its planned use, including the planned operating environment, operating tempo, reliability alternatives, maintenance approaches, and supply chain solutions (JCIDS, 2009, p. B-B- 6). Based on this description the JCIDS System CONOPs is similar to the latter half of the IEEE ConOps. This System CONOPs is an input to the JCIDS process post Milestone B, upon program initiation. JCIDS also acknowledges the analysis of alternatives (AoA) that is part of the larger acquisition process. The AoA is a precursor to the Milestone A decision. The analysis of alternatives, though likely more detailed than what is included in the CONOPs, corresponds to the IEEE ConOps Alternatives section, which discusses system alternatives considered but not selected. Figure 2 provides a rough illustration of the relationship of these documents (to include the concept of employment (COE), which is recognized by DAU as an input to capability development documents) to JCIDS-Acquisition decisions. Figure 2. Relationship of Official CONOPs-Related Documents to JCIDS-Acquisition Activity 8 Though not explicitly defined in the manual, contextually the sustainment CONOPs is a concept of operations specific to maintenance approaches and supply chain solutions. This definition makes the implied position of the sustainment CONOPs somewhat confusing, because maintenance and sustainment concepts communicate desired sustainment instructions that inform system design and planning. The sustainment CONOPs will likely be more detailed post milestone B and C, but per the IEEE format, should be written, if even in an immature state, with the original system-level CONOPs. = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 246=

13 Figure 3. Relationship of Unofficial CONOP-Related Documents to JCIDS- Acquisition Activities In addition to documents described in the JCIDS manual, our research revealed that there are many other documents in use that serve as inputs to the acquisition process. Such inputs include ConOps (described by Nelson), use-cases (as described by Daniels & Bahill, 2004), user CONOPs, and system concepts (see Table 2). The relationship of these inputs to the JCIDS-Acquisition timeline is illustrated in Figure 3. This mapping of CONOPs documents to the acquisition lifecycle suggests that CONOPs documents are developed throughout the acquisition lifecycle. Comprehensive View and Consistent Involvement by Stakeholders. As illustrated above, many separate CONOPs documents are written. A risk to proper use of these CONOPs is that these various CONOPs are independently authored by various individuals or groups that may have different perspectives on the system and on system objectives. Without a single, integrated, system-level CONOPs to draw from, requirements may be unintentionally derived from multiple sources that may or may not include a complete understanding of system uses. This can create [different] and potentially conflicting views of system use [that] will result in a system that only partially meets the user s expectations (IEEE, 2008, para , p. 23). Additionally, breaking the CONOPs into smaller non-integrated documents runs the risk of reducing stakeholder coordination. This practice also reduces the comprehensiveness of the stakeholder input, which in-turn degrades the completeness of perspectives and resulting system requirements. Key to Effective Use of CONOPS in the Acquisition Life Cycle: The Integrated CONOPs Despite the occurrence of the various types of CONOPs documents, Nelson (2007) argues that although many documents go by the name CONOPs, there is only one true = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 247=

14 ConOps 9 and it is the ConOps described in the IEEE 10 standard Nelson goes on to state that the power of the IEEE ConOps comes from its comprehensive assessment of both the what (system identification) and the how (system employment). In our assessment, the IEEE format also includes the why in its section titled Justification. Traceability to the why, or justification, is an important factor in maintaining system validity and verification. 11 Unique to the IEEE format is its emphasis on describing not only the desired capability and end-state, but also the current capability and situation. This end-toend emphasis provides a logic trail from original need to capabilities pursued. Integration of Individual Inputs Because the many types of CONOPs-related documents appear to span the entire lifecycle of the system, we wondered how these individual CONOPs would relate to the IEEE ConOps. To find out, we delineated the relationship of the individual CONOPs documents to sections of the IEEE ConOps (Figure 4). To conduct this assessment, the content of each of the CONOPs documents used as an input to the acquisition process was analyzed. This content was then compared to the content in each section of the IEEE ConOps to identify similarities. Arrows are provided from CONOPs documents to applicable IEEE ConOps sections to show a relationship between the content. Figure 4. Relationship of CONOPs Documents to IEEE An Example of a CONOPs Document IEEE ConOps Relationship. The guiding military concept provides insight into the operational environment, the scope of the mission set, and the need for the system. Although this concept does not provide an 9 Nelson uses the IEEE abbreviation for ConOps to distinguish it from other forms of concept of operations documents, which are often abbreviated as CONOPs (Nelson, 2007). 10 IEEE, pronounced (I-triple E) IEEE is recognized as a leading institution in systems and system standards. Their format for ConOps documents (IEEE, ) is comprehensive and is used by many agencies/ organizations. 11 In their paper on famous failures, Bahill and Henderson note that [system] validation requires consideration of the environment that the system will operate in (2005, p. 3). = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 248=

15 exhaustive list of system user classes, it will provide insight into an initial group of potential system stakeholders. Therefore a relationship is shown between the military concept and the IEEE sections: Current Situation, Background and Scope, Policies and Constraints related to the current situation, User Classes, and Justification for Change. Identification of these relationships confirmed that, while not necessarily an integrated ConOps such as IEEE, elements of the IEEE ConOps are being utilized in the acquisition process via the many, currently occurring CONOPs documents. All IEEE ConOps sections are addressed in the currently occurring CONOPs-related documents with the exception of a detailed explanation of the modes of operation for the current system (this would include modes for legacy systems currently in place) and administrative sections such as referenced documents and document overview. This means there may be an opportunity to integrate elements of each of these documents into an integrated CONOPs, such as the IEEE ConOps. 12 The Value of an Integrated CONOPs over the Current Way of Business Although already occurring independently across the lifecycle, integrating individual CONOPs documents into a single CONOPs document has potential to increase both traceability and continuity. Traceability. According to IEEE, traceability is a key tool for ensuring that the system developed fully meets the needs and requirements defined by the user (IEEE, 2008, para. 4.2, p. 38). An integrated system-level CONOPs resolves, or at least mitigates, the problem of conflicting system views and partially met requirements by using the same resources to create a more complete view of the problem, the proposed solution, the user community, and the intended uses. This pooling of information allows stakeholders an opportunity to recognize requirement needs and contradictions that are otherwise overlooked. 13 Continuity. Both IEEE and ANSI identify a purpose of the CONOPs as a means by which to communicate system characteristics in such a way that is understandable by all system stakeholders. 14 Continuity is an enabler of the required communication and 12 Integrating each of these inputs into a comprehensive CONOPs document does not preclude the use of, or independent improvement of, particular sections. Products, such as an AoA, can continue to stand-alone; in fact AoAs can inform later iterations of the CONOPs document. The point of the integrated CONOPs is not to enforce a rule set but rather to serve as a means for conducting holistic problem and solution space exploration and for providing a document owned by all stakeholders that clearly and logically expresses the system s characteristics. In this way, the CONOPs serves as the baseline document to which all subsequent documents are loyal. 13 According to the INCOSE SE Handbook, version 2a (2004) one use of a CONOPs is [t]o validate requirements at all levels and to discover implicit requirements overlooked in the source documents (para. 8.2 f, p. 104). 14 The CONOPs document is used to communicate overall quantitative and qualitative system characteristics to the user, buyer, developer, and other organizational elements (IEEE, , p. 1). The CONOPs document facilitate[s] understanding of the overall system goals with users, buyers, implementers, architects, testers, and managers (ANSI/AIAA, 1993, p. 1.) = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 249=

16 stakeholder involvement. Similar to what is seen with traceability, continuity suffers when the integrated system-level CONOPs is broken out into multiple documents. The Relationship of the Integrated CONOPs to the Acquisition Lifecycle Current literature provides an illustration of the relationship of military concepts to JCIDS-Acquisition activities (Figure 5). This is in line with our assessment of the relationship of military concepts to acquisition activities (see Figure 1). What appears to be missing though is an illustration of the relationship of the system-level CONOPs to the acquisition lifecycle. If we integrate the many CONOPs documents into a single integrated CONOPs document, what will its relationship to the acquisition lifecycle look like? Figure 5. Requirements and Acquisition Process Flow (Modified from USD(AT&L), 2008) The streamlined CONOPs-Acquisition lifecycle relationship, or CONOPS continuum, is most easily depicted by building upon a baseline graphic (Figure 6) and gradually adding in additional relationships. Initially this illustration depicts two major inputs to the JCIDS- Acquisition process. The first of these is the higher order military concept which is a basis for a CBA and identifies the context in which the proposed system will operate. System-level CONOPs emerge when the CBA process identifies capability gaps for which a materiel solution is the preferred solution. This results in a relationship between higher-order military concepts and acquisition and between higher-order military concepts and system-level CONOPs (which we will term simply CONOPs ). As illustrated in Figure 6, military concepts drive CBAs and Doctrine, Organization, Training, Materiel, Leadership, Personnel, Facilities (DOTMLPF) changes and are the context for CONOPs, which must always support the activities outlined in the military concepts. = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 250=

17 Figure 6. Concept-CONOPs-JCIDS-Acquisition Relationships (CONOPs Continuum) Higher-order military concepts are the basis from which CBAs and systems in development are derived. As such it is imperative that the vision, mission, and goals of military concepts are valid. Invalid or inaccurately assessed military concepts can result in faulty CONOPs and ineffective systems. Therefore, the exercise and evaluation process that validates the military concept is an equally essential part of successful CONOPs development and, ultimately, of successful system development. Figure 7 depicts the relationship of experimentation to military concepts, with military concepts driving experimentation, which informs or validates the military concept. Military concepts that are invalidated should be changed and retested. Once validated, the military concepts then drive CBAs or DOTMLPF changes. Figure 7. Experimentation as Part of the CONOPs Continuum = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 251=

18 Because CONOPs are the basis for system requirements, they also interact with the JCIDS-Acquisition processes. Initially, the CONOPs informs the ICD. As the system progresses through the acquisition lifecycle, events in system develop should inform the CONOPs. This process of revisiting and updating the CONOPS will help ensure it remains relevant. This ongoing cycle of informing and updating is illustrated in Figure The importance of the figure resides in its simplistic illustration of the interconnectedness of many activities. The picture now shows a series of ongoing interconnected processes each reliant on and influencing the other, versus strict swim lanes of disparate processes. Figure 8. Interconnectedness of CONOPs and Acquisition Activities CONOPs Maturation and Phases Ideally, the CONOPs should be updated throughout the acquisition lifecycle, such that as the system matures, the CONOPs increases in specificity. We have identified specific phases of CONOPs maturity each of which coincides with events in the acquisition lifecycle. CONOPs (Initial Phase). Initially, the CONOPs describes the proposed system as a black box and in its most ideal form (i.e., all user desired capabilities). This initial phase of the CONOPs will be used to guide the development of the ICD and serves as a basis for system requirements. 15 The figure also shows the independent role of the AoA. The AoA informs both the higher and system-level concepts. The AoA has potential to influence DOTMLPF solutions that impact the way a Service fights, trains, and equips. The AoA informs the system-level CONOPs section on alternatives considered. = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 252=

19 CONOPs (Discovery Phase). The suggested update cycle is triggered by specific events in the acquisition lifecycle. Initially, the CONOPs informs the Technology Development 16 (TD) phase by communicating desired capabilities, for which technology must be developed. Likewise, the TD process informs the CONOPs, by revealing actual technological possibilities. As TD progresses and technological possibilities become more evident, the CONOPs document should be updated to reflect actual capability. This activity will ensure that the user gets the system expected and that the system, though not as initially envisioned, still meets the operational need(s) described in the military concepts. The updated CONOPs and the ICD are the foundation for the CDD requirements generation process. Following the CDD and Milestone B, the system enters the Engineering, Manufacturing, and Development (EMD) phase. 17 Results from the EMD activities bring additional clarity to the system s operational limitations and advances. Therefore, EMD results should further inform the CONOPs such that it can again be updated to reflect actual system capability. Again, the updated CONOPs will be used as the basis for the requirements captured in the Capability Production Document (CPD). As with previous updates, the updating process will continue to maintain traceability between the user s expectations and the operational mission the system supports. CONOPs (Employment Phase). Shortly after Milestone C system prototypes enter into low rate initial production (LRIP). LRIP models will generate user feedback, which will further inform the CONOPs. LRIP feedback provides the information needed to fully understand how the final system can and will be used. This feedback should be incorporated into a final version of the CONOPs. Throughout the updating cycle, the CONOPs must be compared to the higher order military concept(s) it supports to determine that it still supports the goals, missions and activities of the military concepts. The continuous review of the military concepts and CONOPs relationship is particularly important in long-lead time acquisition programs; over time introduction of new systems, new threats, and new political environments can change the battle field to the point that the system no longer serves a valid mission set. 18 The CONOPs maturity phases align with major phases of the acquisition cycle (i.e. MDD, TD, EMD, and deployment and employment). This results in three distinct phases of the CONOPs document. We have termed these phases: Initial, Discovery, and Employment, the names of which correspond to the level of system understanding discussed above. A graphical representation of these phases as they relate to the JCIDS-Acquisition timeline is provided in Figure During the technology development phase, technologies are developed, matured, and tested (Schwartz, 2009, p. 9) 17 During the EMD phase various subsystems are integrated into one system and a development model or prototype is produced (Schwartz, 2009, p. 10). 18 Although we have identified specific drivers of a CONOPs review, the idea of updating CONOPs is not new. Most guiding documents agree that the CONOPs is a living document (ANSI/AIAA, 1993; IEEE Standards, 1998; Daniels & Bahill, 2004). Even still, in a survey of systems engineers and lead systems integrators, respondents indicated that [over] 50% of [CONOPs] were not updated throughout the entire program lifecycle and of those 49% = were only updated through preliminary design review. (Roberts, 2008, p. 28). do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 253=

20 Figure 9. CONOPs Phases as Related to JCIDS-Acquisition The integrated, maturing CONOPs provides a mechanism for tracing system elements, such as requirements, perspectives, decisions, and solutions. Summary Undeniably, the system-level CONOPs has the ability to influence the success of activities across the entire acquisition lifecycle. To fully realize the benefit of these CONOPs we must first resolve outstanding issues related to barriers of the CONOPs use. First, we must work as a community to promulgate a single, agreed-upon definition for the term CONOPs. Secondly, we must work to combine existing CONOPs documents into an integrated and comprehensive document that speaks to many audiences. The integrated CONOPs will reduce the risks of inconsistent and unmet requirements by ensuring effective collaboration by stakeholders throughout the development life cycle. Once the CONOPs is created, we must remember its alignment with military concepts and acquisition activities and the influence each of these has on the other. Finally, we must remember to revisit the CONOPs and allow it to mature over time. Although a potentially demanding and lengthy process, use of CONOPs will amplify the rapid and cost effective delivery of usable systems that meet warfighter needs. References American Institute of Aeronautics and Astronautics (ANSI/AIAA). (1993). Guide for the preparation of operational concept documents. American national standard. Washington, DC: American Institute of Aeronautics and Astronautics. Ask a Professor (APP). (2009, May 20). Question & answer detail program management. Retrieved November 2009, from Bahill, T.H., & Henderson, S.J. (2005). Requirements development, verification, and validation exhibited in famous failures. Systems Engineering, 8 (1), Retrieved October 2009, from Boardman, J., & Sauser, B. (2008). Systems thinking: Coping with 21 st century problems. Boca Raton, FL: CRC Press Taylor & Francis Group. = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 254=

21 Chairman of the Joint Chiefs of Staff Instruction. (2006, January 27). Joint operations concepts development process (JOpsC-DP), (CJCSI B). Washington, DC: Government Printing Office. Companion, M., & Mortimer, C. (n.d.). Designing for change: A modeling and simulation system approach. Arlington, Texas: Raytheon Systems. Available at Daniels, J. & Bahill, T. (2004, July 7). The hybrid process that combines traditional requirements and use cases. Systems Engineering, 7 (4), Retrieved October 2009, from Edson, R. (2008). Systems thinking. Applied. A primer. Arlington, VA: Analytic Services, Inc. Fleet Forces Command. (2004). CONOPs to doctrine: Shaping the force from idea through implementation (Pre-Decisional-Draft Working Papers). SEA TRIAL Presentation to the Corps of Naval Attaches. Retrieved June 2009, from IEEE Press/Standards Information Network. (2008, November 4). Guide for implementing IEEE Std 1512 TM using a systems engineering process. New York: Institute of Electric and Electronics Engineers. Retrieved from press/ IEEE Standards Board. (1998). IEEE standard , IEEE guide for information technology System definition (Concept of Operations (ConOps) Document). Standard. New York: Institute of Electric and Electronics Engineers. International Council on Systems Engineering (INCOSE). (2004). Systems engineering handbook: A "what to" guide for all SE practitioners (INCOSE-TP ) (Ver. 2a). Joint Capabilities and Development System (JCIDS). (2009). Manual for the operation of the joint capabilities integration and development system. Washington, DC: Author. Joint Publication (2001, April 12, as ammended through October 2008). Department of Defense dictionary of military and associated Terms. Washington, DC: Author. Jost, A. (2007, October). ConOps: The cryptex to operational system mission success. Cross Talk: The Journal of Defense Software Engineering, 20(10), Retrieved from Nelson, G. (2007). The ConOps in a self-similar scale hierarchy for systems engineering (Paper No. 69). In Conference on Systems Engineering Research. Hoboken, NJ. Roberts, N. (2008). An analysis of concept of operation development (Master's Thesis). Hoboken, NJ: Stevens Institute of Technology. Schmitt, J. (2002, December) A Practical Guide for Developing and Writing Military Concepts. (Defense Adaptive Red Team Working Paper). McClean VA: Hicks and Associates, Inc. Retrieved from Schwartz, M. (2009, July 10). Defense acquisitions: How DoD acquires weapon systems and recent efforts to reform the process. Retrieved September 2009, from Turner, R., Verma, D., & Weitekamp, M. (2009). The next generation air transportation system (NextGen). Hoboken, NJ: School of Systems and Enterprises, Stevens Institute of Technology. USD(AT&L). (2008, December 8). Operation of the defense acquisition system (DoD Directive ). Washington, DC: Author. = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 255=

22 US Air Force. (2003, September). Air Force policy directive Air Force Concept Development. Appendix A. Questionnaire Survey The following questionnaire is issued to gain insight into current acquisition processes and to understand perceived shortcomings (if any) and suitable fixes to the identified shortcomings. The ultimate goal of this research is to contribute to efforts to provide the warfighter with needed capabilities in a timely and cost-efficient manner. Answers to this questionnaire are non attributable, meaning answers provided will not be credited to any particular respondent. Furthermore, prior to being included in any published document, any/all answers will be generalized such that specific programs, offices, and/or systems under development will not be identifiable. Your honest answers are greatly appreciated. Respondents who would like a copy of our research end product can request so by ing me at jaime.frittman@anser.org. 1a. In how many programs of record do you currently participate, or have you previously participated (an estimate is ok )? 1b. What is or has been your role in these programs? 2. How do you define the term concept of operations (CONOPs)? 3. In your personal opinion, what is the role/purpose of a CONOPs? 4a. Have you ever written a CONOPs? 4b. If so, what type of content did you include in the CONOPs? 4c. Did you use a certain standard or prescribed template? If so, which one? 4d. Was the CONOPs updated throughout the development cycle? = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 256=

23 4e. Do you believe the CONOPs was properly utilized, or underutilized? 5. How important do you think a CONOPs is to the successful development and employment of the system to which it applies? 6. Off the top of your head, can you think of any shortcomings related to CONOPs as they relate to current systems under development? Please describe these shortcomings. = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 257=

24 Sponsored Research Topics Acquisition Management Acquiring Combat Capability via Public-Private Partnerships (PPPs) BCA: Contractor vs. Organic Growth Defense Industry Consolidation EU-US Defense Industrial Relationships Knowledge Value Added (KVA) + Real Options (RO) Applied to Shipyard Planning Processes Managing the Services Supply Chain MOSA Contracting Implications Portfolio Optimization via KVA + RO Private Military Sector Software Requirements for OA Spiral Development Strategy for Defense Acquisition Research The Software, Hardware Asset Reuse Enterprise (SHARE) repository Contract Management Commodity Sourcing Strategies Contracting Government Procurement Functions Contractors in 21 st -century Combat Zone Joint Contingency Contracting Model for Optimizing Contingency Contracting, Planning and Execution Navy Contract Writing Guide Past Performance in Source Selection Strategic Contingency Contracting Transforming DoD Contract Closeout USAF Energy Savings Performance Contracts USAF IT Commodity Council USMC Contingency Contracting Financial Management Acquisitions via Leasing: MPS case Budget Scoring Budgeting for Capabilities-based Planning = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 258=

25 Capital Budgeting for the DoD Energy Saving Contracts/DoD Mobile Assets Financing DoD Budget via PPPs Lessons from Private Sector Capital Budgeting for DoD Acquisition Budgeting Reform PPPs and Government Financing ROI of Information Warfare Systems Special Termination Liability in MDAPs Strategic Sourcing Transaction Cost Economics (TCE) to Improve Cost Estimates Human Resources Indefinite Reenlistment Individual Augmentation Learning Management Systems Moral Conduct Waivers and First-tem Attrition Retention The Navy s Selective Reenlistment Bonus (SRB) Management System Tuition Assistance Logistics Management Analysis of LAV Depot Maintenance Army LOG MOD ASDS Product Support Analysis Cold-chain Logistics Contractors Supporting Military Operations Diffusion/Variability on Vendor Performance Evaluation Evolutionary Acquisition Lean Six Sigma to Reduce Costs and Improve Readiness Naval Aviation Maintenance and Process Improvement (2) Optimizing CIWS Lifecycle Support (LCS) Outsourcing the Pearl Harbor MK-48 Intermediate Maintenance Activity Pallet Management System PBL (4) Privatization-NOSL/NAWCI RFID (6) = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 259=

26 Risk Analysis for Performance-based Logistics R-TOC AEGIS Microwave Power Tubes Sense-and-Respond Logistics Network Strategic Sourcing Program Management Building Collaborative Capacity Business Process Reengineering (BPR) for LCS Mission Module Acquisition Collaborative IT Tools Leveraging Competence Contractor vs. Organic Support Knowledge, Responsibilities and Decision Rights in MDAPs KVA Applied to AEGIS and SSDS Managing the Service Supply Chain Measuring Uncertainty in Earned Value Organizational Modeling and Simulation Public-Private Partnership Terminating Your Own Program Utilizing Collaborative and Three-dimensional Imaging Technology A complete listing and electronic copies of published research are available on our website: = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 260=

27 THIS PAGE INTENTIONALLY LEFT BLANK = do^ar^qb=p`elli=lc=_rpfkbpp=c=mr_if`=mlif`v= 261=

28 ^Åèìáëáíáçå=êÉëÉ~êÅÜ=mêçÖê~ã= dê~çì~íé=ëåüççä=çñ=äìëáåéëë=c=éìääáå=éçäáåó= k~î~ä=éçëíöê~çì~íé=ëåüççä= RRR=avbo=ol^aI=fkdboplii=e^ii= jlkqbobvi=`^ifclokf^=vpvqp=

29 Unclassified Illustrating the CONOPs Continuum and its Relationship to the Acquisition Lifecycle Presented to: Acquisition Research Symposium Presenter: Jaime Frittman Authors: Jaime Frittman & Robert Edson Date May 12, 2010 Unclassified

30 Overview Motivation for research Research goals Methodology Discussion CONOPs definition Perceptions of CONOPs and barriers to usage Integrating many CONOPs documents Alignment of CONOPs Evolving CONOPs Summary Unclassified 2

31 Motivation As noted by a recent FAA sponsored study, cost, schedule and performance breeches continue to plague large scale programs The FAA study noted the importance of the CONOPs in avoiding programmatic pitfalls one of the most significant artifacts is the creation of a CONOPs. Once created, there is a need to have alignment between the evolving CONOPs, the enterprise architecture, and the governance system (Turner et. al., 2009, p 32). Unclassified 3

32 Research Goals Assess current use of CONOPs Identify any disconnect between use and perceived usefulness Assess current alignment of CONOPs to DOD governance and EA processes Explore maturity phases of CONOPs Unclassified 4

33 Methodology Literature review DoD instructions and manuals Industry standards Websites Academic papers and surveys Analysis 4-way data analysis of: usage, terms, purposes, and relationships Systems thinking Conceptagon application (Edson, 2008) Unclassified 5

34 A CONOPs Is. IEEE Std A user-oriented document that describes system characteristics for a proposed system from the users viewpoint. Joint Pub 1-02 A verbal or graphic statement that clearly and concisely expresses what the joint force commander intends to accomplish and how it will be done using available resources... designed to give an overall picture of the operation. CJCSI B How a joint force commander may organize and employ forces in the near term (now through 7 years into the future) in order to solve a current or emerging military problem CONOPs provide the operational context needed to examine and validate current Unclassified 6

35 Perceptions of CONOPs Use Government community survey Respondents indicated CONOPs as Critical to system success and Underutilized Industry community survey (Roberts, 2008) 108 respondents primarily engineers 100% of respondents said they found a CONOPs useful 1/3 of programs surveyed did not have a CONOPs 18% of CONOPs generated after requirements Unclassified 7

36 Barriers to Effective CONOPs Use Disconnect: perceived importance vs. use Research indicated 4 related causes of the disconnect Definition and purpose Targeted audience Timing and placement in the acquisition development lifecycle Comprehensive view and consistent involvement by stakeholders Unclassified 8

37 Relationship of CONOPs to Acquisition JCIDS and DoD, CONOPs usually refers to a military concept Unclassified 9

38 Relationship of CONOPs to Acquisition DoD Validated assessment of the relationship of Military Concepts Did not specify relationship of system level CONOPs CONOPs alignment within acquisition management system Unclassified 10

39 Relationship of CONOPs to Acquisition DoD literature review described several more CONOPs related documents These were plotted on the existing enterprise architecture/ governance framework Unclassified 11

40 Relationship of CONOPs to Acquisition Plot was increased to include documents referenced in literature Substantial increase in documents spanning lifecycle Unclassified 12

41 Integration of Individual Inputs and IEEE s standard The main reason we overlook the central role of the CONOP is that we give different names to the same thing at different scales (Nelson, 2007) Unclassified 13

42 Value of Integrated CONOPs Traceability Key tool for ensuring that the system developed fully meets the needs and requirements defined by the user (IEEE, 2008, para, 4.2, p., 38) Integration resolves, or mitigates, potentially conflicting views by creating a one stop complete view of the problem, the proposed solution, the user community, and the intended uses. Continuity Key tool for stakeholder involvement and communication Retains comprehensive view of stakeholder input Unclassified 14

43 Alignment of the Integrated CONOPs Unclassified 15

44 CONOPs Maturity Phases Alignment of CONOPs, EA, and governance systems, brought to light specific phases of CONOPs maturity Black box to white box description CONOPs matures in concert with system Maturity phases align with major phases of lifecycle Unclassified 16

45 CONOPs Maturity Phases Initial Phase Describes the system as a black box and in its most ideal form. Guides development of ICD requirements Discovery Phase Informed by the Technology Development & EMD Basis for requirements captured in the CDD & CPD Employment Phase Informed by user feedback Most specific version of the CONOPs Unclassified 17

46 CONOPs Maturity Phases Unclassified 18