, Practices, and Impications for Modeing and imuation Amy Henninger The Probem The act of identifying, enumerating, evauating, and mapping known technoogies to inferred program requirements is an important foundation to enterprise panning activities. The Department of Homeand ecurity mission requires an enterprise s systems-of-systems (o) anaytic capabiity to aow DH eaders to gain understanding of the combined effects of cross-component capabiities and processes from an o perspective, and to enhance DH enterprise panning activities (e.g., joint assessment of requirements, strategic programming, acquisition decisions, operationa assessments). Background Virtuay a anayses currenty performed in DH whether to justify an investment, assess the adequacy of an existing capabiity, or for some other reason center, if not entirey then amost excusivey, on the individuaized assessment of the foca system, patform, or capabiity. Few, if any, satisfactoriy account, in a hoistic way, for the mission contributions of reated systems or combined effects of the overa o. Mutipe Government Accountabiity Office (GAO) reports recognized that DH core missions woud benefit from joint assessments that consider competing and compementary patforms, systems, and activities across the Doctrine, Organization, Training, Materie, Leadership and Education, Personne, Faciities and Poicy (DOTMLPF) spectrum. To address this gap, the DH cience and Technoogy Directorate is standing up a ystem of ystems Operationa Anaytics (ooa) investment to estabish an anaytic framework, designed and deveoped in partnership with the components and headquarters organizations, through the integration of existing and emerging anaytic, modeing, and simuation (M&) technoogies. We describe the ooa in terms of an anaysis use case, aong with some of the anaytic and technica chaenges the program wi need to address. Use Case In genera, there are three sources of activities that may resut in anaysis due to the identification of a gap: a poicy directive, an acquisition initiative, and an Inspector Genera or GAO request. In a three cases, the directive for anaysis is assigned to a sponsor or stakehoder responsibe for responding to the directive (usuay with some kind of anaytic activity). The stakehoder often seeks support (e.g., Federay Funded Research and Deveopment Center, University Affiiated Research Center, 62 Research Notes
or interna support) for the anaysis, and a fair amount of interpay (e.g., probem definition/scoping, negotiation for resources) must take pace to pan and execute the anaysis. Typicay, and in the as-is case (see Figure 1), the directive does not identify a o view, ony a singepatform, singe-soution view. The needs anayses for the MQ-9 and the Muti-mission Enforcement Aircraft (MEA) are exampes of this singe patform approach. The quantities and aydown of these compementry aircraft, with overapping capabiities, were anayzed without regard for each other. This is a common anaytic chaenge at DH, where reated anayses may spawn mutipe directives for mutipe studies or anayses, designed and executed by independent organizations using unique methods, toos, or data that are not normaized, not interoperabe, and in some cases not even formay assessed for their fitness for use. In cases such as these, decision makers are faced with the difficut task of tudy Method 1 using independenty derived and inherenty incomparabe anaytic resuts to envisage the combined effects of mutipe systems. In the to-be case (see Figure 2), on the other hand, the ooa intends to provide a capabiity set that heps to structure the study panning process to foster the use of normaized and vaidated toos, methods, and data. In this case, the anaytica questions and supporting toos and data can be used to assess the interactions of a systems and their contributions to the overa mission. For exampe, if the mission contributions of Unmanned Ground Vehices (UGs) interact with the mission contributions of Integrated Fixed Cameras, the anaysis of the two systems jointy wi revea the reationship and aow for a morerefined characterization of the trade space. This insight aows better informed investments not decided on a system-by-system isoated basis but on the contribution of the pieces to the overa capabiity. Anayst1 Resuts1 Quaity1 Resuts2 Quaity2 Anaytic Toos 1 Mission Essentia Tasks ystem Data 1 Geospatia Data 1 tudy Method 2 Anayst2 Anaytic Toos 2 ystem Data 2 Geospatia Data 2 tudy Method 3 Anaytic Toos 3 ystem Data 3 Geospatia Data 3 Anayst3 Resuts3 Decision Maker Quaity3 Figure 1. As-Is Anaytica Ecosystem at DH ida.org 63
Mission Essentia Tasks Comprehensive ooa Baseines Credibe VV&A ooa Toos Interoperabe ooa Data Anayst1 Resuts1 Quaity1 Resuts2 Quaity2 Resuts3 Quaity3 ooa Tookit ef-instructiona Tutorias Training Guidebook Tutorias User Identity Access Interface Data Toos Piot tudy Repository Information LYR-001 User Interface LYR-002 ef-instructiona Layer LYR-0031 Toos Layer LYR-004 Data Layer Decision Maker Anaytic Toos Toos Appication Baseine Anayst2 Data Data Information Fow Anayst3 Figure 2. To-Be Anaytica Ecosystem with ooa Technoogy Assessment and Roadmap IDA has assisted the ooa with initia program mission anaysis and programmatic documentation by scoping the project and defining higheve technica chaenges, identifying and assessing reevant research that may hep mitigate technica chaenges, and composing a high-eve technoogy roadmap to achieve ooa objectives. These documents are argey organized around three technica chaenges: ystems of systems modeing Anaytic toos and methodoogies Computing paradigms. ystems of ystems Modeing The maturity of o M& and the maturity of the soutions to its 64 Research Notes reated technica gaps, incuding a review of existing o engineering and integration standards, are described in the ooa Apex Tech couting napshot. Many successfu exampes of existing soutions and standards provide some assurance that the ooa is technicay feasibe. Part of the technica chaenge wi be preserving component-specific toos to anayze the capabiities offered by the individua components whie simutaneousy accuratey representing cross-component missions that buid on the combined, synergistic effects of these individua capabiities. This wi require a carefu systems engineering/ integration anaysis. Above and beyond the reuse of existing M& capabiities, other technica chaenges that coud infuence the effectiveness and efficiency of any given system s modeing soution incude semantic
interoperabiity, correated representation of the environment, fair fight anomaies, and entity aggregation and disaggregation. Anaytic Toos and Methodoogies In genera, the ooa tookit shoud comprise a variety of toos to provide for robust anaysis (Davis and Henninger 2007). Beyond the ooa M& infrastructure, ooa is intended to incude a number of methodoogica advancements both to improve anaytic forecasts and to serve as a catayst in striking the right business mode for enterprise participation. One of these methodoogica advancements is ensembe modeing (Henninger, Pratt, and Roske 2006). Ensembe modeing is the process of running a number of reated but phenomenoogicay diverse anaytica modes and then synthesizing the resuts to improve the accuracy of the overa system. The maturity of these anaytic capabiities and the maturity of the soutions to its reated technica gaps are described in the ooa Tech couting napshot. Computing Paradigms Finay, the patform on which the ooa wi be impemented is a technica choice that sti must be evauated. Contemporary efforts simiar in scope to ooa have used coud patforms (Henninger 2016), high-performance computing patforms (Bouwens et a. 2012), and o modeing efforts in distributed environments based on cient-server architectures (Henninger et a. 2008). After identifying reevant capabiities and appicabe technoogies across a of these areas and expressing them in terms of maturity and degree of interest to ooa, IDA prepared a high-eve Technoogy Roadmap. The Roadmap additionay identified a number of APEX engines and programs that may contribute to the ooa capabiity, and highighted some of the interreationships between the various instantiations of these three high-eve technica areas. For exampe, both the simuation architecture and the ensembe architecture woud change depending on the computing paradigm adopted. Concusion Whie ony an initia step, the act of identifying, enumerating, evauating, and mapping known technoogies to inferred program requirements is an important foundation to the program. The maturity of these technoogies and, in some cases, the existence of simiar capabiities, provide some degree of confidence that the undertaking is indeed feasibe and achievabe within the estimated bounds of program costs, and that the potentia payoff in improved capabiity is worthy of continued research investment at the institutiona eve. ida.org 65
References Bouwens, Christina, Amy Henninger, Goria Fowers, and Aicia Pasche. 2012. OneAF as a imuation ervice Using High Performance Computing. Paper presented at the ALAIM 2012 Conference, Huntsvie, AL, May 1 3. Davis, Pau K. and Henninger, Amy. 2007., Practices, and Impications for Modeing and imuation. RAND Occasiona Paper. Arington, VA: RAND Corporation. Henninger, A., Pratt, D., and Roske, V. 2006. Using Ensembes to Reduce Uncertainty. In Proceedings of the 74th MOR ymposium (MOR). Coorado prings, CO: United tates Air Force Academy. Henninger, Amy, Dannie Cutts, Margaret Loper, Robert Lutz, Robert Richbourg, Randy aunders, and teve wenson. 2008. Live Virtua Constructive Architecture Roadmap (LVCAR). Fina Report. Aexandria, VA: Institute for Defense Anayses, eptember. Henninger, Amy. 2016. Impications of Coud Computing on Modeing and imuation. Arington, VA: Nationa Defense Industria Association (NDIA) ystems Engineering Committee, Apri 19. Dr. Amy Henninger is a former Research taff Member in IDA s Information Technoogy and ystems Division. he hods a Doctor of Phiosophy in computer engineering from the University of Centra Forida. 66 Research Notes