Air Force Materiel Command

Transcription

1 Air Force Materiel Command Developing, Fielding, and Sustaining America s Aerospace Force Track 2: Integration, Test and Verification Planning and Executing an Integration and Test Strategy for a Complex Aerospace Dr. Ed Kraft AEDC Chief Technologist AIAA Complex Aerospace s Exchange September 11, 2012 I n t e g r i t y S e r v i c e E x c e l l e n c e

2 We Have Met the Enemy Runaway development cycle time not inherent to complexity Architecture choices Processes Process ownership Accountability Capacity Source: Dr. Kirstie L. Bellman Making DARPA META Goals Come True: How do we Revolutionize Verification and Validation for Complex s? S5 2010, WPAFB, June 17, 2010

3 Keep People Busy Intensity of Competition Obscure Economics The Complexity of Environment and Process (At Least for DoD s) Fill the Factory Must Win Mentality Unqualifiable Mission Value Mission Challenge Prime s Capabilities Future Proofing Technological Prowess Over Design Quality of Early Work Lobbying Drive to Build & Maintain Capability Incumbency Requirements Existing & Required Capabilities DoD Optimism High Cost Of Failure KPP & Demonstrated Performance Existing Updates Over Promise Enemy Capability Rush to Milestone B PreMileston B Attractiveness Need to Reset At Each Milestone KPP Objectives Performance Problems Performance National Security Vulnerability National Security Capability DoD PM Advocacy Geopolitical Environmental Change Don t Fail Mentality Time Development Schedule Number of s DoD Focus On Late Stage s Cancellations Cost and Funding Profile Development Costs Relevance to NSV Portfolio Budget And Required Funds Funds for Other s Delay Other s Congressional Directive Approved POM Budget Budget And Required Funds Cost Unit Production Costs Perceived Value of Sunk Costs Priority In DPG Quantity Strategic Requirement Service/Org Advocacy Requirements Definition Management Design & Mfg Funding

4 Complicated or Complex? Complicated Not simple, but knowable Clear links between causes and effects Tend to give predictable results, as long as the same thing is done the same way, each time. Focus needs to be on diligence and accuracy when following particular steps in a process Complex Parts are interdependent, often changing, and outcomes are less predictable Hard to see the whole, but only what s available from your vantage point Complex systems are not constructed, they are grown Complex systems are dynamic that is, they re constantly changing and adapting to current conditions. My car key is simple My car is complicated Driving my car in traffic is complex Driving a car in some cities is chaotic The art is to not treat complicated as complex and not treat complex as complicated

5 Separating Complicated From Complex Materiel components/subsystems of a system are complicated Materiel systems of integrated components are complicated because they still obey the laws of physics albeit the interfaces are difficult New technologies inserted into a system are complicated still obey the law of physics and are knowable Software can be bipolar An algorithm is complicated because it is deterministic, i.e., get the same computed result with the same inputs and the outputs are knowable if enough time is spent understanding the logic An algorithm is also complex if it is scaled to a degree (SLOC, nodes, interactions) that its logic becomes inscrutable within a reasonable time s of s are also bipolar Interoperability of a material system with other material systems or an SoS of material systems (ignoring independent agents) is complicated, i.e., the physics are knowable Interoperability of SoS with different agents/actors is complex because the actions of the agents are generally unpredictable/nonreproducible Interaction of systems or SoS with external agents (e.g., terrorists, cyber warriors) is complex because the agents are unpredictable and dynamic

6 Different Domains Require Different Design, Integration and Testing Approaches Complicated Design, test to requirements Well defined boundaries Physicsbased modeling Probability based design Precision measurements Statistically defensible testing Regression testing Materiel CNI C4ISR Component Subsystem Flight Simulator Cyber Warfare Independent Agents SoS Complex Manage outcomes vs delivering requirements Ill defined boundaries Soft, stochastic modeling Dynamic environment Experiential learning Requires holistic, collaborative approach Future built on scenarios not predictions

7 However We Also Need to Integrate the Complicated and the Complex Complex Resilient design space against scenarios Interactive physicsbased and operational modeling Early interaction between complicated/complex at requirements setting Cyberphysics interface, testing Materiel CNI C4ISR Complicated Sans independent agents Software V&V Comm links Sensor to shooter path EW physics Interactive Modeling Component Subsystem Flight Simulator Cyber Warfare SoS Common Integrator Independent Agents

8 Panel Members Ming Chang Lockheed Martin Fellow complexity from the defense OEM perspective Barry King Director, Space Test & Operations, Dynetics Inc. commercial view of complex systems Tom Irvine Deputy Associate Administrator, NASA, Aeronautics Research Mission Directorate complexity and the research community Ward Johnson General Manager, Jacobs Technology Group test facilities development Eileen Bjorkman Chief Technologist, 412 Test Wing, AFTC flight testing interface between complicated and complex Derrick Hinton Principal Deputy Director, Test Resource Management Center, OSD cyber testing