A New Approach to the Design and Verification of Complex Systems Research Scientist Palo Alto Research Center Intelligent Systems Laboratory Embedded Reasoning Area Tolga Kurtoglu, Ph.D.
Complexity Highly integrated system Large number of interacting components Mostly nonlinear component interactions Design constraints are rigid Feasible design space is tight Challenges Beyond the grasp of any one individual Difficult to consider all potential system states Hard to capture and model all component interactions Difficult to quantify risk and uncertainty Hard to handle disturbances safely and effectively
Complexity Demands a systematic approach to the design process waterfall model fountain model V-model spiral model agile model Standard practice MIL-STD STD- 449A [1969] Aerospace Automotive Defense
Is the V-model V good enough? Various government programs (NASA, NSF, DARPA) are attempting to improve current systems engineering practices DARPA s s new META (I&II) program is focused solely on revolutionizing methods for design and verification of complex systems s
How can we achieve this vision? How can we maximize performance and time to deliver while minimizing waste and inefficiency? Cost Identification and Elimination of Design Flaws Design Test & Certification Cost Design Identification and Elimination of Design Flaws Test & Certification Time Time By introducing an enabling suite of automated design tools to manage design complexity A novel model-based approach to systems engineering eliminates most hardware-in in-the loop testing for verification
ARRoW (Adaptive, Reflective, Robust Workflow) Objective: Compress design and verification timelines for complex cyber-physical systems by reducing delay-inducing design issues and by accelerating redesign. Model-based qualitative/ quantitative testing that tolerates abstraction imperfection, progressively builds certificate of correctness Design assisted by optimized model-based composition and diagnosis information Model-based diagnosis identifies conflicts directs redesign Approach: Three interlaced workflow threads continuously design, test, and diagnose the complete system through progressively more concrete abstraction layers.
ARRoW (Adaptive, Reflective, Robust Workflow) DARPA024r3 A novel model-based approach to systems engineering
ARRoW Unique Features Multi-Level Hierarchy of Design Abstractions Qualitative Reasoning Engine for Design Look Ahead Metrics Development for Assessment of Dynamic Design Complexity and Adaptability Model-Based Diagnosis for Design Flaw Analysis Model Integration and Meta Design-Language Probabilistic Verification for Issuance of Design Certificate of Correctness
ARRoW Qualitative Reasoning Engine Enables the tractable construction of exhaustive behavior traces It creates the scaffolding necessary to incrementally and efficiently derive probabilistic certificates of correctness Enables us to compute practical measures of complexity based on dynamic behaviors Avoids relying solely on static enumerative measurements
ARRoW Adaptability and Complexity Metrics Metrics capture complexity, robustness, and adaptability of candidate designs at varying levels of abstraction ability to better understand the potential impacts on reliability, maintainability, cost, and system performance Metrics address the full range of lifecycle uncertainties of concern to developers requirements creep, design space complexity, structural complexity, behavior complexity (number of distinct behavioral paths) development complexity (likelihood of design failures)
ARRoW Multi-Objective Design Optimization Assists the designers in selecting the components and design that achieve an optimal tradeoff between performance, complexity, and adaptabilit Tradeoff is better framed as a multi-objective problem Key goal for a design system is the continuous computation and presentation of the Pareto front ARRoW supports early design space exploration ARRoW adopts and improves upon the constraint satisfaction- based approaches ARRoW also builds on research in using surrogate models
ARRoW Probabilistic Certificate of Correctness Probability that a particular design will successfully pass a use case, or set of use cases Examples: probability that a vehicle will move at a top speed > 60 mph for one hour will accelerate to 60 mph in less than 3 seconds will successfully execute a mission plan in an allotted time Compute probabilistic certificate of correction given use case, U, and plant model, M Use case is expressed as a flexible plan Plant model represents dynamic behavior of design
Future Research Questions Can Qualitative Reasoning (QR) live up to the task? How to effectively generate auto/assisted design alternatives for domain specific complex design problems? How to evaluate the value of early architectural designs? How to perform functional (probabilistic) verification? How to guarantee correct by construction designs to eliminate/reduce testing and verification burden?
Thank you! Questions?