Sound Methods and Effective Tools for Engineering Modeling and Analysis

Transcription

1 Sound Methods and Effective Tools for Engineering Modeling and Analysis David Coppit Kevin Sullivan The College of William and Mary The University of Virginia Dept. of Computer Science Dept. of Computer Science

2 Summary Overview 2

3 Modeling & Analysis in Engineering Build model of system using tool Model expressed in modeling language Expression analyzed by tool Result influence design decisions Example: fault tree model for system reliability 3

4 What Does it Take to Succeed? Success Technical Success Economic Success Method Tool Validated Formal Semantics Verified Implementation Functionality and Usability Low Engineering Costs 4

5 Problems Reconciling conflicting requirements Usability/Function vs. Cost Dependability vs. Cost Obvious approaches not clearly feasible Lack proven, cost-effective CBSD approaches Lack proven, cost-effective formal methods Consequences Risks to public [Knight 2002] Cost barriers to development & acquisition Tools hard to learn and use 5

6 Technical Approach Novel, synergistic combination of techniques Package-oriented CBSD Targeted formal methods Hypothesis: technical & economic feasibility End-to-end experimental systems methodology Assess feasibility & profitability for real system End-to-End: ultimate evaluation by real users 6

7 Nova Screenshot 7

8 Case Study Dynamic Fault Tree Modeling 8

9 Case Study: Reliability Engineering Reliability modeling and analysis Fault-tolerant computer systems Complex failure modes Redundancy management 9

10 Traditional Static Fault Tree Overflow Faucet Stuck Drain Clogged Combinatorial failure modes: And, Or 10

11 Dynamic Fault Tree Explode 1 2 Arm Fire Dynamic construct: Order-dependent And 11

12 Markovian Semantics A Dynamic Fault Tree B 1 2 C Markov Chain A B C.1.01 A XB C A B CX.01.1 XA XB CX A XB CX 12

13 Problematical Example FDEP Explode 1 2 Power Loss Arm Fire 13

14 Problem: Semantics & Dependability 14

15 What Does a DFT Mean? How to map DFT s to MC s, in general? Typical approach A few small published examples Prototype proof of concept implementation Not good enough Significant errors [Coppit, Sullivan, Dugan 2000] Commercial tools share fault [Amari et. al 1999] 15

16 Formalizing Methods & Languages Formalize method and validate Formalize language syntax and semantics Informal and formal validation Hypothesis: Technically and cost effective Formalized DFTs DFT, MC, semantic mapping in Z [Spivey 1992] Informal SE/RE reviews Formal validation [Z/Eves] 16

17 A Snippet: Fault Tree È_FaultTree basicevents: F Event gates: F Event Explode Arm Fire inputs: Event seq Event, Ç dom inputs = gates Ag: gates ran (inputs(g)) z basicevents U gates Ag: gates! (IsInputTo(g, g, inputs)) Ð 17

18 18

19 Cost/Benefit Assessment Benefits Significant specification, coding errors found Language design significantly improved Cost 6 months part-time to develop spec (Coppit/Sullivan) 24 hours in group technical reviews 3 months learning Z/Eves & proving theorems Theorem proving too expensive (for us) 19

20 Problem: Usability and Function 20

21 CBSD for Tool Superstructures Usable tools are large & complex Core engine often small (10 4 LOC) Superstructure large (10 6 LOC) Interoperate, print, views, large models Packaging is problem [Shaw 1990] Component-based software development? 21

22 Package-Oriented Programming Packages as components [Sullivan&Knight 1995] Examples: MS Word, Visio, Mathcad Rich feature sets at very low cost Significant investments in usability Customizable, open, and composable Widely deployed and familiar to users Is this CBSD model viable? Proven for simple office automation tasks Can it support industrially viable tool development? 22

23 Promising but Risky [Coppit/Sullivan 2000] Galileo Word + Visio + Core, < 50 KLOC Integrated programmatic & user interfaces Multiple editing views, modular analysis, But Undocumented limitations Standards non-compliance Mismatch issues (in testing) Runtime Performance Issues Rapid component evolution No end-to-end evaluation 23

24 Function/Usability: End-User Evaluation Evaluated in three NASA-wide DFT workshops Surveyed 16 engineers and program managers from 12 different NASA groups and contractors Most engineers use such tools frequently Consequences of failure: billions, lives ~100 questions, designed according to standard practices [Dillman 1999] 24

25 End-User Evaluation Features and usability results Usability good or better than commercial tools Package familiarity significantly eased learning & use Several cited usability as the biggest surprise Adoption by NASA engineers (JSC/ISS) Expanded commitment to develop tool New version for LaRC and JSC NASA encouraging commercialization 25

26 Problem: Cost? 26

27 Nova: Putting the Pieces Together Synergistic combination of techniques POP can significantly reduce features & usability cost Provide resources for judicious formalization Galileo uses POP but not formal spec New, from scratch tool combines approaches Implements formally defined, revised DFT language More advanced POP-based interface About 30 KLOC, unoptimized solver 27

28 Bottom Line 28

29 Results Demonstrated feasibility & profitability Formally-based, easy to use tool at low cost Significant impact on NASA PRA activities 29

30 Conclusion Advance ability to deliver trustworthy, usable, and cost-effective modeling & analysis tools Potential to catalyze development, use of new approaches to engineering modeling & analysis Open question: verification of implementation Forthcoming: specification-based testing using Alloy to leverage investment in Z specification 30

31 31