NILS SHONAN MEETING 048 INTEGRATION OF FORMAL METHODS AND TESTING FOR MODEL BASED SYSTEM-ENGINEERING. Werner Damm

Transcription

1 NILS SHONAN MEETING 048 INTEGRATION OF FORMAL METHODS AND TESTING FOR MODEL BASED SYSTEM-ENGINEERING Werner Damm

2 Objective of meeting This meeting aims to provide a forum for researchers and practitioners working on Formal Methods (FM) and Testing to identify research issues toward bringing related ideas, techniques, theories and tools in both research areas to real industrial solutions.

3 Success Criteria 1. Gap analysis We have identified the major obstacles in industrial deployment of existing formal methods and testing and major gaps potentially seen by industry demanding new formal methods and tests for covering these

4 Success Criteria 2. Technology Push analysis We have identified the major new technology break-throughs in academic research in the areas of formal methods and testing

5 Success Criteria 3. Roadmap We have developed a technology roadmap, which identifies key technologies for resolving obstacles in industrial deployment key technology evolutions from break-thru results to close gaps and characterized their realizability as short term, medium term, long term

6 Resulting Flow of the Meeting Session 1 Gap Analysis (Chair: Prof. W. Damm) Day 1 09:00 15:30 Introductory presentation and open discussion to share understandings of major gaps and key obstacles Capatilize on key experience of industrial participants Take into account findings of relevant strategy documents

7 Resulting Flow of Meeting Technology Push Sessions initial proposal to consider four main vectors of technology innovations requirement-based modelling and verification, fault-based modelling and testing, compositional approaches to testing, learning approaches to testing Other vectors detected on-line!

8 Technology Push Sessions Short presentation (10 min) from each participants which contains brief explanation of his/her work and/or position how his/her work relates to one (or two) of the 4 vectors as shown below how his/her work contribute to the issues discussed in Day 1 what is missing to solve the issues discussed in Day 1

9 Technology Push Sessions Group discussion to consider how to reach the solution based on the findings and ideas from the members presentations. Organized in two blocks with two parallel groups each Each group elects moderator and reporter Reporter to present findings in roadmap plenary sessions, grouping findings along technology vectors

10 Technology Push Session Part I (2 group discussions I.A, I.B) Day 1 16:00 18:00 presentations/discussion Day 2 09:00 12:00 presentations/discussion Part II (2 group discussions II.A, II.B) Day 2 13:30 15:30 presentations/discussion Day 3 09:00 12:00 presentations/discussion Organizers have proposed teams for the four groups I.A, I.B, II.A, II.B mixing in particular research and industry Feel free to change group

11 Proposal for Groups Group A this room 1. Toshiaki Aoki, JAIST 2. Nikolaj Bjorner, Microsoft Research 3. Udo Brockmeyer, BTC Embedded Systems AG 4. Werner Damm, OFFIS 5. Wolfgang Grieskamp, Google 6. Reiner Haehnle, Technical University of Darmstadt 7. Tomoyuki Kaga, Toyota Motor Corporation 8. Takashi Kitamura, AIST 9. Alexandre Petrenko, CRIM 10. Jun Sun, Singapur University of Technology and Design 11. Kenji Taguchi, CAV Technologies/ AIST 12. Willem Visser, Stellenbosch University 13. Burkhart Wolff, University Paris-Sud 14. Matthias Woehrle, Bosch Group B room Cyrille Arth, AIST 2. Achim D. Brucker, SAP Research 3. Rance Cleaveland, University of Maryland 4. Darren Cofer, Rockwell Collins 5. Benjamin Krämer, Technical University of Munich 6. Takuro Kutsuna, Toyota Central R&D Labs. Inc. 7. Darko Marinov, University of Illinois 8. Mark Utting, University of Waikato 9. Jan Peleska, University of Bremen 10. Alexander Pretschner, Technical University of Munich 11. Makoto Takeyama, Kanagawa University 12. Tetsuya Tohdo, Denso Corporation 13. Margus Veanes, Microsoft Research 14. Michael Whalen, University of Minnesota (UMSEC)

12 Flow of Meeting Roadmap (Chair: Prof. A. Pretschner) Day 3 13:30 15:30 Presentations by reporters of Technology Push Sessions Day 4 09:30 12:00 Consolidating the results of the subgroups into a single roadmap

13 GAP ANALYSIS

14 Structure of presentation The following information is based on numerous discussion with automotive OEM/suppliers e.g. in my role as Chairman of SafeTRANS ( Focus on three gaps coping with complex environments coping with complex systems coping with disruptive technology changes

15 Coping with complex environments overarching challenge: assuring sufficient levels of consistency between world model: what the car believes to be true about its environment (surrounding traffic: position, speed, intention,., road: road surface, lanes, ) and the real world NOTE: both directions critical: not in real world not in world model in real world in world model

16 Understanding the world in which we act 16 Copyright Prevent Project

17 what aspects of the real world are essential?

18 The discrepancy between the real world and what the aircraft perceives as real decide over life and death Aircraft thought it was still airborne, because only two tons weight lasted on the wheels due to a strong side wind and the landing maneuver. The computer did not allow braking. The plane ran over the runway into a rampart.

19

20 why traditional testing fails classical approaches with verification using test vehicles do not scale to this level of complexity, are no longer cost effective (would require 100 Million km test driving)

21 can the driver take over? conditional automation (SAE terminology) how can we test whether the driver is able to take over within 10 secs? given that sufficient coverage by vehicle testing is commercially not viable

22 COPING WITH COMPLEX SYSTEMS

23 Copyright Prevent Project Complex Systems 23 Source: Aramis Project

24

25 Challenge I Heterogeneity and plurality of test objectives hundreds of classes of different test objectives Each class addresses specific business needs, such as reliability, availability, safety, robustness, maintainability, costs, etc, Each class requires specific methods to guide the transition from (typically informal) requirements to test cases executable on test platforms

26 Challenge II Pushing the technology frontier for testautomation and analysis methods Expressivity: covering all classes of mathematic models required to cover all test-objectives (probabilistic, timed, hybrid systems) covering all forms of representations of test models E.g. Matlab/Simulink Stateflow/Scade/UML, C-code,... Scalability To complexity of industrial applications Quality Reduction in indeterminate values for static analysis High model coverage for automatic test generation

27 COPING WITH DISRUPTIVE TECHNOLOGIES

28 Examples 1. from 1 function = 1 ECU to Autosar based implementations a) one ECU hosts several functions b) one function distributed over several ECUs 2. from single core to multi core processors 3. from 2d transistor realization to 3d transistor realization 4. massive cross-platform re-use

29 The challenge: Implicit assumptions no longer valid 1a interference between SW functions with different criticality levels on one ECU 1b loss of stability of control laws due to unpredictable jitter 2 unpredictability of Worst Case Execution times due to e.g. cache interferences 3 new types and distribution of failure models, e.g. aging, dependency on power intensity of application on surrounding cores

30 The problem record number of recalls e.g. in US alone 40 mill vehicles in 2013 Warranty Week estimates costs for recalls to reach 40,000 Million US $ in 2013

31 The need how can we mature processes such that implicit assumptions become explicit? how can we assure that a particular integration context meets such assumptions?