Test And Validation: Coping With Complexity The state of play in vehicle software and system validation

Transcription

1 David Bailey, ETAS GmbH Test And Validation: Coping With Complexity The state of play in vehicle software and system validation 1 05.Jun.2008 Copyright 2004, ETAS GmbH LiveDevices Ltd. Vetronix Corp. All rights reserved. The names and designations used in this document are trademarks or brands belonging to the respective owners.

2 Agenda Test & Validation How far have we come? What is driving complexity? How can demands for greater complexity & reliability be reconciled? 2

3 Cars: From Yesterday to Today Form & Function versus Content Wheels Powertrain Steering 3

4 Today Example: BMW series (E65) Offboard diagnostics interface Central gateway ECU Low-Speed CAN Comfort Low-Speed CAN Comfort MOST Multimedia Byteflight Passive Safety High-Speed CAN Powertrain/ Chassis Over 65 networked ECUs 4 bus-systems ~ 116 MB code up to 900 functions 4

5 The drivers of Complexity Warranty costs Increasing number of variants per platform 5 Ever tighter emissions with safety regulations coming soon! Increasingly complex subsystems

6 Test Systems: From Yesterday to Today Simulation (Hard Wired) Electrical Connection to the ECU Computing nodes User Interface Automation Virtual Environments Which drivers have driven this developments? LABCAR-Network test system

7 Drivers in 3 Dimensions Application Test-processes Test System Configuration Test Execution Yesterday Focus on Technology Today Strong focus on economics Process-orientation Technology Increasing use of PC technology and Open Buses Open Models Open run-time environments Economics Cost-Benefit Relationship Cost over lifetime Re-usability Process integration 8

8 Test & Validation: from Yesterday to Today Trends and consequences Application Technology Economics Trends Increasing ECU functionality (Quantitive & Qualitative) Yesterday Manual Testing Simple Models Slow real-time micros Today Automated Testing Complex Models Fast real-time micros Increasing Networking (Quantitive & Qualitative Sinkendes Budget 9

9 10

10 Increasing ECU functionality: Fast real-time µs replace slow ones Non-Real Time Technology [ms] Quelle: ETAS ES1130 (PowerPC) 0.70 RTPC (2.0 GHz) 0.40 RTPC (3.2 GHz, Pentium 4) Improved Price/Performance Ratio 0.27 RTPC (2.4 GHz, Core2Duo) Calculation time [ms] Euler-integrations Vehicle Dynamic Model LABCAR-VDYM V5.0 Excludes I/O turnaround 11

11 Multi-Core Processors Triggered Sub-Systems Benefits of multi-core processors in model execution Real-time model running in two timing loops including hardware access Slow Model Part µs Fast/Slow Signal IO Technology µs Signal Flow between Fast/Slow Model Part ECU Fast Model Part Electronic Control Unit 12

12 Embedded Software Testing Requirements TASK: Increase test cost & coverage with lower budgets Economics Requirement Test Stand Independence Independence from Development language Parametrised (logical) test cases Test Project Management Integration in Testprocesses Advantages & Characteristics Guaranteed re-usability of test-cases between projects, systems, versions and variants Protection of Intellectual Property Reduction of implementation effort (once, for ever) Reduction of learning effort Re-usable development environment Simple but powerful adaptations possible for specific test applications Limiting the development effort one test-case per specification Lower the administration effort Management of different combinations of test cases with parametersets with minimal effort Increase Transparency & Traceability Open interfaces (eg to requirements management tools (Doors, SVN ) Dedicated roles (There is no universal test-environment!) 13

13 Industry Standardization Economics 14

14 Industry Standardization ISO ECU Description in ODX Format ODX ISO22901 Application Economics ODX Format Converter ODX Runtime Data D-Server API MVCI Diagnostic Server (D-Server) ISO (Hardware Specification) D-PDU API MVCI Protocol Module Software ECU ECU ISO

15 Example Test Environment for OBD Development & Validation HIL Fault Insertion PC J1699 / OCT Application ODX-Link D-Server ODX-Link EDC7 ETK ES590 INCA ES1000 ASCET Rapid prototype of new OBD strategies 16

16 Description and allocation of roles Test automation completes or replaces manual testing Application Test Specification Test Plan Test Case Library Test Projects Reports Test Case Developer Test Manager Project Handling Test Parameterization Test Campaigning Test Handler Execution Report Viewer Units under Test UuT Description Test Environment Test Bench Configuration Result & Found Issues Management 17

17 Requirements of the Test Developer There is no universal test language! Application Language Type Test oriented Script Graphical Software Development Example TTCN-3 Python, Perl, m Simulink, UML tools, NI TestStand C#, VB, C++ Comment Rather complex Has specific Test constructs ( e.g. Verdicts ) Small and specialized user circle Little learning effort Limited Process security (no compiled test-cases) Normally not strong typed (a big problem for operational consistency) User friendly with guided operations Guaranteed syntactical correctness Sub-optimal for sequential test architectures Possible loss of oversight with big projects Lack of transparency Very powerful development environment (also low cost) Large user group Higher learning effort No specific Test Constructs (e.g. verdicts) 18

18 LABCAR-RTPC RESET POWER HDD BUZZER ALARM HOLD OUT Example: Test Parameterization Fundamental for re-usability Application Use of the parameter Adaptation of Test Systems Setting and definition of the working point of the tests Description Data for specific Test-system configurations, time-outs & tool options. Environmental Data, Fine-tuning of the plant-models to the test conditions (i.e. dependent on the Unit under Test) Parameter Data Data A 5 B A A = true 5 5 B B true = true A 5 B A A = true 5 2 B B true = false Test Test Case Behaviour A = Param B = Param MA.doThis(A) EA.doThat(B) Setting the Test-points and Test -vectors Describing the Test scenario. (e.g, arc diameter, entrance speed, exist speed, braking force for ESP test) Settings for Evaluation Method e.g. Selection of method and definition of pass/fail thresholds 19

19 Increasing ECU functionality Complex Real-Time models replace simple RT models Technology < 10 ODE < 150 ODE Yesterday: Simple two-track model for Vehicle Dynamics (Source: Dynamik der Kraftfahrzeuge, Mitschke) Today: Complex MBS-vehicle dynamics model with Axel-geometry (Source: INTEC, LABCAR-VDYM V5.0) ODE = Ordinary Differential Equations, MBS = Multi-body simulation 20

20 Model Optimization using both physical & statistical, data driven methods (e.g.tlrnns, SOM/TFA/LLM) The Problem There is a break in the tool-chain between CAE tools used for basic powertrain design and plant models used for controls development How to assess design impact on emissions? How can complex models still run in real time? Solutions Complex logical optimization of the physical model (eg:wave RT von Ricardo) Statistical Modelling Tools (Gamma Technologies, The Mathworks) Engineering solutions. Technology 21

21 HiL-Simulation: from Yesterday to Today Trends and consequences Application Technology Economics Trends Increasing ECU functionality (Quantitive & Qualitative) Yesterday Today Increasing Networking (Quantitive & Qualitative) Sinking Budget Budget Single HiL Systems Slow Bus-systems Sequential Development process Networked HiL Systems Fast Bus-Systems Iterative Development Process 22

22 Increasing Networking Indicators Technology 23

23 Increased Networking HiL- & PC worlds Fast bus systems replace slow ones Quelle: National Instruments White Paper - Bus Performance.pdf 24

24 Increasing Networking Iterative Processes replace Sequential ones Economics Quelle: POSTER_SW-Qual_EntwProz.pdf Release Management Integration Test Version Planning Requirements analysis Use-Case storage Vehicle Development is largely software development The Development process should adapt accordingly Codierung Code-Generierung Testautomatisierung Design sample System Partitioning Release-Planning Yesterday: Sequential Development Proces Today: Iterative Development process. Increasing coherence MiL-SiL-HiL 25

25 HiL-Simulation: From Yesterday to Today Trends & Consequences Application Technology Economics Trends Increasing ECU functionality (Quantitive & Qualitative) Yesterday Today Increasing Networking (Quantitive & Qualitative) Sinking Budget Budget System-based service Proprietary models Technology-driven product solution Solution-based service Open models Application driven Ppoducts Re-usable test-cases Outsourcing of test activity 26

26 Sinking Budget, rising requirements Open model solutions integrate with complex models Technology Economics Project-specific adaptation and parameterization Project-specific adaptation and parameterization Models from Tier 1s & OEMs HiL- specific model-products (Vehicle Dynamics, Powertrain, Drive-train, Driver, Environnment etc.) Models from specialized model vendors Classical off-line models HiL specific model components Yesterday: HiL-specific real-time models Today: HiL specific models extended with a variety of specialized models from different sources 27

27 Sinking Budget Rising Requirements Open integration example Technology Economics Real Time Linux PC (Customer Model) Off-the-shelf PC interface (reflective memory card) allows real-time signal exchange ETAS LABCAR with ETAS ( ) Modell ECU 28

28 Sinking Budget Application-driven solutions replace technology-driven ones Economics Power- Train Chassis Body Others High-End HiL-System Standard HiL-System Yesterday: Technology Driven (z.b. ETAS VME-LABCAR) Basic HiL-System 29

29 Sinking Budget Application-driven solutions replace technology-driven ones Economics Power- Train Chassis Body Others High-End HiL-System Standard HiL-System Basic HiL-System Today Application Driven e.g. ETAS PT-LABCAR) 30

30 What will the future bring? Application Application New Applications Technology Increasingly virtual testing Reusability & Interoperability Common Test Environments deployed at new stages of development Economics Standardization Test languages, Tool APIs Automation Increasing Test Automation, also other development processes automation (e.g. calibration) 31

31 What will the future bring? Technology Technology Increasing Processing Power DualCore, QuadCore, Faster PC Buses PCI-Express, US Virtualization Increasingly high fidelity,realtime capable models Increasing model Types (e.g. Processor, ECU models, network models) 32

32 What will the future bring? Application Economics Further Efficiency Drives 24/7 Test Houses Technology Economics Further Cost Pressure Road to Lab to Math Outsourcing Reducing costs but driving requirement for MUCH improved process security Standardization Further industry wide standardization moves Increased utilization of de facto standard technology (PC,.NET ) 33

33 Thanks for your attention! 34

34 David Bailey Business Development Manager ETAS GmbH David is responsible for business development at ETAS for Test & Validation solutions. He has been working at ETAS for 4 years. Previously David has worked for Dearborn Group Inc where aside from business management in Europe he participated in a number of standardisation committees related to vehicle bus protocols & ECU reprogramming. 35