http://www.mbat-artemis.eu/

ARTEMIS Joint Undertaking
ARTEMIS Joint Undertaking

The public private partnership for R&D a...
ARTEMIS Joint Undertaking

2

© MBAT, ARTEMIS project 269335
Embedded Systems in a Car (ECUs)
as example Targets of MBAT

© Daimler

3‐4 networks
100+ ECUs
2017 autonomous
2020 driver...
Automotive Test Environment
Example application domain for MBAT:
automotive HIL integration test
environment for model-bas...
MBAT’s overall Challenges




V&V technologies are still not effective and efficient
enough
V&V costs for Embedded Syst...
MBAT‘s European impact
MBAT outcomes will contribute
to increase the competitiveness
of European transportation
products i...
MBAT’s Project Character

MBAT is
 an ARTEMIS project, thus focussing on embedded
systems
 an industrial-oriented R&D pr...
MBAT’s Market Impact
MBAT will increase the competitiveness of European key players in 
transportation domain by
 reducin...
MBAT Technological 
Innovation 
MBAT = Combined Model‐based Static Analysis and 
Dynamic Testing of Embedded Systems
Embed...
MBAT Outcomes




Industrial‐approved MBAT Reference Technology Platform 
(MBAT RTP) supporting Validation & Verificatio...
MBAT Use Cases
UC No.

Use case name
Automotive Use Cases

Use case driver

UC A1

Brake-by-Wire

VOLVO

UC A2

Common Pow...
Classification of Techniques
(Automated) 
V&V 
Techniques

Dynamic 
Techniques

Static 
Techniques

Testing

Monitor‐
ing
...
Main MBAT Method
Analysis 
Cases

Req.

Verification
Plan/Status

Results,
T&A Model(s) Coverage

Test Cases

ARTEMIS Join...
ok
Success
(verified)

Engineering
Artifacts

ModelAnalysis

3

Verify req using sim/test as
approximation,
or re-verify u...
AVL’s HCU Initial Combination

Hybrid powertrain control unit (HCU) that is responsible for coordinating 
the energy flows...
Refinement Process
•

Overall
Methodology

•
•

•

Workflow /
Combination
Patterns

•
•

Framework describing general work...
Reduce warnings from Static 
Code Analysis
report

Purpose: Reduce number of warnings from static
code analysis by more ex...
Instance (Work in progress)
Purpose: Reduce number of warnings from static
code analysis by more exact analysis
Pre-condit...
Reduce warnings from Static 
Code Analysis
report

4

Confirmed defects

Report
merger

Confirmed defects 
Remaining warni...
Increase Coverage by Analysis
3

Test
Model

(Counter
example
based) Test
Case

Model-checker

2

1

Simulation
based test...
MBT with analysis 1
Analysis
Objectiv
es

3

Model-checker

Test
Model

(Formalized)
Requirement
s
1

2

Model-based
test ...
MBT with analysis 2
Purpose: Rule out further
defects along known failing
test

Analysis
Objectiv
es

3

Model-checker

An...
Target MBT to failing test case
Purpose: Generate additional 
related test cases in the same 
model neighborhood  due to b...
Target MBT to suspect areas
Test
Cases

Purpose: Target suspected parts 
of SUV&V with  additional 
analysis and test case...
MBAT RTP in more Detail
A Reference Technology Platform (RTP), like the ARTEMIS MBAT RTP,
provides a set of management or ...
MBAT 2nd Full Plenary Meeting in 
Copenhagen May 2012

ARTEMIS Joint Undertaking

31

© MBAT, ARTEMIS project 269335
Upcoming SlideShare
Loading in …5
×

ARTEMIS Project MBAT: Advanced Validation & Verification of Embedded Systems af Brian Nielsen, CISS/AAU

1,150 views
923 views

Published on

Oplægget blev holdt ved et seminar i InfinIT-interessegruppen Softwaretest. Læs mere om interessegruppen her: http://infinit.dk/dk/interessegrupper/softwaretest/softwaretest.htm

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,150
On SlideShare
0
From Embeds
0
Number of Embeds
147
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

ARTEMIS Project MBAT: Advanced Validation & Verification of Embedded Systems af Brian Nielsen, CISS/AAU

  1. 1. http://www.mbat-artemis.eu/ ARTEMIS Joint Undertaking ARTEMIS Joint Undertaking The public private partnership for R&D actors in embedded systems  © MBAT, ARTEMIS project 269335
  2. 2. ARTEMIS Joint Undertaking 2 © MBAT, ARTEMIS project 269335
  3. 3. Embedded Systems in a Car (ECUs) as example Targets of MBAT © Daimler 3‐4 networks 100+ ECUs 2017 autonomous 2020 driverless ARTEMIS Joint Undertaking 3 © MBAT, ARTEMIS project 269335
  4. 4. Automotive Test Environment Example application domain for MBAT: automotive HIL integration test environment for model-based testing of embedded systems (interactions) © Daimler © Daimler ARTEMIS Joint Undertaking 4 © MBAT, ARTEMIS project 269335
  5. 5. MBAT’s overall Challenges    V&V technologies are still not effective and efficient enough V&V costs for Embedded Systems are too high (still up to 50% of Embedded System’s total development costs) V&V technologies should improve the error detection rate ARTEMIS Joint Undertaking HIL Test Environment © Daimler 5 © MBAT, ARTEMIS project 269335
  6. 6. MBAT‘s European impact MBAT outcomes will contribute to increase the competitiveness of European transportation products industry  MBAT will support higher quality European transportation products at reduced development costs  ARTEMIS Joint Undertaking 6 © MBAT, ARTEMIS project 269335
  7. 7. MBAT’s Project Character MBAT is  an ARTEMIS project, thus focussing on embedded systems  an industrial-oriented R&D project to transfer tool innovations and academic research into industrial application  strongly driven and evaluated by industrial use cases ARTEMIS Joint Undertaking 7 © MBAT, ARTEMIS project 269335
  8. 8. MBAT’s Market Impact MBAT will increase the competitiveness of European key players in  transportation domain by  reducing V&V costs for embedded systems by at least 20 % (keeping the  planned level of quality)  shortening time‐to‐market by at least 20 %  increasing the coverage of the embedded system under V&V by at least 30  %  significantly increasing the probablitity to uncover errors  enabling higher quality and safer embedded systems & embedded  systems based products ARTEMIS Joint Undertaking 8 © MBAT, ARTEMIS project 269335
  9. 9. MBAT Technological  Innovation  MBAT = Combined Model‐based Static Analysis and  Dynamic Testing of Embedded Systems Embedded Systems Descriptions Analysis Cases Static Analysis Analysis Results Analysis Test & Analysis Models Test Test Cases ARTEMIS Joint Undertaking 10 Dynamic Tests Test Results © MBAT, ARTEMIS project 269335
  10. 10. MBAT Outcomes   Industrial‐approved MBAT Reference Technology Platform  (MBAT RTP) supporting Validation & Verification of Embedded  Systems Experience Packages describing the usage of the RTP in  industrial domains (automotive, aerospace, rail) ARTEMIS Joint Undertaking 11 © MBAT, ARTEMIS project 269335
  11. 11. MBAT Use Cases UC No. Use case name Automotive Use Cases Use case driver UC A1 Brake-by-Wire VOLVO UC A2 Common Powertrain Control (CPC) DAI UC A3 UC A4 UC A5 UC A6 UC A7 UC A8 Adaptive Brake Light (ABL) Turn Indicator Control (TIC) Transmission Controller Product Line Passive Balancing Hybrid Power Train Control Unit Virtual Prototype Airbag ECU DAI DAI RIC AVL AVL IFAT Aerospace Use Cases UC AE1 UC AE2 UC AE3 UC AE4 UC AE5 UC AE6 UC AE7 UC AE8 UC AE9 Flight Control Program ACSL Component for Flight Control Computer Flight Warning Program Flight Management System/UAV Degraded Vision Landing Aid System DeViLA System TALARION - Unmanned Aerial Vehicle (UAV) Flight Guidance System (FGS) Attitude and Altitude (A&A) for Helicopters Spacecraft Central Software-Sentinel 3 AIR AIR AIR ASIA EADS DE EADS IW RCF RCF TAS Rail Use Cases UC T1 UC T2 UC T3 ARTEMIS Joint Undertaking Automatic Train Control Rapid Transit Metro System (Ansaldo STS) Validator of the ZLB ATOP System 12 ALSTOM ANSALDO SIEMENS © MBAT, ARTEMIS project 269335
  12. 12. Classification of Techniques (Automated)  V&V  Techniques Dynamic  Techniques Static  Techniques Testing Monitor‐ ing Simulation Model‐ checking MiL Testing Runtime  Verifi‐ cation Statistical  Model  Checking Software  Model‐ checking ARTEMIS Joint Undertaking Refinement‐ checking Abstract  Interpreta‐ tion Symbolic  Exec Theorem  Proving (hybrids) © MBAT, ARTEMIS project 269335
  13. 13. Main MBAT Method Analysis  Cases Req. Verification Plan/Status Results, T&A Model(s) Coverage Test Cases ARTEMIS Joint Undertaking Analysis Test © MBAT, ARTEMIS project 269335
  14. 14. ok Success (verified) Engineering Artifacts ModelAnalysis 3 Verify req using sim/test as approximation, or re-verify using strengthen assumption (or refine req.) Failed Reqs V&V Planning 1 2 (MB) Testing V&V Objectives to be tested 5 V&V Objectives to be code checked T&A model(s) Inconclusive V&V Objectives to be analysed Suspected or new case 4 Feedback Success (pass) 3 Correct system, and analyze model in context of trace and test case to rule out similar errors Failed Reqs Refine req and test case Inconclusive 5 4 Feedback ok 1) Make initial V&V Plan that map requirements/V&V objectives to most suitable techique 5) Update V&V plan and status based on ARTEMIS Joint Undertaking results Success (verified 3 Failed Reqs Inconclusive Suspected or new case 2) Construct analysis and/or test model(s) Define new analysis or test cases for model-analysis (or invariants for static code analysis) ok Suspected or new case CodeAnalysis Correct model, design, code, and repeat V&V of all impacted sys. Add cases for regression check 3) Execute 4) evaluate results Define analysis cases for modelanalysis Define invariants for static code analysis Uncovered items: try to target these using model analysis Correct code, or verify weaker invariant, and analyse implication on models level use testing ”Maybe satisfied” property: derive test High warning density High complexity © MBAT, ARTEMIS project 269335
  15. 15. AVL’s HCU Initial Combination Hybrid powertrain control unit (HCU) that is responsible for coordinating  the energy flows between engine, electrical motor, and the battery. ARTEMIS Joint Undertaking © MBAT, ARTEMIS project 269335
  16. 16. Refinement Process • Overall Methodology • • • Workflow / Combination Patterns • • Framework describing general workflow and most  A&T combination strategies Holistic view Domain‐ and tool‐independent Pattern=reusable solution to a commonly occurring  problem  Pattern for common A&T Combination strategies Typically focuses on only a part of the V&V flow • (sub) Method Instance RTP Instance ARTEMIS Joint Undertaking A specific chosen set of notations (reqs, models,  traces, etc.) Specific type of results and data to be exchanged  (syntax and semantics) Specific set of tools   • • Workflow, and data exchange supported by the RTP Tools integrated/interoperable • • © MBAT, ARTEMIS project 269335
  17. 17. Reduce warnings from Static  Code Analysis report Purpose: Reduce number of warnings from static code analysis by more exact analysis 4 Confirmed defects Report merger Confirmed defects  Remaining warnings 1 Code + config Static Code Analyzer warnings 2 Model generator • Program slice • Path precondition 3 ModelAnalyzer • Semantic preserving Model • Property UC T3 SIE “ZLB ATOP System”: SAT‐solving using RTT+ UC A2 DAI “CPC”:  model‐checking ARTEMIS Joint Undertaking © MBAT, ARTEMIS project 269335
  18. 18. Instance (Work in progress) Purpose: Reduce number of warnings from static code analysis by more exact analysis Pre-condition: first step conducted by abstract interpretation (over-approximation) Maturity: research Variants: Notes: Confirmed defects Code + config report Report merger Astrée +slicer+excha nge format for invariants Confirmed defects  Remaining warnings Uppaal Warnings Precondition Slice Static Code Analyzer Model generator Model Analyzer Significant effort!! UC A2 DAI “CPC”:  model‐checking ARTEMIS Joint Undertaking © MBAT, ARTEMIS project 269335
  19. 19. Reduce warnings from Static  Code Analysis report 4 Confirmed defects Report merger Confirmed defects  Remaining warnings 1 Code + config Static Code Analyzer warnings 2 Test input generator • Program slice • Path precondition 3 Test Execution • Instrumente d Program (Oracle) • Test case UC T3 / SIE “ZLB ATOP System” UC AE8 RWC “Attitude and Altitude for Helicopters”  ARTEMIS Joint Undertaking © MBAT, ARTEMIS project 269335
  20. 20. Increase Coverage by Analysis 3 Test Model (Counter example based) Test Case Model-checker 2 1 Simulation based test generator Two Patterns?  • Model‐coverage • White‐box SUT/Code  Coverage Coverage report Coverage Analyzer Same test suite / test  format? Is it possible to transfer a path  synthesized  test case to model  level? Test suite Test Case 4 SUT Executor + Coverage Evaluator Coverage report oracle 6 Test Input 5 Path synthesizer Can model serve as  Oracle? Alternative: Coverage based  test generation + Coverage completion by  simulation UC_AE6 EADS TALARION UC_AE7 RWC “Flight Guidance System”: MC/DC Coverage UC T2 ? ASTS 3.1.9 ANSALDO “Rapid Transit Metro System” ARTEMIS Joint Undertaking © MBAT, ARTEMIS project 269335
  21. 21. MBT with analysis 1 Analysis Objectiv es 3 Model-checker Test Model (Formalized) Requirement s 1 2 Model-based test generator Test objecti ves Test suite 4 SUT Report 5 Executor e.g. UC A1 Volvo BBW ARTEMIS Joint Undertaking Fail: Hypothesis: most  likely impl is wrong  because model was  checked wrt req’s  Pass: Hypothesis: Impl  satisfies requirements  because model satisfy  reqs and impl refines  model © MBAT, ARTEMIS project 269335
  22. 22. MBT with analysis 2 Purpose: Rule out further defects along known failing test Analysis Objectiv es 3 Model-checker Analysis Model (Env + SUT) Model-based test generator 2 SUT Executor ARTEMIS Joint Undertaking (Formalized) Requirement s Test objecti ves Abstracted failed test trace (or observed suspect behavior) Test suite 1 Report 1. 2. 3. Model‐check could not verify  all requirements on analysis  model (spate space too large) Model‐check in context (as  environment model/input) of  failing test case (to reduce  state space) Higher confidence, targeted  analysis: confirm/exclude  “similar” errors © MBAT, ARTEMIS project 269335
  23. 23. Target MBT to failing test case Purpose: Generate additional  related test cases in the same  model neighborhood  due to bug  cluster assumption Pre-condition: failed test case, notion of neighborhood Analysis Objectiv es Model-checker Test Model (Formalized) Requirement s 1 Model-based test generator Test objecti ves 3 failed test case Test suite 2 SUT neighborhood • A related test path found by  choosing alternative  outcome at branching point  in the original path [Peled, in FME 2001] • Small “trace distance”  Report Executor ARTEMIS Joint Undertaking © MBAT, ARTEMIS project 269335
  24. 24. Target MBT to suspect areas Test Cases Purpose: Target suspected parts  of SUV&V with  additional  analysis and test cases  Pre-condition: notion of neighborhood, mapping 3b Test Objectives Model-based test generator Model 3a Model-check Analysis Objectives Model  Mapping 1 SU V&V Report 2 Static analysis High warning density High (cyclomatic) complexity Model Mapping: links  code level element (“defect  area”  e.g., function/statement) to Model‐level  element (eg. component or transition) • Traceability info?  • Auto generated code (e.g. Daimler impact  analysis for Simulink) • Manual inspection ARTEMIS Joint Undertaking © MBAT, ARTEMIS project 269335
  25. 25. MBAT RTP in more Detail A Reference Technology Platform (RTP), like the ARTEMIS MBAT RTP, provides a set of management or engineering methods and processes, as well as engineering tools, which will be used to compose/build a complete engineering environment Integrated subset of RTP components. The interoperability approach is based on the IOS (RTP – Tailoring) An Interoperability Specification (IOS) will guarantee these needs for interoperability and collaboration between tools across the entire engineering lifecycle ARTEMIS Joint Undertaking 30 © MBAT, ARTEMIS project 269335
  26. 26. MBAT 2nd Full Plenary Meeting in  Copenhagen May 2012 ARTEMIS Joint Undertaking 31 © MBAT, ARTEMIS project 269335

×