Speech slides about describing a Model Based Testing approach for the validation of Automotive embedded system with Hardware In the Loop simulators. Tools used are MaTeLo & EXAM.

1. Usage
Model-Based Testing
for Hardware In the Loop
THE SOLUTION TO BOOST
YOUR TEST EFFICIENCY

2. ALL4TEC
Since 1998
French: 45 experts
Turnover 2009: 4 M€
Expertise offer:
QUALITY
SAFETY ENGINEERING EMBEDDED
SYSTEM ENGINEERING SYSTEMS
TESTING

3. QUESTION YOUR TEST DEPARTMENT
What is your cost dealing with various:
Environment Models Test Tools Specification Approaches
Test Platforms Measurement Devices
How long do you spend trying to:
Reuse thru different engineers
Reuse thru different projects
Reuse thru different car brands
How hard do you fight to:
Estimate the quality Calculate the functional coverage
Make documents, reports
Imagine you can use all this invest for:
INCREASING TRUST YOU HAVE IN YOUR PRODUCTS

4. MODEL BASED TESTING PROCESS
System
Requirements
Requirements Management
Usage Usage Model Editor
Model
Requirements
Test
Cases
Traceability Test-Cases Generator
Model
Coverage Test Campaign Analysis
Test
Scripts
Adaptor
Reliability
Platform API
Test Execution
Platform
Mean Time
To Failure
Test Runner
System
Intensity Hardware In the Loop
of Test
Under
Test System Under Test

5. WHAT IS MaTeLo?
Formal Test
Specification
System
Requirements
Product Enhanced
point of view Coverage
Usage
Model
Fully
Test
Requirements
Traceability
Measured
Cases
Model process
Coverage
Test
Test environment
Scripts
Compatibility
Adaptor
Reliability
Test Execution Mean Time
Platform To Failure
Requirements System
Intensity
of Test
Traceability Under
Test Better Engineering
Productivity

6. WHAT IS EXAM?
Formal
Better
Test Cases
Usage of
Specification Abstraction Simulators
Layers & Devices
Centralized
Test Repository
Method designed
& consolidate
Automotive
Ready Library
Focused on Reuse
& Collaboration

7. MATELO USAGE MODEL
(1) ALL POSSIBLE
USAGES SCENARIO (3) FORMAL
REQUIREMENT TEST
SPECIFICATION
(2) DYNAMIC (4) CONSISTENCY BETWEEN
REPRESENTATION 1. TESTS
2. REQUIREMENTS
3. USE CASES

8. TESTING SCOPE
 Process
 Black Box testing
 Field Application
 Functional Testing
 Integration Testing
 Acceptance Testing
 System Under Test
 Model
 Software
 System
 Product

9. PRODUCT SET
 MaTeLo Editor: Model Based Test Specification
 Design the usage model
 Qualify the possible path with profiles
 Assign test operations and requirements
 MaTeLo Testor: Model Based Test Cases Generation Start
 Define the test strategy
 Generate the test cases
Close
 EXAM: Test Automation
 Manage the test cases on the test repository
 Manage the test environment heterogeneity
 Execute an log automatically the test sequences
 MaTeLo Test Campaign Analysis: Model Based Quality Insurance
 Analysis of the Test Campaign
 Quality & Reliability assessment

10. System
Requirements
Usage
Model
Requirements
Traceability
MaTeLo EDITOR
Test
Cases
Model
Coverage
Usage Model Design
Test
Scripts
Adaptor
Reliability
Test Execution Mean Time
Platform To Failure
Intensity
System
of Test
Under
Test

11. MATELO SCREENSHOT

12. MATELO USAGE MODELS
 Probability on every Choices BEGINNING
 Association of Requirements
 Configuration of Test Operations
TRANSITION
MACRO CHAIN
STATE
END

13. MODEL TRANSITION = TEST STEP
Select_Gear(5)
Accelerate(100%)
Check_Speed(185)
EXAM operations
Doors Req.
Stimulations
Inputs Stimulation
Classes of Equivalence Output ports
Network signals
Input ports Environ. Variables
Test Oracle
Networks signals Wave Form
Variables …
GUI objects Outputs = f(Inputs)
… Expected Results
Checks Outputs

14. MATELO PROFILE
 Profiles can be embedded to qualify the usage model
 Operational profiles
 Test profiles
Data distribution Usage path probability
87 (+/- 25)
347
0-100-150-200-500

15. System
Requirements
Usage
Model
MaTeLo EDITOR
Test
Cases
Requirements
Traceability
Model
Requirements Management
Coverage
Test
Scripts
Adaptor
Reliability
Test Execution Mean Time
Platform To Failure
Intensity
System
of Test
Under
Test

16. REQUIREMENTS MANAGEMENT
IBM Doors MaTeLo Req Library MaTeLo Usage Model
Import Model
Update Association
CSV
(Reqtify, Excel …)
Model Object
-Transition
-Input, Expected Result
-Value, Class of Equivalence
XML Requirement Role
-Necessary
-Sufficient
MaTeLo Project

17. System
Requirements
Usage
Model
MaTeLo TESTOR
Test
Cases
Requirements
Traceability
Model
Test Cases Generation
Coverage
Test
Scripts
Adaptor
Reliability
Test Execution Mean Time
Platform To Failure
Intensity
System
of Test
Under
Test

18. MaTeLo TEST STRATEGY
Most probable Field application Arcs coverage Usage
approach approach approach approach
Start Start Start Start
Custom
Test profile
Usage
Test profile
Close Close Close Close
FREQUENCY RISK, CRITICISM COVERAGE USAGE
UPDATE… (Determinist, Stochastic)
 DEFINE THE TEST STRATEGY, BY CHOOSING
 Test Algorithm
 Test Profile Generate
 Part of model to test

19. MaTeLo TESTOR: HTML TEST PLAN
State
Requirement Expected
Input
Result

20. System
Requirements
Usage
Model
MaTeLo EDITOR
Test
Cases
Requirements
Traceability
Model
Test Bench Management
Coverage
Test
Scripts
Adaptor
Reliability
Test Execution Mean Time
Platform To Failure
Intensity
System
of Test
Under
Test

21. MaTeLo FOR EXAM
Test Repository Requirements
 SUT Interface  Name
 Test Operations  UUID
 Stimulation  Description
 Measurement
 Administration
 Sub Test Sequences
Association
Automatic Test Cases
 Automatic call of Test Operation
 Automatic parameterisation
 Requirements association
Association  Test Case description generation
 Usage model respect
Generate

22. System
Requirements
Usage
Model
EXAM
Test
Cases
Requirements
Traceability
Model
Extended & Flexible
Coverage
Test Automation
Test
Scripts
Adaptor
Reliability
Test Execution Mean Time
Platform To Failure
Intensity
System
of Test
Under
Test

23. EXAM SCREENSHOT

24. ABSTRACTION LAYERS
Usage Scenario Markov-Chain
MaTeLo Description Usage Model
<< derive >>
Test Case UML Sequence
Specification Diagram
<< generate >>
Python
Test Flow Control
Precompiled
Device Driver
Application
<< control >>
Third Party Hardware In the
System Under Test
Equipment Loop Test Bench

25. COLLABORATIVE ARCHITECTURE
Multi Users Test Cases Implementer
Test Environnement Python Code Generator
Test Repository
Project, Libraries, Test Campaign

26. System
Requirements
Usage
Model
EXAM Modeller
Test
Cases
Requirements
Traceability
Model
Test Cases Description
Coverage
Test
Scripts
Adaptor
Reliability
Test Execution Mean Time
Platform To Failure
Intensity
System
of Test
Under
Test

27. EXAM UML TEST SEQUENCE
BEGINNING
Test Operation
Sub Sequence
END

28. EXAM INTERFACE CONCEPT
Test Case Description
INTERFACE
Test Case Implementations
CLASS
Sub Sequence Interface Implementation
 Virtual Function
 Virtual Car
 Sub Sequence
 Test Environment Access
Python Code  …

29. System
Requirements
Usage
Model
EXAM Modeller
Test
Cases
Requirements
Traceability
Model
Test Cases Management
Coverage
Test
Scripts
Adaptor
Reliability
Test Execution Mean Time
Platform To Failure
Intensity
System
of Test
Under
Test

30. EXAM ABSTRACTION MANAGEMENT
Variable Mapping Short-name Mapping
Simulation Model Abstraction Embedded Network Abstraction
Parameter Set Composing System Configuration
Concrete Test Case Abstraction Test Platform Abstraction

31. EXAM ABSTRACTION MANAGEMENT
Possible Usage Scenarios
Strategic Test Cases
Test Case 1 Test Case 2 Test Case n
Available Test Configurations
HIL Plateform Network Functions Test Tools Env. Model
dSpace CAN_1 Manual Gear CANoe Gasoline
NovaSim CAN_2 Auto Gear CANape Diesel
Carts LIN_X Hand Free INCA Turbo
ASAM HIL API FlexRay_1 ACC MS Excel Hybrid
Proprietary FlexRay_2 StartStop Diag Tool Electric
… … … … …

32. SYSTEM CONFIGURATION 1
Possible Usage Scenarios
Strategic Test Cases
Test Case 1 Test Case 2 Test Case n
Available Test Configurations
HIL Plateform Network Functions Test Tools Env. Model
dSpace CAN_1 Manual Gear CANoe Gasoline
NovaSim CAN_2 Auto Gear CANape Diesel
Carts LIN_X Hand Free INCA Turbo
ASAM HIL API FlexRay_1 ACC MS Excel Hybrid
Proprietary FlexRay_2 StartStop Diag Tool Electric
… … … … …

33. SYSTEM CONFIGURATION 2
Possible Usage Scenarios
Strategic Test Cases
Test Case 1 Test Case 2 Test Case n
Available Test Configurations
HIL Plateform Network Functions Test Tools Env. Model
dSpace CAN_1 Manual Gear CANoe Gasoline
NovaSim CAN_2 Auto Gear CANape Diesel
Carts LIN_X Hand Free INCA Turbo
ASAM HIL API FlexRay_1 ACC MS Excel Hybrid
Proprietary FlexRay_2 StartStop Diag Tool Electric
… … … … …

34. System
Requirements
Usage
Model
EXAM Test Runner
Test
Cases
Requirements
Traceability
Model
Test Cases Automatic Execution
Coverage
Test
Scripts
Adaptor
Reliability
Test Execution Mean Time
Platform To Failure
Intensity
System
of Test
Under
Test

35. EXAM TEST RUNNER
AUTOMATIC TEST SCHEDULER

36. System
Requirements
Usage
Model
EXAM Report Manager
Test
Cases
Requirements
Traceability
Model
Verdict and Analyze Test Runs
Coverage
Test
Scripts
Adaptor
Reliability
Test Execution Mean Time
Platform To Failure
Intensity
System
of Test
Under
Test

37. ANALYZE TEST RUN REPOSITORY
TEST CAMPAIGN RESULT & VERDICT MANAGEMENT

38. EXAM TEST RECORD MANAGEMENT
REAL TIME RECORD ANALYSIS AND VERDICTS

39. System
Requirements
Usage
Model
MaTeLo TCA
Test Campaign Analysis
Requirements
Test Traceability
Cases
Model
Coverage
Test
Scripts
Adaptor
Reliability
Test Execution Mean Time
Platform To Failure
Intensity
System
of Test
Under
Test

40. 2.
1.
Coverage
Reliability
Trust Indicators
New
Target Quality Criteria
Test
Test
Test
System
System
Model
Cases
Under
Usage
Scripts
Package
System
Adaptor
Platform
Requirements
Requirements
Test Execution
System
Requirements
Model
Usage
of Test
Intensity
Coverage
Reliability
To Failure
Mean Time
Model
Traceability
Usage
Model
Test
Test
Test
System
Model
Cases
Under
Usage
Requirements Scripts
System
Adaptor
Platform
Requirements
Test Execution
System System
TEST CAMPAIGN PROCESS
New version
Requirements
Requirements
Requirements
Test Traceability
Cases Requirements
Model Test Traceability
Coverage Cases
Model
Coverage
Model
of Test
Intensity
Coverage
Reliability
To Failure
Mean Time
Traceability
Requirements
Usage
Test Usage
Model
Scripts Model
Cases
Test
New Test
Adaptor Scripts
Generation
Reliability Adaptor
Test Execution Mean Time Reliability
Bugs
Platform Requirements
To Failure Requirements
Test Test Traceability Traceability Test Execution Mean Time
Cases Cases Intensity Platform To Failure
System
of Test
Model Model
Under
Coverage Coverage Intensity
Test System
of Test
Under
Test
Test Test
Scripts Scripts
Adaptor Adaptor
Reliability Reliability
Metrics
Test Execution
Test Execution Mean Time Mean Time
Platform Platform To Failure To Failure
Intensity Intensity
System System
Quality/Version
of Test of Test
Under Under
Test Test

41. COVERAGE CAPABILITIES
T 3-4 (Req_34)
Transitions Inputs:
Requirements Req 341
coverage
States coverage Signal_A: [0-10;11-50;51-100]
Req_3411; Req_3412; Req_3413
coverage Expected Result
Signal_B: > 47 & <= 62
S0 T 0-3 Req_342
T 0-1 T 3-4 Classes of Equivalence
T 0-2 S3
coverage
T 2-3 S4
T 3-5 Global
S1 coverage
S2 S5
 Indicators by SUT versions
 Unitary by version
 Cumulated over all SUT versions

42. TEST CAMPAIGN REPORT
Model Coverage
Requirements
Database
Requirements Coverage
MaTeLo
Test Campaign Analysis
Trust Indicators
Results
Database

43. MATELO & EXAM
System
Requirements
1) Provide all needed test layers
1) Usage Model
2) Test Case Description
Usage
3) Test Case Implementation
Model
2) Optimize all your test investments
1) Hardware In the Loop & Test tools
Requirements
Test Traceability
Cases 2) Test Engineers
Model
Coverage
3) Provide an open platform focused on
Test
1) Formalization and abstraction
Scripts
Adaptor
2) Reuse and collaboration
Reliability
3) Automatic generation
Test Execution Mean Time
Platform To Failure
System
Intensity 4) Optimize the test campaign for
of Test
Under
Test
1) Functional Coverage
2) Product Availability

44. QUESTION ?
www.all4tec.net
Booth 1720
Technical contact
MaTeLo@all4tec.net
Sales contact
anthony.faucogney@all4tec.net
+33 6 80 88 40 59
Wiki & Forum & Documentation
www.all4tec.net
www.exam-ta.de