The document discusses model-driven development (MDD) and its impact on testing for a nanotech case study involving the redesign of a vacuum system. MDD uses formal modeling and verification to generate defect-free source code from verified models, reducing the need for component testing but not replacing integration and system testing. For the vacuum redesign project, MDD improved quality by finding design errors earlier and automating reliability testing, leading to smoother integration and fewer test cycles.

1. MDD and its impact on testing...
A nanotech case study
Bryan Bakker
bryan.bakker@sioux.eu
@Bryan_Bakker

2. Contents
Intro
MDD
MDD and Testing
Vacuum redesign
Experiences
© Sioux 2014 | Confidential |
2

3. About Bryan Bakker
Test Expert
Certifications: ISTQB, TMap, Prince2
Member of ISTQB Expert Level on Test Automation
Tutor of several test related courses
Domains: medical systems, professional security systems, semi-industry, electron microscopy
Specialties: test automation, integration testing, design for testability, reliability testing
© Sioux 2014 | Confidential |
3

4. About Sioux
HERENTALS
NEDERWEERT
EINDHOVEN
UTRECHT
MOSCOW
© Sioux 2014 | Confidential |
4
DANANG

5. About FEI
World leader in electron microscopes
Light microscope: 1000x  200nm (limited by the wavelength of light)
Electron microscope: 4Mx  0.05 nm
Nm = a billionth of a meter (10-9 meter)
© Sioux 2014 | Confidential |
5

6. About FEI
22 nnm 00.. 55 nnm © Sioux 2014 | Confidential | 6
Atomic structur of Ge
(Germanium). Distance
is 0.5 nm
Breast cancer cell.
Magnification 5.000x Salmonella bacteria.
Magnification 80.000x

7. What is a model?
These models are not meant:
© Sioux 2014 | Confidential |
7

8. What is a model?
Simplification of the reality
By definition wrong, but can be useful
Do not try to put everything in the model
© Sioux 2014 | Confidential |
8

9. Traditional model
© Sioux 2014 | Confidential |
9
Traditional ModelSource codeIs the model correct? Is the source code correct?

10. ASD (Analytical Software Design)
Model Driven Development technology
Component Based Development
Models are verified mathematically at design time
Extensive model checker:
Deadlock
Live-lock
Starvation
Race conditions, etc
© Sioux 2014 | Confidential |
10

11. Model
© Sioux 2014 | Confidential |
11
Model: - Precise- Complete- Correct

12. Model verification
© Sioux 2014 | Confidential |
12
Model: - Precise- Complete- CorrectFormal model and verificationGenerate formal modelDesign errors

13. Code generation
© Sioux 2014 | Confidential |
13
Model: - Precise- Complete- CorrectFormal model and verificationSource code: - Java- MISCRA C- C++ - C# Generate formal modelDesign errorsGenerate defect free source code from verified modelGuaranteed equivalence

14. What is covered?
Interface model:
Specification of interfaces of components
“What”
Design model:
Implementation of interfaces
Internal implementation
“How”
Also the outside interfaces are specified
Interfaces to not-modelled components
© Sioux 2014 | Confidential |
14

15. Handwritten vs. generated
© Sioux 2014 | Confidential |
15
InterfaceInterfaceInterfaceInterfaceInterfaceInterface

16. Handwritten vs. generated
© Sioux 2014 | Confidential | 16
Interface
Interface Interface
Handwritten code
Generated
code
Part of the model

17. Scope of model
Typically used to model behaviour described in:
State machines
Sequence diagrams
Algorithms
© Sioux 2014 | Confidential |
17

18. State machines
© Sioux 2014 | Confidential |
18
Large (encapsulated) state machines can be modelled

19. Sequence diagrams
© Sioux 2014 | Confidential |
19

20. Scope of model
Typically used to model behaviour described in:
State machines
Sequence diagrams
Algorithms
What is modelled?
All normal functionality (good weather)
All exceptional functionality (bad weather)
All illegal behaviour
© Sioux 2014 | Confidential |
20

21. MDD and Testing
© Sioux 2014 | Confidential | 21
Implementation
Design Component Test
Architecture Integration Test
System
Requirements
System Test
User
Requirements
Acceptance Test
Wish, need
Release
Formal
methods

22. MDD and Testing
Formal methods typically:
Applied at Design level
Only in Software
For part of the software
Not for legacy and 3rd party software
© Sioux 2014 | Confidential |
22
ImplementationComponent TestDesignIntegration TestArchitectureSystem RequirementsSystem TestUser RequirementsAcceptance TestWish, needReleaseFormal methods

23. MDD and Testing
Part of the component tests are not needed anymore (Software Engineer)
Test activities of Test Engineer:
From integration level and up
Are still needed
But:
MDD has impact on quality of deliverables
© Sioux 2014 | Confidential |
23 Implementation
Design Component Test
Architecture Integration Test
System
Requirements
System Test
User
Requirements
Acceptance Test
Wish, need
Release
Formal
methods

24. Microscope Vacuum redesign
Vacuum software + firmware redesign
ASD has been used
Functionality tested manually (by experts)
Vacuum is critical part of system  verify reliability separately
AutoIT used for test automation
Freeware, mainly GUI tool
Only scripting used (no recording)
© Sioux 2014 | Confidential |
24

25. Test automation
First steps: quick and dirty
Fast feedback
Automation via GUI
© Sioux 2014 | Confidential |
25

26. Test automation
© Sioux 2014 | Confidential |
26

27. UI-Controls and graphics used for
Test actions
Test verifications
Reliability tests
System tests on vacuum system
Executed for long time (days)
Not only software but whole system is tested
Test automation
© Sioux 2014 | Confidential |
27

28. Testing experiences
Testing is still necessary
Integration (with other SW, with HW)
System testing
Model incorrect  Rubbish in, rubbish out New insights, so model will change
Functionality must still be verified (and errors are found)
Programming errors or integration issues:
In handwritten code
On boundary between generated and handwritten code (hard to analyse)
Not in generated code!
© Sioux 2014 | Confidential |
28

29. Testing experiences
Reliability of generated code: very high!
But crashes can still occur... (but not in generated code)
Reliability tests  HW failures/wear detected much earlier Days instead of months
Integration and system test  not hampered by reliability issues  big improvement, less cycles
© Sioux 2014 | Confidential |
29

30. Testing experiences
Models are much easier to review, understand and adapt than code
Design discussions are held early in project/sprint (as it should be!)  defects prevented!
Refactoring (relatively) simple and reliable (less regression)
Fits well in agile approach
© Sioux 2014 | Confidential |
30

31. Product Risk Analysis
© Sioux 2014 | Confidential |
31
331515  Impact  Likelihood Comp1Comp2Comp399

32. Product Risk Analysis
© Sioux 2014 | Confidential |
32
331515  Impact  Likelihood Comp1Comp2Comp399IIIIIVIICould testMust test“Won´t test”Should test

33. Product Risk Analysis
© Sioux 2014 | Confidential |
33
331515  Impact  Likelihood Comp1Comp2Comp399

34. Product Risk Analysis
© Sioux 2014 | Confidential |
34
331515  Impact  Likelihood Comp1Comp2Comp399 MDD

35. Conclusions
Component tests (white box) could be skipped
Internal working is correct
Although functionality can be incorrect
Integration and system testing a lot smoother
Less interface issues
Less reliability issues
Reliability testing  reliability measurement
Integration and system test more efficient (less cycles)
Architecture and design takes more effort
More re-designs with ASD are planned
© Sioux 2014 | Confidential |
35

36. Questions
© Sioux 2014 | Confidential | 36

37. 37
© Sioux 2014 | Confidential |
www.sioux.eu
bryan.bakker@sioux.eu
+31 (0)40 26 77 100