This document summarizes Altran's approach to software development and testing. It discusses: - Altran's Correctness by Construction approach using formal methods and tools like Z, CSP, SPARK to specify, design, implement and verify software. - Methods like REVEAL to generate test cases from requirements and verification conditions. - Advances in testing approaches including using a "test oracle" to automatically check test outputs against formal specifications rather than manual scripting and checking. - The conclusion is that automating test case generation and execution from formal specifications reduces time, cost and errors compared to traditional manual testing.

1. 1
ALTRAN, GLOBAL LEADER
IN INNOVATION
Advances in Practical
Techniques for Critical Software
Development
5 t h N o v e m b e r 2 0 1 5

2. Agenda
• Who am I? Who are Altran?
• Our approach to building software
• Methods and tool support for building software
• Strengthening the weakest link – advances in test approaches
• Conclusion

3. Agenda
• Who am I? Who are Altran?
• Our approach to building software
• Methods and tool support for building software
• Strengthening the weakest link – advances in test approaches
• Conclusion

4. Who am I?
4
Neil White
Head of Engineering
Altran UK

5. 5
Bath, UK Paris, France Annecy, France
Munich, Germany
Hamburg, Germany
Shanghai, China
Bangalore, India
Sophia-Antipolis, France
Toulouse, France
Milan, Italy
Turin, Italy
Atlanta, USA
Barcelona, Spain
Madrid, Spain
Lille, France
Pisa, Italy
Bologna, Italy
Rome, Italy
Naples, Italy
Who are Altran?
CMMI L3, ISO 9001, ISO 27001, EN 9100,
ISO 13485
SYSTEMS ENGINEERING
SOFTWARE ENGINEERING
ELECTRONICS
SAFETY
SECURITY
CONNECTIVITY

6. UK FOOTPRINT
6
Bristol
Bristol
London
Warwick
Cambridge*
Reading
* Cambridge Consultants, a member of the Altran Group
Bath
€112.3M
Revenues (2014)
Offices in
12 locations
850+
FTEs
Derby
Slough
Penrith
Manchester
Glasgow

7. Agenda
• Who am I? Who are Altran?
• Our approach to building software
• Methods and tool support for building software
• Strengthening the weakest link – advances in test approaches
• Conclusion

8. The cost of errors Correctness by Construction
Source: CMM Data from Jones,
Caspers: Software Assessments,
Benchmarks and Best Practices.
Reading, MA: Addison-Wesley,
2002
Source: C By C data from
Correctness by Construction: A
manifesto for High-Integrity
software, Croxford and
Chapman 2005
Source:Leffingwell
http://www.rational.com/m
edia/whitepapers/roi1.pdf
Source: IEEE Software.
Correctness by Construction:
Developing a Commercial Secure
System, Hall and Chapman, Jan
2002

9. Principles
Avoid introducing defects
Introducing defects is easy –
removing them is hard, and
expensive
Generate evidence as you go
Evidence needed for certification is
produced naturally as a by-product of
the process
Remove defects early
Defects removed early when
changes are cheap
Correctness by Construction
Testing is a
demonstration of
correctness
Not the point where we start
debugging.
Prediction over observation.
Better can be cheaper
Safety is given. How you get there
determines the cost.
Zero tolerance of defects
We cannot claim zero defects but
we can have a zero tolerance
attitude to them.

10. Strategy
• Use precise or formal notations for each step
• Design the software to simplify verification and validation
• Small steps verified at every stage
• Use strong, tool-supported methods to verify each step
• Say things only once
• Do the hard / risky things first
Correctness by Construction

11. Agenda
• Who am I? Who are Altran?
• Our approach to building software
• Methods and tool support for building software
• Strengthening the weakest link – advances in test approaches
• Conclusion

12. Building Blocks
13
Requirements Specification Design Implement
Analyse &
Prove
Test

13. Building Blocks
14
Requirements Specification Design Implement
Analyse &
Prove
Test
REVEAL

14. Building Blocks
15
Requirements Specification Design Implement
Analyse &
Prove
Test
REVEAL
Z, CSP, UML, SCADE,
Matlab/Simulink

15. Building Blocks
16
Requirements Specification Design Implement
Analyse &
Prove
Test
REVEAL
Z, CSP, UML, SCADE,
Matlab/Simulink
INFORMED

16. Building Blocks
17
Requirements Specification Design Implement
Analyse &
Prove
Test
REVEAL
Z, CSP, UML, SCADE,
Matlab/Simulink
INFORMED
SPARK, MISRA C,
SCADE, QGen

17. Building Blocks
18
Requirements Specification Design Implement
Analyse &
Prove
Test
REVEAL
Z, CSP, UML, SCADE,
Matlab/Simulink
INFORMED
SPARK, MISRA C,
SCADE, QGen
SPARK,
CodePeer

18. Building Blocks
19
Requirements Specification Design Implement
Analyse &
Prove
Test
REVEAL
Z, CSP, UML, SCADE,
Matlab/Simulink
INFORMED
SPARK, MISRA C,
SCADE, QGen
SPARK,
CodePeer
??

19. Building Blocks
20
Requirements Specification Design Implement
Analyse &
Prove
Test
REVEAL
Z, CSP, UML, SCADE,
Matlab/Simulink
INFORMED
SPARK, MISRA C,
SCADE, QGen
SPARK,
CodePeer
ConTestor

20. Agenda
• Who am I? Who are Altran?
• Our approach to building software
• Methods and tool support for building software
• Strengthening the weakest link – advances in test approaches
• Conclusion

21. Traditional Dynamic Test Approach
Requirements
Verification
Conditions
System Under
Test
Comparator
Test Scripts
Inputs
Expected
outputs
Actual
outputs
Code
Coverage
VC
Coverage
Results22

22. Dynamic Test Approach with an Oracle
Requirements
Verification
Conditions
System Under
Test
Comparator
Test Scripts
Inputs
Expected
outputs
Actual
outputs
Code
Coverage
Test Oracle
Inputs
Expected
outputs
VC
Coverage
Results23

23. Test Oracle
Test
Oracle
The ConTestor Approach
24
Requirements
Verification
Conditions
System Under
Test
Comparator
Test Descriptions
Inputs
Actual
outputs
Code
Coverage
Inputs
Expected
outputs
VC
Coverage
VC
Checker
Results
VC
Coverage
Test Scripts

24. Agenda
• Who am I? Who are Altran?
• Our approach to building software
• Methods and tool support for building software
• Strengthening the weakest link – advances in test approaches
• Conclusion

25. Conclusion
› Automating the running of Test Scripts has been standard practice for
years.
› Automating the production of Test Scripts for Safety Critical software is
now possible
› Reduces time
› Reduces cost
› Reduces the opportunity for human error
› Improves depth of testing with brute force
› Reduces maintenance costs
› Why ever write Test Scripts again?
26

26. 27