HIS 2015: Neil White - Advances in Practical Techniques for Critical Development Software
1. 1
ALTRAN, GLOBAL LEADER
IN INNOVATION
Advances in Practical
Techniques for Critical Software
Development
5 t h N o v e m b e r 2 0 1 5
2. Agenda
• Who am I? Who are Altran?
• Our approach to building software
• Methods and tool support for building software
• Strengthening the weakest link – advances in test approaches
• Conclusion
3. Agenda
• Who am I? Who are Altran?
• Our approach to building software
• Methods and tool support for building software
• Strengthening the weakest link – advances in test approaches
• Conclusion
5. 5
Bath, UK Paris, France Annecy, France
Munich, Germany
Hamburg, Germany
Shanghai, China
Bangalore, India
Sophia-Antipolis, France
Toulouse, France
Milan, Italy
Turin, Italy
Atlanta, USA
Barcelona, Spain
Madrid, Spain
Lille, France
Pisa, Italy
Bologna, Italy
Rome, Italy
Naples, Italy
Who are Altran?
CMMI L3, ISO 9001, ISO 27001, EN 9100,
ISO 13485
SYSTEMS ENGINEERING
SOFTWARE ENGINEERING
ELECTRONICS
SAFETY
SECURITY
CONNECTIVITY
7. Agenda
• Who am I? Who are Altran?
• Our approach to building software
• Methods and tool support for building software
• Strengthening the weakest link – advances in test approaches
• Conclusion
8. The cost of errors Correctness by Construction
Source: CMM Data from Jones,
Caspers: Software Assessments,
Benchmarks and Best Practices.
Reading, MA: Addison-Wesley,
2002
Source: C By C data from
Correctness by Construction: A
manifesto for High-Integrity
software, Croxford and
Chapman 2005
Source:Leffingwell
http://www.rational.com/m
edia/whitepapers/roi1.pdf
Source: IEEE Software.
Correctness by Construction:
Developing a Commercial Secure
System, Hall and Chapman, Jan
2002
9. Principles
Avoid introducing defects
Introducing defects is easy –
removing them is hard, and
expensive
Generate evidence as you go
Evidence needed for certification is
produced naturally as a by-product of
the process
Remove defects early
Defects removed early when
changes are cheap
Correctness by Construction
Testing is a
demonstration of
correctness
Not the point where we start
debugging.
Prediction over observation.
Better can be cheaper
Safety is given. How you get there
determines the cost.
Zero tolerance of defects
We cannot claim zero defects but
we can have a zero tolerance
attitude to them.
10. Strategy
• Use precise or formal notations for each step
• Design the software to simplify verification and validation
• Small steps verified at every stage
• Use strong, tool-supported methods to verify each step
• Say things only once
• Do the hard / risky things first
Correctness by Construction
11. Agenda
• Who am I? Who are Altran?
• Our approach to building software
• Methods and tool support for building software
• Strengthening the weakest link – advances in test approaches
• Conclusion
20. Agenda
• Who am I? Who are Altran?
• Our approach to building software
• Methods and tool support for building software
• Strengthening the weakest link – advances in test approaches
• Conclusion
21. Traditional Dynamic Test Approach
Requirements
Verification
Conditions
System Under
Test
Comparator
Test Scripts
Inputs
Expected
outputs
Actual
outputs
Code
Coverage
VC
Coverage
Results22
22. Dynamic Test Approach with an Oracle
Requirements
Verification
Conditions
System Under
Test
Comparator
Test Scripts
Inputs
Expected
outputs
Actual
outputs
Code
Coverage
Test Oracle
Inputs
Expected
outputs
VC
Coverage
Results23
23. Test Oracle
Test
Oracle
The ConTestor Approach
24
Requirements
Verification
Conditions
System Under
Test
Comparator
Test Descriptions
Inputs
Actual
outputs
Code
Coverage
Inputs
Expected
outputs
VC
Coverage
VC
Checker
Results
VC
Coverage
Test Scripts
24. Agenda
• Who am I? Who are Altran?
• Our approach to building software
• Methods and tool support for building software
• Strengthening the weakest link – advances in test approaches
• Conclusion
25. Conclusion
› Automating the running of Test Scripts has been standard practice for
years.
› Automating the production of Test Scripts for Safety Critical software is
now possible
› Reduces time
› Reduces cost
› Reduces the opportunity for human error
› Improves depth of testing with brute force
› Reduces maintenance costs
› Why ever write Test Scripts again?
26