•0 likes•3,807 views

Report

Share

Download to read offline

Yannick Moy's presentation on the Hi-Lite project at the ERTS 2012 event in Toulouse France. The paper "Integrating Formal Program Verification with Testing" can be found at http://www.erts2012.org/Site/0P2RUC89/7A-1.pdf

Follow

- 1. Integrating Formal Program Verification with Testing Cyrille Comar, Johannes Kanig and Yannick Moy
- 2. Integrating Formal Program with Testing Verification Cyrille Comar, Johannes Kanig and Yannick Moy
- 3. Integrating Formal Program with Verification Cyrille Comar, Johannes Kanig and Yannick Moy
- 4. Motivation
- 5. Cost of testing • Cost of testing greater than cost of development • 10% increase each year for avionics software (Boeing META Project) • Uneven repartition: 20% 80% of effort! 80% • Uneven quality: 80% of errors traced to 20% of code (NASA Software Safety Guidebook) • Need to reduce and focus the cost of testing
- 6. DO-178C: formal methods can replace testing Formal methods […] might be the primary source of evidence for the satisfaction of many of the objectives concerned with development and verification. 2011: Formal Methods Supplement (DO-333)
- 7. Myths of formal methods • Myth 4: Formal methods require highly trained mathematicians • Myth 5: Formal methods increase the cost of development • Myth 6: Formal methods are unacceptable to users • Myth 7: Formal methods are not used on real, large-scale software (Anthony Hall, Praxis Systems, 1990)
- 8. Practice of formal methods Since 2001, Airbus has been integrating several tool supported formal verification techniques into the development process of avionics software products. 2009: Formal Verification of Avionics Software Products (Souyris, Wiels, Delmas, Delseny)
- 9. Cost of verification 20% 80% of 20% 80% of 80% testing effort 80% formal effort Hi-Lite goal: using formal verification first, then testing… 4% 16% testing 80% formal … to reduce and focus the cost of verification
- 10. Proof + Test
- 11. Programming Contracts {P}C{Q} Hoare logic (1969) logic contracts executable contracts for proofs for tests SPARK (1987) Eiffel DbC (1986) Hi-Lite: executable annotation language???
- 12. Project
- 13. Ada 2012
- 14. Testing vs. Formal Verification prove pre of Q use Q code assume post of Q cover P constructs P calls Q P calls Q P P Q Q assume pre of Q actual body of Q prove post of Q or stub… local exhaustivity argument: global soundness argument: each function covered P all functions proved enough behaviors all assumptions justified explored Q R
- 15. Combining tests and proofs P is tested P calls Q How so we justify assumptions made P during proof? Q Q calls P Q is proved verification combining tests and proofs should be AT LEAST AS GOOD AS verification based on tests only
- 16. Caution: contracts are not only pre/post! strong typing parameters not aliased )… parameters initialized data dependences
- 17. Combination 1: tested calls proved P is tested P calls Q during testing: check that P precondition of Q Q is respected Q is proved assumption for proof: precondition of Q is respected
- 18. Combination 2: proved calls tested P is tested during testing: check that P postcondition of P Q is respected Q calls P Q is proved assumption for proof: postcondition of P is respected
- 19. Testing + Formal Verification tested P proved Q R proved local exhaustivity argument: global soundness argument: - test: function covered - proof: assumptions proved - proof: by nature of proof - test: assumptions tested Testing must check additional properties Done by compiler instrumentation
- 20. GNAT toolsuite executable GNAT GNATtest compiler unit testing aggregated verification results GNATprove unit proof
- 21. Conclusion
- 22. Airbus 5 “must-have” of formal methods • Soundness • Applicability to the code • Usability by normal engineers on normal computers • Improve on classical methods current work • Certifiability
- 23. Benefits of openness .org • announcements • public: • all code • meeting slides meeting minutes • dev docs • articles / docs technical work • user docs 69 members • private: management partner code external collaborations with industry and academia
- 24. Project Partners