Hi-Lite erts2012


Published on

Yannick Moy's presentation on the Hi-Lite project at the ERTS 2012 event in Toulouse France. The paper "Integrating Formal Program Verification with Testing" can be found at http://www.erts2012.org/Site/0P2RUC89/7A-1.pdf

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Hi-Lite erts2012

  1. 1. Integrating Formal Program Verification with TestingCyrille Comar, Johannes Kanig and Yannick Moy
  2. 2. Integrating Formal Program with Testing VerificationCyrille Comar, Johannes Kanig and Yannick Moy
  3. 3. Integrating Formal Program with VerificationCyrille Comar, Johannes Kanig and Yannick Moy
  4. 4. Motivation
  5. 5. Cost of testing• Cost of testing greater than cost of development• 10% increase each year for avionics software (Boeing META Project)• Uneven repartition: 20%  80% of effort! 80%• Uneven quality: 80% of errors traced to 20% of code (NASA Software Safety Guidebook)• Need to reduce and focus the cost of testing
  6. 6. DO-178C: formal methods can replace testingFormal methods […] might be theprimary source of evidence forthe satisfaction of many of theobjectives concerned withdevelopment and verification. 2011: Formal Methods Supplement (DO-333)
  7. 7. Myths of formal methods• Myth 4: Formal methods require highly trained mathematicians• Myth 5: Formal methods increase the cost of development• Myth 6: Formal methods are unacceptable to users• Myth 7: Formal methods are not used on real, large-scale software (Anthony Hall, Praxis Systems, 1990)
  8. 8. Practice of formal methodsSince 2001, Airbus has beenintegrating several tool supportedformal verification techniques intothe development process ofavionics software products. 2009: Formal Verification of Avionics Software Products (Souyris, Wiels, Delmas, Delseny)
  9. 9. Cost of verification 20%  80% of 20%  80% of 80% testing effort 80% formal effort Hi-Lite goal: using formal verification first, then testing… 4% 16% testing 80%formal … to reduce and focus the cost of verification
  10. 10. Proof + Test
  11. 11. Programming Contracts {P}C{Q} Hoare logic (1969)logic contracts executable contracts for proofs for testsSPARK (1987) Eiffel DbC (1986)Hi-Lite: executable annotation language???
  12. 12. Project
  13. 13. Ada 2012
  14. 14. Testing vs. Formal Verification prove pre of Q use Q code assume post of Qcover P constructs P calls Q P calls Q P P Q Q assume pre of Q actual body of Q prove post of Q or stub…local exhaustivity argument: global soundness argument:each function covered P all functions proved enough behaviors  all assumptions justified explored Q R
  15. 15. Combining tests and proofs P is tested P calls Q How so we justify assumptions made P during proof? Q Q calls P Q is provedverification combining tests and proofs should be AT LEAST AS GOOD AS verification based on tests only
  16. 16. Caution: contracts are not only pre/post! strong typing parameters not aliased )… parameters initializeddata dependences
  17. 17. Combination 1: tested calls proved P is tested P calls Q during testing: check that Pprecondition of Q Q is respected Q is proved assumption for proof: precondition of Q is respected
  18. 18. Combination 2: proved calls tested P is tested during testing: check that Ppostcondition of P Q is respected Q calls P Q is proved assumption for proof: postcondition of P is respected
  19. 19. Testing + Formal Verification tested P proved Q R provedlocal exhaustivity argument: global soundness argument:- test: function covered - proof: assumptions proved- proof: by nature of proof - test: assumptions tested Testing must check additional properties Done by compiler instrumentation
  20. 20. GNAT toolsuite executable GNAT GNATtestcompiler unit testing aggregated verification results GNATprove unit proof
  21. 21. Conclusion
  22. 22. Airbus 5 “must-have” of formal methods• Soundness• Applicability to the code• Usability by normal engineers on normal computers• Improve on classical methods current work• Certifiability
  23. 23. Benefits of openness .org • announcements • public: • all code • meeting slides  meeting minutes • dev docs • articles / docs  technical work • user docs  69 members • private:  management  partner code external collaborations with industry and academia
  24. 24. Project Partners
  25. 25. www.open-do.org/projects/hi-lite