Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Integrating Formal Program  Verification with TestingCyrille Comar, Johannes Kanig and Yannick Moy
Integrating        Formal Program            with Testing           VerificationCyrille Comar, Johannes Kanig and Yannick ...
Integrating        Formal Program             with            VerificationCyrille Comar, Johannes Kanig and Yannick Moy
Motivation
Cost of testing•   Cost of testing greater than cost of development•   10% increase each year for avionics software (Boein...
DO-178C: formal methods can replace testingFormal methods […] might be theprimary source of evidence forthe satisfaction o...
Myths of formal methods•   Myth 4: Formal methods require highly trained mathematicians•   Myth 5: Formal methods increase...
Practice of formal methodsSince 2001, Airbus has beenintegrating several tool supportedformal verification techniques into...
Cost of verification          20%      80% of                       20%      80% of 80%              testing effort     ...
Proof + Test
Programming Contracts                  {P}C{Q}       Hoare logic (1969)logic contracts             executable contracts   ...
Project
Ada 2012
Testing vs. Formal Verification                                        prove pre of Q   use Q code                        ...
Combining tests and proofs           P is tested                              P calls Q                                   ...
Caution: contracts are not only pre/post!   strong typing                parameters                                not ali...
Combination 1: tested calls proved                    P is tested                                        P calls Q  during...
Combination 2: proved calls tested                      P is tested  during testing:    check that                   Ppost...
Testing + Formal Verification                   tested                               P         proved                     ...
GNAT toolsuite               executable GNAT            GNATtestcompiler         unit testing   aggregated                ...
Conclusion
Airbus 5 “must-have” of formal methods•   Soundness•   Applicability to the code•   Usability by normal engineers on norma...
Benefits of openness                       .org   • announcements            • public:             • all code   • meeting ...
Project Partners
www.open-do.org/projects/hi-lite
Upcoming SlideShare
Loading in …5
×

Hi-Lite erts2012

9,424 views

Published on

Yannick Moy's presentation on the Hi-Lite project at the ERTS 2012 event in Toulouse France. The paper "Integrating Formal Program Verification with Testing" can be found at http://www.erts2012.org/Site/0P2RUC89/7A-1.pdf

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Hi-Lite erts2012

  1. 1. Integrating Formal Program Verification with TestingCyrille Comar, Johannes Kanig and Yannick Moy
  2. 2. Integrating Formal Program with Testing VerificationCyrille Comar, Johannes Kanig and Yannick Moy
  3. 3. Integrating Formal Program with VerificationCyrille Comar, Johannes Kanig and Yannick Moy
  4. 4. Motivation
  5. 5. Cost of testing• Cost of testing greater than cost of development• 10% increase each year for avionics software (Boeing META Project)• Uneven repartition: 20%  80% of effort! 80%• Uneven quality: 80% of errors traced to 20% of code (NASA Software Safety Guidebook)• Need to reduce and focus the cost of testing
  6. 6. DO-178C: formal methods can replace testingFormal methods […] might be theprimary source of evidence forthe satisfaction of many of theobjectives concerned withdevelopment and verification. 2011: Formal Methods Supplement (DO-333)
  7. 7. Myths of formal methods• Myth 4: Formal methods require highly trained mathematicians• Myth 5: Formal methods increase the cost of development• Myth 6: Formal methods are unacceptable to users• Myth 7: Formal methods are not used on real, large-scale software (Anthony Hall, Praxis Systems, 1990)
  8. 8. Practice of formal methodsSince 2001, Airbus has beenintegrating several tool supportedformal verification techniques intothe development process ofavionics software products. 2009: Formal Verification of Avionics Software Products (Souyris, Wiels, Delmas, Delseny)
  9. 9. Cost of verification 20%  80% of 20%  80% of 80% testing effort 80% formal effort Hi-Lite goal: using formal verification first, then testing… 4% 16% testing 80%formal … to reduce and focus the cost of verification
  10. 10. Proof + Test
  11. 11. Programming Contracts {P}C{Q} Hoare logic (1969)logic contracts executable contracts for proofs for testsSPARK (1987) Eiffel DbC (1986)Hi-Lite: executable annotation language???
  12. 12. Project
  13. 13. Ada 2012
  14. 14. Testing vs. Formal Verification prove pre of Q use Q code assume post of Qcover P constructs P calls Q P calls Q P P Q Q assume pre of Q actual body of Q prove post of Q or stub…local exhaustivity argument: global soundness argument:each function covered P all functions proved enough behaviors  all assumptions justified explored Q R
  15. 15. Combining tests and proofs P is tested P calls Q How so we justify assumptions made P during proof? Q Q calls P Q is provedverification combining tests and proofs should be AT LEAST AS GOOD AS verification based on tests only
  16. 16. Caution: contracts are not only pre/post! strong typing parameters not aliased )… parameters initializeddata dependences
  17. 17. Combination 1: tested calls proved P is tested P calls Q during testing: check that Pprecondition of Q Q is respected Q is proved assumption for proof: precondition of Q is respected
  18. 18. Combination 2: proved calls tested P is tested during testing: check that Ppostcondition of P Q is respected Q calls P Q is proved assumption for proof: postcondition of P is respected
  19. 19. Testing + Formal Verification tested P proved Q R provedlocal exhaustivity argument: global soundness argument:- test: function covered - proof: assumptions proved- proof: by nature of proof - test: assumptions tested Testing must check additional properties Done by compiler instrumentation
  20. 20. GNAT toolsuite executable GNAT GNATtestcompiler unit testing aggregated verification results GNATprove unit proof
  21. 21. Conclusion
  22. 22. Airbus 5 “must-have” of formal methods• Soundness• Applicability to the code• Usability by normal engineers on normal computers• Improve on classical methods current work• Certifiability
  23. 23. Benefits of openness .org • announcements • public: • all code • meeting slides  meeting minutes • dev docs • articles / docs  technical work • user docs  69 members • private:  management  partner code external collaborations with industry and academia
  24. 24. Project Partners
  25. 25. www.open-do.org/projects/hi-lite

×