Hi-Lite erts2012

Integrating Formal Program Verification with TestingCyrille Comar, Johannes Kanig and Yannick Moy

Integrating Formal Program with Testing VerificationCyrille Comar, Johannes Kanig and Yannick Moy

Integrating Formal Program with VerificationCyrille Comar, Johannes Kanig and Yannick Moy

Motivation

Cost of testing• Cost of testing greater than cost of development• 10% increase each year for avionics software (Boeing META Project)• Uneven repartition: 20%  80% of effort! 80%• Uneven quality: 80% of errors traced to 20% of code (NASA Software Safety Guidebook)• Need to reduce and focus the cost of testing

DO-178C: formal methods can replace testingFormal methods […] might be theprimary source of evidence forthe satisfaction of many of theobjectives concerned withdevelopment and verification. 2011: Formal Methods Supplement (DO-333)

Myths of formal methods• Myth 4: Formal methods require highly trained mathematicians• Myth 5: Formal methods increase the cost of development• Myth 6: Formal methods are unacceptable to users• Myth 7: Formal methods are not used on real, large-scale software (Anthony Hall, Praxis Systems, 1990)

Practice of formal methodsSince 2001, Airbus has beenintegrating several tool supportedformal verification techniques intothe development process ofavionics software products. 2009: Formal Verification of Avionics Software Products (Souyris, Wiels, Delmas, Delseny)

Cost of verification 20%  80% of 20%  80% of 80% testing effort 80% formal effort Hi-Lite goal: using formal verification first, then testing… 4% 16% testing 80%formal … to reduce and focus the cost of verification

Proof + Test

Programming Contracts {P}C{Q} Hoare logic (1969)logic contracts executable contracts for proofs for testsSPARK (1987) Eiffel DbC (1986)Hi-Lite: executable annotation language???

Project

Ada 2012

Testing vs. Formal Verification prove pre of Q use Q code assume post of Qcover P constructs P calls Q P calls Q P P Q Q assume pre of Q actual body of Q prove post of Q or stub…local exhaustivity argument: global soundness argument:each function covered P all functions proved enough behaviors  all assumptions justified explored Q R

Combining tests and proofs P is tested P calls Q How so we justify assumptions made P during proof? Q Q calls P Q is provedverification combining tests and proofs should be AT LEAST AS GOOD AS verification based on tests only

Caution: contracts are not only pre/post! strong typing parameters not aliased )… parameters initializeddata dependences

Combination 1: tested calls proved P is tested P calls Q during testing: check that Pprecondition of Q Q is respected Q is proved assumption for proof: precondition of Q is respected

Combination 2: proved calls tested P is tested during testing: check that Ppostcondition of P Q is respected Q calls P Q is proved assumption for proof: postcondition of P is respected

Testing + Formal Verification tested P proved Q R provedlocal exhaustivity argument: global soundness argument:- test: function covered - proof: assumptions proved- proof: by nature of proof - test: assumptions tested Testing must check additional properties Done by compiler instrumentation

GNAT toolsuite executable GNAT GNATtestcompiler unit testing aggregated verification results GNATprove unit proof

Conclusion

Airbus 5 “must-have” of formal methods• Soundness• Applicability to the code• Usability by normal engineers on normal computers• Improve on classical methods current work• Certifiability

Benefits of openness .org • announcements • public: • all code • meeting slides  meeting minutes • dev docs • articles / docs  technical work • user docs  69 members • private:  management  partner code external collaborations with industry and academia

Project Partners

www.open-do.org/projects/hi-lite

http://www.open-do.org	3614
http://libre.adacore.com	569
http://www.adacore.com	511
http://www2.adacore.com	5
http://www1.adacore.com	4
http://corp3.adacore.com	4
http://translate.googleusercontent.com	3
http://new.open-do.org	1
http://127.0.0.1	1
http://homi.abakus-systemhaus.de	1

Hi-Lite erts2012

by AdaCore on Mar 13, 2012

Accessibility

Categories

Upload Details

Usage Rights

10 Embeds 4,713

Statistics

Hi-Lite erts2012 Presentation Transcript