Open-DO Update

7,557 views
7,449 views

Published on

A review of the latest news and developments with the Open-DO effort.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
7,557
On SlideShare
0
From Embeds
0
Number of Embeds
5,909
Actions
Shares
0
Downloads
29
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • 01/03/10
  • 01/03/10
  • 01/03/10
  • 01/03/10
  • 01/03/10
  • Open-DO Update

    1. 1. Open DO update Open DO & Formality Cyrille Comar [email_address] www.open-do.org
    2. 2. Summary <ul><ul><li>Reminder on Open-DO Concepts </li></ul></ul><ul><ul><li>What has been Happening within Open-Do </li></ul></ul><ul><ul><li>Couverture & Formal Methods </li></ul></ul><ul><ul><li>Hi-Lite </li></ul></ul>
    3. 3. Reminder on Open-DO Concepts
    4. 4. <ul><li>How to solve the “big Freeze” problem ? </li></ul><ul><li>How to manage exposed life-cycle Data ? </li></ul><ul><li>How to reduce cost & time-to-market ? </li></ul><ul><li>How to augment quality & reduce residual problems ? </li></ul>Open–DO
    5. 5. FLOSS Freely Licensed Open Source Software High-Integrity Certification Agile Lean The meeting of 3 worlds
    6. 6. FLOSS Freely Licensed Open Source Software High-Integrity Certification Agile Lean Longevity Visibility Resilience Cost-sharing Reuse Iterative requirements Continuous integration Test driven development Executable specifications Reducing waste Qualified tools Life cycle traceability Req based testing The meeting of 3 worlds
    7. 7. FLOSS Freely Licensed Open Source Software High-Integrity Certification Security Agile Lean Longevity Visibility Resilience Cost-sharing Reuse Iterative requirements Continuous integration Test driven development Executable specifications Reducing waste Qualified tools Life cycle traceability Requirement based testing The meeting of 3 worlds Formal methods verification verification 4 ?
    8. 8. Open-DO Document Templates Qualifiable Tools Education Materials Certifiable Components Open DO Components Life Cycle Management
    9. 9. <ul><ul><li>What has been Happening within Open-Do ? </li></ul></ul>
    10. 10. <ul><ul><li>Awareness </li></ul></ul>Stages <ul><ul><li>Infrastructure </li></ul></ul><ul><ul><li>Initial projects </li></ul></ul><ul><ul><li>Community management </li></ul></ul>Kick-off projects
    11. 11. <ul><ul><li>Awareness </li></ul></ul>Stages <ul><ul><li>Infrastructure </li></ul></ul><ul><ul><li>Initial projects </li></ul></ul><ul><ul><li>Community management </li></ul></ul><ul><li>Conferences & Papers </li></ul><ul><ul><li>OpenCert conference </li></ul></ul><ul><ul><li>Avionics 2009 </li></ul></ul><ul><ul><li>DASIA 2009 </li></ul></ul><ul><ul><li>Safety-Critical Systems Club </li></ul></ul><ul><ul><li>Ada Europe 2009 </li></ul></ul><ul><ul><li>International SPICE days </li></ul></ul><ul><ul><li>Eclipse Embedded Day </li></ul></ul><ul><ul><li>2009 IET System Safety Conf. </li></ul></ul><ul><ul><li>Ada UK Conference 2009 </li></ul></ul><ul><ul><li>Agile Tour 2009 </li></ul></ul><ul><li>Presentations </li></ul><ul><ul><li>SC-205 (DO-178c) </li></ul></ul><ul><ul><li>Boeing/Airbus/Embraer </li></ul></ul><ul><li>Social networks </li></ul><ul><ul><li>Linked-In (80 members) </li></ul></ul>Kick-off projects
    12. 12. <ul><ul><li>Awareness </li></ul></ul>Stages <ul><ul><li>Infrastructure </li></ul></ul><ul><ul><li>Initial projects </li></ul></ul><ul><ul><li>Community management </li></ul></ul>Kick-off projects <ul><li>Website </li></ul><ul><ul><li>increasing #s of visits </li></ul></ul><ul><li>Forge </li></ul><ul><ul><li>- ½ dozen hosted projects </li></ul></ul><ul><li>Mailinglists/forums </li></ul><ul><ul><li>more than a 100 registrations </li></ul></ul><ul><ul><li>mostly from mil-aero </li></ul></ul>
    13. 13. <ul><ul><li>Awareness </li></ul></ul>Stages <ul><ul><li>Infrastructure </li></ul></ul>Kick-off projects <ul><ul><li>Community management </li></ul></ul><ul><li>Couverture </li></ul><ul><li>Qualification Machine </li></ul><ul><li>Hi-Lite </li></ul><ul><li>… and a few others </li></ul>
    14. 14. <ul><ul><li>Awareness </li></ul></ul>Stages <ul><ul><li>Infrastructure </li></ul></ul>Kick-off projects <ul><ul><li>Community management </li></ul></ul><ul><li>Still a bit early… </li></ul><ul><li>Concentrate on relationship with related initiatives </li></ul>
    15. 15. Couverture … Hi-Lite … The Qualifying Machine <ul><ul><li>An agile infrastructure to support: </li></ul></ul><ul><ul><ul><ul><li>Delta qualification </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Continuous qualification </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Internally used at AdaCore </li></ul></ul></ul></ul><ul><ul><li>Availability of partial qualification material for GNATcheck </li></ul></ul><ul><ul><ul><ul><li>A coding standard tools </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Tool Qualification Plan </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Quality Assurance Plan </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Configuration Management Plan </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Example of Tool Operational Requirements </li></ul></ul></ul></ul>Some Open-DO Projects (1)
    16. 16. Other projects <ul><ul><li>HiberSource </li></ul></ul><ul><ul><ul><ul><li>Configuration Management System </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Support for full life cycle </li></ul></ul></ul></ul><ul><ul><ul><ul><li>DO-178 compliant </li></ul></ul></ul></ul><ul><ul><li>Gene-Auto/Ada </li></ul></ul><ul><ul><ul><ul><li>A model compiler for data-flow and state machine languages </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Supports Simulink and Stateflow </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Generates Ada 2005 </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Final goal: qualification as DO-178C development tool </li></ul></ul></ul></ul>Some Open-DO Projects (2)
    17. 17. <ul><li>Embarquez Agile (Embed Agility) </li></ul><ul><li>Bordeaux- March 18 th , 2010 </li></ul><ul><li>Cyrille Comar, AdaCore: “Open-DO: open source and agility for critical software” </li></ul><ul><li>Matteo Bordin, AdaCore: “The Qualifying Machine: agile DO-178 qualification” </li></ul><ul><li>ERTS 2 2010: Embedded and Real-Time Systems 2010 </li></ul><ul><li> </li></ul><ul><li>Toulouse, May 19 th -21 st </li></ul><ul><li>FM+AM 2010 </li></ul><ul><li>Pisa – September 17 th , 2010 </li></ul><ul><li>2 nd Internation Workshop on Formal Methods and Agile Methods </li></ul><ul><li>Co-located with the 8 th IEEE Conference on Software Engineering and Formal Methods </li></ul>Upcoming Events
    18. 18. <ul><ul><li>Good visibility in the avionics industry </li></ul></ul><ul><ul><li>Open Development in a certification context is a challenge </li></ul></ul><ul><ul><li>Importance of the quick-off projects </li></ul></ul>Remarks
    19. 19. <ul><ul><li>Couverture & Formal Methods </li></ul></ul>
    20. 20. <ul><li>Couverture provides either Object or Source coverage </li></ul><ul><li>Source coverage: </li></ul><ul><ul><li>Statement </li></ul></ul><ul><ul><li>Decision </li></ul></ul><ul><ul><li>MC/DC </li></ul></ul><ul><ul><li>pros: </li></ul></ul><ul><ul><ul><li>Simple for user </li></ul></ul></ul><ul><ul><ul><li>DO-178 </li></ul></ul></ul><ul><li>Object coverage: </li></ul><ul><ul><li>Instruction </li></ul></ul><ul><ul><li>Branch </li></ul></ul><ul><ul><li>pros: </li></ul></ul><ul><ul><ul><li>on the final code </li></ul></ul></ul><ul><ul><ul><li>bounded traces </li></ul></ul></ul><ul><ul><ul><li>lang independent </li></ul></ul></ul>
    21. 21. <ul><li>Object Branch coverage output example </li></ul>
    22. 22. <ul><li>Decision and MC/DC coverage </li></ul>function P (A, B, C : Boolean) return Boolean is begin if ( A and then B ) or else C then return True; end if ; end P; Decision Conditions Decision Coverage At least n+1 tests n = number of conditions MC/DC Coverage statements Statement Coverage A B C if statement T T ? T F ? F F A B C if statement T T ? T F ? F F F ? T T T F F F A C B A B C if statement T T ? T
    23. 23. <ul><li>Seems a reasonable assumption when </li></ul><ul><ul><li>boolean operator  branch in the object </li></ul></ul><ul><li>Has been assumed true for years </li></ul><ul><li>Recent FAA study (J. Chelinsky from Boeing) shows experimentally that it is not always the case </li></ul><ul><li>So what is the story? </li></ul>is MCDC implied by object branch coverage?
    24. 24. <ul><li>Counter-example </li></ul>if ( A and then B ) or else C then … end if ; Object Branch Coverage 4 tests MC/DC Coverage 3 tests Binary Decision Diagram (BDD) A B C if statement T T ? T F ? F F T F T T A B C if statement T T ? T F ? F F F ? T T T F F F A C B A B C True False T F F F T T
    25. 25. <ul><li>Verify new conjectures </li></ul><ul><li>Only when a single kind of operator? </li></ul><ul><li>No diamond in the BDD ? </li></ul><ul><li>Are the 2 above equivalent? </li></ul>if A and then B and then C and then D … then … end if ; A B C True False T F F F T T D F T
    26. 26. <ul><li>What is Alloy? </li></ul><ul><ul><li>a specification language for relational (first order) logic </li></ul></ul><ul><ul><li>specifications are executable </li></ul></ul><ul><li>What does it bring? </li></ul><ul><ul><li>exhaustive exploration in (small) user-defined scope </li></ul></ul><ul><ul><li>produces counter-examples </li></ul></ul>Alloy in the loop model checking
    27. 27. <ul><li>Modelisation of BDDs & MC/DC requirements </li></ul><ul><li>Verification of conjectures in a limited scope: </li></ul><ul><ul><li>Decisions with less than N conditions </li></ul></ul><ul><ul><li>with N = 5, 6, … </li></ul></ul><ul><li>Alloy generated counter-examples were key to find the proper equivalence </li></ul>Alloy in the loop (2)
    28. 28. <ul><li>Even in the proper context </li></ul><ul><ul><li>Boolean ops limited to: not, and then, or else </li></ul></ul><ul><ul><li>one branch in the object per condition </li></ul></ul><ul><ul><li>Normalized Decisions (NNF) </li></ul></ul><ul><ul><li>… </li></ul></ul><ul><li>Object Branch Coverage  MC/DC </li></ul><ul><li>For decisions limited to “and then” (or “or else”) OBC is sufficient for MC/DC … but not necessary… </li></ul>The results
    29. 29. <ul><li>There are forms of NNF decisions where </li></ul><ul><ul><li>OBC  MC/DC </li></ul></ul><ul><ul><ul><ul><li>(sub-decision1) and then (sub-decision2) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>with no “or else” in sub-decision1 </li></ul></ul></ul></ul><ul><ul><ul><ul><li>(sub-decision1) or else (sub-decision2) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>with no “and then” in sub-decision1 </li></ul></ul></ul></ul><ul><li>Alloy shows this is true for Nb_Conditions <= 7 </li></ul><ul><li>Manual Proof was built to show it for any conditions </li></ul>The results (2)
    30. 30. <ul><ul><li>Hi-Lite </li></ul></ul>
    31. 31. Overlap of existing techniques Hi-Lite: Verifying Program Properties (1) SPARK: decades of experience in proof of: - absence of RT errors - functional properties Programming by contract in Ada CodePeer: - detection of RT errors - implicit contracts Testing Static Analysis Formal methods
    32. 32. <ul><ul><li>Properties : </li></ul></ul><ul><ul><ul><li>Absence of classes of errors </li></ul></ul></ul><ul><ul><ul><li>Invariants maintained </li></ul></ul></ul><ul><ul><ul><li>Function contracts </li></ul></ul></ul><ul><ul><li>Verification : </li></ul></ul><ul><ul><ul><li>Testsuite passes ok </li></ul></ul></ul><ul><ul><ul><li>No critical warnings (compiler, static analyzer) </li></ul></ul></ul><ul><ul><ul><li>100% VC proved (VC = Verification Condition) </li></ul></ul></ul>Hi-Lite: Verifying Program Properties (2)
    33. 33. <ul><ul><li>NO method-specific expression of properties </li></ul></ul><ul><ul><li>ex: </li></ul></ul><ul><ul><ul><ul><ul><li>Oracles for tests </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Annotations for static analysis </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Logical formulas for proof </li></ul></ul></ul></ul></ul><ul><ul><li>Instead, ONE executable annotation language </li></ul></ul><ul><ul><ul><li>ex : assertions </li></ul></ul></ul><ul><ul><li>TOOLS do the translation </li></ul></ul>Hi-Lite: Verifying Program Properties (3)
    34. 34. <ul><li>ONE artifact for program and properties </li></ul><ul><li>ONE language for program and properties </li></ul><ul><li>MANY eyes for reviewing both </li></ul><ul><li>MANY ways to contribute properties </li></ul><ul><ul><li>Manually added </li></ul></ul><ul><ul><li>Inferred by static analyzer </li></ul></ul><ul><ul><li>Generated from higher-level description (model) </li></ul></ul><ul><li>MANY different workflows </li></ul><ul><ul><li>Dynamic vs. static verification </li></ul></ul><ul><ul><li>Various techniques to generate and prove formulas </li></ul></ul>Hi-Lite: Verifying Program Properties (4)
    35. 35. Conclusion <ul><li>Formal methods are useful in various ways </li></ul><ul><li>They need to be democratized </li></ul><ul><li>They need to be integrated in the Dev Cycle … in an agile way </li></ul>

    ×