Open-DO Update

Open DO update Open DO & Formality Cyrille Comar [email_address] www.open-do.org

Summary
Reminder on Open-DO Concepts
What has been Happening within Open-Do
Couverture & Formal Methods
Hi-Lite

Reminder on Open-DO Concepts

How to solve the “big Freeze” problem ?
How to manage exposed life-cycle Data ?
How to reduce cost & time-to-market ?
How to augment quality & reduce residual problems ?
Open–DO

FLOSS Freely Licensed Open Source Software High-Integrity Certification Agile Lean The meeting of 3 worlds

FLOSS Freely Licensed Open Source Software High-Integrity Certification Agile Lean Longevity Visibility Resilience Cost-sharing Reuse Iterative requirements Continuous integration Test driven development Executable specifications Reducing waste Qualified tools Life cycle traceability Req based testing The meeting of 3 worlds

FLOSS Freely Licensed Open Source Software High-Integrity Certification Security Agile Lean Longevity Visibility Resilience Cost-sharing Reuse Iterative requirements Continuous integration Test driven development Executable specifications Reducing waste Qualified tools Life cycle traceability Requirement based testing The meeting of 3 worlds Formal methods verification verification 4 ?

Open-DO Document Templates Qualifiable Tools Education Materials Certifiable Components Open DO Components Life Cycle Management

What has been Happening within Open-Do ?

Awareness
Stages
Infrastructure
Initial projects
Community management
Kick-off projects

Awareness
Stages
Infrastructure
Initial projects
Conferences & Papers
OpenCert conference
Avionics 2009
DASIA 2009
Safety-Critical Systems Club
Ada Europe 2009
International SPICE days
Eclipse Embedded Day
2009 IET System Safety Conf.
Ada UK Conference 2009
Agile Tour 2009
Presentations
SC-205 (DO-178c)
Boeing/Airbus/Embraer
Social networks
Linked-In (80 members)
Kick-off projects

Awareness
Stages
Infrastructure
Initial projects
Kick-off projects
Website
increasing #s of visits
Forge
- ½ dozen hosted projects
Mailinglists/forums
more than a 100 registrations
mostly from mil-aero

Awareness
Stages
Infrastructure
Kick-off projects
Couverture
Qualification Machine
Hi-Lite
… and a few others

Awareness
Stages
Infrastructure
Kick-off projects
Still a bit early…
Concentrate on relationship with related initiatives

Couverture … Hi-Lite … The Qualifying Machine
An agile infrastructure to support:
Delta qualification
Continuous qualification
Internally used at AdaCore
Availability of partial qualification material for GNATcheck
A coding standard tools
Tool Qualification Plan
Quality Assurance Plan
Configuration Management Plan
Example of Tool Operational Requirements
Some Open-DO Projects (1)

Other projects
HiberSource
Configuration Management System
Support for full life cycle
DO-178 compliant
Gene-Auto/Ada
A model compiler for data-flow and state machine languages
Supports Simulink and Stateflow
Generates Ada 2005
Final goal: qualification as DO-178C development tool
Some Open-DO Projects (2)

Embarquez Agile (Embed Agility)
Bordeaux- March 18 th , 2010
Cyrille Comar, AdaCore: “Open-DO: open source and agility for critical software”
Matteo Bordin, AdaCore: “The Qualifying Machine: agile DO-178 qualification”
ERTS 2 2010: Embedded and Real-Time Systems 2010

Toulouse, May 19 th -21 st
FM+AM 2010
Pisa – September 17 th , 2010
2 nd Internation Workshop on Formal Methods and Agile Methods
Co-located with the 8 th IEEE Conference on Software Engineering and Formal Methods
Upcoming Events

Good visibility in the avionics industry
Open Development in a certification context is a challenge
Importance of the quick-off projects
Remarks

Couverture & Formal Methods

Couverture provides either Object or Source coverage
Source coverage:
Statement
Decision
MC/DC
pros:
Simple for user
DO-178
Object coverage:
Instruction
Branch
pros:
on the final code
bounded traces
lang independent

Object Branch coverage output example

Decision and MC/DC coverage
function P (A, B, C : Boolean) return Boolean is begin if ( A and then B ) or else C then return True; end if ; end P; Decision Conditions Decision Coverage At least n+1 tests n = number of conditions MC/DC Coverage statements Statement Coverage A B C if statement T T ? T F ? F F A B C if statement T T ? T F ? F F F ? T T T F F F A C B A B C if statement T T ? T

Seems a reasonable assumption when
boolean operator  branch in the object
Has been assumed true for years
Recent FAA study (J. Chelinsky from Boeing) shows experimentally that it is not always the case
So what is the story?
is MCDC implied by object branch coverage?

Counter-example
if ( A and then B ) or else C then … end if ; Object Branch Coverage 4 tests MC/DC Coverage 3 tests Binary Decision Diagram (BDD) A B C if statement T T ? T F ? F F T F T T A B C if statement T T ? T F ? F F F ? T T T F F F A C B A B C True False T F F F T T

Verify new conjectures
Only when a single kind of operator?
No diamond in the BDD ?
Are the 2 above equivalent?
if A and then B and then C and then D … then … end if ; A B C True False T F F F T T D F T

What is Alloy?
a specification language for relational (first order) logic
specifications are executable
What does it bring?
exhaustive exploration in (small) user-defined scope
produces counter-examples
Alloy in the loop model checking

Modelisation of BDDs & MC/DC requirements
Verification of conjectures in a limited scope:
Decisions with less than N conditions
with N = 5, 6, …
Alloy generated counter-examples were key to find the proper equivalence
Alloy in the loop (2)

Even in the proper context
Boolean ops limited to: not, and then, or else
one branch in the object per condition
Normalized Decisions (NNF)
…
Object Branch Coverage  MC/DC
For decisions limited to “and then” (or “or else”) OBC is sufficient for MC/DC … but not necessary…
The results

There are forms of NNF decisions where
OBC  MC/DC
(sub-decision1) and then (sub-decision2)
with no “or else” in sub-decision1
(sub-decision1) or else (sub-decision2)
with no “and then” in sub-decision1
Alloy shows this is true for Nb_Conditions <= 7
Manual Proof was built to show it for any conditions
The results (2)

Hi-Lite

Overlap of existing techniques Hi-Lite: Verifying Program Properties (1) SPARK: decades of experience in proof of: - absence of RT errors - functional properties Programming by contract in Ada CodePeer: - detection of RT errors - implicit contracts Testing Static Analysis Formal methods

Properties :
Absence of classes of errors
Invariants maintained
Function contracts
Verification :
Testsuite passes ok
No critical warnings (compiler, static analyzer)
100% VC proved (VC = Verification Condition)
Hi-Lite: Verifying Program Properties (2)

NO method-specific expression of properties
ex:
Oracles for tests
Annotations for static analysis
Logical formulas for proof
Instead, ONE executable annotation language
ex : assertions
TOOLS do the translation

ONE artifact for program and properties
ONE language for program and properties
MANY eyes for reviewing both
MANY ways to contribute properties
Manually added
Inferred by static analyzer
Generated from higher-level description (model)
MANY different workflows
Dynamic vs. static verification
Various techniques to generate and prove formulas

Conclusion
Formal methods are useful in various ways
They need to be democratized
They need to be integrated in the Dev Cycle … in an agile way

http://www.open-do.org	4093
http://hi-lite.forge.open-do.org	56
http://translate.googleusercontent.com	19
http://translate.yandex.net	3
http://webcache.googleusercontent.com	2
http://www.slideshare.net	1
http://honyaku.yahoofs.jp	1
http://cache.baidu.com	1
https://hi-lite.forge.open-do.org	1

Open-DO Update

by AdaCore

Accessibility

Categories

Upload Details

Usage Rights

9 Embeds 4,177

Statistics

Open-DO Update Presentation Transcript

Open-DO Update

by AdaCore

Accessibility

Categories

Upload Details

Usage Rights

9 Embeds 4,177

Statistics

Open-DO Update Presentation Transcript

Let LinkedIn power your SlideShare experience

Let LinkedIn power your SlideShare experience

Customize SlideShare content based on your interests

Keep up to date when your LinkedIn contacts post on SlideShare