DO-178C : upcoming guidance for OOS Cyrille Comar SafeComp2009, Humburg, Sept 15, 2009

DO-178C : upcoming guidance for OOS

Cyrille Comar

SafeComp2009, Humburg, Sept 15, 2009

Summary A few words on DO-178B Toward DO-178C Glimpse at the Tools Qualification Supplement Glimpse at the OOT Supplement Glimpse at the Formal Methods Supplement

A few words on DO-178B

Toward DO-178C

Glimpse at the Tools Qualification Supplement

Glimpse at the OOT Supplement

Glimpse at the Formal Methods Supplement

DO-178B Software Aspects of commercial aircraft certification (Aeronautic Industry vs FAA & EASA) Evidence must be shown for 80+ objectives encompassing: Planning & liaison with authorities Software Development Software Verification Configuration Management Quality Assurance Strong emphasis on Requirement-based Testing Traceability of life cycle data

Software Aspects of commercial aircraft certification (Aeronautic Industry vs FAA & EASA)

Evidence must be shown for 80+ objectives encompassing:

Planning & liaison with authorities

Software Development

Software Verification

Configuration Management

Quality Assurance

Strong emphasis on

Requirement-based Testing

Traceability of life cycle data

DO-178B 5 levels (A to E) depending on the criticality of the software Higher levels means more Detailed evidence Independence in verification Testing (Statement Coverage, Decision Coverage, MCDC) ad hoc Standard verifying good Software Engineering Practices (of the 90s…) Well adapted to traditional development in Ada83 or C

5 levels (A to E) depending on the criticality of the software

Higher levels means more

Detailed evidence

Independence in verification

Testing (Statement Coverage, Decision Coverage, MCDC)

ad hoc Standard verifying good Software Engineering Practices (of the 90s…)

Well adapted to traditional development in Ada83 or C

DO-178C / ED-12C Working group: SC-205 (RTCA) WG-71 (EUROCAE) Final documents are owned jointly by RTCA & EUROCAE Goals: Fix known issues in the official documents Take into account new trends in Software Engineering Past revisions: 1982: DO-178 / ED-12 1985: DO-178A / ED-12A 1992: DO-178B / ED-12B

Working group: SC-205 (RTCA) WG-71 (EUROCAE)

Final documents are owned jointly by RTCA & EUROCAE

Goals:

Fix known issues in the official documents

Take into account new trends in Software Engineering

Past revisions:

1982: DO-178 / ED-12

1985: DO-178A / ED-12A

1992: DO-178B / ED-12B

TimeLine First plenary meeting early 2005 (in London) 14 plenary meetings between 2005 & 2010 Meetings alternate between US & Europe Last Meeting in 06-2009 in Hartford CT Next Meeting in 10-2009 in Paris 2 0 1 0 Sarasota, Florida, Feb 8 Integration committee Final Plenary

First plenary meeting early 2005 (in London)

14 plenary meetings between 2005 & 2010

Meetings alternate between US & Europe

Last Meeting in 06-2009 in Hartford CT

Next Meeting in 10-2009 in Paris

Participants The group is completely open, based on volunteer’s participation About 110 regular participants in 2009 Many one-time visitors Some plenary had more than 200 participants About ½ from the US and ½ from Europe Europe: mostly GB, France, Germany, Sweeden Some participation from Brazil and recently China 3 main types of Participants Industry (Boeing, Airbus, Lockeed, RC, Honeywell, Thales, Eurocopter, …) Authorities & DERs (FAA, EASA, Transport Canada, DGAC…) Tool vendors (LDRA, Esterel, Mathworks, AdaCore, Verocel, …)

The group is completely open, based on volunteer’s participation

About 110 regular participants in 2009

Many one-time visitors

Some plenary had more than 200 participants

About ½ from the US and ½ from Europe

Europe: mostly GB, France, Germany, Sweeden

Some participation from Brazil and recently China

3 main types of Participants

Industry (Boeing, Airbus, Lockeed, RC, Honeywell, Thales, Eurocopter, …)

Authorities & DERs (FAA, EASA, Transport Canada, DGAC…)

Tool vendors (LDRA, Esterel, Mathworks, AdaCore, Verocel, …)

Organization 2 chairs (Jim Krodel & Gérard Ladier) 7 Subgroups: SG1: SCWG Document Integration SG2: Issues & Rationale SG3: Tool Qualification SG4: Model Based Design & Verification SG5: Object Oriented Technology SG6: Formal Methods SG7: Safety Related Considerations (ground based systems) Subgroups work specific items Based on original TOR & Issue list New supplements or changes to the core document Plenary vote for acceptance “ almost-unanimity” is the rule

2 chairs (Jim Krodel & Gérard Ladier)

7 Subgroups:

SG1: SCWG Document Integration

SG2: Issues & Rationale

SG3: Tool Qualification

SG4: Model Based Design & Verification

SG5: Object Oriented Technology

SG6: Formal Methods

SG7: Safety Related Considerations (ground based systems)

Subgroups work specific items

Based on original TOR & Issue list

New supplements or changes to the core document

Plenary vote for acceptance

“ almost-unanimity” is the rule

Expected Result(s) DO-178B DO-278 DO-248B DO-248C DO-178C core 12.3 guidance guidelines Clarification fixes Clarification fixes MBD Suppl. OOT Suppl. Formal methods Suppl. Tools Qual Suppl Overriding supplements DO-278A core changes Equivalent to DO-178c

DO-178B

Status as of Hartford Meeting Core DO-178c: some moderate changes to come Tools supplement : almost complete MBD supplement: ½ of the text approved OO supplement : ¾ of the text approved FM supplement : ¾ of the text approved Most of the guidance approved

Core DO-178c: some moderate changes to come

Tools supplement : almost complete

MBD supplement: ½ of the text approved

OO supplement : ¾ of the text approved

FM supplement : ¾ of the text approved

Example of change in the Main Document : fig 2-1 DO-178B DO-178C

Tool Qualification Supplement DO-178B 2 cases Development Tool Verification Tool DO-178C 3 criteria & 5 levels (TQL) Criteria 1 Criteria 2 Criteria 3 Could insert an error Could fail to detect an error and is used to reduce other development or verification activities Could fail to detect an error Qualification needed when processes are eliminated, reduced or automated without manual verification

DO-178B

2 cases

Development Tool

Verification Tool

Tool Qualification Supplement (2) Examples of “Criteria 2 vs Criteria 3” Proof Tool Static Analysis Tool Source code verification + reduce robustness testing Level 3 Level 2 Source code review + Remove defensive code Level 3 Level 2

Examples of “Criteria 2 vs Criteria 3”

Proof Tool

Static Analysis Tool

Tool Qualification Supplement (3) Mostly for formal methods & MBD TQL 1: do-178 level A TQL 2: do-178 level B TQL3: do-178 level C TQL4 : complete requirements describe architecture more verifications TQL5 : TOR verification Software Level Criteria 1 2 3 A TQL-1 TQL-4 TQL-5 B TQL-2 TQL-4 TQL-5 C TQL-3 TQL-5 TQL-5 D TQL-4 TQL-5 TQL-5

Mostly for formal methods & MBD

OOT Supplement Very few changes related to DO-178B Addresses more than pure OOT stuff Memory management (e.g. garbage collection) Exception management Generics (parametric polymorphism) Virtualization techniques One significant additional objective: “ Local Type Consistency Verification” Many guidelines Can be addressed by proper Design/Coding standards

Very few changes related to DO-178B

Addresses more than pure OOT stuff

Memory management (e.g. garbage collection)

Exception management

Generics (parametric polymorphism)

Virtualization techniques

One significant additional objective:

“ Local Type Consistency Verification”

Many guidelines

Can be addressed by proper Design/Coding standards

Local Type Consistency Verification 3 choices are given: Formally verify substitutability Ensure that each class pass all the tests of its parent types that the class can replace For each call point, test every method that can be invoked (pessimistic testing approach) Rationale: Usual Structural Coverage Analysis not sufficient in presence of dynamic dispatch Liskov Substitution Principle

3 choices are given:

Formally verify substitutability

Ensure that each class pass all the tests of its parent types that the class can replace

For each call point, test every method that can be invoked (pessimistic testing approach)

Rationale:

Usual Structural Coverage Analysis not sufficient in presence of dynamic dispatch

Memory Management The bar is pretty high but it makes it possible to use sound real-time garbage collectors All typical vulnerabilities of memory managers are to be verified: Ambiguous references Fragmentation Deallocation starvation Heap memory exhanstion Premature deallocation Time bound allocation & deallocation …

The bar is pretty high but it makes it possible to use sound real-time garbage collectors

All typical vulnerabilities of memory managers are to be verified:

Ambiguous references

Fragmentation

Deallocation starvation

Heap memory exhanstion

Premature deallocation

Time bound allocation & deallocation

…

Virtualization What is code and what is data ? Many objectives apply to code (… and not to data…) “ Any time that data, when interpreted, provides control flow for the executable program, virtualization is being used” Each virtualization layer must be verified on its own

What is code and what is data ?

Many objectives apply to code (… and not to data…)

“ Any time that data, when interpreted, provides control flow for the executable program, virtualization is being used”

Each virtualization layer must be verified on its own

Formal Methods Supplement Many activities can be achieved by formal methods Requirements consistency, correctness & review Source code review and compliance with LLR Test cases covering the LLR … Some (but not all) testing can be replaced by formal verification

Many activities can be achieved by formal methods

Requirements consistency, correctness & review

Source code review and compliance with LLR

Test cases covering the LLR

…

Some (but not all) testing can be replaced by formal verification

What about formal verification of the source code? Can it be used to alleviate testing? No. What runs of the target hardware is the object code not the source code! … but … there is an escape clause: << Formal analysis of source code can be used to reach [compliance with LLR] provided that complementary analyses show the property preservation between source code and object code>>

Can it be used to alleviate testing?

No. What runs of the target hardware is the object code not the source code!

… but … there is an escape clause:

<< Formal analysis of source code can be used to reach [compliance with LLR] provided that complementary analyses show the property preservation between source code and object code>>