Three Years of ECSS Software Standards An Appraisal and Outlook OPS-G Forum 20th January 2006 D. Ponz & M. Spada

Outline Why Software Engineering The Standardisation Process ECSS Software Standards applicable to ESA Software Key issues in application of the ECSS Standards Tailoring of ECSS Standards for Ground Segments First step: SEMG Second step: SETG Next Steps Conclusions

Why Software Engineering

The Standardisation Process

ECSS Software Standards applicable to ESA Software

Key issues in application of the ECSS Standards

Tailoring of ECSS Standards for Ground Segments

First step: SEMG

Second step: SETG

Next Steps

Conclusions

What is Software Engineering? Software engineering is about systematic development, evaluation and maintenance of software Surveys have shown that the average corporation pays 3 to 7% of its annual revenues for hardware and software for corporate data processing and another 2 to 5% for maintenance: that’s 5-10% of the economy in the developed world Another measure: software project failures cost the US economy about 60 B$ annually – double this for the whole world* Information technology is mission critical for many businesses - rely on the efficiency and integrity of their IT systems for their business needs Means that software engineering is vital * Source 2002 study by America's National Institute of Standards (NIST), quoted in Economist of Nov 25 2004

Software engineering is about systematic development, evaluation and maintenance of software

Surveys have shown that the average corporation pays 3 to 7% of its annual revenues for hardware and software for corporate data processing and another 2 to 5% for maintenance: that’s 5-10% of the economy in the developed world

Another measure: software project failures cost the US economy about 60 B$ annually – double this for the whole world*

Information technology is mission critical for many businesses - rely on the efficiency and integrity of their IT systems for their business needs

Means that software engineering is vital

* Source 2002 study by America's National Institute of Standards (NIST), quoted in Economist of Nov 25 2004

Software Engineering Standards Many people have to develop or maintain software or manage the related processes It follows that software engineering standards are used by many people The standards must therefore be readily understandable to a wide community This is not the case for many other engineering standards: e.g. we all use TCP/IP many times on a daily basis, most of us without any special knowledge of TCP/IP, but implementing TCP/IP packages and tuning TCP/IP communications systems is very specialised activity

Many people have to develop or maintain software or manage the related processes

It follows that software engineering standards are used by many people

The standards must therefore be readily understandable to a wide community

This is not the case for many other engineering standards:

e.g. we all use TCP/IP many times on a daily basis, most of us without any special knowledge of TCP/IP, but implementing TCP/IP packages and tuning TCP/IP communications systems is very specialised activity

The Standardisation Process Standardisation organisations like the European Cooperation for Space Standardisation (ECSS) and the Consultative Committee for Space Data Systems (CCSDS) exist to produce and maintain standards They have established structures/procedures for reviewing and approving standards WGs, Independent peer review, committee hierarchy etc. In addition, some technical standards can be verified by prototyping, bread-boarding or an initial implementation before release of the standard E.g. CCSDS SLE standards But not possible for a software engineering standard

Standardisation organisations like the European Cooperation for Space Standardisation (ECSS) and the Consultative Committee for Space Data Systems (CCSDS) exist to produce and maintain standards

They have established structures/procedures for reviewing and approving standards

WGs, Independent peer review, committee hierarchy etc.

In addition, some technical standards can be verified by prototyping, bread-boarding or an initial implementation before release of the standard

E.g. CCSDS SLE standards

But not possible for a software engineering standard

Standardisation Process - Software Engineering Standards Not practical to “test “a software engineering standard before release because Would involve a significant implementation project – would take too long, would not easily find volunteer projects A software engineering standard reflects best practices as understood by the writers of the standards Incorporates the authors’ development cultures Users of the standard are in effect verifying it! Users’ development cultures may differ from those of authors It can result in interpretation problems Tailoring complicates it further: Users may exercise the processes in ways which were not thought about by the authors or only subset of the processes

Not practical to “test “a software engineering standard before release because

Would involve a significant implementation project – would take too long, would not easily find volunteer projects

A software engineering standard reflects best practices as understood by the writers of the standards

Incorporates the authors’ development cultures

Users of the standard are in effect verifying it!

Users’ development cultures may differ from those of authors

It can result in interpretation problems

Tailoring complicates it further:

Users may exercise the processes in ways which were not thought about by the authors or only subset of the processes

Standards Applicable to ESA Software ECSS-E-40B Space Engineering – Software E-40 Part1B: Principles and Requirements (Nov. 2003) Covers all aspects of space software engineering Complete life cycle Is defined in terms of Engineering processes,which may be broken down into activities and tasks Process interfaces with management and product assurance Inputs and outputs to processes E-40 Part2B: Document Requirements Definitions (DRDs) (Mar. 2005) Defines the contents of documents in line with expected outputs of processes/actrivities defined in E-40 Part1

ECSS-E-40B Space Engineering – Software

E-40 Part1B: Principles and Requirements (Nov. 2003)

Covers all aspects of space software engineering

Complete life cycle

Is defined in terms of

Engineering processes,which may be broken down into activities and tasks

Process interfaces with management and product assurance

Inputs and outputs to processes

E-40 Part2B: Document Requirements Definitions (DRDs) (Mar. 2005)

Defines the contents of documents in line with expected outputs of processes/actrivities defined in E-40 Part1

Standards Applicable to ESA Software (Cont.) ECSS-E-40B is or will be complemented by the following level 3 (technical domain specific) standards:

ECSS-E-40B is or will be complemented by the following level 3 (technical domain specific) standards:

Standards Applicable to ESA Software (Cont.) ECSS-Q80-B: Space Product Assurance – Software (Oct. 2003) Covers SW product assurance programme implementation: organisation, responsibilities, etc. for SW PA SW process assurance: lifecycle and requirements applicable to different phases SW product quality assurance: metrification, product quality requirements, etc. Management Standards – not software specific M-00B Policy and Principles (August 2003) M-10B Project Breakdown Structures (June 2003) M-20B Project Organisation (June 2003) M-30A Project Phasing and Planning (April 1996) M-40A Configuration Management (April 1996) M-50A Information/Document Management (April 1996) M-60A Cost and Schedule Management (April 1996) M-00-03A Risk Management (April 2000)

ECSS-Q80-B: Space Product Assurance – Software (Oct. 2003)

Covers

SW product assurance programme implementation: organisation, responsibilities, etc. for SW PA

SW process assurance: lifecycle and requirements applicable to different phases

SW product quality assurance: metrification, product quality requirements, etc.

Management Standards – not software specific

M-00B Policy and Principles (August 2003)

M-10B Project Breakdown Structures (June 2003)

M-20B Project Organisation (June 2003)

M-30A Project Phasing and Planning (April 1996)

M-40A Configuration Management (April 1996)

M-50A Information/Document Management (April 1996)

M-60A Cost and Schedule Management (April 1996)

M-00-03A Risk Management (April 2000)

Application of Standards Solution : Use prescriptive standards but allow tailoring ECSS Approach: Tailoring of the standards to Programme/Project needs Tailoring influencing factors Programme/Project characteristics e.g. Programmatic and financial constraints Technical parameters Scope and type of procurement

Solution : Use prescriptive standards but allow tailoring

ECSS Approach: Tailoring of the standards to Programme/Project needs

Tailoring influencing factors

Programme/Project characteristics e.g.

Programmatic and financial constraints

Technical parameters

Scope and type of procurement

The Challenge of Tailoring a Software Engineering Standard Requires detailed understanding of the whole standard, Average software developer or project manager does not have this Standard is open to interpretation - ambiguous Almost no guiding literature available – also for ISO 12207, the “parent” standard of E-40 Limited advice in ECSS literature, e.g. a short annex to ECSS-E-40B

Requires detailed understanding of the whole standard,

Average software developer or project manager does not have this

Standard is open to interpretation - ambiguous

Almost no guiding literature available – also for ISO 12207, the “parent” standard of E-40

Limited advice in ECSS literature, e.g. a short annex to ECSS-E-40B

Tailoring Options Use a tailoring tool Questionnaire based The answers are used to define a table with Retained requirements Output documents required TEC-SWE have developed a useful tailoring tool for E-40 Organizational tailoring Suited for an organisation with similar projects The same tailoring could be used for all No tailoring at all! Let the contractor do it In conflict with ESA Policy (IPOL(2004)6) – customer should do it Only works where there is no competition – otherwise “playing field not level”

Use a tailoring tool

Questionnaire based

The answers are used to define a table with

Retained requirements

Output documents required

TEC-SWE have developed a useful tailoring tool for E-40

Organizational tailoring

Suited for an organisation with similar projects

The same tailoring could be used for all

No tailoring at all!

Let the contractor do it

In conflict with ESA Policy (IPOL(2004)6) – customer should do it

Only works where there is no competition – otherwise “playing field not level”

Particular problems of tailoring E-40 E-40 is defined in terms of Processes Inputs and outputs to processes Processes may be broken down into activities with their own inputs and outputs Problem: Have to map outputs into a set of supplier deliverable documents Outputs must be aggregated into manageable set of deliverables If not done can get large number of deliverables Not practicable, very costly, difficult configuration management Problem for both customer and supplier The E-40 DRDs were not available for a long time

E-40 is defined in terms of

Processes

Inputs and outputs to processes

Processes may be broken down into activities with their own inputs and outputs

Problem:

Have to map outputs into a set of supplier deliverable documents

Outputs must be aggregated into manageable set of deliverables

If not done can get large number of deliverables

Not practicable, very costly, difficult configuration management

Problem for both customer and supplier

The E-40 DRDs were not available for a long time

The “ Many Standards” Issue for Software Problem: ECSS-E-40 addresses the technical processes needed to define, design, build and maintain software Producing and maintaining software also involves the disciplines of management and product assurance In the ECSS scheme need to use standards from the PA branch (ECSS-Q series) Management branch (the ECSS-M series) Solutions possible: Use E-40 only or E-40 and Q-80 E-40 and Q-80 contain basic management elements Combine all relevant ECSS standards into a single volume (ESOC and Galileo approach)

Problem:

ECSS-E-40 addresses the technical processes needed to define, design, build and maintain software

Producing and maintaining software also involves the disciplines of management and product assurance

In the ECSS scheme need to use standards from the

PA branch (ECSS-Q series)

Management branch (the ECSS-M series)

Solutions possible:

Use E-40 only or E-40 and Q-80

E-40 and Q-80 contain basic management elements

Combine all relevant ECSS standards into a single volume (ESOC and Galileo approach)

Stated with BSSC ESA Ground Segment Software Engineering and Management Guide – SEMG , BSSC(2002)4 adopted by ESOC in 2002 became applicable to new projects, e.g. GOCE and Herschel/Planck, TMTCS Backend System Structured in three Parts: Part A : Software Engineering Software lifecycle processes in accordance with the E-40 Standard Part B : Software PA & Project Management Software project management, configuration and documentation management and software product assurance in accordance with ECSS Q-80 and M-series Standards Part C : Document Templates First ECSS Software Engineering Standards Tailoring: the SEMG

Stated with BSSC ESA Ground Segment Software Engineering and Management Guide – SEMG , BSSC(2002)4

adopted by ESOC in 2002

became applicable to new projects, e.g. GOCE and Herschel/Planck, TMTCS Backend System

Structured in three Parts:

Part A : Software Engineering

Software lifecycle processes in accordance with the E-40 Standard

Part B : Software PA & Project Management

Software project management, configuration and documentation management and software product assurance in accordance with ECSS Q-80 and M-series Standards

Part C : Document Templates

Experience with the SEMG ECSS-E-40 and ECSS-Q-80 are consistent E.g. Tailoring of Q-80 flows naturally from that of E-40 ECSS-M standards are Focused more on overall space project management Not adapted for software engineering Stylistically inconsistent with E-40 and Q-80 Following a period of usage of the SEMG, the following needs arose To align the SEMG with the latest versions of the ECSS Standards To provide traceability to the ECSS applicable standards To retrofit the users experience with the usage of SEMG To address some specific problems with the first tailoring e.g.: far too many (over 50) deliverables Due to lack of clarity on mapping of deliverables to process outputs, especially in ECSS-E-40 Tailoring not critical enough (did not go far enough)

ECSS-E-40 and ECSS-Q-80 are consistent

E.g. Tailoring of Q-80 flows naturally from that of E-40

ECSS-M standards are

Focused more on overall space project management

Not adapted for software engineering

Stylistically inconsistent with E-40 and Q-80

Following a period of usage of the SEMG, the following needs arose

To align the SEMG with the latest versions of the ECSS Standards

To provide traceability to the ECSS applicable standards

To retrofit the users experience with the usage of SEMG

To address some specific problems with the first tailoring e.g.: far too many (over 50) deliverables

Due to lack of clarity on mapping of deliverables to process outputs, especially in ECSS-E-40

Tailoring not critical enough (did not go far enough)

Tailoring of ECSS Software Standards for Ground Segments in ESA – SETG , BSSC 2005(1) BSSC established a support contract with Fraunhofer Institute for Experimental Software Engineering (IESE) in July 2004 IESE has experience in the software standards domain IESE is an experimental institute which brings together a solid theoretical background with a down-to-earth mindset Main process steps Interview of main stakeholders: feedback on SEMG, proposals for improvement Review of latest ECSS standard versions against SEMG Review by BSSC and interviewees New tailoring started in June 2004 SETG published in June 2005 A step forward – The SETG

Tailoring of ECSS Software Standards for Ground Segments in ESA – SETG , BSSC 2005(1)

BSSC established a support contract with Fraunhofer Institute for Experimental Software Engineering (IESE) in July 2004

IESE has experience in the software standards domain

IESE is an experimental institute which brings together a solid theoretical background with a down-to-earth mindset

Main process steps

Interview of main stakeholders: feedback on SEMG, proposals for improvement

Review of latest ECSS standard versions against SEMG

Review by BSSC and interviewees

New tailoring started in June 2004

SETG published in June 2005

The process descriptions must be streamlined and made more structured, with clear identification of the activities to be performed Explicit identification of process outputs is required The review/update cycle for outputs/documents should be made more explicit Explicit identification of reviews inputs and outputs should be provided The processes outputs should be mapped to documents as far as possible The number of documents to be produced (about 50 in SEMG) was excessive to be reduced as far as possible by elimination of redundancies and aggregation of outputs No traceability to ECSS requirements. Tailoring is not transparent Feedback on SEMG

The process descriptions must be streamlined and made more structured, with clear identification of the activities to be performed

Explicit identification of process outputs is required

The review/update cycle for outputs/documents should be made more explicit

Explicit identification of reviews inputs and outputs should be provided

The processes outputs should be mapped to documents as far as possible

The number of documents to be produced (about 50 in SEMG) was excessive

to be reduced as far as possible by elimination of redundancies and aggregation of outputs

No traceability to ECSS requirements. Tailoring is not transparent

Traceability to ECSS requirements has been provided The processes and the terminology have been aligned with the latest relevant ECSS standards version T he processes have been consolidated in line with ESA Ground Segment practice T he processes outputs have been detailed and associated with documents and reviews as far as necessary Reviews inputs and outputs have been detailed Traceability of outputs vs. documents has been provided The number of mandated documents has been considerably reduced ( from 50 to 16 !) and the document life-cycles have been made explicit Main changes in the SETG

Traceability to ECSS requirements has been provided

The processes and the terminology have been aligned with the latest relevant ECSS standards version

T he processes have been consolidated in line with ESA Ground Segment practice

T he processes outputs have been detailed and associated with documents and reviews as far as necessary

Reviews inputs and outputs have been detailed

Traceability of outputs vs. documents has been provided

The number of mandated documents has been considerably reduced ( from 50 to 16 !) and the document life-cycles have been made explicit

The SETG has four parts: Part A: Software Engineering Software lifecycle processes in accordance with the E-40 Standard Part B: Product Assurance and Management Software project management, configuration and document management and software product assurance in accordance with ECSS Q-80 and M-series Standards Part C: Document Templates Packaging of processes outputs Part D: Traceability Against ECSS Standards Traceability Matrix with respect to ECSS Standards Requirements Outputs vs. Documents The SETG

The SETG has four parts:

Part A: Software Engineering

Software lifecycle processes in accordance with the E-40 Standard

Part B: Product Assurance and Management

Software project management, configuration and document management and software product assurance in accordance with ECSS Q-80 and M-series Standards

Part C: Document Templates

Packaging of processes outputs

Part D: Traceability Against ECSS Standards

Traceability Matrix with respect to ECSS Standards Requirements

Outputs vs. Documents

The SETG Structure Development Maintenance Verification Validation Project management Configuration management Document management Product assurance Process assurance Primary life cycle processes Support life cycle processes Part B Part A

ECSS processes and the SETG SETG

ECSS processes and the SETG (Cont.) ESTEC Ground Segment Manager/ Final User ESOC engineering OPS Industry Customer that performs the System requirement engineering Customer that performs the System requirement engineering Customer that makes only System/ Software requirement engineering Supplier that does only development Supplier that does only development software spec - CDR system software spec – A R 1 system software spec – AR 2 Customer that delegates the system requirement engineering to its Supplier Supplier that does everything Supplier that does everything Supplier that does the rest Supplier that does the rest E40 applicable SETG as a valid tailoring of E40 SETG as OPS/ industry sw std

Unchanged 529 Tailored 20 Tailored out 78 Tailoring Statistics 85% 3% 12% unchanged Tailored Tailored out

Unchanged 529

Tailored 20

Tailored out 78

Table indicates when documents are produced/updated/reviewed Documents Life-cycle

Table indicates when documents are produced/updated/reviewed

The SETG is a tailoring of ECSS Standards for ESA Ground Segment Software It is a more user-friendly version of the BSSC SEMG, aligned with the latest versions of the ECSS Standards The traceability to the ECSS Standards is fully documented [SETG-Part-D] The SETG is intended to be used as the applicable tailoring for ESOC software development and maintenance contracts ESRIN is considering adopting the SETG for their Ground Segment Software The SETG is available On the BSSC Web Site http://www. estec . esa . int / wmwww /EME/ Bssc / stds . htm On the BSSC LN Database Summary

The SETG is a tailoring of ECSS Standards for ESA Ground Segment Software

It is a more user-friendly version of the BSSC SEMG, aligned with the latest versions of the ECSS Standards

The traceability to the ECSS Standards is fully documented [SETG-Part-D]

The SETG is intended to be used as the applicable tailoring for ESOC software development and maintenance contracts

ESRIN is considering adopting the SETG for their Ground Segment Software

The SETG is available

On the BSSC Web Site

http://www. estec . esa . int / wmwww /EME/ Bssc / stds . htm

On the BSSC LN Database

Review OPS-QMS procedures to align them with SETG WG established in Q4 2005 WG reviewed 39 procedures DCRs raised on 19 documents Work has been completed Tailoring required for small projects and studies Can be based on tailoring of ECSS-E-40 only Approach will be to keep this tailoring consistent with the SETG Will be a BSSC activity for 2006 Next Steps

Review OPS-QMS procedures to align them with SETG

WG established in Q4 2005

WG reviewed 39 procedures

DCRs raised on 19 documents

Work has been completed

Tailoring required for small projects and studies

Can be based on tailoring of ECSS-E-40 only

Approach will be to keep this tailoring consistent with the SETG

Will be a BSSC activity for 2006

Conclusions Improvement of ECSS software standards needed These standards are not easy to use Not “ readily understandable” and unambiguous, inconsistent in some cases Tailoring of ECSS software engineering standards is a significant challenge E-40 and Q-80 give all options, but not what actually required in given project Solutions have been identified, but there is still some way to go Proliferation of tailorings must be avoided Define families of software applications Have tailoring done by experts Promising work done by TEC-SWE : Tailoring tool http://www.estec.esa.nl/wmwww/EME/index.htm BSSC : Tailoring of ECSS Software Engineering Standards for Ground Segments

Improvement of ECSS software standards needed

These standards are not easy to use

Not “ readily understandable” and unambiguous, inconsistent in some cases

Tailoring of ECSS software engineering standards is a significant challenge

E-40 and Q-80 give all options, but not what actually required in given project

Solutions have been identified, but there is still some way to go

Proliferation of tailorings must be avoided

Define families of software applications

Have tailoring done by experts

Promising work done by

TEC-SWE : Tailoring tool

http://www.estec.esa.nl/wmwww/EME/index.htm

BSSC : Tailoring of ECSS Software Engineering Standards for Ground Segments

Conclusions (Cont.) Software engineering combines the E, Q and M disciplines. The ECSS standards separation into E, Q and M documents is not helpful for software Promising work done by BSSC Integrated tailoring of software engineering related ECSS standards Reviews of standards should not be for “elite” People with experience in the field should be included as far as possible Reviews should be means to expose the standards to the people who will use them Feedback Awareness Commitment

Software engineering combines the E, Q and M disciplines. The ECSS standards separation into E, Q and M documents is not helpful for software

Promising work done by BSSC

Integrated tailoring of software engineering related ECSS standards

Reviews of standards should not be for “elite”

People with experience in the field should be included as far as possible

Reviews should be means to expose the standards to the people who will use them

Feedback

Awareness

Commitment