Lean Software Production and Qualification Infrastructures

Lean Software Production and Qualification Infrastructures Florian Villoing – AdaCore villoing@adacore.com Grenoble – October 20, 2009 Valence – October 22, 2009

What is it about? ● A software production infrastructure ● A software qualification infrastructure In a lean fashion! 2

Lean Software Production 3

About AdaCore's products ● Ada compilation toolchain ➔ Compiler (based on GCC), debugger ➔ Interfacing with C, C++ and Java ➔ 2 IDEs: GPS + GNATbench (Eclipse plug-in) ➔ Coverage analysis ● Add-ons: ASIS, XML/Ada, GtkAda, AWS, PolyORB ● Static analysis tools ➔ Metrics computation ➔ Stack analyzer ➔ Coding standard checker ➔ Automatic peer review 4

What is a compiler? “A compiler is a computer program (or set of programs) that transforms source code written in a computer language (the source language) into another computer language (the target language, often having a binary form known as object code).” Wikipedia 5

Native and cross compilers ● A native compiler generates code for the same machine (the host) ● A cross compiler generates code for another machine (the target) native cross Host Target 6

Native compilers ● Different operating systems ● Windows, Linux, Solaris, Mac OS X, HP-UX, VMS, LynxOS, Tru64, AIX, IRIX, etc. ● Different Versions ● Windows XP, 2000, Vista, etc. ● Linux Red Hat 3/4/5, SuSe 8/9/10, etc. ● Different processors ● x86, x86 64bits, Itanium, SPARC, PowerPC, Alpha, PA-RISC, etc. 7

Cross compilers ● Host environment (OS, OS version, processor) ➔ Linux, Windows, Solaris ● Target environment (Target OS, Target OS Version, processor) ➔ VxWorks, PikeOS, ElinOS, Lynx OS, Nucleus OS, etc. ➔ VxWorks 5, VxWorks 6, VxWorks 653, etc. ➔ PowerPC, ARM, AVR, LEON, ERC32, etc. ● Run-time system ➔ ZFP, Ravenscar, Full, etc. 8

Combinatory explosion ● Native toolchains ● OS x OS version x processor = 46 ● Cross toolchains ● Host OS x host OS version x host processor x Target OS x target OS version x target processor x run-time system = 49 9

And things are getting more complex... ● New platforms are added each year ● Few are removed ● Long-term projects need to be supported for up 10-15 years How do I deal with that? 10

Some figures ● 95 different platforms (native and cross) ● 15 independent test suites ➢ 15000 tests and 10 million lines of code ➢ 1.4 million tests run each night ● 96 scripts to control the infrastructure ➢ About 2300 scripts run each day 11

How to solve the equation? ● Adopting a lean strategy ● Introducing agile tactics Let's move on! 12

The Toyota Production System TPS Jidoka Just-in-Time ● Automatic defects ● Making only what is detection needed, when it's needed, in the amount needed + ● Stop the line ● Limited stocks with all ● Identify the problem parts ● Andon (problem display ● Replace used parts from board) the preceding process ● An operator for several ● Limited stocks for the machines preceding process 13

Adopting a lean strategy ● Identify the value ● Identify and remove waste (muda) ● Automate ● Detect defects early ● Fix the cause of defects 14

The value ● Provide high quality software ● Fixed release schedule ● Provide pre-release versions as soon as possible ● Support new platforms ● Add new features ● Add new tools 15

Quality & Open-Source ● All AdaCore's products are Open-Source ● It's possible to achieve high quality ● AdaCore is an active member of the GCC community ● We identify a reasonably stable version of GCC ● We run it through our QA process ● We contribute our changes back to the community 16

Lean testing strategy ● A fundamental piece toward achieving high quality ● Goals: ● Detect defects as early as possible ● Adopt different testing tactics to minimize risk ● Keep high level of productivity ● Balance the machine load 17

How to detect defects? 1. Local testing 2. Mailserver 3. Check-in 5. Nightly builds SVN 4. Continuous Builder 18

The mailserver system ● Email-based interface Less than 20 minutes! ● To test patches ● On a remote machine ● Incremental build ● Full testing even on exotic platforms ● Clean result mandatory before check-in 19

The continuous builder ● Every check-in triggers an automatic build ● Fast machines ➔ quick feedback ● Stop-the-line: should an error be reported, immediate attention/action is required ● Limit the risk of nightly builds failures ➔ Delays in intermediate releases delivery ➔ More waste, less value ➔ Customer satisfaction decreases 20

Nightly builds SVN Packaging Linux Windows Building Testing Red Hat 5 SuSe 9 XP Server Vista 21

Nightly builds ● For cross platforms ➔ We test a different version of the target OS everyday ➔ Use of simulators ➔ More stability ➔ Better performances ● Heavy use of virtual machines ➔ Ease maintenance of obsolescent platforms 22

Lean production infrastructure ● Shippable software is produced every day ● Defects are detected early ● Allows to achieve high quality ● Allows to deliver intermediate releases quickly 23

Official releases ● Only 2 releases a year ➢ One major release (January/February) ➢ One corrective release (June/July) ● Customers can plan their own release cycle in advance ● Intermediate releases are still available ➢ Provide quick fixes ➢ Allows to experiment with new features 24

Official releases ● Failed builds are relaunched ● Manual review of each test suite report ● Human sanity check of each package ● Cross toolchains are tested on real boards ➢ 48h to run the ACATS on some cross platforms! 25

The release schedule 6.2 branch 6.3 branch 6.2.2 Release 6.3.2 Release 6.2.1 Release 6.3.1 Release 6.2.0 Beta 6.3.0 Beta Development branch 26 2009 2010

New features and known problems ● New features are advertised to everybody on our web site ● Known problems are listed on the customer web interface ● Improve traceability ● Provide easy access to workarounds ● Raises interest in evolution of the technology 27

Visual management with GAIA ● Monitor scripts ● Everything is stored ● Monitor machines in a database ● Monitor test suites ● Human-readable weather reports ● Submit analysis ● Use of the django framework 28

GAIA: How does it work? Test machines GAIA Server GAIA User's Interface DB 29

GAIA: Test suite overview 30

GAIA: Weather reports 31

Lean Software Qualification 32

Critical software certification ● Aerospace ● ECSS-E-ST-40C/ECSS-Q-ST-80C ● Civil Avionics ● DO-178B/ED-12B ● Air Traffic Management ● DO-278B ● Security ● Common Criteria ● Railway ● EN-50126/EN-50128/EN-50129 ● Others... 33

DO-178B/ED-12B ● “DO-178B, Software Considerations in Airborne Systems and Equipment Certification” ● Last revision: December 1, 1992 ● DO-178C is in the pipeline ● 5 Design Assurance Levels (DAL) ● A => Catastrophic ● B => Hazardous ● C => Major ● D => Minor ● E => No effect 34

What is in DO-178B? ● Process oriented: ● Planning ● Development ● Verification ● Configuration management ● Quality assurance ● Certification liaison 35

DO-178B: The Waterfall Model Not very agile... Source: Wikipedia 36

DO-178B: The V-Model Not agile either... Source: Wikipedia 37

Certification and Qualification ● Embedded software is certified ● A tool is qualified ● As a development tool – Output is part of the airborne software – Can introduce errors – Certification-like process ● As a verification tool – May fail to detect an error – Lightweight process – Tool Operational Requirements – Requirements based testing 38

Qualification of verification tools ● A tool may be reused in different contexts ● Operational requirements may change ● Different part of the tool may be used in different context ● The tool may evolve Let's do agile qualification! 39

The qualification machine ● Based on FitNesse ● Centralize all qualification artifacts ● Ensure consistency between requirements, test cases and test data ● Automatic generation of qualification documents It's now easy to add new requirements and associated tests! 40

The qualification machine Let's see it in action! 41

References ● Lean Software Development, An Agile Toolkit by Mary and Tom Poppendieck ● Implementing Lean Software Development by Mary and Tom Poppendieck ● Lean Software Strategies, Proven Techniques for Managers and developpers by and Peter Middelton and Jim Sutton ● http://www2.toyota.co.jp/en/vision/production_s ystem/ 42

http://www.open-do.org	160
http://www.slideshare.net	9
http://translate.googleusercontent.com	2

Lean Software Production and Qualification Infrastructures

by AdaCore on Oct 27, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 171

Statistics

Lean Software Production and Qualification Infrastructures — Presentation Transcript