Practical Experience of Applying Agile Techniques to the Development of a Safety-Critical System 
David Nicoll 
Project Realization
When do we get value from software? 
When we get the software into…
Business Benefit 
Typical Waterfall Development 
Requirements 
Design 
Code 
Test 
Traceability between phases 
Completion of a Phase is often a contract payment milestone. 
Review 
Review 
Review 
End of Phase reviews form a stage-gate 
When do we get the ROI? 
Right at the end….. 
But will it deliver what we want ?
Reputation & Outcomes 
•Software is always late 
•It doesn’t work 
•Costs too much 
•Quality is poor 
•Changes are slow and costly
Model of Traditional Development 
Quality 
Time 
Cost 
Functionality 
Fixed 
Variable
Status of Agile 
•Is by far the fastest growing methodology today 
•Is being taken up across all sectors both public and private 
•Is becoming the de-facto standard approach 
•Originally for small development teams 
–Typical team of 7 people (± 2) 
–Is now being scaled to teams of 1000+ 
•Number of approaches for organisational levels 
–Development level (includes Scrum, XP) 
–Project & Programme level (includes DSDM, SAFe)
Basis of Agile and ROI 
•Agile is concerned with getting the fastest ROI 
•Continuous iterative development 
•Progressive incremental delivery 
–to provide Business Benefit throughout the development 
•Driven by costs and timescales 
–Functionality is removed or deferred 
•Assumes not everything is known 
–Anticipates Change will happen 
•Fast feedback supports continuous improvement 
•Collaborative working between 
–between Client and Supplier 
–Development teams
Deliveries are Fixed In Time 
Time 
Incremental 
Delivery 
#1 
Incremental 
Delivery 
#2 
Incremental 
Delivery 
#3 
Planned Delivery Dates based on Timescale NOT content
Increment #1 
Incremental Functionality 
Time 
Incremental Delivery #1 
Incremental 
Delivery 
#2 
Incremental Delivery #3 
Build incrementally on firm foundations 
Increment #1 
Increment #1 
Increment #2 
Increment #2 
Increment #3
Increment #2 
Increment #1 
Timescale Takes Precedence 
Time 
Incremental 
Delivery 
#1 
Incremental Delivery #2 
Incremental 
Delivery 
#3 
Continuous delivery – functionality deferred 
PRIORITISE 
PRIORITISE 
Increment #3 
Increment #1 
Increment #2 
Increment #3
Completeness of Increments 
Time 
Incremental Delivery #1 
Incremental 
Delivery 
#2 
Incremental Delivery #3 
Design 
Code 
Test 
Design 
Code 
Test 
Design 
Code 
Test 
Review 
Plan 
Review 
Plan 
Review 
Plan
Incremental Safety Assurance 
Time 
Increment 
#1 
Increment #2 
Increment 
#3 
Increment #2 
Increment #1 
Safety 
Audit 
#1 
Safety 
Audit 
#2 
Increment #1 
Safety 
Audit 
#1 
Increment #2 
Safety 
Audit 
#2 
Increment #1 
Safety 
Audit 
#1 
Increment #3 
Safety 
Audit 
#3
Model of Agile Development 
Quality 
Time 
Cost 
Functionality 
Fixed 
Variable
Agile Development 
No change to existing best practice 
•Full traceability (requirements, design, code, test) 
•Coding standards 
–Static analysis 
–Complexity 
–Module size 
•Unit Test – full path coverage 
•Independent reviews 
•Test Driven Development (TDD) 
•Automated overnight build and test 
•Strict configuration control and change control 
14
Management of Risk 
•Agile provides early tangible working product 
–Evidence based progress 
–Avoids the “90% complete” syndrome 
–Provides for re-prioritisation 
•Overall risk is progressively reduced throughout the development 
•Risk exposure is limited to the cost of the current increment 
•Lessons Learned from one increment are passed to the next
Risk Over Time 
Time 
Risk 
Delivery 
Deadline 
Waterfall 
Agile 
Increment #1 
Increment 
#2 
Increment 
#3
Progress Monitoring 
•Traditional Gantt Charts are only useful at a high level 
•Daily Stand-Up meetings 
–Provides an environment for communication and team building 
–Each team member provides a verbal update to the rest of the team 
•Wall boards show 
–The Workflow 
–Who is doing what 
–Where the progress blocks are 
•Burn-down charts 
–Shows how fast work is being performed (velocity) 
–Provide a forecast completion date
Progress Monitoring: Burn-Down Charts 
To Do 
Complete 
In Progress 
Estimated Finish #1 
Estimated Finish #2 
Date 
Number of Reqmts 
Estimated 
Finish #3 
Actual Finish
Agile for Safety-Critical Rail 
•Autonomous Underground Train Control system 
•Real-Time Safety-Critical System 
–CENELEC 50128 (SIL4) – Loss of Life 
•Automatic control of 
–train, signals, points 
•Radio based communication between 
–Trains 
–Timetable (including local speed restrictions) 
•Doppler radar 
–Provides speed, distance & direction 
•Axle counters 
–to determine train position in station
FBP: System Layout 
Radio 
Interlocking 
Control Centre 
ATP 
ATO 
Radio Links 
Communications 
Signals & Points 
Radio 
BP 
BP 
Fixed Block Processor 
Interlocking 
ATP 
ATO 
Train driving
Initial Development Process 
Detailed 
Design 
Code 
Review & 
Safety Audit 
Review & Safety Audit 
Requirements 
Definition 
High-Level 
Design 
Unit Test 
Integration 
Test 
Acceptance 
Test 
System 
Test 
Review & 
Safety Audit 
Review & 
Safety Audit 
Review & 
Safety Audit 
Review & 
Safety Audit 
Review & 
Safety Audit
Development Process 
Detailed 
Design 
Code 
Requirements 
Definition 
High-Level 
Design 
Unit Test 
Integration 
Test 
Acceptance 
Test 
System 
Test 
Code 
Detailed 
Design 
Unit Test 
Detailed Design 
Code 
SPARK 
Unit Test 
Increments 
SPARK 
Detailed Design 
Detailed Design 
Code 
SPARK 
Unit Test 
Detailed Design 
SPARK 
Unit Test 
Unit Test 
Code 
Code 
Detailed Design 
Code 
Unit Test 
SPARK
Incremental Development 
Time 
Review Previous Increment 
Plan this Increment 
Overall 
Requirements 
Backlog 
Design 
Code 
Test 
SPARK 
Increment 
Requirements 
Backlog 
Process Improvements
Incremental Development 
Design 
Code 
Test 
SPARK 
•Full Traceability 
–Requirements 
–Design 
–Code 
–Test 
•Independent Unit test 
–100% path coverage 
–MC/DC testing 
–Boundary values 
•Independent Formal Reviews 
•Incremental Hazard Analysis
25 
Conventional Safety Analysis Process 
System Hazards & Safety Constraints 
New Hazards 
New Hazards 
Vertical Slice Analysis 
Known Hazards 
Known Hazards 
Design Verification Safety Analysis 
Code Verification Safety Analysis 
Safety Requirements Verification Analysis 
Safety Audit Report 
Phase Specific Safety Reports 
Safety Analysis 
Safety Analysis 
Design 
Code 
Test
Safety 
Analysis 
FBP: Increment based Safety Analysis 
System Hazards & Safety Constraints 
Vertical Slice Analysis 
Increment 
Unit Test Safety Analysis 
Code Verification Safety Analysis 
Design Verification Safety Analysis 
Safety Audit Report (Increment) 
Design 
Code 
Test 
New Hazards 
Safety 
Analysis 
New Hazards 
Phase Specific Safety Reports 
Safety 
Analysis 
New Hazards
FBP Burn-Down: Testing 
0 
100 
200 
300 
400 
500 
600 
700 
800 
900 
No. of Module Tests 
Weeks 
FBP: Module Testing (Formal) 
Complete 
Progressing 
To Go 
Total Tests
28 
FBP: Retrospective Project Analysis 
Development Team Size = 70+ (35 UK, 35 India, 4 Spain) 
Primary project objective = Timescale 
Crude industry standard = 22 ↔ 24 months duration 
Actual development = 18 months 
Cost (Effort) 
Time 
FBP
Approaching Agile 
•Fundamentally Agile is a mind-set 
•It is about managing project risk in order to deliver business benefit 
•Agile is not proscriptive 
–Best practice in all activities 
–No conflict with current industry practice 
•Agile advocates a number of methods, techniques and approaches that deliver business benefit 
•It is up to you to tailor these to your need
Any Questions? 
David Nicoll 
www.project-realization.com

Practical Application of Agile Techniques in Developing Safety Related Systems

  • 1.
    Practical Experience ofApplying Agile Techniques to the Development of a Safety-Critical System David Nicoll Project Realization
  • 2.
    When do weget value from software? When we get the software into…
  • 3.
    Business Benefit TypicalWaterfall Development Requirements Design Code Test Traceability between phases Completion of a Phase is often a contract payment milestone. Review Review Review End of Phase reviews form a stage-gate When do we get the ROI? Right at the end….. But will it deliver what we want ?
  • 4.
    Reputation & Outcomes •Software is always late •It doesn’t work •Costs too much •Quality is poor •Changes are slow and costly
  • 5.
    Model of TraditionalDevelopment Quality Time Cost Functionality Fixed Variable
  • 6.
    Status of Agile •Is by far the fastest growing methodology today •Is being taken up across all sectors both public and private •Is becoming the de-facto standard approach •Originally for small development teams –Typical team of 7 people (± 2) –Is now being scaled to teams of 1000+ •Number of approaches for organisational levels –Development level (includes Scrum, XP) –Project & Programme level (includes DSDM, SAFe)
  • 7.
    Basis of Agileand ROI •Agile is concerned with getting the fastest ROI •Continuous iterative development •Progressive incremental delivery –to provide Business Benefit throughout the development •Driven by costs and timescales –Functionality is removed or deferred •Assumes not everything is known –Anticipates Change will happen •Fast feedback supports continuous improvement •Collaborative working between –between Client and Supplier –Development teams
  • 8.
    Deliveries are FixedIn Time Time Incremental Delivery #1 Incremental Delivery #2 Incremental Delivery #3 Planned Delivery Dates based on Timescale NOT content
  • 9.
    Increment #1 IncrementalFunctionality Time Incremental Delivery #1 Incremental Delivery #2 Incremental Delivery #3 Build incrementally on firm foundations Increment #1 Increment #1 Increment #2 Increment #2 Increment #3
  • 10.
    Increment #2 Increment#1 Timescale Takes Precedence Time Incremental Delivery #1 Incremental Delivery #2 Incremental Delivery #3 Continuous delivery – functionality deferred PRIORITISE PRIORITISE Increment #3 Increment #1 Increment #2 Increment #3
  • 11.
    Completeness of Increments Time Incremental Delivery #1 Incremental Delivery #2 Incremental Delivery #3 Design Code Test Design Code Test Design Code Test Review Plan Review Plan Review Plan
  • 12.
    Incremental Safety Assurance Time Increment #1 Increment #2 Increment #3 Increment #2 Increment #1 Safety Audit #1 Safety Audit #2 Increment #1 Safety Audit #1 Increment #2 Safety Audit #2 Increment #1 Safety Audit #1 Increment #3 Safety Audit #3
  • 13.
    Model of AgileDevelopment Quality Time Cost Functionality Fixed Variable
  • 14.
    Agile Development Nochange to existing best practice •Full traceability (requirements, design, code, test) •Coding standards –Static analysis –Complexity –Module size •Unit Test – full path coverage •Independent reviews •Test Driven Development (TDD) •Automated overnight build and test •Strict configuration control and change control 14
  • 15.
    Management of Risk •Agile provides early tangible working product –Evidence based progress –Avoids the “90% complete” syndrome –Provides for re-prioritisation •Overall risk is progressively reduced throughout the development •Risk exposure is limited to the cost of the current increment •Lessons Learned from one increment are passed to the next
  • 16.
    Risk Over Time Time Risk Delivery Deadline Waterfall Agile Increment #1 Increment #2 Increment #3
  • 17.
    Progress Monitoring •TraditionalGantt Charts are only useful at a high level •Daily Stand-Up meetings –Provides an environment for communication and team building –Each team member provides a verbal update to the rest of the team •Wall boards show –The Workflow –Who is doing what –Where the progress blocks are •Burn-down charts –Shows how fast work is being performed (velocity) –Provide a forecast completion date
  • 18.
    Progress Monitoring: Burn-DownCharts To Do Complete In Progress Estimated Finish #1 Estimated Finish #2 Date Number of Reqmts Estimated Finish #3 Actual Finish
  • 19.
    Agile for Safety-CriticalRail •Autonomous Underground Train Control system •Real-Time Safety-Critical System –CENELEC 50128 (SIL4) – Loss of Life •Automatic control of –train, signals, points •Radio based communication between –Trains –Timetable (including local speed restrictions) •Doppler radar –Provides speed, distance & direction •Axle counters –to determine train position in station
  • 20.
    FBP: System Layout Radio Interlocking Control Centre ATP ATO Radio Links Communications Signals & Points Radio BP BP Fixed Block Processor Interlocking ATP ATO Train driving
  • 21.
    Initial Development Process Detailed Design Code Review & Safety Audit Review & Safety Audit Requirements Definition High-Level Design Unit Test Integration Test Acceptance Test System Test Review & Safety Audit Review & Safety Audit Review & Safety Audit Review & Safety Audit Review & Safety Audit
  • 22.
    Development Process Detailed Design Code Requirements Definition High-Level Design Unit Test Integration Test Acceptance Test System Test Code Detailed Design Unit Test Detailed Design Code SPARK Unit Test Increments SPARK Detailed Design Detailed Design Code SPARK Unit Test Detailed Design SPARK Unit Test Unit Test Code Code Detailed Design Code Unit Test SPARK
  • 23.
    Incremental Development Time Review Previous Increment Plan this Increment Overall Requirements Backlog Design Code Test SPARK Increment Requirements Backlog Process Improvements
  • 24.
    Incremental Development Design Code Test SPARK •Full Traceability –Requirements –Design –Code –Test •Independent Unit test –100% path coverage –MC/DC testing –Boundary values •Independent Formal Reviews •Incremental Hazard Analysis
  • 25.
    25 Conventional SafetyAnalysis Process System Hazards & Safety Constraints New Hazards New Hazards Vertical Slice Analysis Known Hazards Known Hazards Design Verification Safety Analysis Code Verification Safety Analysis Safety Requirements Verification Analysis Safety Audit Report Phase Specific Safety Reports Safety Analysis Safety Analysis Design Code Test
  • 26.
    Safety Analysis FBP:Increment based Safety Analysis System Hazards & Safety Constraints Vertical Slice Analysis Increment Unit Test Safety Analysis Code Verification Safety Analysis Design Verification Safety Analysis Safety Audit Report (Increment) Design Code Test New Hazards Safety Analysis New Hazards Phase Specific Safety Reports Safety Analysis New Hazards
  • 27.
    FBP Burn-Down: Testing 0 100 200 300 400 500 600 700 800 900 No. of Module Tests Weeks FBP: Module Testing (Formal) Complete Progressing To Go Total Tests
  • 28.
    28 FBP: RetrospectiveProject Analysis Development Team Size = 70+ (35 UK, 35 India, 4 Spain) Primary project objective = Timescale Crude industry standard = 22 ↔ 24 months duration Actual development = 18 months Cost (Effort) Time FBP
  • 29.
    Approaching Agile •FundamentallyAgile is a mind-set •It is about managing project risk in order to deliver business benefit •Agile is not proscriptive –Best practice in all activities –No conflict with current industry practice •Agile advocates a number of methods, techniques and approaches that deliver business benefit •It is up to you to tailor these to your need
  • 30.
    Any Questions? DavidNicoll www.project-realization.com