Better, faster, cheaper. Lean and agile approaches to IT development

Marc McNeill, ThoughtWorks

IT doesn’t have a track record

“Two thirds of (IT) projects are challenged.”. Source: The Standish Group CHAOS Reports

Over budget by 189% . Source: The Standish Group CHAOS Reports

Over schedule by 220% . Source: The Standish Group CHAOS Reports

Only 61% of features delivered. Source: The Standish Group CHAOS Reports

66% of projects fail or are ‘marginal’. Source: The Standish Group CHAOS Reports

$3.45 billion tax-credit overpayment UK Inland Revenue $2.6 billion spent system cancelled US Federal Aviation Administration $160 million lost because of ERP system problems Hewlett-Packard $527 million written off supply chain system abandoned J Sainsbury plc

Poor Quality Slow to Deliver Expensive

Better Faster Cheaper

What if…

Lean

Agile

but first...

Context just a little

A Company with a purpose

To be the home for the best knowledge workers in the world

To revolutionise the way software is delivered to make a positive difference to business and to society

Consultancy Global Delivery Training Products 1000+ employees 6 countries 15 years

A story

The business has an idea Increase revenue Decrease costs Regulatory pressures Cartoons courtesy of http://designcomics.org/

The idea is analysed High level design Detailed design

This takes time

The business sign off on the requirements

(Lots of requirements)

Developers start writing code

It gets tested

Then deployed

But it doesn’t always work like this

Long feedback cycles

The business changes

Dates slip, scope gets cut, effort wasted

Value is not delivered

High project failure rate

High cost of change

Permanent state of maintenance

Little innovation

Unhappy stakeholders

Poor customer experience

A different story?

01. Agile

http://agilemanifesto.org/

02. Lean

Scientific management “ It is only through enforced standardization of methods, enforced adoption of the best implements and working conditions and enforced cooperation that this faster work can be assured. And the duty of enforcing the adoption of standards and enforcing this cooperation rests with management alone.” Frederick Winslow Taylor Any colour so long as it is black

Lean Just in time “ All we are doing is looking at the time line, from the point the customer gives us an order to the point when we collect the cash. And we are reducing the time line by reducing the non-value added wastes.” Taiichi Ohno

Just in time Build in quality People and teamwork Reduce waste Regular cadence Stabilise and standardise Make problems visible Better quality Lower cost Shorter lead times Higher morale Continuous improvement

The story revisited

(v) Value-Added Time (e) Elapsed Time (c) Cumulative Elapsed Time Adapted from http://www.thoughtworks.com/pdfs/lean-qtb-jun08.pdf Requirements Gathering & Analysis (v) 1 Week (e) 4 Weeks (c) 4 Weeks Review & Approve Req. Spec. (v) 1 Day (e) 2 Weeks (c) 6 Weeks Solution Design (v) 2 Weeks (e) 6 Weeks (c) 12 Weeks Review & Approve Tech. Spec. (v) 1 Day (e) 2 Weeks (c) 14 Weeks Build Solution (Coding) (v) 8 Weeks (e) 14 Weeks (c) 28 Weeks System testing (v) 3 Weeks (e) 3 Weeks (c) 31 Weeks User Acceptance Testing (UAT) (v) 3 Weeks (e) 3 Weeks (c) 34 Weeks Deploy (v) 1 Week (e) 2 Weeks (c) 36 Weeks

Requirements Gathering & Analysis Review & Approve Req. Spec. Solution Design Review & Approve Tech. Spec. (v) 1 Week (e) 4 Weeks (c) 4 Weeks (v) 1 Day (e) 2 Weeks (c) 6 Weeks (v) 2 Weeks (e) 6 Weeks (c) 12 Weeks (v) 1 Day (e) 2 Weeks (c) 14 Weeks Deploy User Acceptance Testing (UAT) System testing Build Solution (Coding) (v) 1 Week (e) 2 Weeks (c) 36 Weeks (v) 3 Weeks (e) 3 Weeks (c) 34 Weeks (v) 3 Weeks (e) 3 Weeks (c) 31 Weeks (v) 8 Weeks (e) 14 Weeks (c) 28 Weeks

Total Elapsed Time = 36 Weeks

Estimated Project Cost (at $50k per week) = $1.8M

Cycle Time Efficiency = 53%

Systemic Faults:

Inefficient

Slow, expensive & poor quality

Inflexible

System often fails

Requirements Gathering & Analysis Review & Approve Req. Spec. Solution Design Review & Approve Tech. Spec. (v) 1 Week (e) 4 Weeks (c) 4 Weeks (v) 1 Day (e) 2 Weeks (c) 6 Weeks (v) 2 Weeks (e) 6 Weeks (c) 12 Weeks (v) 1 Day (e) 2 Weeks (c) 14 Weeks Deploy User Acceptance Testing (UAT) System testing Build Solution (Coding) (v) 1 Week (e) 2 Weeks (c) 36 Weeks (v) 3 Weeks (e) 3 Weeks (c) 34 Weeks (v) 3 Weeks (e) 3 Weeks (c) 31 Weeks (v) 8 Weeks (e) 14 Weeks (c) 28 Weeks Extra Features Defects Defects Waiting (v) Value-Added Time (e) Elapsed Time (c) Cumulative Elapsed Time Movement Waiting Excess Inventory Movement Waiting Excess Inventory

The lean and agile way

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Collaborative workshops

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Visioning

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Shared understanding

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Lo-fi prototyping

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Collaborative prioritisation

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Group estimation

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Lightweight documentation

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance Build in quality People and teamwork Reduce waste Regular cadence Stabilise and standardise Make problems visible Better quality Lower cost Shorter lead times Higher morale Continuous improvement Just in time

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance Just in time Build in quality People and teamwork Reduce waste Regular cadence Stabilise and standardise Make problems visible Better quality Lower cost Shorter lead times Higher morale Continuous improvement User stories

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance Just in time Build in quality People and teamwork Reduce waste Regular cadence Stabilise and standardise Make problems visible Better quality Lower cost Shorter lead times Higher morale Continuous improvement Adaptive planning

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance Just in time Build in quality People and teamwork Reduce waste Regular cadence Stabilise and standardise Make problems visible Better quality Lower cost Shorter lead times Higher morale Continuous improvement Spike to solve hard technical problems through real experiments

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance Build in quality People and teamwork Reduce waste Regular cadence Stabilise and standardise Make problems visible Better quality Lower cost Shorter lead times Higher morale Continuous improvement Test driven development Just in time Build in quality

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance Build in quality People and teamwork Reduce waste Regular cadence Stabilise and standardise Make problems visible Better quality Lower cost Shorter lead times Higher morale Continuous improvement Continuous integration Just in time Build in quality

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance Build in quality People and teamwork Reduce waste Regular cadence Stabilise and standardise Make problems visible Better quality Lower cost Shorter lead times Higher morale Continuous improvement Paired programming Just in time Build in quality

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance Build in quality People and teamwork Reduce waste Regular cadence Stabilise and standardise Make problems visible Better quality Lower cost Shorter lead times Higher morale Continuous improvement The right documentation Just in time

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance Build in quality People and teamwork Reduce waste Regular cadence Stabilise and standardise Make problems visible Better quality Lower cost Shorter lead times Higher morale Continuous improvement Continuous feedback - showcases Just in time

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance Build in quality People and teamwork Regular cadence Stabilise and standardise Make problems visible Better quality Lower cost Shorter lead times Higher morale Continuous improvement Collective ownership Just in time Reduce waste

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance Build in quality People and teamwork Regular cadence Stabilise and standardise Make problems visible Better quality Lower cost Shorter lead times Higher morale Continuous improvement Stand-ups Just in time Reduce waste

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance Build in quality Regular cadence Stabilise and standardise Make problems visible Better quality Lower cost Shorter lead times Higher morale Continuous improvement Time boxed increments Just in time Reduce waste People and teamwork

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance Build in quality Regular cadence Stabilise and standardise Make problems visible Better quality Lower cost Shorter lead times Higher morale Continuous improvement Sustainable pace Just in time Reduce waste People and teamwork

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance Build in quality Regular cadence Stabilise and standardise Make problems visible Better quality Lower cost Shorter lead times Higher morale Continuous improvement Patterns, standards and ubiquitous language Just in time Reduce waste People and teamwork

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance Build in quality Regular cadence Stabilise and standardise Make problems visible Better quality Lower cost Shorter lead times Higher morale Continuous improvement Information radiators Just in time Reduce waste People and teamwork

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance (v) 8 Weeks (e) 10 Weeks (c) 14 Weeks (v) Value-Added Time (e) Elapsed Time (c) Cumulative Elapsed Time

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance (v) 8 Weeks (e) 10 Weeks (c) 14 Weeks (v) Value-Added Time (e) Elapsed Time (c) Cumulative Elapsed Time Deploy User Acceptance Testing (UAT) System testing (v) 1 Week (e) 2 Weeks (c) 18 Weeks (v) 1 Week (e) 1 Week (c) 16 Weeks (v) 1 Week (e) 1 Week (c15 Weeks

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance (v) 8 Weeks (e) 10 Weeks (c) 14 Weeks Deploy User Acceptance Testing (UAT) System testing (v) 1 Week (e) 2 Weeks (c) 18 Weeks (v) 1 Week (e) 1 Week (c) 16 Weeks (v) 1 Week (e) 1 Week (c15 Weeks (v) Value-Added Time (e) Elapsed Time (c) Cumulative Elapsed Time

Requirements Gathering & Analysis (v) 3 Week (e) 4 Weeks (c) 4 Weeks Testing Design & Build User Acceptance (v) 8 Weeks (e) 10 Weeks (c) 14 Weeks Deploy User Acceptance Testing (UAT) System testing (v) 1 Week (e) 2 Weeks (c) 18 Weeks (v) 1 Week (e) 1 Week (c) 16 Weeks (v) 1 Week (e) 1 Week (c15 Weeks