Improving Software
Development at Scale:
Promise and Pitfalls
Dr Andy Carmichael
Head of Agile Services, Clearvision-CM.com
@andycarmich - #LLKD14
Software development at scale
is particularly hard
Scale changes the problem
Software is hard
Outline of this session…
Improving Software
Development at Scale:
Promise and Pitfalls
Improvement
• Improvements suffer the
J-Curve syndrome
• Small step improvements
(Kaizen) may alleviate deep dips
in performance
• Radical change might also be
needed (Kaikaku) but may not
“stick”
• Kanban is an IMPROVEMENT
method based on the
Lean flow paradigm
Improving Software
Development at Scale:
Promise and Pitfalls
What is Kanban?
An approach for managing and improving the
flow of value from knowledge work?
Process
WORK FLOWS
(not all work does flow – but concentrate on the “work that flows”)
A short definition of Kanban needed
in order to be scale-free
1. See work as flow
2. Start from here and evolve
3. Make work and policies visible;
Make validated improvements
See also: “The Shortest Possible Definition of Kanban and why it matters for scaling”
#KLRAT13 and #LKUK13 – Slideshare: http://slidesha.re/1mbvNsb
Lean Flow
Paradigm
Foundational
Principles
Core
Practices
The Twitter Version
Essence of Kanban
see flow
start here
with visible work & policies
validate improvements
Also see: "How to Adopt Kanban" @andycarmich
Improving Software
Development at Scale:
Promise and Pitfalls
2 scaling mechanisms
• Scaling by “not scaling”
o use service-orientation concept to build a network of
independently operating but interdependent services
o balance work flowing between different kanban systems
• Scale through “scale-free” understanding
o same approach applied to different units of flow at different time
scales
3 (or 4) Scales of Kanban
• Personal / small team
• unit of flow: Task
• time scale: hours
• Service delivery / workflow
• unit of flow: Work item e.g. User Story
• time scale: days
• Product
• unit of flow: Project, Epic, MVP or MMF
• time scale: months
• Portfolio
• unit of flow: “Product Holding”
• time scale: months/year(s)
Decisionmakingat
differentlevelshave
differingscopeand
purpose
Decision Flows between Scales
• Personal / small team
• Service delivery / workflow
• Product
• Portfolio
Decisionmakingat
differentlevelshave
differingscopeand
purpose
 Personal forecasts and blockages – Daily Stand-up
 Team goals and priorities
 Cost-Schedule forecasts and blockages – Operations Meeting
 Product goals and priorities
 Cost-Benefit-Schedule forecasts (“P/E ratios”)
 Investment goals and priorities
Implementation
options
Product
options
Investment
options
Outline of this session…
Improving Software
Development at Scale:
Promise and Pitfalls
Pitfalls: Mild insults & non-
communication
• keep the „chickens‟ silent while the „pigs‟ speak
Subtext: management is not committed
• keep the „geeks‟ away from the „suits‟
Subtext: the technical concerns are for lower-level discussions
Pitfall 1: Adopting Agile
Frameworks without the Values or
Enabling Practices
• Frequent integration and/or delivery without
automated testing
• “Agile planning”… fixed scope and schedule
• Water-scrum-fall
• Hierarchical management/communication
Pitfall 2: Ignoring
dis-economies of scale
• Inside every large project there are small
ones trying to get out
• Inside impossibly-massive projects (just say
no!) there may be feasibly-large ones
@MartinBurnsSV
Architecture
• Components and acyclic
dependency graph
• Policies to facilitate non-
functional requirements
compliance
• Processes to improve time to
quality
• Architecture is not “arm-waving”
• It‟s not “non-technical”
• It‟s not disconnected from
organisation structures
(Conway‟s Law)
Pitfall 3: Thinking architecture is
primarily about function…
or that it‟s optional
Pitfall 4: Thinking dependencies in
plans are there to be managed
Most must be eliminated!
Pitfall 5: Thinking agility is a
quality without a cost
Is it valued by the organisation?
Is predictability valued more?
Pitfalls 6 & 7: Interventions…
• We have done those things that we ought not
to have done (Instruction Giving)
• We have left undone those interventions we
ought to have done (System Thinking)
RESPECT
CONTINUOUS
IMPROVEMENT
Improving Software
Development at Scale:
Promise and Pitfalls
Dr Andy Carmichael
Head of Agile Services, Clearvision-CM.com
@andycarmich - #LLKD14
http://xprocess.blogspot.co.uk/
http://www.slideshare.net/andycarmichaeluk/

Improving software development at scale llkd14

  • 1.
    Improving Software Development atScale: Promise and Pitfalls Dr Andy Carmichael Head of Agile Services, Clearvision-CM.com @andycarmich - #LLKD14
  • 2.
    Software development atscale is particularly hard Scale changes the problem Software is hard
  • 3.
    Outline of thissession… Improving Software Development at Scale: Promise and Pitfalls
  • 4.
    Improvement • Improvements sufferthe J-Curve syndrome • Small step improvements (Kaizen) may alleviate deep dips in performance • Radical change might also be needed (Kaikaku) but may not “stick” • Kanban is an IMPROVEMENT method based on the Lean flow paradigm
  • 5.
    Improving Software Development atScale: Promise and Pitfalls
  • 6.
    What is Kanban? Anapproach for managing and improving the flow of value from knowledge work? Process
  • 7.
    WORK FLOWS (not allwork does flow – but concentrate on the “work that flows”)
  • 8.
    A short definitionof Kanban needed in order to be scale-free 1. See work as flow 2. Start from here and evolve 3. Make work and policies visible; Make validated improvements See also: “The Shortest Possible Definition of Kanban and why it matters for scaling” #KLRAT13 and #LKUK13 – Slideshare: http://slidesha.re/1mbvNsb Lean Flow Paradigm Foundational Principles Core Practices
  • 9.
    The Twitter Version Essenceof Kanban see flow start here with visible work & policies validate improvements Also see: "How to Adopt Kanban" @andycarmich
  • 10.
    Improving Software Development atScale: Promise and Pitfalls
  • 11.
    2 scaling mechanisms •Scaling by “not scaling” o use service-orientation concept to build a network of independently operating but interdependent services o balance work flowing between different kanban systems • Scale through “scale-free” understanding o same approach applied to different units of flow at different time scales
  • 12.
    3 (or 4)Scales of Kanban • Personal / small team • unit of flow: Task • time scale: hours • Service delivery / workflow • unit of flow: Work item e.g. User Story • time scale: days • Product • unit of flow: Project, Epic, MVP or MMF • time scale: months • Portfolio • unit of flow: “Product Holding” • time scale: months/year(s) Decisionmakingat differentlevelshave differingscopeand purpose
  • 13.
    Decision Flows betweenScales • Personal / small team • Service delivery / workflow • Product • Portfolio Decisionmakingat differentlevelshave differingscopeand purpose  Personal forecasts and blockages – Daily Stand-up  Team goals and priorities  Cost-Schedule forecasts and blockages – Operations Meeting  Product goals and priorities  Cost-Benefit-Schedule forecasts (“P/E ratios”)  Investment goals and priorities Implementation options Product options Investment options
  • 14.
    Outline of thissession… Improving Software Development at Scale: Promise and Pitfalls
  • 15.
    Pitfalls: Mild insults& non- communication • keep the „chickens‟ silent while the „pigs‟ speak Subtext: management is not committed • keep the „geeks‟ away from the „suits‟ Subtext: the technical concerns are for lower-level discussions
  • 16.
    Pitfall 1: AdoptingAgile Frameworks without the Values or Enabling Practices • Frequent integration and/or delivery without automated testing • “Agile planning”… fixed scope and schedule • Water-scrum-fall • Hierarchical management/communication
  • 17.
    Pitfall 2: Ignoring dis-economiesof scale • Inside every large project there are small ones trying to get out • Inside impossibly-massive projects (just say no!) there may be feasibly-large ones
  • 18.
    @MartinBurnsSV Architecture • Components andacyclic dependency graph • Policies to facilitate non- functional requirements compliance • Processes to improve time to quality • Architecture is not “arm-waving” • It‟s not “non-technical” • It‟s not disconnected from organisation structures (Conway‟s Law) Pitfall 3: Thinking architecture is primarily about function… or that it‟s optional
  • 19.
    Pitfall 4: Thinkingdependencies in plans are there to be managed Most must be eliminated!
  • 20.
    Pitfall 5: Thinkingagility is a quality without a cost Is it valued by the organisation? Is predictability valued more?
  • 21.
    Pitfalls 6 &7: Interventions… • We have done those things that we ought not to have done (Instruction Giving) • We have left undone those interventions we ought to have done (System Thinking) RESPECT CONTINUOUS IMPROVEMENT
  • 22.
    Improving Software Development atScale: Promise and Pitfalls Dr Andy Carmichael Head of Agile Services, Clearvision-CM.com @andycarmich - #LLKD14 http://xprocess.blogspot.co.uk/ http://www.slideshare.net/andycarmichaeluk/

Editor's Notes

  • #2 Monday 28th April, 2014 4:30pm to 5:10pm (GMT)Software (as frequently observed) is hard. And software development at scale is particularly hard. Evidence suggests a strong inverse relationship between the likelihood of a software project delivering its planned benefits (within budgeted costs) and the project's size. While this is nothing new, we should ask why has there been so little improvement over the years.Agile methods undoubtedly contributed much over their first two decades to the effectiveness of software teams - particularly "coffee-pot-sized" teams developing new products. Agile methods were primarily designed with this sized team in mind, and agile process frameworks are still defined almost entirely with reference to this scale. In their third decade however, the question of how these methods scale can no longer be avoided. This presentation, rather than focusing on the new frameworks that are now emerging, reviews anecdotal evidence as well as theoretical ideas on what improves (or degrades) performance of large initiatives… in particular the management behaviours that have proved helpful or counter-productive in real projects.Large scale does not invalidate strategies that work at small scale, however it does introduce management problems that are new - problems that are not overcome by simply "keeping the geeks away from the suits" (or keeping the "chickens" silent while the "pigs" speak)!
  • #19 You can build a shed without an architectA cathedral (or an office building) needs at least oneArchitectureComponents and acyclic dependency graphPolicies to facilitate non-functional requirements complianceProcesses to improve time to qualityArchitecture is a technical discipline – not proverbial arm-waving
  • #23 Monday 28th April, 2014 4:30pm to 5:10pm (GMT)Software (as frequently observed) is hard. And software development at scale is particularly hard. Evidence suggests a strong inverse relationship between the likelihood of a software project delivering its planned benefits (within budgeted costs) and the project's size. While this is nothing new, we should ask why has there been so little improvement over the years.Agile methods undoubtedly contributed much over their first two decades to the effectiveness of software teams - particularly "coffee-pot-sized" teams developing new products. Agile methods were primarily designed with this sized team in mind, and agile process frameworks are still defined almost entirely with reference to this scale. In their third decade however, the question of how these methods scale can no longer be avoided. This presentation, rather than focusing on the new frameworks that are now emerging, reviews anecdotal evidence as well as theoretical ideas on what improves (or degrades) performance of large initiatives… in particular the management behaviours that have proved helpful or counter-productive in real projects.Large scale does not invalidate strategies that work at small scale, however it does introduce management problems that are new - problems that are not overcome by simply "keeping the geeks away from the suits" (or keeping the "chickens" silent while the "pigs" speak)!