This document discusses the importance of understanding product development flow using metrics. It argues that organizations should not be dogmatic about which metrics they use and should trust people over data unless data is being used for a good reason. The document suggests we may be experiencing a paradigm shift in how metrics are used, from a focus on performance metrics to using scientific methods to discover what actually affects outcomes. It maintains that organizations that can discover and understand their actual product development flow will have a competitive advantage.
8. Time
Performance
“There is nothing new to
be discovered in physics
now. All that remains is
more and more precise
measurement.”
Lord Kelvin
1900
“Hmm….”
Einstein, publisher of
a paper on relativity
theory a few years
later.
Are we experiencing a
paradigm shift in how
metrics are being
used?
9. Current paradigm:
Working software is the primary measure of progress*
Next paradigm:
Do not stay dogmatic about which metrics that are being
used
* Principles behind the Agile Manifesto: http://agilemanifesto.org/principles.html
10. Current paradigm:
“In God we trust, all other bring data”*
Next paradigm:
In people we trust, only bring data you care about for a very
good reason
*Famous quote, often attributed to Deming
11. Current paradigm:
Performance is reviewed with metrics
Next paradigm:
The Death of Reporting: Use scientific methods to discover
and teach to stakeholders what actually affects outcome
Tip: Arne Roock - Learning from Fake Charts: http://vimeo.com/80365303
12. Automatic Creation
and Maintenance
Powerful data automation
and visualization solutions
Easier and Cheaper Technology
High Cost Low Cost
Whiteboards, stickers etc
Manual Creation
and Maintenance
Data warehousing,
traditional reporting
solutions
Improved Agile/Lean Practices
13. The usual suspects in current paradigm when
asking about metrics (outside of code stuff)
Time Time
Work remaining
# New Bugs
19. Take an Economic Perspective
Revenue /
Delivered story points
Time
Cost of
delay
_____________
_____________
_____________
_____________
_____________
_____________
_____________
_____________
_____________
$$$
$
Time
Size
S
XXL
Grooming process +
WSJF
Yesterday’s
Weather
Outcome
20. Measure validity of advice
(example DEEP)
Average Size
No
Prio
Low
Prio
Medium
Prio
High
Prio
21. Do not stay
dogmatic
Truly informed
decision making
The death of
Reporting
Organizations that discover their actual flow will
be kings and queens in the next paradigm.
22. Inspiration in addition to our Customers:
Donald G. Reinertsen
The Principles of Product
Development Flow
2012
Douglas W. Hubbard
How to Measure Anything:
Finding the Value of
Intangibles in Business
2014
Thomas S. Kuhn
The Structure Of Scientific
Revolutions
1970
Michael Lewis
Moneyball
2011
Editor's Notes
Difficult to talk about metrics without mentioning the lean startup
Erick Ries - Lean Startup
Actionable metrics
Possibly first covered in 1620 by Francis Bacon?
Knowledge also moves in paradigms
Famous example from Physics
There is something going on, on a paradigm level, on how we look at these numbers.
Thought leaders:
Douglas Hubbard (How to Measure Anything) - Science was never about having data, it was about getting data. Model your measures around decisions you must make.
Examples from the real world:
Story: “the only thing we know, is that the things we have been looking at does not help us any more.”
Own experience:
Many teams struggle with things like burn-downs etc (sometimes for the wrong reasons) but many find it much more useful with Cumulative Flow Diagrams and similar.
Thought leaders:
Datensparsamkeit, Martin Fowler has a good blog post on subject: http://martinfowler.com/bliki/Datensparsamkeit.html
Examples from the real world:
Story: Moneyball
Own experience:
The first thing many agile teams do is to throw out old reports, KPI’s and stuff out the door.
Thought leaders:
Deming: Wrong practice: Management by results. (Action based on outcome is not action on the causes of the outcome. Emphasis on cost reduction is an example. Costs are not causes.)
Better practice: Use the concepts and tools related to variation to understand and improve the system. Deming estimates that around 94% of the possible improvements belong to the system - the responsibility of management.
Examples from the real world:
80+ data scientists working at King
Happiness indexes