Measuring software development is difficult. Measuring the return on investment (ROI) of complex initiatives (such as adopting agile) is even more difficult. One department at IBM struggled to reduce a list of 150 metrics down to a top twenty to use in consulting engagements. Through the years, nearly every one of Anthony Crain’s clients has asked him for help in proving that their teams were “getting better” at development. Nearly all of their measures were “adoption” measures showing how teams were doing agile. However, none of the metrics was focused on whether the development was getting better. Today, Anthony has it narrowed down to four measures that can be used for everything—Quality, Productivity, Predictability, and Engagement. Code named QPPE (pronounced kewpee), these measures work in every situation. When your teams agree to this single measurement framework, they can start running experiments to improve those measures. Anthony shares the definitions of these metrics and gives real examples—at the team, release, and portfolio level.

1. BW2
Concurrent Session
11/8/17 11:30 AM
Measure Anything: The Quality,
Productivity, Predictability, and
Engagement Model
Presented by:
Anthony Crain
Blue Agility, LLC
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073
888---268---8770 ·· 904---278---0524 - info@techwell.com - https://www.techwell.com/

2. Anthony Crain
Blue Agility, LLC
Since 1999, Anthony Crain has led enterprise organizational change initiatives for
IBM and now for Blue Agility. He is known for his outstanding mentoring and
teaching abilities, clearly explaining the practical side of theoretical concepts in
an exciting and actionable manner. A dynamic speaker who energizes his
audience, Anthony has introduced thousands of people to agile, portfolio
management, requirements, architecture, and numerous other engineering
topics. He has led transformations in diverse industries—commercial banking,
software development, automotive, healthcare, financial, government, retail,
automated controls, manufacturing, power, telecom, home mortgage, and more.

3. 11/2/17
1
BLUEAGILITYEmpower the Enterprise
How to Measure Anything
QPPE Metrics Model
Anthony Crain
acrain@blue-agility.com
Anthony R. Crain
Agenda
Ø What is your current state?
• What are the QPPE metrics?
• How do you measure the QPPE metrics?
• Case Study: QPPE PorAolio Level Rollups
• Conclusion

4. 11/2/17
2
Anthony R. Crain
What is your current state?
• What do you measure today?
• How many measures?
• Can most people recite them from memory?
• Is there consensus on what to measure?
• Do you invent new measures for each situaIon?
• Would things go faster if you had an agreed upon
model for all experiments and iniIaIves?
Anthony R. Crain
Agenda
ü What is your current state?
Ø What are the QPPE metrics?
• How do you measure the QPPE metrics?
• Case Study: QPPE PorAolio Level Rollups
• Conclusion

5. 11/2/17
3
Anthony R. Crain
The QPPE (“kewpee”) Model
• Get your organizaIon to agree to a
measurement framework for all things!
• Quality
• ProducIvity
• Predictability
• Engagement
• Are measures missing from model?
• Use QPPE metrics for everything!
• Every iniIaIve. Is agile beSer than waterfall? Offshore vs onshore?, etc
• Every level: Team, Program, PorAolio
• Back of the napkin experiments: fast esImaIon, short form story wriIng, etc
Anthony R. Crain
The QPPE Metrics are in a Specific Order
• ProducIvity
• What most organizaIons zealously focus on!
• Tend to be easiest to measure
• But when focused on… what tends to suffer?
• Quality
• Most important metric
• If we improve any others, but quality drops,
usually not a win!
• Predictability
• What’s more important: producIvity or predictability?
• Engagement
• If you run an experiment to improve QPP and the needles don’t move…
• Is it because the soluIon isn’t good? Or did we just do it poorly? Or not at all?
• Think “go agile” – Why aren’t we seeing the QPPs move? Is agile a bad idea?

6. 11/2/17
4
Anthony R. Crain
Agenda
ü What is your current state?
ü What are the QPPE metrics?
Ø How do you measure the QPPE metrics?
• Case Study: QPPE PorAolio Level Rollups
• Conclusion
Anthony R. Crain
But how do you measure these?
• But how do you actually measure:
• Quality?
• Predictability?
• ProducIvity?
• Engagement?
• It depends on what the target to be
measured is…
• But definitely paSerns that
work regardless of subject…

7. 11/2/17
5
Anthony R. Crain
The PaFerns
• Quality – how good are our soluIons?
• Defects, value
• Compliance violaIons
• Customer saIsfacIon
• Predictability – how accurate are our plans?
• Accuracy, Time to accuracy, Variance
• ProducIvity – how much do we get done?
• Lead Ime, cost, scope, throughput
• Engagement – are our innovaIons ideas
being adopted and having an effect?
• InnovaIon SaIsfacIon
• Skill growth
• Agile Maturity
Agile Health Assessments – where do they fit in QPPE?
Anthony R. Crain
QPPEs Change Based on Context
• How would a Scrum team measure QPPE?
• Quality?
• Predictability?
• ProducIvity?
• Engagement?
• How about a Kanban team?
• How about a Program (Team of Teams)?
• How about a Release?
• How about a PorAolio?
• How about an Agile TransformaIon?

8. 11/2/17
6
Anthony R. Crain
Example: Release QPPEs
• If you only measure one thing… measure your releases
• Quality – Release Value, Release Quality
• Predictability – On Ime, on budget, on scope, on value, on quality
• ProducIvity – Mean Ime between releases, cost per release, scope delivered
• Engagement – agile health, good agile, impeded agile
Anthony R. Crain
Agenda
ü What is your current state?
ü What are the QPPE metrics?
ü How do you measure the QPPE metrics?
Ø Case Study: QPPE PorAolio Level Rollups
• Conclusion

9. 11/2/17
7
Anthony R. Crain
PorKolio Level Rollups – Release Quality
Anthony R. Crain
PorKolio Level Rollups – Release Predictability
As planned: within 5% of planned Ime, cost, scope and quality; within 20% of planned ROI
Impacted: within 20% of planned Ime, cost, scope, quality and ROI
Severely Impacted: >20% of planned Ime, cost, scope and quality; value gained > cost to build
Failed: Cancelled, zero value gained or cost to build > value gained
% = sum of project costs per category vs total porAolio cost
NOT count of projects!

10. 11/2/17
8
Anthony R. Crain
PorKolio Level Rollups – Release Predictability
When did we know what the actuals were?
0% e.g. planned 1 yr, $1M, actuals 1 yr, $1M
50% e.g. updated plan at 6 mos to 1Y, $1M
100% e.g. didn’t know it was 1yr, $1M Ill project ended
300% e.g. planned 1 yr, $1M, at 3 yr, $3M updated plan to 5y, $5M, actuals were $5y, $5M
Goal: 33% or lower
Anthony R. Crain
PorKolio Level Rollups – Release ProducPvity

11. 11/2/17
9
Anthony R. Crain
PorKolio Level Rollups – Release Engagement
IniIaIve: Go Agile!
Later added: Good
Agile using Health
Assessment metrics
Anthony R. Crain
Agenda
ü What is your current state?
ü What are the QPPE metrics?
ü How do you measure the QPPE metrics?
ü Case Study: QPPE PorAolio Level Rollups
Ø Conclusion

12. 11/2/17
10
Anthony R. Crain
Final Thoughts
• QPPE can be used to measure
• InnovaIon ideas
• Teams
• Programs
• PorAolios
• IniIaIves
• The order of QPPE is deliberate
• If you only measure one thing:
Measure release QPPEs!