How many times have you been asked to deliver on metrics that did not make sense, that were counterproductive to the team’s effectiveness, or the organization’s effectiveness? Did those metrics seem impossible to collect? Often times, the metrics being collected are the ones that are easy, and focused on individual “productivity”. How do we collect data that drives continuous improvement and promotes an open and trust-filled environment. How does that change at scale? When it comes to software delivery, lean and agile practices have clearly taken the lead. This session will take a look at the measures we can and should collect across teams and organizations. We’ll dig into metrics that are relevant, interesting, AND useful, and discuss some of the common traps.

1. Collecting the Right Metrics for the Right Reasons
Angela Dugan
Director

2. Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

3. Lines of Code
# Bugs Found# Bugs Fixed Velocity
Utilization
Lead Time
Bug Reactivations
Cyclomatic Complexity
# Features Delivered
WIP
Planned vs. Actual Budget Overrun
Sprint Burndown
Actuals vs Estimate
Build Quality
Code Coverage

4. @OakParkGirl
Angela@Polarissolutions.comTwitter: @OakParkGirl Email: Angela@PolarisSolutions.com

5. Director
Agile coach
Change agent
ALM MVP
SAFe program Consultant
A *possibly* unhealthy love of Halloween
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

6. Angela@PolarisSolutions.com Twitter: @OakParkGirl
Yes, this Polaris! ☺
Not this Polaris! ☺

7. What are you trying to learn?
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

8. @OakParkGirl
Angela@Polarissolutions.com
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

9. @OakParkGirl
Angela@Polarissolutions.comTwitter: @OakParkGirl Email: Angela@PolarisSolutions.com

10. This Photo by Unknown Author is licensed under CC BY-SA-NC

11. witter: @OakParkGirl Email: Angela@PolarisSolutions.com

12. That which is
measured will
improve…
At a cost
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

13. When a measure
becomes a target, it
ceases to be a good
measure
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

14. Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

15. Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

16. What are we supposed to measure?!?
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

17. @OakParkGirl
Angela@Polarissolutions.comTwitter: @OakParkGirl Email: Angela@PolarisSolutions.com

18. Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

19. organizational culture that values “being busy”
dependencies on other teams
team’s skill/confidence at saying no
people’s willingness/ability to swarm
unexpected outages and unplanned work
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

20. WIP at scale has awesome applications
Portfolio Kanban
Program Kanban
Team Kanban
Previously mentioned challenges are
compounded!
@OakParkGirl
Angela@Polarissolutions.com
http://v4.scaledagileframework.com/metrics/

21. Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

22. time poorly spent
dependencies on other teams
team’s skill/confidence at estimating
too much WIP
unexpected outages and unplanned work
People updating their remaining work on a consistent
basis
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

23. Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

24. Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

25. Burndown is hard to interpret at scale!
• Teams may not estimate tasks in hours
• Teams aren’t necessarily using same sprint
cadences
• Burndown resets every sprint boundary
The only way to create a functioning organizational burndown is
to pre-estimate everything everyone is ever going to work on!
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

26. http://v4.scaledagileframework.com/metrics/

27. Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

28. Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

29. @OakParkGirl
Angela@Polarissolutions.com

30. time poorly spent
dependencies on other teams
team’s skill/confidence at estimating
too much WIP
unexpected outages and unplanned work
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

31. Velocity is not useful across teams* at scale!
• Teams may not use same units
• Teams will have different scales
• Teams aren’t necessarily using same sprint cadences
At BEST you’re causing toxic competition, encouraging system
gaming, and learning nothing useful
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

32. Angela@PolarisSolutions.com Twitter:
@OakParkGirlTwitter: @OakParkGirl Email: Angela@PolarisSolutions.com

33. Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

34. Is effected by all of the same challenges as before
Look for bottlenecks and wild variations and INSPECT
and ADAPT
Focus on shrinking lead time and cycle time across
teams in the organization to reap huge benefits!
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

35. Most common ones are LAGGING indicators
They are not a reliable predictor for future performance
Asking for more/better metrics may come at a cost
Teams, products, and components are not interchangeable
Trends can be valuable, but can also be gamed
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

36. Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

37. Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

38. Is the team confident that they understand what they are
being asked to deliver?
Is the team confident they have the support they neam
from the groups they are dependant on?
Is the customer confident that the organization
understands what they want?
Is the business confident they can deliver a high quality
product on time to the customer?
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

39. Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

40. Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

41. Twitter: @OakParkGirl Email:
Angela@PolarisSolutions.com
Reliability
Openness
Collaboration
Innovation
And not just for individual teams.
Measure it for business units, divisions, and the organization!

42. Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

43. 1. Job Satisfaction
a. Score: (1 - 10)
b. Biggest contributor to satisfaction?
c. Biggest detractor from satisfaction?
2. Compensation Satisfaction
a. Score: (1-10)
3. Push/Pull
a. What factors could push you away from our team/company?
b. What factors could pull you away from out team/company?
4. Career Goals (short and long term)
a. What are they?
b. Are you making progress toward them?
https://medium.com/@mmanela/quarterly-questions-e92cba9b2480

44. Measure
LOTS
of
things!
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

45. VersionOne - Annual State of Agile Report https://explore.versionone.com/state-of-agile/versionone-12th-annual-state-of-agile-report

46. @OakParkGirl
Angela@Polarissolutions.com
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

47. 1. Escape Velocity (via Lean Pub) by Doc Norton
2. Version One: State of Agile Report
3. Scaled agile team and organizational metrics:
https://www.scaledagileframework.com/metrics/
Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

48. Email: Angela@PolarisSolutions.com
LinkedIn: https://www.linkedin.com/company/polaris-solutions/
Twitter: @TeamPolaris, @OakParkGirl
Polaris Solutions website - http://www.polarissolutions.com/
Meetups and Briefings: http://www.polarissolutions.com/events/
@OakParkGirl Angela@Polarissolutions.com

49. Angela@PolarisSolutions.com Twitter:
@OakParkGirl