Data-driven performance feedback from customer satisfaction metrics like Net Promoter Score can help teams improve outcomes. Frameworks like DORA provide guidance on measuring what matters, like deployment frequency and change failure rates. Involving teams in selecting metrics and tools promotes transparency. While individual contributions are difficult to measure, tracking team metrics over time shows progress and areas for improvement. Development analytics tools analyze source code management data to provide insights.

1. 1

2. 2

3. 3

4. 4

5. Data-Driven Performance Feedback
Helps Teams Improve Customer Outcomes
with the authors of
ManagingTheUnmanageable.net
Ron Lichty | Ron Lichty Consulting
Ron@RonLichty.com – ronlichty.com
Mickey W. Mantle | Mantle Consulting
mickey@mmantle.com – mmantle.com
5

6. From Programmers to
Managing Software People & Teams
Mickey Ron
6

7. Managing (People and Teams) is Hard!
7

8. Managing (People and Teams) is Hard!
So we wrote:
8

9. 10 Chapters to Help Make Any Manager
More Successful
9

10. Judging and Improving Performance is so important
that it makes up ~25%
of our Chapter on Managing Down
10

11. Survey - How frequently does your organization
require performance reviews to be given:
o Annually
o Semi-annually
o Quarterly
o Never
11

12. Performance Reviews
• A key topic in our book: the importance of feedback
o Studies: employees want feedback, to “know where they stand”
▪ best: continual feedback: weekly, even daily, always timely
• Formal Performance Reviews:
o most organizations require them at least annually
o must be clear regarding how a person is performing
o encourage good outcomes
o discourage inappropriate behavior and bad results
o provide a written record should performance become an issue
o provide guidance and direction for growth
o the jury is out
12

13. The Jury Is Out
…if performance reviews were a drug, they would not receive FDA approval: about
half the time they make things better; about half the time they make things worse.
-Robert Sutton, author of Good Boss, Bad Boss
…traditional performance reviews and approaches to feedback are often so bad that
they actually make performance worse about one-third of the time.
-Gallup, “More Harm Than Good: The Truth About Performance Reviews”
13

14. Caution Using Data
14

15. The Tyranny of Metrics
Metric fixation… the belief that it is possible – and desirable – to replace professional
judgment (acquired through personal experience and talent) with numerical
indicators of performance.
--Jerry Muller
What gets measured is what is most easily measured, and since the outcomes are more
difficult to measure than their inputs, ... The urge to measure… leads to a focus not on
outcomes but on measurable inputs.
--Jerry Muller
What gets measured gets manipulated.
--a corporate client of Ron’s
Never, ever, ever, let story points or velocity be used as a performance metric.
--Ron
15

16. But Data Can Help to…
• Identify a team member with little or no activity
• Supplement managerial judgment with data
• See if guidance/coaching/motivators working
• Suggest where to help your folks grow
• Make trends, changes & deltas visible
16

17. Data-Driven Customer Feedback
Can Help Teams Improve Outcomes
• Measure what matters: customer satisfaction
• NPS: Net Promoter Score
• Would you recommend our product?
○ Why/Why not?
• Pay attention to:
○ trends
○ comments
17

18. Data and Team Performance
• Spot team trends, changes and deltas!
• Follow team-owned business measures
o (e.g., time in app, progress thru app, app abandonment,
repeat visits, app results)
• Are team process experiments working?
• Track DORA / Accelerate metrics
18

19. Survey - How familiar are you with DORA /Accelerate
metrics:
o Currently using them
o Very familiar
o Heard of them
o Not aware of them
19

20. DORA Metrics
(DevOps Research and
Assessment )
(Accelerate)
20

21. DORA Metrics
(2021: added a 5th Metric*)
The fifth metric: from availability to reliability
The fifth metric represents operational performance and is a measure
of modern operational practices. The primary metric for operational
performance is reliability, which is the degree to which a team can
keep promises and assertions about the software they operate.
*https://cloud.google.com/devops/state-of-devops
21

22. DORA Is Aspirational
• Requires a significant commitment across your
dev and ops teams
• May be too big a bite for startups or smaller,
less mature companies
• Emphasizes end-to-end processes
• Requires / can drive organizational maturity
• Can yield dramatic results
22

23. DORA’s Demonstrated Performance Improvement
23

24. Using Data to Improve Customer
Outcomes
• Consciously decide what to measure
24

25. Using Data to Improve Customer
Outcomes
• Consciously decide what to measure
• Adopt, adapt or devise a framework to guide the
measurements that matter for your organization
25

26. The SPACE
Framework for
Developer Productivity
*Excerpted from “The SPACE of Developer Productivity, by Fosgren, Storey, Madilla and Zimmerman,”
https://queue.acm.org/detail.cfm?id=3454124
Satisfaction and well-being
Performance
Activity
Communication & Collaboration
Efficiency & Flow
26

27. Using Data to Improve Customer Outcomes
• Consciously decide what to measure
• Adopt, adapt or devise a framework to guide the
measurements that matter for your organization
• Survey your teams to understand how they feel
27

28. Using Data to Improve Customer Outcomes
• Consciously decide what to measure
• Adopt, adapt or devise a framework to guide the
measurements that matter for your organization
• Survey your teams to understand how they feel
• Involve your team(s) in the decisions
28

29. Using Data to Improve Customer Outcomes
• Consciously decide what to measure
• Adopt, adapt or devise a framework to guide the
measurements that matter for your organization
• Survey your teams to understand how they feel
• Involve your team(s) in the decisions
• Transparency
29

30. Using Data to Improve Customer Outcomes
• Consciously decide what to measure
• Adopt, adapt or devise a framework to guide the
measurements that matter for your organization
• Survey your teams to understand how they feel
• Involve your team(s) in the decisions
• Transparency
• Begin gathering data
30

31. Gathering Data
• Track it yourself
31

32. Gathering Data
• Track it yourself
• Build Tools
32

33. Gathering Data
• Track it yourself
• Build Tools
33

34. Gathering Data
• Track it yourself
• Build Tools
• Use off-the-shelf Tools
34

35. Measure Individual Contributions
or Overall Results?
35

36. DORA’s Demonstrated Performance
Improvement
36

37. DORA’s Demonstrated Performance
Improvements
37

38. Development Tools
38

39. Development Analytics Tools
Git = Git compatible
- Gitlab
- Bitbucket
- AWS CodeCommit
- Cloud Source (Google)
- etc.
39

40. Survey - Development Analytics Tools:
o Not aware of any
o Interested in them
o Experimenting with a few or more
o Using successfully
40

41. Development Productivity Metrics
Picking the Right Tool(s) is important!
41
41

42. DORA Metrics
Software Delivery Performance
42

43. DORA Metrics
43

44. lead time for changes
44
Dashboard powered by Waydev

45. deployment frequency
45
Dashboard powered by Waydev

46. change failure rate
change failure rate
46
Dashboard powered by Waydev

47. time to restore service
47
Dashboard powered by Waydev

48. Team Metrics
48
Dashboard powered by Waydev

49. Personal Metrics in Context of Team
49
Dashboard powered by Waydev

50. Dev Analytics - Powerful Tools!
• Focus on team metrics
• Use personal metrics to see potential issues
o Use as a coaching opportunity
o Never use punitively
• Track progress
• Celebrate team and individual improvements
• Be careful sharing information “up” and “out”
50

51. Dev Analytics – Getting Started
• Identify what to measure
• Evaluate tools – ask others, look at reviews
• Do a “test drive”
o time consuming to “get up to speed” with each Tool
• Start small
o use a small project to get started
• Log and track your learnings
51

52. 52
https://cloud.google.com/devops/state-of-devops

53. 53

54. "It’s unbelievable how much you don’t know about
the game you’ve been playing all your life.“
—Mickey Charles Mantle
54

55. "It’s unbelievable how much you don’t know about
the game you’ve been playing all your life.“
—Mickey Charles Mantle
55

56. "It’s unbelievable how much you don’t know about
the game you’ve been playing all your life.“
—Mickey Charles Mantle
THE ART OF
WINNING AN
UNFAIR GAME
56

57. Lots of Lessons from Moneyball
• Watch it (again)
57

58. Lots of Lessons from Moneyball
• Watch it (again)
• Learn from it
58

59. Lots of Lessons from Moneyball
• Watch it again
• Learn from it
• Think about it
59

60. Lots of Lessons from Moneyball
• Watch it again
• Learn from it
• Think about it
60

61. ManagingTheUnmanageable.net
Ron Lichty | Ron Lichty Consulting
Ron@RonLichty.com – ronlichty.com
Mickey W. Mantle | Mantle
Consulting
Mickey@MMantle.com – mmantle.com 61

62. Audience Q&A
62