The document discusses techniques for measuring key metrics in software development and delivery to improve visibility, manage costs, and ensure success. It provides examples of metrics to measure at different stages of the development lifecycle, including code coverage, team velocity, defect rates, testing times, and story completion rates. The document emphasizes establishing a measurement process to identify problems, take action, and continuously improve product quality and delivery.

1. Software Development and Delivery
Wm. Barrett Simms
barrett@wbsimms.com
@wbsimms

2. What are you interested in?
New Product
Construction
Product
Maintenance
Manage Cost
IT Visibility
Success
Employee
Management
Customer
Satisfaction

3. What’s in it for you?
• Techniques
• Managing technical product delivery
• Managing product maintenance
• Metrics
• Measure Constraints for Visibility
• Philosophy
• The big picture has tons of little details

4. About Me
Software
Developer
Agile Team
Member
Team Lead
Agile
Advocate
SDLC
Implementer

5. Why do we want to measure?
Product
Development/QA
Physical Infrastructure
Project Management
COST

6. What are your variable costs?
Requirements Quality Defects / Testing
Technical Skill Infrastructure
These are under
your control

7. The measurement process
Take
Action
Identify
Problems
Measure

8. What to measure?
Development
• Code Coverage
• Team Velocity
• Developer
Monitoring
• Defect Kill Rate
QA
• Defects Found
• Testing Time :
Manual vs.
Automated
• Kickback Rate
• Quality Tracing
Business
• Story Creation
Rate
• Story Unblock
Rate
• Release Progress

9. Actionable Stories
0
20
40
60
80
100
120
140
160
180
Actionable
Added Unblocked
0
20
40
60
80
100
120
140
160
Blocked
Added Blocked

10. Unblock Rate
𝑈𝑛𝑏𝑙𝑜𝑐𝑘𝑒𝑑 − 𝐵𝑙𝑜𝑐𝑘𝑒𝑑
𝐴𝑑𝑑𝑒𝑑
-1.5
-1
-0.5
0
0.5
1
1.5
Iteration 1 Iteration 2 Iteration 3 Iteration 4 Iteration 5 Iteration 6
Unblock Score Unblock Rate
New Story Count Story Blocked Story Unblocked
10 3 7
10 10 0
10 6 4
10 0 10
10 5 5
10 7 3

11. The Hawthorne Effect
• Knowing you are being monitored will change your behavior

12. Quality Tracing
• Add findable defects in product
each week
• Track how long it takes to find all
defects
• Reveal the find/miss rate each
week
• Automated UI Testing
• QA Define Requirements
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Trace Defects Found

13. Iteration Flow (QA Constraint)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 2 3 4 5 6 7
Defined In Progress Completed Accepted
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 2 3 4 5 6 7
Defined In Progress Completed Accepted

14. Team Velocity

15. Developer Monitoring
0
200
400
600
800
1000
1200
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Developer #1
Code Added Code Modified
0
200
400
600
800
1000
1200
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Developer #2
Code Added Code Modified

16. Defect Kill Rate
𝐴𝑣𝑒𝑟𝑎𝑔𝑒(𝐷𝑒𝑓𝑒𝑐𝑡 𝐶𝑙𝑜𝑠𝑒𝑑 − 𝐷𝑒𝑓𝑒𝑐𝑡 𝑂𝑝𝑒𝑛𝑒𝑑)
0
1
2
3
4
5
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Difference
0
2
4
6
8
10
12
14
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Average
Average
Implementation of Unit Testing!

17. SDLC

18. Code Coverage

19. Velocity versus Coverage
0
2
4
6
8
10
12
14
16
18
0%
10%
20%
30%
40%
50%
60%
70%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Coverage Velocity

20. Putting it all together