Implementing a successful metrics system in an IT company involves defining goals and key performance indicators from both a top-down and bottom-up approach. Quality metrics should track factors like cost of quality, defects, and rework trends over time. Regular collection and analysis of metrics on projects, processes, and people can provide insights into trends and areas for improvement. Both business objectives and individual project success can be better managed through an iterative process of planning, collecting, analyzing, and evolving organizational metrics.

1. Implementing a successful metrics system in an IT company
Muntean Ramona
Measurements & Best Practices @ISDC

2. IT’S IMPORTANT TO YOU
IT’S REDUCING UNCERTAINTY
TELLS YOU MORE THAN YOU KNEW BEFORE
Better informed decisions
Tracking the progress of the objectives
Managing the achievements of goals
WE CAN MANAGE WHAT WE CAN EASILY MEASURE

3. GOALS
Goal
Business growth rate
Profit margin
Bids success rate
Personnel turnover
Success
Factors
SUCCES FACTORS
Key Performance
Indicators
Productivity
Predictability
Quality
Competitiveness
Client Satisfaction
Employee Satisfaction
Metrics / Data
Projects, Processes, People
TOP DOWN APPROACH – COMPANY GOALS

4. QUALITY
Cost of Quality
(CoQ)
Cost of Appraisal
(CoA)
Reviews (1st time)
Code
Requirements
Design
Test plans, test scripts
Testing (1st time)
Audits
Cost of Prevention
(CoP)
Training
Procedures, methods
Tools, systems
Planning
Quality improvement
Measurements
Cost of Non-Conformance
(CNC)
Fixing defects
Re-work code review
Re-work doc review
Re-reviews
Re-testing

5. QUALITY TRENDS

6. Improve
KPI’s
Productivity
Predictability
Quality
Client Satisfaction
Team Satisfaction
Metrics,
Trends
Key Performance
Indicators
METRICS ANALIZED
Velocity: Story Points/sprint
Estimation deviation
%Re-work, DDR, CoQ trends
Demo / Release success score
Tooling, Data
Projects, Processes, People
BOTTOM UP APPROACH – PROJECT’S KPI

7. TRENDS
Story Points done / sprint
60
55
50
45
40
35
30
25
20
15
10
55
43
41
43
32
29
23
22
S4
S5
S6
S8
S9
S10
22
S11
S12
S13

8. TRENDS
Defect Detection Rate (DDR)
90.00%
We can learn how
review is
influencing the
product quality in
correlation with
defects found by
testers.
80.00%
70.00%
60.00%
50.00%
40.00%
30.00%
20.00%
10.00%
0.00%
S4
S5
S6
DDR_Review
S7
S8
DDR_Testing
S9
S10
S11
DDR_Customer
S12
S13

9. DEVELOPING MEASUREMENT PROCESS
DO
PLAN
Define procedures:
Collect metrics:
- Metrics to be collected
- Collection practices & tooling
- Analysis & reporting
- Feedback integration
- Tailor procedures (if the case)
- Record and store data
- Use proper tools & templates
- Check data accuracy
Identify scope:
- Goals, objectives
- “Problems” that need answer
- Measurements needs
Evolve process:
- Evaluate progress
- Evaluate process
- Recommend improvements
ACT
Analyze metrics:
- Aggregate, correlate data
- Prepare reports
- Discuss trends, identify thresholds
- Revise procedures
CHECK

10. MEASUREMENT PARADOX?
I want a car that uses as little petrol as possible.
I have chosen 2 cars and will do 3 test drives with both of them.
Test drive 1
Test drive 2
Test drive 3
Average
Car A
1 km/l
1 km/l
4 km/l
2 km/l
Car B
1.5 km/l
1.5 km/l
1.5 km/l
1.5 km/l
Test drive 1
Test drive 2
Test drive 3
Average
Car A
1 l/km
1 l/km
0.25 l/km
0.75 l/km
Car B
0.667 l/km
0.667 l/km
0.667 l/km
0.667 l/km
The car that uses less
petrol is car A which does
more kilometres per litre.
But, if I had measured
litres per kilometre, car B
uses less litres per
kilometre and is the better
car.