The document discusses the importance of measuring both extrinsic and intrinsic metrics related to software development and product quality, emphasizing the need for deeper insight into the quality of work to reduce technical debt. It highlights various types of quality measurements such as test progress, product quality, and process quality. The text proposes a metaphorical 'fishnet' to illustrate how to evaluate different metrics and improve efficiency in development practices.

1. 1

2. "We track a lot of metrics related to progress of development & quality every
sprint, like backlogs, technical debt, velocity, task status etc.
What is not very evident is the “quality of movement”
i.e. how well done, so that we create less debt as we move.
How can I get a better insight of the quality of tests done
and a more objective measure of product quality?”
asked a Senior Engineering Manager

3. 3
Lots of metrics, yet lots of work and tech debt?
would you like a better insight to do less and accomplish more?

4. 4
Most often we measure what is easy and visible (EXTRINSIC)
TEST PROGRESS
stories validated
tests done …
these give greater insight into effectiveness of outcome, to do less & accomplish more

5. 5
Most often we measure what is easy and visible (EXTRINSIC)
TEST PROGRESS
stories validated
tests done …
PRODUCT QUALITY
#bugs found, fixed
distribution, rates..
these give greater insight into effectiveness of outcome, to do less & accomplish more

6. 6
Most often we measure what is easy and visible (EXTRINSIC)
TEST PROGRESS
stories validated
tests done …
PRODUCT QUALITY
#bugs found, fixed
distribution, rates..
PROCESS QUALITY
effort, time
automation…
these give greater insight into effectiveness of outcome, to do less & accomplish more

7. 7
Most often we measure what is easy and visible (EXTRINSIC)
TEST PROGRESS
stories validated
tests done …
PRODUCT QUALITY
#bugs found, fixed
distribution, rates..
PROCESS QUALITY
effort, time
automation…
TEST QUALITY
coverage…
these give greater insight into effectiveness of outcome, to do less & accomplish more

8. 8
Most often we measure what is easy and visible (EXTRINSIC)
TEST PROGRESS
stories validated
tests done …
PRODUCT QUALITY
#bugs found, fixed
distribution, rates..
PROCESS QUALITY
effort, time
automation…
TEST QUALITY
coverage…
these give visibility of direction, progress, speed and external feel of product quality
these give greater insight into effectiveness of outcome, to do less & accomplish more

9. 9
Most often we measure what is easy and visible (EXTRINSIC)
Go deeper to gain better insight into quality of work (INTRINSIC)
TEST PROGRESS
stories validated
tests done …
PRODUCT QUALITY
#bugs found, fixed
distribution, rates..
PROCESS QUALITY
effort, time
automation…
TEST QUALITY
coverage…
these give visibility of direction, progress, speed and external feel of product quality

10. 10
Most often we measure what is easy and visible (EXTRINSIC)
Go deeper to gain better insight into quality of work (INTRINSIC)
TEST PROGRESS
stories validated
tests done …
PRODUCT QUALITY
#bugs found, fixed
distribution, rates..
PROCESS QUALITY
effort, time
automation…
TEST QUALITY
coverage…
these give visibility of direction, progress, speed and external feel of product quality
quality levels
reached

11. 11
Most often we measure what is easy and visible (EXTRINSIC)
Go deeper to gain better insight into quality of work (INTRINSIC)
TEST PROGRESS
stories validated
tests done …
PRODUCT QUALITY
#bugs found, fixed
distribution, rates..
PROCESS QUALITY
effort, time
automation…
TEST QUALITY
coverage…
these give visibility of direction, progress, speed and external feel of product quality
quality levels
reached
defect type
distribution

12. 12
Most often we measure what is easy and visible (EXTRINSIC)
Go deeper to gain better insight into quality of work (INTRINSIC)
TEST PROGRESS
stories validated
tests done …
PRODUCT QUALITY
#bugs found, fixed
distribution, rates..
PROCESS QUALITY
effort, time
automation…
TEST QUALITY
coverage…
these give visibility of direction, progress, speed and external feel of product quality
quality levels
reached
defect type
distribution
dev escapes
product escapes
yield, focus areas

13. 13
Most often we measure what is easy and visible (EXTRINSIC)
Go deeper to gain better insight into quality of work (INTRINSIC)
TEST PROGRESS
stories validated
tests done …
PRODUCT QUALITY
#bugs found, fixed
distribution, rates..
PROCESS QUALITY
effort, time
automation…
TEST QUALITY
coverage…
these give visibility of direction, progress, speed and external feel of product quality
quality levels
reached
defect type
distribution
dev escapes
product escapes
yield, focus areas
test cases
by levels, entities
+ve : -ve indicators

14. 14
Most often we measure what is easy and visible (EXTRINSIC)
Go deeper to gain better insight into quality of work (INTRINSIC)
TEST PROGRESS
stories validated
tests done …
PRODUCT QUALITY
#bugs found, fixed
distribution, rates..
PROCESS QUALITY
effort, time
automation…
TEST QUALITY
coverage…
these give visibility of direction, progress, speed and external feel of product quality
quality levels
reached
defect type
distribution
dev escapes
product escapes
yield, focus areas
test cases
by levels, entities
+ve : -ve indicators
these give greater insight into effectiveness of outcome, to do less & accomplish more

15. 15
A simple fishnet to gain deeper insight

16. 16
A simple fishnet to gain deeper insight
TEST PROGRESS
how are we moving?
scripted, unscripted,
automated

17. 17
A simple fishnet to gain deeper insight
TEST PROGRESS
how are we moving?
scripted, unscripted,
automated
PRODUCT QUALITY
what types of fish
caught where &
when

18. 18
A simple fishnet to gain deeper insight
TEST PROGRESS
how are we moving?
scripted, unscripted,
automated
PRODUCT QUALITY
what types of fish
caught where &
when
TEST QUALITY
net breadth, depth

19. 19
A simple fishnet to gain deeper insight
TEST PROGRESS
how are we moving?
scripted, unscripted,
automated
PRODUCT QUALITY
what types of fish
caught where &
when
PROCESS QUALITY
types of fish caught
net porosity
TEST QUALITY
net breadth, depth

20. 20
A simple fishnet to gain deeper insight
Lower technical debt, accelerate faster
TEST PROGRESS
how are we moving?
scripted, unscripted,
automated
PRODUCT QUALITY
what types of fish
caught where &
when
PROCESS QUALITY
types of fish caught
net porosity
TEST QUALITY
net breadth, depth

21. stagsoftware.com
Home of SmartQA.
Do LESS.Accomplish MORE.
Brilliant thinking, diligent action and adept tooling
to deliver clean code rapidly and cost effectively.
Check out our SmartQA Practice consulting.
stagsoftware.com/smartqa
“Keen on insightful quality metrics?”
Unshackle your practice to see clearly and do far better.