Introduction to Bugs Measurementby V. Novostavsky
Yes, it’s about metrics
TOC• What we’re talking about• Why its good, and when it’s evil• “Easy way”
Definition• Distance function  – from “anything” to numbers :)
Metrics in life: School
Metrics in life: Traders
Metrics in life: dou.ua
… in testing– Usually we measure bugs  • because there’re a lot of bugs   We can measure barely anything in testing, but ...
… in testing– Number of open bugs– Bugs per module– Bugs reopened– Bugs posted in a week– Bugs rejected / accepted– A lot ...
… in testing– Number of open bugs– Bugs per module– Bugs reopened– Bugs posted in a week– Bugs rejected / accepted– A lot ...
Why its important• testers job is info support for the project
Why its important• testers job is info support for the project• more and diverse info -> better conclusions ->  easier dec...
Why its important• testers job is info support for the project• more and diverse info -> better conclusions ->  easier dec...
Why its important• testers job is info support for the project• more and diverse info -> better conclusions ->  easier dec...
Why its important• testers job is info support for the project• more and diverse info -> better conclusions ->  easier dec...
Example500 open bugs this week…         A lot of?
Example
ExampleMore bugs in a module…   Does it mean it’s worse?
Example
ExampleLess bugs this week…      Is it good?
Example
Top issues
Top issues• time waste!
Top issues• time waste!• wrong conclusions and politics!
Top issues• time waste!make it easier to generate metrics• wrong conclusions and politics!
Top issues• time waste!make it easier to generate metrics• wrong conclusions and politics!make it clear what they mean
“Easy way”• Step 1: choose correct metrics for you
“Easy way”• Step 1: choose correct metrics for you  – Not so easy!
“Easy way”• Step 2: ensure everybody gets what they  mean
“Easy way”• Step 3: automate them (+ use SQL)
“Easy way”• Step 4: be prepared for change requests
“Easy way”• Step 5: use often
“Easy way”• Step 6: publish online
“Easy way”• Step 6: publish online  – Publish charts!
“Easy way”1.   choose correct metrics for you2.   ensure everybody gets what they mean3.   automate them4.   be prepared f...
Example: my metrics• No classic test cases & project is not big  – Bug metrics only• Custom bugtracker  – No access to DB•...
Example: tools• MS XLS –> MS SQL
Example: tools• MS XLS –> MS SQL• Google Docs
Example: my trender
Example: my rollup
Thanks! Q&A
Upcoming SlideShare
Loading in …5
×

Introduction to bugs measurement

1,634 views
1,576 views

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,634
On SlideShare
0
From Embeds
0
Number of Embeds
637
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Poll
  • Introduction to bugs measurement

    1. 1. Introduction to Bugs Measurementby V. Novostavsky
    2. 2. Yes, it’s about metrics
    3. 3. TOC• What we’re talking about• Why its good, and when it’s evil• “Easy way”
    4. 4. Definition• Distance function – from “anything” to numbers :)
    5. 5. Metrics in life: School
    6. 6. Metrics in life: Traders
    7. 7. Metrics in life: dou.ua
    8. 8. … in testing– Usually we measure bugs • because there’re a lot of bugs  We can measure barely anything in testing, but that’s another story…
    9. 9. … in testing– Number of open bugs– Bugs per module– Bugs reopened– Bugs posted in a week– Bugs rejected / accepted– A lot of other stuff…
    10. 10. … in testing– Number of open bugs– Bugs per module– Bugs reopened– Bugs posted in a week– Bugs rejected / accepted– A lot of other stuff…
    11. 11. Why its important• testers job is info support for the project
    12. 12. Why its important• testers job is info support for the project• more and diverse info -> better conclusions -> easier decisions
    13. 13. Why its important• testers job is info support for the project• more and diverse info -> better conclusions -> easier decisions• metrics are facts, operate with facts, not blah- blah-blah
    14. 14. Why its important• testers job is info support for the project• more and diverse info -> better conclusions -> easier decisions• metrics are facts, operate with facts, not blah- blah-blah• example: “big” manager
    15. 15. Why its important• testers job is info support for the project• more and diverse info -> better conclusions -> easier decisions• metrics are facts, operate with facts, not blah- blah-blah• example: “big” manager• NOTE: dont judge too quickly!
    16. 16. Example500 open bugs this week… A lot of?
    17. 17. Example
    18. 18. ExampleMore bugs in a module… Does it mean it’s worse?
    19. 19. Example
    20. 20. ExampleLess bugs this week… Is it good?
    21. 21. Example
    22. 22. Top issues
    23. 23. Top issues• time waste!
    24. 24. Top issues• time waste!• wrong conclusions and politics!
    25. 25. Top issues• time waste!make it easier to generate metrics• wrong conclusions and politics!
    26. 26. Top issues• time waste!make it easier to generate metrics• wrong conclusions and politics!make it clear what they mean
    27. 27. “Easy way”• Step 1: choose correct metrics for you
    28. 28. “Easy way”• Step 1: choose correct metrics for you – Not so easy!
    29. 29. “Easy way”• Step 2: ensure everybody gets what they mean
    30. 30. “Easy way”• Step 3: automate them (+ use SQL)
    31. 31. “Easy way”• Step 4: be prepared for change requests
    32. 32. “Easy way”• Step 5: use often
    33. 33. “Easy way”• Step 6: publish online
    34. 34. “Easy way”• Step 6: publish online – Publish charts!
    35. 35. “Easy way”1. choose correct metrics for you2. ensure everybody gets what they mean3. automate them4. be prepared for change requests5. use regularly6. publish online
    36. 36. Example: my metrics• No classic test cases & project is not big – Bug metrics only• Custom bugtracker – No access to DB• 2 metrics - weekly – Open bugs: Trend in time – Open bugs: By modules
    37. 37. Example: tools• MS XLS –> MS SQL
    38. 38. Example: tools• MS XLS –> MS SQL• Google Docs
    39. 39. Example: my trender
    40. 40. Example: my rollup
    41. 41. Thanks! Q&A

    ×