Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Introduction to Bugs Measurementby V. Novostavsky
Yes, it’s about metrics
TOC• What we’re talking about• Why its good, and when it’s evil• “Easy way”
Definition• Distance function  – from “anything” to numbers :)
Metrics in life: School
Metrics in life: Traders
Metrics in life: dou.ua
… in testing– Usually we measure bugs  • because there’re a lot of bugs   We can measure barely anything in testing, but ...
… in testing– Number of open bugs– Bugs per module– Bugs reopened– Bugs posted in a week– Bugs rejected / accepted– A lot ...
… in testing– Number of open bugs– Bugs per module– Bugs reopened– Bugs posted in a week– Bugs rejected / accepted– A lot ...
Why its important• testers job is info support for the project
Why its important• testers job is info support for the project• more and diverse info -> better conclusions ->  easier dec...
Why its important• testers job is info support for the project• more and diverse info -> better conclusions ->  easier dec...
Why its important• testers job is info support for the project• more and diverse info -> better conclusions ->  easier dec...
Why its important• testers job is info support for the project• more and diverse info -> better conclusions ->  easier dec...
Example500 open bugs this week…         A lot of?
Example
ExampleMore bugs in a module…   Does it mean it’s worse?
Example
ExampleLess bugs this week…      Is it good?
Example
Top issues
Top issues• time waste!
Top issues• time waste!• wrong conclusions and politics!
Top issues• time waste!make it easier to generate metrics• wrong conclusions and politics!
Top issues• time waste!make it easier to generate metrics• wrong conclusions and politics!make it clear what they mean
“Easy way”• Step 1: choose correct metrics for you
“Easy way”• Step 1: choose correct metrics for you  – Not so easy!
“Easy way”• Step 2: ensure everybody gets what they  mean
“Easy way”• Step 3: automate them (+ use SQL)
“Easy way”• Step 4: be prepared for change requests
“Easy way”• Step 5: use often
“Easy way”• Step 6: publish online
“Easy way”• Step 6: publish online  – Publish charts!
“Easy way”1.   choose correct metrics for you2.   ensure everybody gets what they mean3.   automate them4.   be prepared f...
Example: my metrics• No classic test cases & project is not big  – Bug metrics only• Custom bugtracker  – No access to DB•...
Example: tools• MS XLS –> MS SQL
Example: tools• MS XLS –> MS SQL• Google Docs
Example: my trender
Example: my rollup
Thanks! Q&A
Upcoming SlideShare
Loading in …5
×

Introduction to bugs measurement

1,751 views

Published on

  • Be the first to comment

Introduction to bugs measurement

  1. 1. Introduction to Bugs Measurementby V. Novostavsky
  2. 2. Yes, it’s about metrics
  3. 3. TOC• What we’re talking about• Why its good, and when it’s evil• “Easy way”
  4. 4. Definition• Distance function – from “anything” to numbers :)
  5. 5. Metrics in life: School
  6. 6. Metrics in life: Traders
  7. 7. Metrics in life: dou.ua
  8. 8. … in testing– Usually we measure bugs • because there’re a lot of bugs  We can measure barely anything in testing, but that’s another story…
  9. 9. … in testing– Number of open bugs– Bugs per module– Bugs reopened– Bugs posted in a week– Bugs rejected / accepted– A lot of other stuff…
  10. 10. … in testing– Number of open bugs– Bugs per module– Bugs reopened– Bugs posted in a week– Bugs rejected / accepted– A lot of other stuff…
  11. 11. Why its important• testers job is info support for the project
  12. 12. Why its important• testers job is info support for the project• more and diverse info -> better conclusions -> easier decisions
  13. 13. Why its important• testers job is info support for the project• more and diverse info -> better conclusions -> easier decisions• metrics are facts, operate with facts, not blah- blah-blah
  14. 14. Why its important• testers job is info support for the project• more and diverse info -> better conclusions -> easier decisions• metrics are facts, operate with facts, not blah- blah-blah• example: “big” manager
  15. 15. Why its important• testers job is info support for the project• more and diverse info -> better conclusions -> easier decisions• metrics are facts, operate with facts, not blah- blah-blah• example: “big” manager• NOTE: dont judge too quickly!
  16. 16. Example500 open bugs this week… A lot of?
  17. 17. Example
  18. 18. ExampleMore bugs in a module… Does it mean it’s worse?
  19. 19. Example
  20. 20. ExampleLess bugs this week… Is it good?
  21. 21. Example
  22. 22. Top issues
  23. 23. Top issues• time waste!
  24. 24. Top issues• time waste!• wrong conclusions and politics!
  25. 25. Top issues• time waste!make it easier to generate metrics• wrong conclusions and politics!
  26. 26. Top issues• time waste!make it easier to generate metrics• wrong conclusions and politics!make it clear what they mean
  27. 27. “Easy way”• Step 1: choose correct metrics for you
  28. 28. “Easy way”• Step 1: choose correct metrics for you – Not so easy!
  29. 29. “Easy way”• Step 2: ensure everybody gets what they mean
  30. 30. “Easy way”• Step 3: automate them (+ use SQL)
  31. 31. “Easy way”• Step 4: be prepared for change requests
  32. 32. “Easy way”• Step 5: use often
  33. 33. “Easy way”• Step 6: publish online
  34. 34. “Easy way”• Step 6: publish online – Publish charts!
  35. 35. “Easy way”1. choose correct metrics for you2. ensure everybody gets what they mean3. automate them4. be prepared for change requests5. use regularly6. publish online
  36. 36. Example: my metrics• No classic test cases & project is not big – Bug metrics only• Custom bugtracker – No access to DB• 2 metrics - weekly – Open bugs: Trend in time – Open bugs: By modules
  37. 37. Example: tools• MS XLS –> MS SQL
  38. 38. Example: tools• MS XLS –> MS SQL• Google Docs
  39. 39. Example: my trender
  40. 40. Example: my rollup
  41. 41. Thanks! Q&A

×