Agile KPIs

49,526 views

Published on

my presentation at IAD10, Italian Agile Day 2010, Genova, November 19, 2010

Published in: Technology
2 Comments
133 Likes
Statistics
Notes
  • Good One, Can i have a copy of the deck please - raju_imi@rediffmail.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • presented ad IAD10, Italian Agile Day 2010.
    an introduction to general key concepts, benefits and pitfalls in measuring software.
    after an overview about classic KPIs, some typical agile metrics are presented.
    more to follow ...
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
49,526
On SlideShare
0
From Embeds
0
Number of Embeds
4,953
Actions
Shares
0
Downloads
1,015
Comments
2
Likes
133
Embeds 0
No embeds

No notes for slide

Agile KPIs

  1. 1. Agile KPIs Gaetano Mazzanti Gama-Tech
  2. 2. are you measuring your performances ?
  3. 3. a metric is a measure or a combination of measures for quantitatively assessing, controlling or improving a process, a product, a team a KPI, Key Perfomance Indicator, is a (aggregate) metric that: is tied to a strategic objective; have at least one defined time-bound target value (number, range, limit, percentage, trend, variation) Metric vs. KPI
  4. 4. self-induced vs. enforced internal vs. external snapshot vs. forward looking different perspectives
  5. 5. actionable understandable accessible Many “recipes” and characteristics, i.e. SMART Specific Measurable Achievable Relevant Timely INVEST Immediately actionable Negotiable Valuable Estimable Sized to fit Testable a good KPI 3 are enough
  6. 6. Agile vs. Traditional KPIs
  7. 7. classic KPIs efforttimescope quality
  8. 8. some examples (time, effort, scope) Is planning accurate ? How long does it take for a requirement to be delivered to customers ? Are effort and cost estimates accurate ? How is effort split between design, coding and testing ? Are requirements satisfied ? Are requirements changing ? How often ? How “big” is the software ? Schedule Adherence & Variance Lead Time, Cycle Time Slip Charts Effort & Cost Adherence Cost per Phase Amount of Rework Requirements adherence Requirements volatility (churn) Code size (KLOCs :( , #modules, #classes, …)
  9. 9. a typical problem time Is planning accurate ? slip chart 33% 200% 66% 300%
  10. 10. Defects Code Architecture Usability Documentation Installation Support etc. metrics explosion Quality
  11. 11. exponential metric growth: i.e. Defects total # defects # defects by category (i.e. critical, major, minor) # non-functional defects (usability, performance) # new defects / time # defects fixed / time # critical defects / time # re-opened defects (regression) # tests / defect time required to fix a defect # defects found in-house / total # defects (DRE) # defects found / # test hours spent etc.
  12. 12. too many KPIs are useless
  13. 13. keep it simple, one step at a time
  14. 14. self-observation problem #1 neglect shortcomings problem #2 excuses benchmarking problem #3 who do you compare to? evaluating: easier said than done
  15. 15. Metrics should not scare or threaten people Enforced metrics are often cheated or ignored
  16. 16. Cheating copy & paste some code, copy & paste unit tests making slight modifications result: increased code coverage write some buggy code and quickly fix it result: increased number of fixed bugs (maybe you also get credit for additional LOCs!)
  17. 17. Sex, Lies & Statistics (beware of wrong/biased numbers) 1920: “most criminals are farmers” actually most people where farmers at that time 1940: “twins more likely if mothers are in their 25s-30s” most mothers gave birth in their 25s-30s at that time growth much easier when starting from a small base
  18. 18. Code Quality Metrics KLOCs code reviews & specific analysis tools needed how fragile you are Complexity Duplication Smells Churn
  19. 19. Code Analysis Tools are Fun! source: source:
  20. 20. some Agile principles: Create Value for the customer as early as possible Eliminate Waste (WIP, YAGNI) Drive and Respond to Change, quickly Time/Capacity Boxing (see Scrum and Kanban) Provide Visibility into project progress Enter Agile
  21. 21. productivity defects efficiency effort customer satisfaction velocity time requirements size trend quality burndown roi schedule business value capacity complexity cumulative flow effectiveness Googling with KPIs
  22. 22. Story Point an arbitrary value to express effort, complexity and risk associated to a user story, feature, task Velocity S Velocity # story points / interval (where interval is usually a sprint or more in general time) Velocity is a team specific metric, it cannot be used to compare different teams Velocity Improvement is a key goal for Agile teams
  23. 23. 0 100 200 300 400 500 600 1 2 3 4 5 6 7 8 9 10 11 ideal SP left Release Burndown Chart sprint Average velocity = 50 Story Points / Sprint 500 Story Points (SP) => 10 Sprints
  24. 24. -100 0 100 200 300 400 500 600 1 2 3 4 5 6 7 8 9 10 11 12 13 14 added SP SP carried over from previous sprint SP todo from initial list ideal trend velocity added SP trend actual trend Release Burndown Chart sprint velocity delay new expected completion date SPs have to be dropped to get back on track
  25. 25. 0 50 100 150 200 250 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 real SP left real Xtra time left ideal burndown ideal Xtra time left Sprint Burndown Chart bad baddays delay look, ma instant feedback! Xtra time = time reserved for other tasks (emergencies, bug fixing, reviews, etc.)
  26. 26. each story has one or more automated tests when tests pass the story has been completed => you are forced to become agile: cannot do BDUF*, must do automated testing/ continuous integration must deliver early value Compile Build Test Deploy Running Tested FeaturesRon Jeffries C.I. Quality Feedback Loop*Big Design Up Front
  27. 27. Backlog Story Points Value For Customer Value Delivered (*100) story 1 13 21 162 story 2 21 13 62 story 3 34 21 62 story 4 5 8 160 story 5 5 5 100 story 6 13 3 23 story 7 8 8 100 story 8 3 1 33 story 9 13 5 38 story 10 21 5 24 Early Value Delivery (again) Business Value Points Desire of Customer Value = Cost of providing functionality 68.3% of value already delivered here same velocity greater value! story points
  28. 28. Iteration Related Metrics (%) Stories & Story Points Completed (%) Stories added/removed (Sprint & Release) Stories unfinished/moved to Next Sprint Sprints Moved to Next Release Lead & Cycle Time (Stories & Defects) Average Age of Stories and Defects (%) Failed Builds, (%) Failed Tests Defects Added & Fixed (absolute & trend)
  29. 29. WIP Cumulative Flow Diagram to do doing done cycle time cycle time WIP WIP is increasing done doing to do Work In Process (WIP) and Cycle Time should be minimized
  30. 30. Some Kanban Specific Metrics Cycle Time = Number of Things in Process/ Average Completion Rate Little’s Law time spent in each lane ? bottlenecks ? cycle time lead time Flow = Speed * Density, Density Speed => Traffic Jam 40 60 25 ouch! ouch!
  31. 31. first thing in the morning: dashboard! Short Term vs. Long Term Dashboard. Short Term: build ok, automated tests ok, value earned, schedule (burndown), critical bugs entered, critical bugs still there, impediments, ... Long Term: trends: failed builds, failed tests, # of tests, velocity, earned value per month, # and type of bugs, … i.e. increase test coverage
  32. 32. Essential Tools a big board and lots of cards (hard to extract metrics though) source code management (obvious) automated build & test suites (CI) issue tracking* some/many of these tools may be part of integrated product suites *unless you belong to a small elite code analysis agile management system dashboard
  33. 33. Take-Home Points
  34. 34. measure, evaluate, improve
  35. 35. communicate clearly @you: are you getting this ?
  36. 36. transparency: KPIs visible to everyone
  37. 37. communicate visually
  38. 38. use simple tools (but use them!)
  39. 39. build a simple but effective dashboard
  40. 40. aim higher
  41. 41. Gaetano Mazzanti info@gama-tech.net Gama-Tech Photo Credit: iStockphoto.com S.Bouchard Bruce McBroom/@Apple Corps Ltd

×