Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Simple Measurements                     Schalk W. Cronjé                    ysb33r @ gmail.com                         @ys...
What is Agile?CT SPIN 2012                    © Schalk W. Cronjé
“What is Agile?” - Responses       ●   “Continuous development done in small iterations”       ●   “Adapting to rapid chan...
Being an Agile Fundamentalist                                                      ●   Increased revenue                  ...
"The engineering method is the use of         heuristics to cause the best change in a           poorly understood situati...
Accuracy is to precision                             as               engineering is to mathematicsCT SPIN 2012           ...
Accuracy is to precision                             as               engineering is to mathematics               Be accur...
Software Engineering       ●   Implies measurement       ●   Implies Kolb / Shewhart / Deming Cycles       ●   Is not code...
Simple Measurements:               A foundation of software engineering,                                NOT               ...
Why are you not measuring                 your process today?CT SPIN 2012                               © Schalk W. Cronjé
How do you know when your                delivery was successful?CT SPIN 2012                               © Schalk W. Cr...
Signature Skills       ●   Three parts               –   Preferred Task               –   Preferred Cognitive Approach    ...
Batch Development        Feature 1 Dev                     Feature 1 QA        Feature 2 Dev                     Feature 2...
Batch Development    Total Time = devt1 .. devt4 + qat1 …qat4 + fixdelay + fixtime + retest time         Feature 1 Dev    ...
Time Factors                                                                                Building test                 ...
Taking an Economic View       ●   It helps to quantify the effects of multiple interacting           variables       ●   I...
"If you only quantify one thing,                  quantify the cost of delay."                                        - Do...
Cost of Delay – Quality Issue       ●   One small feature, not properly tested       ●   Found post-release       ●   Time...
"A pilot who sees from afar               will not make his boat a wreck."                                              - ...
David Andersons Six Steps       ●   Focus on quality       ●   Reduce Work-in-Progress       ●   Deliver often       ●   B...
Business Value Increments       ●   Delivery is in business value increments           (BVIs)       ●   Only “done” when t...
Properties of a Kanban SystemCT SPIN 2012                          © Schalk W. Cronjé
Visualisation       ●   Honesty!       ●   Makes it transparent to everyone that is going           on       ●   Visual cu...
VisualisationCT SPIN 2012                   © Schalk W. Cronjé
VisualisationCT SPIN 2012                   © Schalk W. Cronjé
VisualisationCT SPIN 2012                   © Schalk W. Cronjé
Limit Work-in-Progress           R e a d y   S p e c   S p e c         D e v   D e v           Q A   Q A              R e ...
Policies           R e a d y   S p e c   S p e c         D e v   D e v           Q A   Q A              R e le a s e d    ...
Feedback & Batch Size       ●   Faster feedback makes learning faster and           more efficient       ●   Faster feedba...
Measuring Cycle Time       ●   Measure average time in system of each BVI       ●   Less time is spent on estimating futur...
Historic Data vs EstimationCT SPIN 2012                                 © Schalk W. Cronjé
Cadences       ●   Delivery is done at regular cadences.       ●   What is ready, is delivered, what is not, is left      ...
Case study               3 Teams – 3 Locations – 18 monthsCT SPIN 2012                                       © Schalk W. C...
Simple Flow Model               Specify                            Develop                                  QA     Kanban ...
Reduced Learning Time       ●       Writing test specifications after the               development is costly in time     ...
Reduced Learning Time               Specify             Develop               QA                                          ...
Changed the Flow Model               Specify    Develop & Check   Verify       ●   Ensure the all automated checks are inc...
Real-world measurements               Specify            Develop & Check             Verify                            S1 ...
Pathology                        S1           S2           S3      S4  READY                  4            6              ...
Pathology  Excessive long time between feature availability and testing. Is there enough  testing bandwidth? Or is this te...
Kanban fails when people               don’t want to face the truth.                                         - Hillel Glaz...
It all fell apart ...       ●       Restructure – 3 teams + 3 managers       ●       2 managers rejected Kanban           ...
It fell apart even more ...       ●       1 team decided on 4 wks dev + 4 wks testing           –   Quality was worse than...
What did the team members      say?       ●   “I have lost visualisation. I have no idea what is going           on”      ...
Quantifying BVIsCT SPIN 2012                      © Schalk W. Cronjé
Quantifying BVIs       ●   Know what you want to achieve       ●   Establish a scale for measurement       ●   Decide how ...
Quantify BVIs - Example          For all customer submissions that meets          the following criteria, 80% should be   ...
Planguage       ●   Created by Tom Gilb       ●   Provides a structured way of quantifying           requirements       ● ...
Quantify BVI - Planguage      Name: Customer submissions resolved by automation systems      Scale: Percentage of submissi...
Quantified BVI - Applying the                        Economic Model       ●   Compare to average human time to respond    ...
End gameCT SPIN 2012              © Schalk W. Cronjé
Apply Lean Principles                 to your Team               Specify        Develop & Check       Verify       ●   Sta...
Apply Lean Principles                 to your Team       ●   Pair up people with different skills to work on           sam...
Apply Lean Principles               Simple Measurements       ●   Measure cycle time (now predict delivery)       ●   Quan...
Thank youCT SPIN 2012               © Schalk W. Cronjé
Upcoming SlideShare
Loading in …5
×

Simple Measurements #2

954 views

Published on

Use of three simple measurements in to aid with improving software delivering. Includes real world data and a case study from three geographicaly distributed teams.

An updated version of Simple Measurements as delivered at the CT-SPIN group in 2012.

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

Simple Measurements #2

  1. 1. Simple Measurements Schalk W. Cronjé ysb33r @ gmail.com @ysb33rCT SPIN 2012 © Schalk W. Cronjé
  2. 2. What is Agile?CT SPIN 2012 © Schalk W. Cronjé
  3. 3. “What is Agile?” - Responses ● “Continuous development done in small iterations” ● “Adapting to rapid change, predictable periodic delivery of usable software” ● “Developers and testers work together from beginning to end” ● “Youd only get a cynical answer from me”CT SPIN 2012 © Schalk W. Cronjé
  4. 4. Being an Agile Fundamentalist ● Increased revenue ● Reduction in costs ● Faster response times ● Less reworkAgile is the continuous delivering of valueto stakeholders ● Customers ● Users ● Developers & Testers ● Governments & Legislators ● Societal InstitutionsCT SPIN 2012 © Schalk W. Cronjé
  5. 5. "The engineering method is the use of heuristics to cause the best change in a poorly understood situation within the available resources." - Billy V KoenCT SPIN 2012 © Schalk W. Cronjé
  6. 6. Accuracy is to precision as engineering is to mathematicsCT SPIN 2012 © Schalk W. Cronjé
  7. 7. Accuracy is to precision as engineering is to mathematics Be accurate enough to get there, but no more accurateCT SPIN 2012 © Schalk W. Cronjé
  8. 8. Software Engineering ● Implies measurement ● Implies Kolb / Shewhart / Deming Cycles ● Is not code craftsmanship ● SE receives bad press – Perceived difficulty in measurement – Lack of measurement in S/W devCT SPIN 2012 © Schalk W. Cronjé
  9. 9. Simple Measurements: A foundation of software engineering, NOT a tool for measuring humansCT SPIN 2012 © Schalk W. Cronjé
  10. 10. Why are you not measuring your process today?CT SPIN 2012 © Schalk W. Cronjé
  11. 11. How do you know when your delivery was successful?CT SPIN 2012 © Schalk W. Cronjé
  12. 12. Signature Skills ● Three parts – Preferred Task – Preferred Cognitive Approach – Preferred Technology ● This may affect how/what you measure! ● Combat with – Appropriate team composition – Reviews / quality measuresCT SPIN 2012 © Schalk W. Cronjé
  13. 13. Batch Development Feature 1 Dev Feature 1 QA Feature 2 Dev Feature 2 QA Feature 3 Dev Feature 3 QA Feature 4 Dev Feature 4 QA delivered buildCT SPIN 2012 © Schalk W. Cronjé
  14. 14. Batch Development Total Time = devt1 .. devt4 + qat1 …qat4 + fixdelay + fixtime + retest time Feature 1 Dev Feature 1 QA Feature 2 Dev Feature 2 QA Feature 3 Dev Feature 3 QA Feature 4 Dev Feature 4 QA delivered build defect trickle feed Bug Feature 5+ QA DB Retest fixes QA delivered build Feature 5+ DevCT SPIN 2012 © Schalk W. Cronjé
  15. 15. Time Factors Building test understanding infrastructure specs Feature 1 Dev Feature 1 QA Feature 2 Dev Feature 2 QA Feature 3 Dev Feature 3 QA Feature 4 Dev Feature 4 QA basic build verification delivered build time from raising defect until it is available for testing defect trickle feed time to re-test Bug Feature 5+ QA DB Retest fixes QA delivered build Feature 5+ DevCT SPIN 2012 © Schalk W. Cronjé
  16. 16. Taking an Economic View ● It helps to quantify the effects of multiple interacting variables ● It helps us to understand that the customer is not the only judge of value ● By using an economic framework it can allow us to maximise value, including – cycle time – product cost – development expense ● It helps to communicate with non-technical decision makersCT SPIN 2012 © Schalk W. Cronjé
  17. 17. "If you only quantify one thing, quantify the cost of delay." - Don ReinertsenCT SPIN 2012 © Schalk W. Cronjé
  18. 18. Cost of Delay – Quality Issue ● One small feature, not properly tested ● Found post-release ● Time wasting by users having to manual work quantified => £35,000 TotalManualTimeSpent x AvgCostToCompanyPerUser + TimeToRework x AvgCostToCompanyPerDevCT SPIN 2012 © Schalk W. Cronjé
  19. 19. "A pilot who sees from afar will not make his boat a wreck." - Amen-en-apt (Egyptian philosopher, ~700BC)CT SPIN 2012 © Schalk W. Cronjé
  20. 20. David Andersons Six Steps ● Focus on quality ● Reduce Work-in-Progress ● Deliver often ● Balance demand against throughput ● Prioritise ● Attack sources of variability to improve predictabilityCT SPIN 2012 © Schalk W. Cronjé
  21. 21. Business Value Increments ● Delivery is in business value increments (BVIs) ● Only “done” when the increment is delivered end-to-end ● No burning down of tasks ! (false economy) ● Each increment must have at least one metric of the value it will deliverCT SPIN 2012 © Schalk W. Cronjé
  22. 22. Properties of a Kanban SystemCT SPIN 2012 © Schalk W. Cronjé
  23. 23. Visualisation ● Honesty! ● Makes it transparent to everyone that is going on ● Visual cue for bottlenecks ● Manual board and easy-to-use online systemCT SPIN 2012 © Schalk W. Cronjé
  24. 24. VisualisationCT SPIN 2012 © Schalk W. Cronjé
  25. 25. VisualisationCT SPIN 2012 © Schalk W. Cronjé
  26. 26. VisualisationCT SPIN 2012 © Schalk W. Cronjé
  27. 27. Limit Work-in-Progress R e a d y S p e c S p e c D e v D e v Q A Q A R e le a s e d C o m p le te C o m p le te C o m p l e te ● Use a pull system ● Managing queues are easier than managing time lines ● Allows for better throughput utilisation ● Identifies bottlenecks in the system ● Measuring cycle time allows better prediction than traditional estimationCT SPIN 2012 © Schalk W. Cronjé
  28. 28. Policies R e a d y S p e c S p e c D e v D e v Q A Q A R e le a s e d C o m p le te C o m p le te C o m p l e te ● Each stage has a policy ● Describes generic activities required ● “Definition of Done” for each stageCT SPIN 2012 © Schalk W. Cronjé
  29. 29. Feedback & Batch Size ● Faster feedback makes learning faster and more efficient ● Faster feedback provides a sense of control ● Large batches leads to slower feedbackCT SPIN 2012 © Schalk W. Cronjé
  30. 30. Measuring Cycle Time ● Measure average time in system of each BVI ● Less time is spent on estimating future delivery times ● Historic data automatically takes into account any disruptions such as – Operational issues – Days off sick AvgTimeInSystem x NoOfBVIs Time = --------------------------------- ConcurrencyCT SPIN 2012 © Schalk W. Cronjé
  31. 31. Historic Data vs EstimationCT SPIN 2012 © Schalk W. Cronjé
  32. 32. Cadences ● Delivery is done at regular cadences. ● What is ready, is delivered, what is not, is left out ● No artificial time-boxing or break points as in SCRUM-like approaches ● Cycle-time leads to better predictability of what can actually be deliveredCT SPIN 2012 © Schalk W. Cronjé
  33. 33. Case study 3 Teams – 3 Locations – 18 monthsCT SPIN 2012 © Schalk W. Cronjé
  34. 34. Simple Flow Model Specify Develop QA Kanban Board Ready Spec Spec Dev Dev Q A Q A R e le a s e d C o m p l e te C o m p le te C o m p le teCT SPIN 2012 © Schalk W. Cronjé
  35. 35. Reduced Learning Time ● Writing test specifications after the development is costly in time – Knowledge decay – Leads to incomplete designs ● Move the test specification up-front before any development starts – This is part of requirements discovery – It broadens the perspective of the programmer – Allows us to distinguish between automated checks and human testingCT SPIN 2012 © Schalk W. Cronjé
  36. 36. Reduced Learning Time Specify Develop QA Test specification +Time to specify -Time to re-work +Time to develop -Time to re-testCT SPIN 2012 © Schalk W. Cronjé
  37. 37. Changed the Flow Model Specify Develop & Check Verify ● Ensure the all automated checks are included in the development stage ● Free up a Verification stage for exploratory testing and validating that process has been fulfilled.CT SPIN 2012 © Schalk W. Cronjé
  38. 38. Real-world measurements Specify Develop & Check Verify S1 S2 S3 S4 READY 4 6 5 1h SPEC 1 0.5 6 0.5 SPEC/Completed 2 0.5 1 0.5 DEV + Check 2 1 13 4 DEV + Check / 3 6 4 18 Completed VERIFY 3 7 8 1 VERIFY/Completed 2 5 6 18 Blockages 0.5 1h 2 1h Total 17 26 43 42 Average cycle days per featureCT SPIN 2012 © Schalk W. Cronjé
  39. 39. Pathology S1 S2 S3 S4 READY 4 6 5 1h SPEC 1 0.5 6 0.5 SPEC/Completed 2 0.5 1 0.5 DEV + Check 2 1 13 4 DEV + Check / 3 6 4 18 Completed VERIFY 3 7 8 1 VERIFY/Completed 2 5 6 18 Blockages 0.5 1h 2 1h Total 17 26 43 42 Average cycle days per featureCT SPIN 2012 © Schalk W. Cronjé
  40. 40. Pathology Excessive long time between feature availability and testing. Is there enough testing bandwidth? Or is this team just cheating? S1 S2 S3 S4 READY 4 6 5 1h SPEC 1 0.5 6 0.5 SPEC/Completed 2 0.5 1 0.5 DEV + Check 2 1 13 4 DEV + Check / 3 6 4 18 Completed VERIFY 3 7 8 1 VERIFY/Completed 2 5 6 18 Blockages 0.5 1h 2 1h Total 17 26 43 42 Average cycle days per featureCT SPIN 2012 © Schalk W. Cronjé
  41. 41. Kanban fails when people don’t want to face the truth. - Hillel GlazerCT SPIN 2012 © Schalk W. Cronjé
  42. 42. It all fell apart ... ● Restructure – 3 teams + 3 managers ● 2 managers rejected Kanban – 1 manager wanted ScrumWorks – 1 manager used Microsoft Project ● 1 manager was forced not to use Kanban – Switched to iteration-based deliveries ● All teams forced back to time-based estimationCT SPIN 2012 © Schalk W. Cronjé
  43. 43. It fell apart even more ... ● 1 team decided on 4 wks dev + 4 wks testing – Quality was worse than before – Cycle time was at least 8 calendar weeks per BVI ● 1 team did not deliver for over 5 months – Finally delivered something the user did not want ● None of teams have any measurements on cycle time; cannot predict how long delivery will takeCT SPIN 2012 © Schalk W. Cronjé
  44. 44. What did the team members say? ● “I have lost visualisation. I have no idea what is going on” ● “We are not using Kanban, because the company forced us to SCRUM” ● “If I was given a choice, my the team and I would definitely go with Kanban” ● “It is not the company standard as such you need to sell the process frequently. “CT SPIN 2012 © Schalk W. Cronjé
  45. 45. Quantifying BVIsCT SPIN 2012 © Schalk W. Cronjé
  46. 46. Quantifying BVIs ● Know what you want to achieve ● Establish a scale for measurement ● Decide how to measure ● Make constraints explicit ● CommunicateCT SPIN 2012 © Schalk W. Cronjé
  47. 47. Quantify BVIs - Example For all customer submissions that meets the following criteria, 80% should be handled by an automation system instead of a human and resolution achieved within 30 minutes. Criteria A Criteria B etc.CT SPIN 2012 © Schalk W. Cronjé
  48. 48. Planguage ● Created by Tom Gilb ● Provides a structured way of quantifying requirements ● Allows for reuse of specificationsCT SPIN 2012 © Schalk W. Cronjé
  49. 49. Quantify BVI - Planguage Name: Customer submissions resolved by automation systems Scale: Percentage of submissions resolved by automation in relation to all submissions Meter: Query on database using the following statement: SELECT … FRO M… WHERE . . . Goal: 80% [Criteria A, Criteria B]CT SPIN 2012 © Schalk W. Cronjé
  50. 50. Quantified BVI - Applying the Economic Model ● Compare to average human time to respond and volume covered ● Extrapolate to time savings or cost savings ● This allows for two measures of success – Technical – FinancialCT SPIN 2012 © Schalk W. Cronjé
  51. 51. End gameCT SPIN 2012 © Schalk W. Cronjé
  52. 52. Apply Lean Principles to your Team Specify Develop & Check Verify ● Start from where you are; model the workflow ● Make the system pull-based ● Reduce batch size; limit the work-in-progress ● One feature end-to-end ● If the flow breaks, fix it immediatelyCT SPIN 2012 © Schalk W. Cronjé
  53. 53. Apply Lean Principles to your Team ● Pair up people with different skills to work on same feature – Feature teams are not a new concept – Reduces mind-set trap, creative abrasion ● Could be perceived to have higher person cost per feature – Instead of distributing the person cost over multiple queues, the cost is combined into a single queue – Can actually lead to reduced cycle time.CT SPIN 2012 © Schalk W. Cronjé
  54. 54. Apply Lean Principles Simple Measurements ● Measure cycle time (now predict delivery) ● Quantify requirements (know that you did deliver) ● Use “Cost of Delay” (prioritise, show impact/risk)CT SPIN 2012 © Schalk W. Cronjé
  55. 55. Thank youCT SPIN 2012 © Schalk W. Cronjé

×