Your SlideShare is downloading. ×
0
Simple Measurements                     Schalk W. Cronjé                    ysb33r @ gmail.com                         @ys...
What is Agile?CT SPIN 2012                    © Schalk W. Cronjé
“What is Agile?” - Responses       ●   “Continuous development done in small iterations”       ●   “Adapting to rapid chan...
Being an Agile Fundamentalist                                                      ●   Increased revenue                  ...
"The engineering method is the use of         heuristics to cause the best change in a           poorly understood situati...
Accuracy is to precision                             as               engineering is to mathematicsCT SPIN 2012           ...
Accuracy is to precision                             as               engineering is to mathematics               Be accur...
Software Engineering       ●   Implies measurement       ●   Implies Kolb / Shewhart / Deming Cycles       ●   Is not code...
Simple Measurements:               A foundation of software engineering,                                NOT               ...
Why are you not measuring                 your process today?CT SPIN 2012                               © Schalk W. Cronjé
How do you know when your                delivery was successful?CT SPIN 2012                               © Schalk W. Cr...
Signature Skills       ●   Three parts               –   Preferred Task               –   Preferred Cognitive Approach    ...
Batch Development        Feature 1 Dev                     Feature 1 QA        Feature 2 Dev                     Feature 2...
Batch Development    Total Time = devt1 .. devt4 + qat1 …qat4 + fixdelay + fixtime + retest time         Feature 1 Dev    ...
Time Factors                                                                                Building test                 ...
Taking an Economic View       ●   It helps to quantify the effects of multiple interacting           variables       ●   I...
"If you only quantify one thing,                  quantify the cost of delay."                                        - Do...
Cost of Delay – Quality Issue       ●   One small feature, not properly tested       ●   Found post-release       ●   Time...
"A pilot who sees from afar               will not make his boat a wreck."                                              - ...
David Andersons Six Steps       ●   Focus on quality       ●   Reduce Work-in-Progress       ●   Deliver often       ●   B...
Business Value Increments       ●   Delivery is in business value increments           (BVIs)       ●   Only “done” when t...
Properties of a Kanban SystemCT SPIN 2012                          © Schalk W. Cronjé
Visualisation       ●   Honesty!       ●   Makes it transparent to everyone that is going           on       ●   Visual cu...
VisualisationCT SPIN 2012                   © Schalk W. Cronjé
VisualisationCT SPIN 2012                   © Schalk W. Cronjé
VisualisationCT SPIN 2012                   © Schalk W. Cronjé
Limit Work-in-Progress           R e a d y   S p e c   S p e c         D e v   D e v           Q A   Q A              R e ...
Policies           R e a d y   S p e c   S p e c         D e v   D e v           Q A   Q A              R e le a s e d    ...
Feedback & Batch Size       ●   Faster feedback makes learning faster and           more efficient       ●   Faster feedba...
Measuring Cycle Time       ●   Measure average time in system of each BVI       ●   Less time is spent on estimating futur...
Historic Data vs EstimationCT SPIN 2012                                 © Schalk W. Cronjé
Cadences       ●   Delivery is done at regular cadences.       ●   What is ready, is delivered, what is not, is left      ...
Case study               3 Teams – 3 Locations – 18 monthsCT SPIN 2012                                       © Schalk W. C...
Simple Flow Model               Specify                            Develop                                  QA     Kanban ...
Reduced Learning Time       ●       Writing test specifications after the               development is costly in time     ...
Reduced Learning Time               Specify             Develop               QA                                          ...
Changed the Flow Model               Specify    Develop & Check   Verify       ●   Ensure the all automated checks are inc...
Real-world measurements               Specify            Develop & Check             Verify                            S1 ...
Pathology                        S1           S2           S3      S4  READY                  4            6              ...
Pathology  Excessive long time between feature availability and testing. Is there enough  testing bandwidth? Or is this te...
Kanban fails when people               don’t want to face the truth.                                         - Hillel Glaz...
It all fell apart ...       ●       Restructure – 3 teams + 3 managers       ●       2 managers rejected Kanban           ...
It fell apart even more ...       ●       1 team decided on 4 wks dev + 4 wks testing           –   Quality was worse than...
What did the team members      say?       ●   “I have lost visualisation. I have no idea what is going           on”      ...
Quantifying BVIsCT SPIN 2012                      © Schalk W. Cronjé
Quantifying BVIs       ●   Know what you want to achieve       ●   Establish a scale for measurement       ●   Decide how ...
Quantify BVIs - Example          For all customer submissions that meets          the following criteria, 80% should be   ...
Planguage       ●   Created by Tom Gilb       ●   Provides a structured way of quantifying           requirements       ● ...
Quantify BVI - Planguage      Name: Customer submissions resolved by automation systems      Scale: Percentage of submissi...
Quantified BVI - Applying the                        Economic Model       ●   Compare to average human time to respond    ...
End gameCT SPIN 2012              © Schalk W. Cronjé
Apply Lean Principles                 to your Team               Specify        Develop & Check       Verify       ●   Sta...
Apply Lean Principles                 to your Team       ●   Pair up people with different skills to work on           sam...
Apply Lean Principles               Simple Measurements       ●   Measure cycle time (now predict delivery)       ●   Quan...
Thank youCT SPIN 2012               © Schalk W. Cronjé
Upcoming SlideShare
Loading in...5
×

Simple Measurements #2

481

Published on

Use of three simple measurements in to aid with improving software delivering. Includes real world data and a case study from three geographicaly distributed teams.

An updated version of Simple Measurements as delivered at the CT-SPIN group in 2012.

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
481
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Many of things discussed has roots in Knowledge Management and ancient philosophy, beliefs and faiths. It is not new, but are generally ignored due to the strong influence of Taylorism especially in western contexts. This is an overview of things that work, that will need adoption in your context and are simple guidelines showing you how to do things.
  • Cynical answers are usually indicative of organisational pathology
  • You are not here to produce software, you are here to provide value to the business Also refer to Dave Nicolette ( http://davenicolette.wordpress.com/2012/08/14/how-to-avoid-the-local-optimization-problem-when-coaching-at-the-team-level-2/ ) “The goal is delivery effectiveness” Some still preach the Agile Manifesto, Some show you methods that will work
  • Credit to Bob Marshall for introducing me to the work of Billy Koen (Texas University)
  • From the work of Dorothy Leonard. Read “Wellsprings of Knowledge”
  • One always has to revisit classic batch development, as it is where most organisations are still stuck.
  • Read Don’s book: The Principles of Product Development. FLOW. Second Generation Lean Product Development Attention Principle: Time counts more than money
  • Use of an online system provides automated measurement. It is important not to negflect having a BIG VISIBLE BOARD
  • From “Quotable Kanban”
  • Eclesiastes 4:9-10 Two are better than one; because they have a good reward for their labour. For if they fall, the one will lift up his fellow: but woe to him that is alone when he falleth; for he hath not another to help him up.
  • Transcript of "Simple Measurements #2"

    1. 1. Simple Measurements Schalk W. Cronjé ysb33r @ gmail.com @ysb33rCT SPIN 2012 © Schalk W. Cronjé
    2. 2. What is Agile?CT SPIN 2012 © Schalk W. Cronjé
    3. 3. “What is Agile?” - Responses ● “Continuous development done in small iterations” ● “Adapting to rapid change, predictable periodic delivery of usable software” ● “Developers and testers work together from beginning to end” ● “Youd only get a cynical answer from me”CT SPIN 2012 © Schalk W. Cronjé
    4. 4. Being an Agile Fundamentalist ● Increased revenue ● Reduction in costs ● Faster response times ● Less reworkAgile is the continuous delivering of valueto stakeholders ● Customers ● Users ● Developers & Testers ● Governments & Legislators ● Societal InstitutionsCT SPIN 2012 © Schalk W. Cronjé
    5. 5. "The engineering method is the use of heuristics to cause the best change in a poorly understood situation within the available resources." - Billy V KoenCT SPIN 2012 © Schalk W. Cronjé
    6. 6. Accuracy is to precision as engineering is to mathematicsCT SPIN 2012 © Schalk W. Cronjé
    7. 7. Accuracy is to precision as engineering is to mathematics Be accurate enough to get there, but no more accurateCT SPIN 2012 © Schalk W. Cronjé
    8. 8. Software Engineering ● Implies measurement ● Implies Kolb / Shewhart / Deming Cycles ● Is not code craftsmanship ● SE receives bad press – Perceived difficulty in measurement – Lack of measurement in S/W devCT SPIN 2012 © Schalk W. Cronjé
    9. 9. Simple Measurements: A foundation of software engineering, NOT a tool for measuring humansCT SPIN 2012 © Schalk W. Cronjé
    10. 10. Why are you not measuring your process today?CT SPIN 2012 © Schalk W. Cronjé
    11. 11. How do you know when your delivery was successful?CT SPIN 2012 © Schalk W. Cronjé
    12. 12. Signature Skills ● Three parts – Preferred Task – Preferred Cognitive Approach – Preferred Technology ● This may affect how/what you measure! ● Combat with – Appropriate team composition – Reviews / quality measuresCT SPIN 2012 © Schalk W. Cronjé
    13. 13. Batch Development Feature 1 Dev Feature 1 QA Feature 2 Dev Feature 2 QA Feature 3 Dev Feature 3 QA Feature 4 Dev Feature 4 QA delivered buildCT SPIN 2012 © Schalk W. Cronjé
    14. 14. Batch Development Total Time = devt1 .. devt4 + qat1 …qat4 + fixdelay + fixtime + retest time Feature 1 Dev Feature 1 QA Feature 2 Dev Feature 2 QA Feature 3 Dev Feature 3 QA Feature 4 Dev Feature 4 QA delivered build defect trickle feed Bug Feature 5+ QA DB Retest fixes QA delivered build Feature 5+ DevCT SPIN 2012 © Schalk W. Cronjé
    15. 15. Time Factors Building test understanding infrastructure specs Feature 1 Dev Feature 1 QA Feature 2 Dev Feature 2 QA Feature 3 Dev Feature 3 QA Feature 4 Dev Feature 4 QA basic build verification delivered build time from raising defect until it is available for testing defect trickle feed time to re-test Bug Feature 5+ QA DB Retest fixes QA delivered build Feature 5+ DevCT SPIN 2012 © Schalk W. Cronjé
    16. 16. Taking an Economic View ● It helps to quantify the effects of multiple interacting variables ● It helps us to understand that the customer is not the only judge of value ● By using an economic framework it can allow us to maximise value, including – cycle time – product cost – development expense ● It helps to communicate with non-technical decision makersCT SPIN 2012 © Schalk W. Cronjé
    17. 17. "If you only quantify one thing, quantify the cost of delay." - Don ReinertsenCT SPIN 2012 © Schalk W. Cronjé
    18. 18. Cost of Delay – Quality Issue ● One small feature, not properly tested ● Found post-release ● Time wasting by users having to manual work quantified => £35,000 TotalManualTimeSpent x AvgCostToCompanyPerUser + TimeToRework x AvgCostToCompanyPerDevCT SPIN 2012 © Schalk W. Cronjé
    19. 19. "A pilot who sees from afar will not make his boat a wreck." - Amen-en-apt (Egyptian philosopher, ~700BC)CT SPIN 2012 © Schalk W. Cronjé
    20. 20. David Andersons Six Steps ● Focus on quality ● Reduce Work-in-Progress ● Deliver often ● Balance demand against throughput ● Prioritise ● Attack sources of variability to improve predictabilityCT SPIN 2012 © Schalk W. Cronjé
    21. 21. Business Value Increments ● Delivery is in business value increments (BVIs) ● Only “done” when the increment is delivered end-to-end ● No burning down of tasks ! (false economy) ● Each increment must have at least one metric of the value it will deliverCT SPIN 2012 © Schalk W. Cronjé
    22. 22. Properties of a Kanban SystemCT SPIN 2012 © Schalk W. Cronjé
    23. 23. Visualisation ● Honesty! ● Makes it transparent to everyone that is going on ● Visual cue for bottlenecks ● Manual board and easy-to-use online systemCT SPIN 2012 © Schalk W. Cronjé
    24. 24. VisualisationCT SPIN 2012 © Schalk W. Cronjé
    25. 25. VisualisationCT SPIN 2012 © Schalk W. Cronjé
    26. 26. VisualisationCT SPIN 2012 © Schalk W. Cronjé
    27. 27. Limit Work-in-Progress R e a d y S p e c S p e c D e v D e v Q A Q A R e le a s e d C o m p le te C o m p le te C o m p l e te ● Use a pull system ● Managing queues are easier than managing time lines ● Allows for better throughput utilisation ● Identifies bottlenecks in the system ● Measuring cycle time allows better prediction than traditional estimationCT SPIN 2012 © Schalk W. Cronjé
    28. 28. Policies R e a d y S p e c S p e c D e v D e v Q A Q A R e le a s e d C o m p le te C o m p le te C o m p l e te ● Each stage has a policy ● Describes generic activities required ● “Definition of Done” for each stageCT SPIN 2012 © Schalk W. Cronjé
    29. 29. Feedback & Batch Size ● Faster feedback makes learning faster and more efficient ● Faster feedback provides a sense of control ● Large batches leads to slower feedbackCT SPIN 2012 © Schalk W. Cronjé
    30. 30. Measuring Cycle Time ● Measure average time in system of each BVI ● Less time is spent on estimating future delivery times ● Historic data automatically takes into account any disruptions such as – Operational issues – Days off sick AvgTimeInSystem x NoOfBVIs Time = --------------------------------- ConcurrencyCT SPIN 2012 © Schalk W. Cronjé
    31. 31. Historic Data vs EstimationCT SPIN 2012 © Schalk W. Cronjé
    32. 32. Cadences ● Delivery is done at regular cadences. ● What is ready, is delivered, what is not, is left out ● No artificial time-boxing or break points as in SCRUM-like approaches ● Cycle-time leads to better predictability of what can actually be deliveredCT SPIN 2012 © Schalk W. Cronjé
    33. 33. Case study 3 Teams – 3 Locations – 18 monthsCT SPIN 2012 © Schalk W. Cronjé
    34. 34. Simple Flow Model Specify Develop QA Kanban Board Ready Spec Spec Dev Dev Q A Q A R e le a s e d C o m p l e te C o m p le te C o m p le teCT SPIN 2012 © Schalk W. Cronjé
    35. 35. Reduced Learning Time ● Writing test specifications after the development is costly in time – Knowledge decay – Leads to incomplete designs ● Move the test specification up-front before any development starts – This is part of requirements discovery – It broadens the perspective of the programmer – Allows us to distinguish between automated checks and human testingCT SPIN 2012 © Schalk W. Cronjé
    36. 36. Reduced Learning Time Specify Develop QA Test specification +Time to specify -Time to re-work +Time to develop -Time to re-testCT SPIN 2012 © Schalk W. Cronjé
    37. 37. Changed the Flow Model Specify Develop & Check Verify ● Ensure the all automated checks are included in the development stage ● Free up a Verification stage for exploratory testing and validating that process has been fulfilled.CT SPIN 2012 © Schalk W. Cronjé
    38. 38. Real-world measurements Specify Develop & Check Verify S1 S2 S3 S4 READY 4 6 5 1h SPEC 1 0.5 6 0.5 SPEC/Completed 2 0.5 1 0.5 DEV + Check 2 1 13 4 DEV + Check / 3 6 4 18 Completed VERIFY 3 7 8 1 VERIFY/Completed 2 5 6 18 Blockages 0.5 1h 2 1h Total 17 26 43 42 Average cycle days per featureCT SPIN 2012 © Schalk W. Cronjé
    39. 39. Pathology S1 S2 S3 S4 READY 4 6 5 1h SPEC 1 0.5 6 0.5 SPEC/Completed 2 0.5 1 0.5 DEV + Check 2 1 13 4 DEV + Check / 3 6 4 18 Completed VERIFY 3 7 8 1 VERIFY/Completed 2 5 6 18 Blockages 0.5 1h 2 1h Total 17 26 43 42 Average cycle days per featureCT SPIN 2012 © Schalk W. Cronjé
    40. 40. Pathology Excessive long time between feature availability and testing. Is there enough testing bandwidth? Or is this team just cheating? S1 S2 S3 S4 READY 4 6 5 1h SPEC 1 0.5 6 0.5 SPEC/Completed 2 0.5 1 0.5 DEV + Check 2 1 13 4 DEV + Check / 3 6 4 18 Completed VERIFY 3 7 8 1 VERIFY/Completed 2 5 6 18 Blockages 0.5 1h 2 1h Total 17 26 43 42 Average cycle days per featureCT SPIN 2012 © Schalk W. Cronjé
    41. 41. Kanban fails when people don’t want to face the truth. - Hillel GlazerCT SPIN 2012 © Schalk W. Cronjé
    42. 42. It all fell apart ... ● Restructure – 3 teams + 3 managers ● 2 managers rejected Kanban – 1 manager wanted ScrumWorks – 1 manager used Microsoft Project ● 1 manager was forced not to use Kanban – Switched to iteration-based deliveries ● All teams forced back to time-based estimationCT SPIN 2012 © Schalk W. Cronjé
    43. 43. It fell apart even more ... ● 1 team decided on 4 wks dev + 4 wks testing – Quality was worse than before – Cycle time was at least 8 calendar weeks per BVI ● 1 team did not deliver for over 5 months – Finally delivered something the user did not want ● None of teams have any measurements on cycle time; cannot predict how long delivery will takeCT SPIN 2012 © Schalk W. Cronjé
    44. 44. What did the team members say? ● “I have lost visualisation. I have no idea what is going on” ● “We are not using Kanban, because the company forced us to SCRUM” ● “If I was given a choice, my the team and I would definitely go with Kanban” ● “It is not the company standard as such you need to sell the process frequently. “CT SPIN 2012 © Schalk W. Cronjé
    45. 45. Quantifying BVIsCT SPIN 2012 © Schalk W. Cronjé
    46. 46. Quantifying BVIs ● Know what you want to achieve ● Establish a scale for measurement ● Decide how to measure ● Make constraints explicit ● CommunicateCT SPIN 2012 © Schalk W. Cronjé
    47. 47. Quantify BVIs - Example For all customer submissions that meets the following criteria, 80% should be handled by an automation system instead of a human and resolution achieved within 30 minutes. Criteria A Criteria B etc.CT SPIN 2012 © Schalk W. Cronjé
    48. 48. Planguage ● Created by Tom Gilb ● Provides a structured way of quantifying requirements ● Allows for reuse of specificationsCT SPIN 2012 © Schalk W. Cronjé
    49. 49. Quantify BVI - Planguage Name: Customer submissions resolved by automation systems Scale: Percentage of submissions resolved by automation in relation to all submissions Meter: Query on database using the following statement: SELECT … FRO M… WHERE . . . Goal: 80% [Criteria A, Criteria B]CT SPIN 2012 © Schalk W. Cronjé
    50. 50. Quantified BVI - Applying the Economic Model ● Compare to average human time to respond and volume covered ● Extrapolate to time savings or cost savings ● This allows for two measures of success – Technical – FinancialCT SPIN 2012 © Schalk W. Cronjé
    51. 51. End gameCT SPIN 2012 © Schalk W. Cronjé
    52. 52. Apply Lean Principles to your Team Specify Develop & Check Verify ● Start from where you are; model the workflow ● Make the system pull-based ● Reduce batch size; limit the work-in-progress ● One feature end-to-end ● If the flow breaks, fix it immediatelyCT SPIN 2012 © Schalk W. Cronjé
    53. 53. Apply Lean Principles to your Team ● Pair up people with different skills to work on same feature – Feature teams are not a new concept – Reduces mind-set trap, creative abrasion ● Could be perceived to have higher person cost per feature – Instead of distributing the person cost over multiple queues, the cost is combined into a single queue – Can actually lead to reduced cycle time.CT SPIN 2012 © Schalk W. Cronjé
    54. 54. Apply Lean Principles Simple Measurements ● Measure cycle time (now predict delivery) ● Quantify requirements (know that you did deliver) ● Use “Cost of Delay” (prioritise, show impact/risk)CT SPIN 2012 © Schalk W. Cronjé
    55. 55. Thank youCT SPIN 2012 © Schalk W. Cronjé
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×