Simple Measurements #2
Upcoming SlideShare
Loading in...5
×
 

Simple Measurements #2

on

  • 618 views

Use of three simple measurements in to aid with improving software delivering. Includes real world data and a case study from three geographicaly distributed teams. ...

Use of three simple measurements in to aid with improving software delivering. Includes real world data and a case study from three geographicaly distributed teams.

An updated version of Simple Measurements as delivered at the CT-SPIN group in 2012.

Statistics

Views

Total Views
618
Views on SlideShare
617
Embed Views
1

Actions

Likes
0
Downloads
10
Comments
0

1 Embed 1

http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Many of things discussed has roots in Knowledge Management and ancient philosophy, beliefs and faiths. It is not new, but are generally ignored due to the strong influence of Taylorism especially in western contexts. This is an overview of things that work, that will need adoption in your context and are simple guidelines showing you how to do things.
  • Cynical answers are usually indicative of organisational pathology
  • You are not here to produce software, you are here to provide value to the business Also refer to Dave Nicolette ( http://davenicolette.wordpress.com/2012/08/14/how-to-avoid-the-local-optimization-problem-when-coaching-at-the-team-level-2/ ) “The goal is delivery effectiveness” Some still preach the Agile Manifesto, Some show you methods that will work
  • Credit to Bob Marshall for introducing me to the work of Billy Koen (Texas University)
  • From the work of Dorothy Leonard. Read “Wellsprings of Knowledge”
  • One always has to revisit classic batch development, as it is where most organisations are still stuck.
  • Read Don’s book: The Principles of Product Development. FLOW. Second Generation Lean Product Development Attention Principle: Time counts more than money
  • Use of an online system provides automated measurement. It is important not to negflect having a BIG VISIBLE BOARD
  • From “Quotable Kanban”
  • Eclesiastes 4:9-10 Two are better than one; because they have a good reward for their labour. For if they fall, the one will lift up his fellow: but woe to him that is alone when he falleth; for he hath not another to help him up.

Simple Measurements #2 Simple Measurements #2 Presentation Transcript

  • Simple Measurements Schalk W. Cronjé ysb33r @ gmail.com @ysb33rCT SPIN 2012 © Schalk W. Cronjé
  • What is Agile?CT SPIN 2012 © Schalk W. Cronjé
  • “What is Agile?” - Responses ● “Continuous development done in small iterations” ● “Adapting to rapid change, predictable periodic delivery of usable software” ● “Developers and testers work together from beginning to end” ● “Youd only get a cynical answer from me”CT SPIN 2012 © Schalk W. Cronjé
  • Being an Agile Fundamentalist ● Increased revenue ● Reduction in costs ● Faster response times ● Less reworkAgile is the continuous delivering of valueto stakeholders ● Customers ● Users ● Developers & Testers ● Governments & Legislators ● Societal InstitutionsCT SPIN 2012 © Schalk W. Cronjé
  • "The engineering method is the use of heuristics to cause the best change in a poorly understood situation within the available resources." - Billy V KoenCT SPIN 2012 © Schalk W. Cronjé
  • Accuracy is to precision as engineering is to mathematicsCT SPIN 2012 © Schalk W. Cronjé
  • Accuracy is to precision as engineering is to mathematics Be accurate enough to get there, but no more accurateCT SPIN 2012 © Schalk W. Cronjé
  • Software Engineering ● Implies measurement ● Implies Kolb / Shewhart / Deming Cycles ● Is not code craftsmanship ● SE receives bad press – Perceived difficulty in measurement – Lack of measurement in S/W devCT SPIN 2012 © Schalk W. Cronjé
  • Simple Measurements: A foundation of software engineering, NOT a tool for measuring humansCT SPIN 2012 © Schalk W. Cronjé
  • Why are you not measuring your process today?CT SPIN 2012 © Schalk W. Cronjé
  • How do you know when your delivery was successful?CT SPIN 2012 © Schalk W. Cronjé
  • Signature Skills ● Three parts – Preferred Task – Preferred Cognitive Approach – Preferred Technology ● This may affect how/what you measure! ● Combat with – Appropriate team composition – Reviews / quality measuresCT SPIN 2012 © Schalk W. Cronjé
  • Batch Development Feature 1 Dev Feature 1 QA Feature 2 Dev Feature 2 QA Feature 3 Dev Feature 3 QA Feature 4 Dev Feature 4 QA delivered buildCT SPIN 2012 © Schalk W. Cronjé
  • Batch Development Total Time = devt1 .. devt4 + qat1 …qat4 + fixdelay + fixtime + retest time Feature 1 Dev Feature 1 QA Feature 2 Dev Feature 2 QA Feature 3 Dev Feature 3 QA Feature 4 Dev Feature 4 QA delivered build defect trickle feed Bug Feature 5+ QA DB Retest fixes QA delivered build Feature 5+ DevCT SPIN 2012 © Schalk W. Cronjé
  • Time Factors Building test understanding infrastructure specs Feature 1 Dev Feature 1 QA Feature 2 Dev Feature 2 QA Feature 3 Dev Feature 3 QA Feature 4 Dev Feature 4 QA basic build verification delivered build time from raising defect until it is available for testing defect trickle feed time to re-test Bug Feature 5+ QA DB Retest fixes QA delivered build Feature 5+ DevCT SPIN 2012 © Schalk W. Cronjé
  • Taking an Economic View ● It helps to quantify the effects of multiple interacting variables ● It helps us to understand that the customer is not the only judge of value ● By using an economic framework it can allow us to maximise value, including – cycle time – product cost – development expense ● It helps to communicate with non-technical decision makersCT SPIN 2012 © Schalk W. Cronjé
  • "If you only quantify one thing, quantify the cost of delay." - Don ReinertsenCT SPIN 2012 © Schalk W. Cronjé
  • Cost of Delay – Quality Issue ● One small feature, not properly tested ● Found post-release ● Time wasting by users having to manual work quantified => £35,000 TotalManualTimeSpent x AvgCostToCompanyPerUser + TimeToRework x AvgCostToCompanyPerDevCT SPIN 2012 © Schalk W. Cronjé
  • "A pilot who sees from afar will not make his boat a wreck." - Amen-en-apt (Egyptian philosopher, ~700BC)CT SPIN 2012 © Schalk W. Cronjé
  • David Andersons Six Steps ● Focus on quality ● Reduce Work-in-Progress ● Deliver often ● Balance demand against throughput ● Prioritise ● Attack sources of variability to improve predictabilityCT SPIN 2012 © Schalk W. Cronjé
  • Business Value Increments ● Delivery is in business value increments (BVIs) ● Only “done” when the increment is delivered end-to-end ● No burning down of tasks ! (false economy) ● Each increment must have at least one metric of the value it will deliverCT SPIN 2012 © Schalk W. Cronjé
  • Properties of a Kanban SystemCT SPIN 2012 © Schalk W. Cronjé
  • Visualisation ● Honesty! ● Makes it transparent to everyone that is going on ● Visual cue for bottlenecks ● Manual board and easy-to-use online systemCT SPIN 2012 © Schalk W. Cronjé
  • VisualisationCT SPIN 2012 © Schalk W. Cronjé
  • VisualisationCT SPIN 2012 © Schalk W. Cronjé
  • VisualisationCT SPIN 2012 © Schalk W. Cronjé
  • Limit Work-in-Progress R e a d y S p e c S p e c D e v D e v Q A Q A R e le a s e d C o m p le te C o m p le te C o m p l e te ● Use a pull system ● Managing queues are easier than managing time lines ● Allows for better throughput utilisation ● Identifies bottlenecks in the system ● Measuring cycle time allows better prediction than traditional estimationCT SPIN 2012 © Schalk W. Cronjé
  • Policies R e a d y S p e c S p e c D e v D e v Q A Q A R e le a s e d C o m p le te C o m p le te C o m p l e te ● Each stage has a policy ● Describes generic activities required ● “Definition of Done” for each stageCT SPIN 2012 © Schalk W. Cronjé
  • Feedback & Batch Size ● Faster feedback makes learning faster and more efficient ● Faster feedback provides a sense of control ● Large batches leads to slower feedbackCT SPIN 2012 © Schalk W. Cronjé
  • Measuring Cycle Time ● Measure average time in system of each BVI ● Less time is spent on estimating future delivery times ● Historic data automatically takes into account any disruptions such as – Operational issues – Days off sick AvgTimeInSystem x NoOfBVIs Time = --------------------------------- ConcurrencyCT SPIN 2012 © Schalk W. Cronjé
  • Historic Data vs EstimationCT SPIN 2012 © Schalk W. Cronjé
  • Cadences ● Delivery is done at regular cadences. ● What is ready, is delivered, what is not, is left out ● No artificial time-boxing or break points as in SCRUM-like approaches ● Cycle-time leads to better predictability of what can actually be deliveredCT SPIN 2012 © Schalk W. Cronjé
  • Case study 3 Teams – 3 Locations – 18 monthsCT SPIN 2012 © Schalk W. Cronjé
  • Simple Flow Model Specify Develop QA Kanban Board Ready Spec Spec Dev Dev Q A Q A R e le a s e d C o m p l e te C o m p le te C o m p le teCT SPIN 2012 © Schalk W. Cronjé
  • Reduced Learning Time ● Writing test specifications after the development is costly in time – Knowledge decay – Leads to incomplete designs ● Move the test specification up-front before any development starts – This is part of requirements discovery – It broadens the perspective of the programmer – Allows us to distinguish between automated checks and human testingCT SPIN 2012 © Schalk W. Cronjé
  • Reduced Learning Time Specify Develop QA Test specification +Time to specify -Time to re-work +Time to develop -Time to re-testCT SPIN 2012 © Schalk W. Cronjé
  • Changed the Flow Model Specify Develop & Check Verify ● Ensure the all automated checks are included in the development stage ● Free up a Verification stage for exploratory testing and validating that process has been fulfilled.CT SPIN 2012 © Schalk W. Cronjé
  • Real-world measurements Specify Develop & Check Verify S1 S2 S3 S4 READY 4 6 5 1h SPEC 1 0.5 6 0.5 SPEC/Completed 2 0.5 1 0.5 DEV + Check 2 1 13 4 DEV + Check / 3 6 4 18 Completed VERIFY 3 7 8 1 VERIFY/Completed 2 5 6 18 Blockages 0.5 1h 2 1h Total 17 26 43 42 Average cycle days per featureCT SPIN 2012 © Schalk W. Cronjé
  • Pathology S1 S2 S3 S4 READY 4 6 5 1h SPEC 1 0.5 6 0.5 SPEC/Completed 2 0.5 1 0.5 DEV + Check 2 1 13 4 DEV + Check / 3 6 4 18 Completed VERIFY 3 7 8 1 VERIFY/Completed 2 5 6 18 Blockages 0.5 1h 2 1h Total 17 26 43 42 Average cycle days per featureCT SPIN 2012 © Schalk W. Cronjé
  • Pathology Excessive long time between feature availability and testing. Is there enough testing bandwidth? Or is this team just cheating? S1 S2 S3 S4 READY 4 6 5 1h SPEC 1 0.5 6 0.5 SPEC/Completed 2 0.5 1 0.5 DEV + Check 2 1 13 4 DEV + Check / 3 6 4 18 Completed VERIFY 3 7 8 1 VERIFY/Completed 2 5 6 18 Blockages 0.5 1h 2 1h Total 17 26 43 42 Average cycle days per featureCT SPIN 2012 © Schalk W. Cronjé
  • Kanban fails when people don’t want to face the truth. - Hillel GlazerCT SPIN 2012 © Schalk W. Cronjé
  • It all fell apart ... ● Restructure – 3 teams + 3 managers ● 2 managers rejected Kanban – 1 manager wanted ScrumWorks – 1 manager used Microsoft Project ● 1 manager was forced not to use Kanban – Switched to iteration-based deliveries ● All teams forced back to time-based estimationCT SPIN 2012 © Schalk W. Cronjé
  • It fell apart even more ... ● 1 team decided on 4 wks dev + 4 wks testing – Quality was worse than before – Cycle time was at least 8 calendar weeks per BVI ● 1 team did not deliver for over 5 months – Finally delivered something the user did not want ● None of teams have any measurements on cycle time; cannot predict how long delivery will takeCT SPIN 2012 © Schalk W. Cronjé
  • What did the team members say? ● “I have lost visualisation. I have no idea what is going on” ● “We are not using Kanban, because the company forced us to SCRUM” ● “If I was given a choice, my the team and I would definitely go with Kanban” ● “It is not the company standard as such you need to sell the process frequently. “CT SPIN 2012 © Schalk W. Cronjé
  • Quantifying BVIsCT SPIN 2012 © Schalk W. Cronjé
  • Quantifying BVIs ● Know what you want to achieve ● Establish a scale for measurement ● Decide how to measure ● Make constraints explicit ● CommunicateCT SPIN 2012 © Schalk W. Cronjé
  • Quantify BVIs - Example For all customer submissions that meets the following criteria, 80% should be handled by an automation system instead of a human and resolution achieved within 30 minutes. Criteria A Criteria B etc.CT SPIN 2012 © Schalk W. Cronjé
  • Planguage ● Created by Tom Gilb ● Provides a structured way of quantifying requirements ● Allows for reuse of specificationsCT SPIN 2012 © Schalk W. Cronjé
  • Quantify BVI - Planguage Name: Customer submissions resolved by automation systems Scale: Percentage of submissions resolved by automation in relation to all submissions Meter: Query on database using the following statement: SELECT … FRO M… WHERE . . . Goal: 80% [Criteria A, Criteria B]CT SPIN 2012 © Schalk W. Cronjé
  • Quantified BVI - Applying the Economic Model ● Compare to average human time to respond and volume covered ● Extrapolate to time savings or cost savings ● This allows for two measures of success – Technical – FinancialCT SPIN 2012 © Schalk W. Cronjé
  • End gameCT SPIN 2012 © Schalk W. Cronjé
  • Apply Lean Principles to your Team Specify Develop & Check Verify ● Start from where you are; model the workflow ● Make the system pull-based ● Reduce batch size; limit the work-in-progress ● One feature end-to-end ● If the flow breaks, fix it immediatelyCT SPIN 2012 © Schalk W. Cronjé
  • Apply Lean Principles to your Team ● Pair up people with different skills to work on same feature – Feature teams are not a new concept – Reduces mind-set trap, creative abrasion ● Could be perceived to have higher person cost per feature – Instead of distributing the person cost over multiple queues, the cost is combined into a single queue – Can actually lead to reduced cycle time.CT SPIN 2012 © Schalk W. Cronjé
  • Apply Lean Principles Simple Measurements ● Measure cycle time (now predict delivery) ● Quantify requirements (know that you did deliver) ● Use “Cost of Delay” (prioritise, show impact/risk)CT SPIN 2012 © Schalk W. Cronjé
  • Thank youCT SPIN 2012 © Schalk W. Cronjé