Simulating Soft ware       Teams       Sagi Smolarski
Purpose of This Presentation
Purpose of This PresentationLearn how to use process simulations to...
Purpose of This PresentationLearn how to use process simulations to...   Gain insights into soft ware development   process
Purpose of This PresentationLearn how to use process simulations to...   Gain insights into soft ware development   proces...
Purpose of This PresentationLearn how to use process simulations to...   Gain insights into soft ware development   proces...
Purpose of This PresentationLearn how to use process simulations to...   Gain insights into soft ware development   proces...
Perspectives       on a soft ware        development         operationThere are many ways to  look at a soft waredevelopme...
Perspectiveson a soft ware development  operation  This is one way
Perspectiveson a soft ware development  operation    Here we are      talking     about this    perspective
Kanban Board                         In Kanban                                               this Machine                 ...
Questions              What are the right WIP limits? (*)              What is the optimal User Story Size?              H...
Visual Simulations
Visual Simulations       Visual Mode =      • validate model      • gain insights
Batch Simulations
Batch Simulations
Batch Simulations
Batch Simulations
Batch Simulations        Batch Mode = • Simulate many options • Answer a specific question
Simulation RulesSimple queue transition mechanics Pull to WIP Limit Process to capacitySome rules from FLOW
Example of such a rule      Value Decay Model      value 10.5                         The later a request is delivered,   ...
Example of such a rule      Value Decay Model      value 10.5                         The later a request is delivered,   ...
Example of such a rule      Value Decay Model      value 10.5                         The later a request is delivered,   ...
Example of such a rule      Value Decay Model      value 10.5                         The later a request is delivered,   ...
Example of such a rule      Value Decay Model      value 10.5                         The later a request is delivered,   ...
Question #1 What is our   Project’sPerformance ?
What is Our Performance?    Looking at a project in progress...
What is Our Performance?      Weekly Profit / Loss
What is Our Performance?       How do We Measure Performance? Profit/Loss is great... But... trailing indicator (after the...
Cumulative Flow                                               Ready                                            Development...
Cycle Time                        On day 47, a work item                        was delivered 43 days                     ...
Lesson #1 It’s hard to know your performance when youdon’t see it. Use cycle time and cumulative flow to  control your pro...
Question #2What Are the Right  WIP Limits?
INCREASING WIP Limit
INCREASING WIP Limit             Cycle Time
INCREASING WIP Limit             Cycle Time              Profit
INCREASING WIP Limit             Cycle Time              Profit            Team members’              Utilization
INCREASING WIP Limit               Cycle Time                Profit      Sweet   Team members’       Spot     Utilization
INCREASING WIP Limit               Cycle Time                              Cycle Time                               Change...
Some Insights From This Simple Case
Some Insights From This Simple Case1. Top financial performance coincides with   reaching top utilization of the team
Some Insights From This Simple Case1. Top financial performance coincides with   reaching top utilization of the team2. To...
Some Insights From This Simple Case1. Top financial performance coincides with   reaching top utilization of the team2. To...
Single Working Queue453015              Cycle Time0     0   20     40         60      80                                WI...
Single Working Queue4530                                       Little’s                                        Formula15  ...
Single Working Queue4530            Lifetime Value                  Little’s                                             F...
Single Working Queue         Team Members’ Utilization4530             Lifetime Value                  Little’s           ...
Single Working Queue                   Team Members’ Utilization4530                       Lifetime Value                 ...
Decision Making
Decision Making   In Agile:
Decision Making   In Agile:  • Frequent inspect & adapt
Decision Making   In Agile:  • Frequent inspect & adapt  • What about long-term decision?
Decision Making   In Agile:  • Frequent inspect & adapt  • What about long-term decision?  • What about irreversible decis...
The CFO has justapproved a new hire   for your team
The CFO has just approved a new hire    for your team Great! We’ll hire anew developer so wecan crank out more     features
The CFO has just  approved a new hire     for your team Great! We’ll hire anew developer so wecan crank out more     featu...
Question #3Should I hire adeveloper or a   tester?
Adding a Team MemberTotal Value Delivered, Simulated Process                                                              ...
Adding a Team MemberTotal Value Delivered, Simulated Process                                                              ...
Lesson #3 In some situations,   hiring an extraDeveloper will reduce   value delivered
Question #4Small User Stories   are good...How small is small    enough?
User Story SizeValue Delivered $1,125,000  $750,000  $375,000         $0 ($375,000) ($750,000)($1,125,000)($1,500,000)    ...
e        User-Story Size         High Resolution Simulation                                         Value                 ...
Lesson #4 When value delivereddecays with time, small batches (user stories)    increase value
From Simple to Complex  From the previous chart, we can see that   a combination of simple rules, yields      bewilderingl...
Aircraft & Crew   An interesting analogy of how we   can deal with  complex systems Simulate?                Why          ...
FlightSimulation
FlightSimulation• Experiment in a safe environment
FlightSimulation• Experiment in a safe environment• Can bring the system to extremes without risk
FlightSimulation• Experiment in a safe environment• Can bring the system to extremes without risk• Can pause, fast for war...
FlightSimulation• Experiment in a safe environment• Can bring the system to extremes without risk• Can pause, fast for war...
FlightSimulation• Experiment in a safe environment• Can bring the system to extremes without risk• Can pause, fast for war...
Soft ware DevelopmentOrganizations         • Complicated System         • Easy to make costly mistakes         • Small inp...
Playing to Learn   Example: GetKanban.com - Looking at the   micro-mechanics of Kanban   Here we are looking at the macro-...
Lesson #5Simulations are a cost- effective way to get   insights into the operation of complex        systems
The Holy Grail of metrics...        Often sought...        Seldom reliably measuredProductivity
The Holy Grail of metrics...        Often sought...        Seldom reliably measuredProductivity     =  Output   Input
Productivity Measurement
Productivity MeasurementBest measure Value Delivered and maximize it
Productivity MeasurementBest measure Value Delivered and maximize itHolistic - Includes Product Management,Operations, Sal...
Productivity MeasurementBest measure Value Delivered and maximize itHolistic - Includes Product Management,Operations, Sal...
Productivity MeasurementBest measure Value Delivered and maximize itHolistic - Includes Product Management,Operations, Sal...
Productivity Measurement Downside of Using Bottom-line Value
Productivity Measurement  Downside of Using Bottom-line ValueHolistic - Measures dev + qa + operations +support + PM + Sal...
Productivity Measurement  Downside of Using Bottom-line ValueHolistic - Measures dev + qa + operations +support + PM + Sal...
Productivity Measurement   Alternative to Bottom-line Value
Productivity Measurement      Alternative to Bottom-line Value Can measure value-add hours vs. paid hours
Productivity Measurement      Alternative to Bottom-line Value Can measure value-add hours vs. paid hours Delta is Waste:
Productivity Measurement      Alternative to Bottom-line Value Can measure value-add hours vs. paid hours Delta is Waste: ...
Productivity Measurement      Alternative to Bottom-line Value Can measure value-add hours vs. paid hours Delta is Waste: ...
Productivity Measurement      Alternative to Bottom-line Value Can measure value-add hours vs. paid hours Delta is Waste: ...
Productivity Measurement      Alternative to Bottom-line Value Can measure value-add hours vs. paid hours Delta is Waste: ...
Productivity Measurement      Alternative to Bottom-line Value Can measure value-add hours vs. paid hours Delta is Waste: ...
Productivity Measurement      Alternative to Bottom-line Value Can measure value-add hours vs. paid hours Delta is Waste: ...
Productivity Measurement      Alternative to Bottom-line Value Can measure value-add hours vs. paid hours Delta is Waste: ...
Productivity Measurement
Productivity MeasurementExample:
Productivity MeasurementExample:5 developers, 2 testers, working for 100 days
Productivity MeasurementExample:5 developers, 2 testers, working for 100 daysInput = 100 * ( 5+2) = 700
Productivity MeasurementExample:5 developers, 2 testers, working for 100 daysInput = 100 * ( 5+2) = 700delivered 70 featur...
Productivity MeasurementExample:5 developers, 2 testers, working for 100 daysInput = 100 * ( 5+2) = 700delivered 70 featur...
Productivity MeasurementExample:5 developers, 2 testers, working for 100 daysInput = 100 * ( 5+2) = 700delivered 70 featur...
The observation itself affects the system under observation
The observation itself         affects the system         under observation This is true insoft ware teams    as well...
Measuring ProductivityWould this measurement work?                 In-situ (real teams) In-vitro (Simulations)Will be game...
Team Composition &   Productivity    Productivity    Cycle time (avg, stdev)                        Productivity = Efficie...
Question #6Is it cost-effective to invest in qualitypractices?
Should We Invest inAutomated Testing?                                                                                 Valu...
Should We Invest inAutomated Testing?                                                                                 Valu...
Best Practices In Creating Simulations
Best Practices In Creating Simulations    Use real historical data as a baseline
Best Practices In Creating Simulations    Use real historical data as a baseline    When real data not available, use indu...
Best Practices In Creating Simulations    Use real historical data as a baseline    When real data not available, use indu...
Best Practices In Creating Simulations    Use real historical data as a baseline    When real data not available, use indu...
Best Practices In Creating Simulations    Use real historical data as a baseline    When real data not available, use indu...
Summary
SummarySimulations are a great addition to our toolset
SummarySimulations are a great addition to our toolset   To gain insight into complex processes
SummarySimulations are a great addition to our toolset   To gain insight into complex processes   To make better decisions
SummarySimulations are a great addition to our toolset   To gain insight into complex processes   To make better decisions...
SummarySimulations are a great addition to our toolset   To gain insight into complex processes   To make better decisions...
SummarySimulations are a great addition to our toolset   To gain insight into complex processes   To make better decisions...
Key Take AwaysUnderstand that there is a machine withinyour team’s processRealize it’s a complex machine, who has asignifi...
More InformationCheck it out at:     http://flower.agilesparks.comTalk to us:  sagi@agilesparks.com  yuval@agilesparks.com ...
More InformationCheck it out at:     http://flower.agilesparks.comTalk to us:  sagi@agilesparks.com  yuval@agilesparks.com ...
Upcoming SlideShare
Loading in...5
×

Simulating Software Teams

469

Published on

Computer simulations of software teams process helps you gain insight into its inherent complexity and assists in making better decisions about process policies

Published in: Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
469
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • One disclaimer to get you in the right mind set...\nOne of the many ways to look at software development operations is to use a perspectives model such as this one\nThis talk is about the “machine” layer. Not because it’s more important... it’s not. \nBecause it is important as it provides insights to the people about the basic mechanics & physics laws of their process,\nThen the people can make better decisions.\n
  • One disclaimer to get you in the right mind set...\nOne of the many ways to look at teams is to use a perspectives model such as this one\nThis talk is about the “machine” layer. Not because it’s more important... it’s not. \nBecause it is important as it provides insights to the people about the basic mechanics & physics laws of their process,\nThen the people can make better decisions.\n
  • One disclaimer to get you in the right mind set...\nOne of the many ways to look at teams is to use a perspectives model such as this one\nThe machine layer is the one who contains lists of things to do for different stages in the process, the policies for how things move from one list to another, for how work items (tasks and bugs etc.) look like. etc.\nThis talk is about the “machine” layer. Not because it’s more important... it’s not. \nBecause it is important as it provides insights to the people about the basic mechanics & physics laws of their process,\nThen the people can make better decisions.\n
  • one of the nice things about Kanban is exposing this machine so we see what’s going on...\nThere are many decisions to make about how the machine operates:\nWIP limits, who works on what, when? policies for how things are transitioning from list to list, classes of service, prioritization. Size of tasks\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Flow is a great book which combines Statistics, Finance & Ops research to get important insights about how to run this machine. \n
  • like in radioactivity\nrelated to the cost of delay\naverage for all items\n
  • like in radioactivity\nrelated to the cost of delay\naverage for all items\n
  • like in radioactivity\nrelated to the cost of delay\naverage for all items\n
  • like in radioactivity\nrelated to the cost of delay\naverage for all items\n
  • like in radioactivity\nrelated to the cost of delay\naverage for all items\n
  • like in radioactivity\nrelated to the cost of delay\naverage for all items\n
  • like in radioactivity\nrelated to the cost of delay\naverage for all items\n
  • like in radioactivity\nrelated to the cost of delay\naverage for all items\n
  • \n
  • Are we doing well?\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • The longer the queue, the slower the system is to respond to change, \nin this case, changes to cycle time following changes in WIP limits.\n100% utilization marks the start of the sweet spot.\n\n\n
  • The longer the queue, the slower the system is to respond to change, \nin this case, changes to cycle time following changes in WIP limits.\n100% utilization marks the start of the sweet spot.\n\n\n
  • The longer the queue, the slower the system is to respond to change, \nin this case, changes to cycle time following changes in WIP limits.\n100% utilization marks the start of the sweet spot.\n\n\n
  • The longer the queue, the slower the system is to respond to change, \nin this case, changes to cycle time following changes in WIP limits.\n100% utilization marks the start of the sweet spot.\n\n\n
  • The longer the queue, the slower the system is to respond to change, \nin this case, changes to cycle time following changes in WIP limits.\n100% utilization marks the start of the sweet spot.\n\n\n
  • The longer the queue, the slower the system is to respond to change, \nin this case, changes to cycle time following changes in WIP limits.\n100% utilization marks the start of the sweet spot.\n\n\n
  • The longer the queue, the slower the system is to respond to change, \nin this case, changes to cycle time following changes in WIP limits.\n100% utilization marks the start of the sweet spot.\n\n\n
  • The longer the queue, the slower the system is to respond to change, \nin this case, changes to cycle time following changes in WIP limits.\n100% utilization marks the start of the sweet spot.\n\n\n
  • The longer the queue, the slower the system is to respond to change, \nin this case, changes to cycle time following changes in WIP limits.\n100% utilization marks the start of the sweet spot.\n\n\n
  • The longer the queue, the slower the system is to respond to change, \nin this case, changes to cycle time following changes in WIP limits.\n100% utilization marks the start of the sweet spot.\n\n\n
  • True for this simple setup. If there’s a team that deals with interruptions it’s a different matter.\n
  • True for this simple setup. If there’s a team that deals with interruptions it’s a different matter.\n
  • True for this simple setup. If there’s a team that deals with interruptions it’s a different matter.\n
  • This is true only for a queue with WIP limits, feeding off of a backlog with a finite size\n(has wip limits itself). This is not an M/M/1/inf. queue. This is an w/x/y/z queue.\nImportant to notice that here we are getting optimum performance at peak utilization.\nThis is at odds with the statement that “full utilization creates waste”.\n
  • This is true only for a queue with WIP limits, feeding off of a backlog with a finite size\n(has wip limits itself). This is not an M/M/1/inf. queue. This is an w/x/y/z queue.\nImportant to notice that here we are getting optimum performance at peak utilization.\nThis is at odds with the statement that “full utilization creates waste”.\n
  • This is true only for a queue with WIP limits, feeding off of a backlog with a finite size\n(has wip limits itself). This is not an M/M/1/inf. queue. This is an w/x/y/z queue.\nImportant to notice that here we are getting optimum performance at peak utilization.\nThis is at odds with the statement that “full utilization creates waste”.\n
  • This is true only for a queue with WIP limits, feeding off of a backlog with a finite size\n(has wip limits itself). This is not an M/M/1/inf. queue. This is an w/x/y/z queue.\nImportant to notice that here we are getting optimum performance at peak utilization.\nThis is at odds with the statement that “full utilization creates waste”.\n
  • This is true only for a queue with WIP limits, feeding off of a backlog with a finite size\n(has wip limits itself). This is not an M/M/1/inf. queue. This is an w/x/y/z queue.\nImportant to notice that here we are getting optimum performance at peak utilization.\nThis is at odds with the statement that “full utilization creates waste”.\n
  • This is true only for a queue with WIP limits, feeding off of a backlog with a finite size\n(has wip limits itself). This is not an M/M/1/inf. queue. This is an w/x/y/z queue.\nImportant to notice that here we are getting optimum performance at peak utilization.\nThis is at odds with the statement that “full utilization creates waste”.\n
  • This is true only for a queue with WIP limits, feeding off of a backlog with a finite size\n(has wip limits itself). This is not an M/M/1/inf. queue. This is an w/x/y/z queue.\nImportant to notice that here we are getting optimum performance at peak utilization.\nThis is at odds with the statement that “full utilization creates waste”.\n
  • This is true only for a queue with WIP limits, feeding off of a backlog with a finite size\n(has wip limits itself). This is not an M/M/1/inf. queue. This is an w/x/y/z queue.\nImportant to notice that here we are getting optimum performance at peak utilization.\nThis is at odds with the statement that “full utilization creates waste”.\n
  • \n
  • \n
  • overhead: queues need to be managed, status needs to be reported...\n\n
  • \n
  • Each point represents an experiment with 500 working days. How much value was generated.\nIf you don’t operate on this plateau you are wasting money\nWhat determines the width? Cost of delay.\nWhy isn’t this sensitive to changes in WIP limits in the testing queue?\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • If we keep everything constant, but do a what-if on both scenarios we can get some data which can support the argument one way or the other\n
  • \n
  • \n
  • \n
  • Keep in mind that this applies to a particular scenario, where there are two working queues,\nthe first queue has 5 workers with an an average daily capacity of 5 work units, a WIP limit of 10 tasks, the second queue has 2 workers with the same average capacity, a WIP limit of 5. Tasks have no variability in size.\nNote that interestingly enough, this does not take into account many of the effects of large stories,\nsuch as reduced estimate accuracy, reduced quality. The main driver here is that it takes more\ntime to get value out, so the value decays more.\n
  • Keep in mind that this applies to a particular scenario, where there are two working queues,\nthe first queue has 5 workers with an an average daily capacity of 5 work units, a WIP limit of 10 tasks, the second queue has 2 workers with the same average capacity, a WIP limit of 5. Tasks have no variability in size.\nNote that interestingly enough, this does not take into account many of the effects of large stories,\nsuch as reduced estimate accuracy, reduced quality. The main driver here is that it takes more\ntime to get value out, so the value decays more.\n
  • Similar analysis, but with a fine-grained story size, with stdev included\nFor example, short stories. This is something I believe in, and always felt it was hard to convince.\nIn this chart every point represents 2 years’ worth of a team process.\nSame process, with only one thing changing - the average size of the story.\nYou can see in red the cycle time and its standard deviation.\nIn blue you can see total value delivered over the life time of the project.\nYou can see how significant the impact is.\nOne thing that surprised me is the complexity of the result. Out of a small set of simple rules, you get such a complex outcome.\nYou can see that there’s a sweet spot around one day. This is where you want to be. Again, this is true only for the particular process I described. you can see that with a smaller stories average size, you hurt the \n
  • \n
  • From that chart, we can see how complexity emerges\nfrom the combination of a set of fairly simple rules.\nComplex systems exhibit unpredictable response to changes:\n - Small change can have a big impact\n - Change will sometimes be unpredicted\nDoes not mean that there is an inconsistent relationship between cause and effect.\nThis is where simulation comes to the rescue - we can gain more insights into the relationship\n
  • One interesting analogy to look at is the one of an aircraft and its crew\n\nand in that world getting a handle of complexity is crucial as the price of making mistakes can be quite high,\nAnd an airplane and its crew form a complex system, where small changes can have big impacts, and not necessarily in the anticipated direction.\nSo simulations are used to provide crews with a safe environment where you can experiment and take the system to its extreme in a cheaper, safer manner, you can pause, replay, fast-forward\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Software development operations are complex systems as well which exhibit similar attributes\nDon’t look as intimidating as a crew in an airliner’s cockpit, with the shiny uniforms and golden badges, but it doesn’t mean that they are different.\nBut somehow simulations are rarely used if at all as part of training of team members and management, and this means that people need to learn from\ncostly mistakes, or from books, and my experience is that the majority of practitioners don’t read books about process, and even when they\ndo, what they learn is sometimes so different from common sense and the existing dogma, that they are likely to face resistance in implementing them.\nSimulations can be a big help there, and I think there’s a lot more than can be done in using those as a tool in that space.\n
  • a shout out to GetKanban which has been an inspiration in creating the simulations\n
  • \n
  • \n
  • Productivity is the holy grail of the metrics & Impossible to measure in real teams.\nPossible to measure in a simulation in a fairly straightforward way: \nThe user story estimates are correlated to value (assuming a smart PM).\nSo let’s measure the amount of value delivered in any time, and measure the time invested by the team. The ratio between them is productivity, or efficiency.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • You probably know that in physical systems, you cannot observe a quantity without affecting another (e.g. the heisenberg uncertainty principle).\nIn software systems we see the same behavior, and it is far from being negligible.\nIf you start measuring something in a real team, it will change the behavior of the team.\nIn a simulation, you are god a creator, and if you program this out, it won’t happen.\n
  • \n
  • Applied to the problem discussed before (dev/qa mix), we can measure productivity (expressed in % here) and keep a straight face.\nThere is no cheating here since we haven’t programmed it it.\nThe insights should port pretty well into reality.\n
  • \n
  • Sometimes we are thinking about investing in quality\nExamples are testing automation, which will slow us down in the short term,\nbut will help us in the long term. It’s always very hard to make the call, but if we simulate to forecast, we can see what is going to be the break-even point...\n5 developers, 2 testers\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Transcript of "Simulating Software Teams "

    1. 1. Simulating Soft ware Teams Sagi Smolarski
    2. 2. Purpose of This Presentation
    3. 3. Purpose of This PresentationLearn how to use process simulations to...
    4. 4. Purpose of This PresentationLearn how to use process simulations to... Gain insights into soft ware development process
    5. 5. Purpose of This PresentationLearn how to use process simulations to... Gain insights into soft ware development process Make better decisions about soft ware process
    6. 6. Purpose of This PresentationLearn how to use process simulations to... Gain insights into soft ware development process Make better decisions about soft ware process Convince ourselves and others
    7. 7. Purpose of This PresentationLearn how to use process simulations to... Gain insights into soft ware development process Make better decisions about soft ware process Convince ourselves and others Bend the laws of physics
    8. 8. Perspectives on a soft ware development operationThere are many ways to look at a soft waredevelopment operation
    9. 9. Perspectiveson a soft ware development operation This is one way
    10. 10. Perspectiveson a soft ware development operation Here we are talking about this perspective
    11. 11. Kanban Board In Kanban this Machine is visualized on a boardReady (5) Development (4) Testing (2) Done
    12. 12. Questions What are the right WIP limits? (*) What is the optimal User Story Size? Hire a developer or a tester? Invest time in code inspections? testing automation? How can I measure Productivity?(*) - WIP = Work in Process. WIP Limits - Limits we impose on the size of queues in order to improve efficiency
    13. 13. Visual Simulations
    14. 14. Visual Simulations Visual Mode = • validate model • gain insights
    15. 15. Batch Simulations
    16. 16. Batch Simulations
    17. 17. Batch Simulations
    18. 18. Batch Simulations
    19. 19. Batch Simulations Batch Mode = • Simulate many options • Answer a specific question
    20. 20. Simulation RulesSimple queue transition mechanics Pull to WIP Limit Process to capacitySome rules from FLOW
    21. 21. Example of such a rule Value Decay Model value 10.5 The later a request is delivered, The less value it brings time Value half-life
    22. 22. Example of such a rule Value Decay Model value 10.5 The later a request is delivered, The less value it brings time Value half-life
    23. 23. Example of such a rule Value Decay Model value 10.5 The later a request is delivered, The less value it brings λ time Value half-life
    24. 24. Example of such a rule Value Decay Model value 10.5 The later a request is delivered, The less value it brings λ time Value half-life
    25. 25. Example of such a rule Value Decay Model value 10.5 The later a request is delivered, The less value it brings λ time Value half-life
    26. 26. Question #1 What is our Project’sPerformance ?
    27. 27. What is Our Performance? Looking at a project in progress...
    28. 28. What is Our Performance? Weekly Profit / Loss
    29. 29. What is Our Performance? How do We Measure Performance? Profit/Loss is great... But... trailing indicator (after the fact) Takes into account many more than the efficiency of the soft ware development operation (Sales & Marketing etc.) Other “gauges” need to be used
    30. 30. Cumulative Flow Ready Development Testing DoneAt any point in time, how many items are in each stage (queue)
    31. 31. Cycle Time On day 47, a work item was delivered 43 days after it was requested This work item wasrequested on day 40 & delivered on day 65. Cycle time is 25
    32. 32. Lesson #1 It’s hard to know your performance when youdon’t see it. Use cycle time and cumulative flow to control your process
    33. 33. Question #2What Are the Right WIP Limits?
    34. 34. INCREASING WIP Limit
    35. 35. INCREASING WIP Limit Cycle Time
    36. 36. INCREASING WIP Limit Cycle Time Profit
    37. 37. INCREASING WIP Limit Cycle Time Profit Team members’ Utilization
    38. 38. INCREASING WIP Limit Cycle Time Profit Sweet Team members’ Spot Utilization
    39. 39. INCREASING WIP Limit Cycle Time Cycle Time Change Lags Profit Sweet Team members’ Spot Utilization
    40. 40. Some Insights From This Simple Case
    41. 41. Some Insights From This Simple Case1. Top financial performance coincides with reaching top utilization of the team
    42. 42. Some Insights From This Simple Case1. Top financial performance coincides with reaching top utilization of the team2. Top financial performance coincides with a low cycle time
    43. 43. Some Insights From This Simple Case1. Top financial performance coincides with reaching top utilization of the team2. Top financial performance coincides with a low cycle time3. The longer the queue, the longer the system will take to respond to a change in WIP limits
    44. 44. Single Working Queue453015 Cycle Time0 0 20 40 60 80 WIP Limit
    45. 45. Single Working Queue4530 Little’s Formula15 Cycle Time0 0 20 40 60 80 WIP Limit
    46. 46. Single Working Queue4530 Lifetime Value Little’s Formula15 Cycle Time0 0 20 40 60 80 WIP Limit
    47. 47. Single Working Queue Team Members’ Utilization4530 Lifetime Value Little’s Formula15 Cycle Time0 0 20 40 60 80 WIP Limit
    48. 48. Single Working Queue Team Members’ Utilization4530 Lifetime Value Little’s Formula15 Cycle Time0 0 Optimal 20 40 60 80 WIP WIP Limit
    49. 49. Decision Making
    50. 50. Decision Making In Agile:
    51. 51. Decision Making In Agile: • Frequent inspect & adapt
    52. 52. Decision Making In Agile: • Frequent inspect & adapt • What about long-term decision?
    53. 53. Decision Making In Agile: • Frequent inspect & adapt • What about long-term decision? • What about irreversible decisions?
    54. 54. The CFO has justapproved a new hire for your team
    55. 55. The CFO has just approved a new hire for your team Great! We’ll hire anew developer so wecan crank out more features
    56. 56. The CFO has just approved a new hire for your team Great! We’ll hire anew developer so wecan crank out more features No way! Weneed a new tester in order to improve quality!
    57. 57. Question #3Should I hire adeveloper or a tester?
    58. 58. Adding a Team MemberTotal Value Delivered, Simulated Process Value [$1,000] 5 Developers, 3 Testers 5 Developers, 2 Testers 6 Developers, 2 Testers Time [days]
    59. 59. Adding a Team MemberTotal Value Delivered, Simulated Process Value [$1,000] $500K More Value 5 Developers, 3 Testers 5 Developers, 2 Testers 6 Developers, 2 Testers Time [days]
    60. 60. Lesson #3 In some situations, hiring an extraDeveloper will reduce value delivered
    61. 61. Question #4Small User Stories are good...How small is small enough?
    62. 62. User Story SizeValue Delivered $1,125,000 $750,000 $375,000 $0 ($375,000) ($750,000)($1,125,000)($1,500,000) Story size 1 2 3 4 5 6 7 [days]
    63. 63. e User-Story Size High Resolution Simulation Value Cycle Time Cycle Time Story Size (total estimated effort in days) High Cycle Time = Noisy Process
    64. 64. Lesson #4 When value delivereddecays with time, small batches (user stories) increase value
    65. 65. From Simple to Complex From the previous chart, we can see that a combination of simple rules, yields bewilderingly complex behavior + =
    66. 66. Aircraft & Crew An interesting analogy of how we can deal with complex systems Simulate? Why To Gain Insights Airplane Simulators • Complicated System • Easy to make costly mistakes • Small input changes can have large unintended consequences
    67. 67. FlightSimulation
    68. 68. FlightSimulation• Experiment in a safe environment
    69. 69. FlightSimulation• Experiment in a safe environment• Can bring the system to extremes without risk
    70. 70. FlightSimulation• Experiment in a safe environment• Can bring the system to extremes without risk• Can pause, fast for ward to interesting parts
    71. 71. FlightSimulation• Experiment in a safe environment• Can bring the system to extremes without risk• Can pause, fast for ward to interesting parts• Track & Analyze during and after
    72. 72. FlightSimulation• Experiment in a safe environment• Can bring the system to extremes without risk• Can pause, fast for ward to interesting parts• Track & Analyze during and after• Cheaper to operate than the real thing
    73. 73. Soft ware DevelopmentOrganizations • Complicated System • Easy to make costly mistakes • Small input changes can have large unintended consequences
    74. 74. Playing to Learn Example: GetKanban.com - Looking at the micro-mechanics of Kanban Here we are looking at the macro-mechanics of flow in product development Looking at the 10,000’ level Kanban trainings & Other settings
    75. 75. Lesson #5Simulations are a cost- effective way to get insights into the operation of complex systems
    76. 76. The Holy Grail of metrics... Often sought... Seldom reliably measuredProductivity
    77. 77. The Holy Grail of metrics... Often sought... Seldom reliably measuredProductivity = Output Input
    78. 78. Productivity Measurement
    79. 79. Productivity MeasurementBest measure Value Delivered and maximize it
    80. 80. Productivity MeasurementBest measure Value Delivered and maximize itHolistic - Includes Product Management,Operations, Sales, Development, Testing...
    81. 81. Productivity MeasurementBest measure Value Delivered and maximize itHolistic - Includes Product Management,Operations, Sales, Development, Testing...Bottom-line
    82. 82. Productivity MeasurementBest measure Value Delivered and maximize itHolistic - Includes Product Management,Operations, Sales, Development, Testing...Bottom-lineBest correlation to what business wants -“It’s all about Benjamins, baby!”
    83. 83. Productivity Measurement Downside of Using Bottom-line Value
    84. 84. Productivity Measurement Downside of Using Bottom-line ValueHolistic - Measures dev + qa + operations +support + PM + Sales - We usually want tomeasure development operation (incl. testing)separately
    85. 85. Productivity Measurement Downside of Using Bottom-line ValueHolistic - Measures dev + qa + operations +support + PM + Sales - We usually want tomeasure development operation (incl. testing)separatelyValue could be derived from solutions whichincorporate multiple products, which requirearbitrary attribution of value to products
    86. 86. Productivity Measurement Alternative to Bottom-line Value
    87. 87. Productivity Measurement Alternative to Bottom-line Value Can measure value-add hours vs. paid hours
    88. 88. Productivity Measurement Alternative to Bottom-line Value Can measure value-add hours vs. paid hours Delta is Waste:
    89. 89. Productivity Measurement Alternative to Bottom-line Value Can measure value-add hours vs. paid hours Delta is Waste: Time spent working on defects
    90. 90. Productivity Measurement Alternative to Bottom-line Value Can measure value-add hours vs. paid hours Delta is Waste: Time spent working on defects Time spent on items that are still in progress
    91. 91. Productivity Measurement Alternative to Bottom-line Value Can measure value-add hours vs. paid hours Delta is Waste: Time spent working on defects Time spent on items that are still in progress Idle time
    92. 92. Productivity Measurement Alternative to Bottom-line Value Can measure value-add hours vs. paid hours Delta is Waste: Time spent working on defects Time spent on items that are still in progress Idle time Manual regression testing
    93. 93. Productivity Measurement Alternative to Bottom-line Value Can measure value-add hours vs. paid hours Delta is Waste: Time spent working on defects Time spent on items that are still in progress Idle time Manual regression testing Process overhead
    94. 94. Productivity Measurement Alternative to Bottom-line Value Can measure value-add hours vs. paid hours Delta is Waste: Time spent working on defects Time spent on items that are still in progress Idle time Manual regression testing Process overhead Support interruptions
    95. 95. Productivity Measurement Alternative to Bottom-line Value Can measure value-add hours vs. paid hours Delta is Waste: Time spent working on defects Time spent on items that are still in progress Idle time Manual regression testing Process overhead Support interruptions Deployment
    96. 96. Productivity Measurement
    97. 97. Productivity MeasurementExample:
    98. 98. Productivity MeasurementExample:5 developers, 2 testers, working for 100 days
    99. 99. Productivity MeasurementExample:5 developers, 2 testers, working for 100 daysInput = 100 * ( 5+2) = 700
    100. 100. Productivity MeasurementExample:5 developers, 2 testers, working for 100 daysInput = 100 * ( 5+2) = 700delivered 70 features whose estimated effortwas an average of 6 days
    101. 101. Productivity MeasurementExample:5 developers, 2 testers, working for 100 daysInput = 100 * ( 5+2) = 700delivered 70 features whose estimated effortwas an average of 6 daysValue = 70*6 = 420
    102. 102. Productivity MeasurementExample:5 developers, 2 testers, working for 100 daysInput = 100 * ( 5+2) = 700delivered 70 features whose estimated effortwas an average of 6 daysValue = 70*6 = 420Productivity = 420/700 = 70%
    103. 103. The observation itself affects the system under observation
    104. 104. The observation itself affects the system under observation This is true insoft ware teams as well...
    105. 105. Measuring ProductivityWould this measurement work? In-situ (real teams) In-vitro (Simulations)Will be gamed? Absolutely Bits don’t cheatDemoralizing? Probably Bits don’t care Yes - Eminently Yes - Insights should Applicable? relevant metric port well to real life
    106. 106. Team Composition & Productivity Productivity Cycle time (avg, stdev) Productivity = Efficiency: Can be measured just fine in a simulation
    107. 107. Question #6Is it cost-effective to invest in qualitypractices?
    108. 108. Should We Invest inAutomated Testing? Value [$1,000] 10% ongoing development effort = save 2 days duration to perform full regression testing, and keep this saving for ever No investment in automation = 1 more day to regression every quarter10% Ongoing Dev Effort on Automated Regression Testing Time [days]
    109. 109. Should We Invest inAutomated Testing? Value [$1,000] $300K More Value Generated 10% ongoing development effort = save 2 days duration to perform full regression testing, and keep this saving for ever No investment in automation = 1 more day to regression every quarter10% Ongoing Dev Effort on Automated Regression Testing Time [days]
    110. 110. Best Practices In Creating Simulations
    111. 111. Best Practices In Creating Simulations Use real historical data as a baseline
    112. 112. Best Practices In Creating Simulations Use real historical data as a baseline When real data not available, use industry data (e.g. Capers Jones)
    113. 113. Best Practices In Creating Simulations Use real historical data as a baseline When real data not available, use industry data (e.g. Capers Jones) Validate model using visual simulation
    114. 114. Best Practices In Creating Simulations Use real historical data as a baseline When real data not available, use industry data (e.g. Capers Jones) Validate model using visual simulation Keep it simple (Low fidelity is often enough)
    115. 115. Best Practices In Creating Simulations Use real historical data as a baseline When real data not available, use industry data (e.g. Capers Jones) Validate model using visual simulation Keep it simple (Low fidelity is often enough) Translate results into $$$
    116. 116. Summary
    117. 117. SummarySimulations are a great addition to our toolset
    118. 118. SummarySimulations are a great addition to our toolset To gain insight into complex processes
    119. 119. SummarySimulations are a great addition to our toolset To gain insight into complex processes To make better decisions
    120. 120. SummarySimulations are a great addition to our toolset To gain insight into complex processes To make better decisions To convince (yourself / others)
    121. 121. SummarySimulations are a great addition to our toolset To gain insight into complex processes To make better decisions To convince (yourself / others) To bend the laws of physics
    122. 122. SummarySimulations are a great addition to our toolset To gain insight into complex processes To make better decisions To convince (yourself / others) To bend the laws of physics To forecast
    123. 123. Key Take AwaysUnderstand that there is a machine withinyour team’s processRealize it’s a complex machine, who has asignificant impact on the outcomeSimulations are but one of many tools thathelp us thereShould not be the main consideration in makingdecisions - People considerations are moreimportant, but can provide a great deal of help
    124. 124. More InformationCheck it out at: http://flower.agilesparks.comTalk to us: sagi@agilesparks.com yuval@agilesparks.com @FLOWerSimulator
    125. 125. More InformationCheck it out at: http://flower.agilesparks.comTalk to us: sagi@agilesparks.com yuval@agilesparks.com @FLOWerSimulator

    ×