This document discusses forecasting techniques for software projects. It introduces Little's Law which relates the number of customers in a system to the arrival rate and time spent in the system. It notes that traditional "agile" forecasting is heuristic while stochastic forecasting uses statistical models based on historic data. Building an accurate stochastic model requires accounting for factors like work in progress, cycle time, bugs, and scope changes. The model is used for Monte Carlo simulations to generate forecasts, with the 85th percentile often quoted as the forecast range. Reducing variance through stable processes improves forecast accuracy.

1. Brett Ansley
Forecasting with
confidence

2. How I got here

3. How I got here

4. Little’s L𝜆w
L=𝜆W

5. Little’s L𝜆w
L=𝜆W
No. of Customers in system = Arrival Rate * Time spent in system

6. Little’s L𝜆w
L=𝜆W
No. of Customers in system = Arrival Rate * Time spent in system
Cycle Time = WIP / Throughput

7. Little’s L𝜆w
L=𝜆W
No. of Customers in system = Arrival Rate * Time spent in system
Cycle Time = WIP / Throughput
ASSUMPTIONS
• All measurement units are consistent
• Conservation of Flow which assumes
✴Average Arrival Rate == Average Departure Rate
✴All work that enters the system flows through to completion and exits
• System is “stable”
✴The average age of WIP is neither increasing or decreasing
✴The total amount of WIP is roughly the same at the beginning and at the end

8. Little’s L𝜆w
• Explicitly Record:
• Cycle time
• WIP
• Throughput

9. Cumulative Flow

10. Heuristic forecasting
• Traditional “Agile” forecasting is heuristic
• Points
• Burn-up / burn-down
• “Planning games”

11. Heuristic forecasting
• Fall foul of cognitive biases
• Confirmation bias
• Optimism bias
• Familiarity bias
• Miss small events that don’t happen often but
cumulatively have a big impact

12. Stochastic forecasting
• Using a statistical model to forecast project
delivery
• Model based on historic data

13. Example of moving to
stochastic forecasting

14. Factors
• WIP
• Cycle time
• Bug rate
• Rate of change of scope
• Team size
• Rate of change of average age of stories

15. Complex forecasting
• Needs a stochastic model
• Uses Monte Carlo simulations based on the model

16. Building a model
• Need less data points than you think
• Start simple
• Layer complexity
• Use history to refine

17. Building a model

18. Building a model

19. Building a model

20. Building a model

21. Building a model

23. Validating a model
• Test against historic data
• Run multiple tests
• Keep validating every time you use the model (in
case something has changed)

24. Forecasting with a model
• Monte Carlo simulation
• 85th percentile
• Quote as a range
• Reducing variance improves forecasts

25. It’s all about variance
ASSUMPTIONS
• All measurement units are consistent
• Conservation of Flow which assumes
✴Average Arrival Rate == Average Departure Rate
✴All work that enters the system flows through to completion and exits
• System is “stable”
✴The average age of WIP is neither increasing or decreasing
✴The total amount of WIP is roughly the same at the beginning and at the end

26. It’s all about variance
• Make assumptions explicit
• 85th Percentile
• 85th to 50th percentile split (indicator of variance)
• Variance introduced by instability (changing
average age)

27. Example of moving to
stochastic forecasting

28. Dashboard

29. Impacts on forecast
accuracy
• Instability in the project team
• Excessively large work items
• Artificial ageing

30. Brett Ansley
Agile coach
CPO C-Change
@pbansley
brett@brettansley.co.uk