Software Development Project Estimation
Oleksandr Katrusha
Senior Engineering Manager
oleksandr.Katrusha@Innovecs.com
“Prediction is very difficult, especially about the future.”
Niels Bohr, Danish physicist
2
Intro
• Estimated length is 60 minutes
• Feel free to ask questions!
• Terminology
• Software Development (Waterfall, my experience)
• Project
• Estimate – number, document
• Effort
• Estimation – process
3
When?
• RFP response / tender bid
• Make go/no-go decisions with a project
• Cost and ROI (return on Investment)
• Plan new phase
• Plan resources / budget
• Calendar-driven
4
Why?
• Define scope and assumptions
• Know price and/or cost
• Justify the price / budget, not to estimate!
• Sell more, sell better
• Input for other processes
• Sales
• Scheduling
• Hiring
• Budgeting
• Correct estimate is crucial for project success!
5
Structure
• Scope. “What?”
• Riskier than underestimation!
• In / out
• Client sees not what you see
• Assumptions and risks. Conditions
• Versions, separation of duties etc..
• Numbers. “How much?”
• Per task in WBS
• Buffers
• Any number is a commitment!
• Wrapping
Experiment - Flower merchant
6
• Have kiosk in Kyiv, ~100 types of flowers
• Ignorant in IT
• 50 to 150 orders per day, private and corporate, wholesale and retail
• Payment in credit card or cash
• Now tracking in Excel - inconvenient, prone to errors and cheating
Need to:
• Track order status (placed, purchased, prepared, delivered, sold)
• Integrate with payment gateway
• In order not to forget delivery and collect payments
• Shelf product does not work, technology does not matter
• How much effort will it take to develop and deploy the
program (in man-hours)?
https://goo.gl/5CCsDs
7
Remember
• Estimation mistakes are very expensive
• The number of (negative) risks is unlimited
• Parkinson's law: ”Work expands so as to fill
the time available for its completion“
• Do not overestimate
• Raise issues and arguments as early as
possible
• Scope creep kills you quick
• And starts from estimate…
8
Prerequisites
• Enough time and other resources
• Know stakeholders and POC
• Read SOW (Statement of Work, contract)
• Deliverables and conditions
• Sales / delivery handover
• Do they know what you develop?
• Do you know what they sell?
• Different KPIs
• Participate in sales process
9
Scope
Unknow
knows
Known
Unknowns
Unknown
Unknowns
Known
knowns
Estimate
Discover
and
estimate
Overestimate
Assume
OR
Put out of scope
Add buffer
10
Define scope
• Collect requirements
• Confirm requirements
• At least by default
• Explicitly state what is out of scope / not estimated
• Or estimate if not sure
• Make it large first (even ridiculous)
• Formal, informal, missing
• Email, talks, references
• Expectations
• Functional
• NFRs
• Legacy artifacts – not only code
• Risky items (not risks)
• “Do it for all forms in the system”
• Captions case
11
Non-functional requirements
• Performance
o Response Time
o Throughput
o …
• Scalability
• Capacity
• Availability
• Reliability
• Recoverability
• Maintainability
• Serviceability
• Security
• Regulatory
(GDPR)
• Manageability
• Environmental
• Data Integrity
• Usability
• Interoperability
12
Create WBS (Work Breakdown structure)
• Types
• Verb – oriented
• Time-phased
• Noun-oriented (features)
• 40 hrs per task max
• 100% of scope
• Non-project activities not included
13
Assumptions
• DoD (Definition of Done)
• Estimation’s “Best before”
• Versions
• Quality bar
• Who does what, support,
subcontractors
• Access to systems (case)
• Hardware requirements &
performance
• Legacy
• Licensing
• Known Unknowns
• Database size (Tb case)
• Integrations
• Legacy code
• …
• Any doubts, comments,
contradictions
• Anything that can be interpreted
wrongly (Tech upgrade case)
14
Units
• Effort vs Duration
• Do you know you team?
• Other projects / tasks
• Man-hours, -days, -weeks
• Billing, time-tracking
• Schedule
• Part-timers, calendars
• Working hours vs effective hours?
• How do you sell it?
• Scheduling /8 /40 /160
• 2 + 2 + 2 = 6 = 8
15
Executors
• Estimate for middle specialist
• Do you know your team?
• Do they have expertise?
• Add to assumptions
• Blends may vary in productivity
by ~50% !
S
M
J
M
J J
S
M
J
M
J
M
1.
2.
16
Who should estimate?
• Trained seniors
• With failure experience 
• But must feel safe
• Parkinson's law
• Cognitive biases
• Planning fallacy
• Probability
• Anchoring
17
Estimate!
 Per WBS item
• Development
• BA/SA ~70% of development
• QA ~50%
• Task contingency 5 - 30% of task time
 Outside of WBS
• Meetings ~20%
• Management ~15%
• DevOps ~5%
 Project buffer 5 - 15% of project time
Tips
• Use powers of two: 0.5, 1, 2, 4, 8, 16, (24), 32, (40)
• Estimate all categories if you can
• Optimistic in small, pessimistic in large
% may vary depending on
domain,
technology,
phase etc.
Sample
estimate,
man-hours
18
19
What is development?
• Read, understand and discuss task
• Code
• Build / Deploy
• Dev test
• Design and discuss solution
• Research and prototype
• Set up IDE, synch with VCS
• Unit test
• Create test data
• Create test scenario
• Fix bugs after dev test
• Check for rules compliance
• Commit / Review
• Fix after review
• Retest (!)
• Commit
• Fix bugs after QA
• Document
• Log status and time
20
Formulae
• Task effort is a random variable with its distribution!
• Three-point estimation
• a - the best case (Optimistic)
• m - the most likely (Realistic)
• b - the worst case (Pessimistic)
• Best case - average amount if task is performed 100 times
• Most likely case given normal problems and opportunities
• Worst case - everything goes wrong
• Weighted average (Expected time in PERT) E = (a + 4m + b) / 6
• Triangular E = (a + m + b) / 3
• Standard deviation S = (b – a) / 6
21
O – R – P
hrs #
1 0 0
2 0 0
3 1 3
4 3 12
5 8 40
6 5 30
7 3 21
8 2 16
9 0 0
10 2 20
11 0 0
12 1 12
25 154 6.16
https://goo.gl/ZUwwVa
What is the “most likely” estimate?
What is the most probable?
22
Expected value
23
Pros and Cons of 3 points
• Takes time and effort
• Still ambiguous what is “Good, Bad, Real”
• Allows to calculate standard deviation
• Better use for risky tasks with high level of uncertainty
• More or less same results with Realistic + Contingency
• Less time
 Agree on what you all mean by “Realistic”
24
Central limit theorem
When independent random variables are added
their properly normalized sum tends toward a normal distribution
even if the original variables themselves are not normally distributed
20–30 tasks for this to be reasonable, and
each of the estimates unbiased
• Tasks are dependent…
• Drucker's Law: "If one thing goes
wrong, everything else will, and
at the same time”
• 6 sigma
 Add (hide) project buffer!
25
Tips
• Be pessimistic first
• “What could go wrong” exercise?
• Use templates, checklists, samples, historical data
• Agree about buffers
• Beware of reverse engineering estimate
• You have no DoD
• Complexity is infinite
26
Checklist – requirements and management
Check if you have not forgotten:
• Planning, time tracking and budget control
• Reporting
• Clarifying and updating requirements and design
• Translation to/from English
• Onboarding and education
• Any system-wide features
• Demo data and environments, demo meetings
• Delivery preparation
• Design update after changes to code
• The actual time spent of estimation (case)
27
Checklist - development
• Post-delivery support and warranty
• Check SOW
• Quality bar
• Code quality
• Unit tests
• Certification
• Environment maintenance and outage
• Receiving access
• Bugs in standard and 3rd-party products
• Backlog from previous phases / projects
28
Checklist - development
• Data management and upgrade
• Security development
• Code review, refactoring and retest
• Performance test and tuning
• Integration with parallel teams or third parties
• Interim upgrades
• Risks of new technologies and frameworks
29
Checklist - test
• Test documentation
• Performance test
• Integration test
• Regression test
• SP / hotfix test
• Test / Demo data setup
• …
30
Wrapping
• Options
• Visible scope and assumptions
• Show / hide details
• Buffers / bug fixing… - client specific
 Elite
 Business
 Economy
31
Recap
Define Scope
Define Assumptions and Risks
Create WBS
Estimate Development per task
Define Contingency per task
Add BA, Test per task
Add Management, Meeting, DevOps, …
Add Project buffer
Check
Pack and sell
32
questions

Software development project estimation

  • 1.
    Software Development ProjectEstimation Oleksandr Katrusha Senior Engineering Manager oleksandr.Katrusha@Innovecs.com “Prediction is very difficult, especially about the future.” Niels Bohr, Danish physicist
  • 2.
    2 Intro • Estimated lengthis 60 minutes • Feel free to ask questions! • Terminology • Software Development (Waterfall, my experience) • Project • Estimate – number, document • Effort • Estimation – process
  • 3.
    3 When? • RFP response/ tender bid • Make go/no-go decisions with a project • Cost and ROI (return on Investment) • Plan new phase • Plan resources / budget • Calendar-driven
  • 4.
    4 Why? • Define scopeand assumptions • Know price and/or cost • Justify the price / budget, not to estimate! • Sell more, sell better • Input for other processes • Sales • Scheduling • Hiring • Budgeting • Correct estimate is crucial for project success!
  • 5.
    5 Structure • Scope. “What?” •Riskier than underestimation! • In / out • Client sees not what you see • Assumptions and risks. Conditions • Versions, separation of duties etc.. • Numbers. “How much?” • Per task in WBS • Buffers • Any number is a commitment! • Wrapping
  • 6.
    Experiment - Flowermerchant 6 • Have kiosk in Kyiv, ~100 types of flowers • Ignorant in IT • 50 to 150 orders per day, private and corporate, wholesale and retail • Payment in credit card or cash • Now tracking in Excel - inconvenient, prone to errors and cheating Need to: • Track order status (placed, purchased, prepared, delivered, sold) • Integrate with payment gateway • In order not to forget delivery and collect payments • Shelf product does not work, technology does not matter • How much effort will it take to develop and deploy the program (in man-hours)? https://goo.gl/5CCsDs
  • 7.
    7 Remember • Estimation mistakesare very expensive • The number of (negative) risks is unlimited • Parkinson's law: ”Work expands so as to fill the time available for its completion“ • Do not overestimate • Raise issues and arguments as early as possible • Scope creep kills you quick • And starts from estimate…
  • 8.
    8 Prerequisites • Enough timeand other resources • Know stakeholders and POC • Read SOW (Statement of Work, contract) • Deliverables and conditions • Sales / delivery handover • Do they know what you develop? • Do you know what they sell? • Different KPIs • Participate in sales process
  • 9.
  • 10.
    10 Define scope • Collectrequirements • Confirm requirements • At least by default • Explicitly state what is out of scope / not estimated • Or estimate if not sure • Make it large first (even ridiculous) • Formal, informal, missing • Email, talks, references • Expectations • Functional • NFRs • Legacy artifacts – not only code • Risky items (not risks) • “Do it for all forms in the system” • Captions case
  • 11.
    11 Non-functional requirements • Performance oResponse Time o Throughput o … • Scalability • Capacity • Availability • Reliability • Recoverability • Maintainability • Serviceability • Security • Regulatory (GDPR) • Manageability • Environmental • Data Integrity • Usability • Interoperability
  • 12.
    12 Create WBS (WorkBreakdown structure) • Types • Verb – oriented • Time-phased • Noun-oriented (features) • 40 hrs per task max • 100% of scope • Non-project activities not included
  • 13.
    13 Assumptions • DoD (Definitionof Done) • Estimation’s “Best before” • Versions • Quality bar • Who does what, support, subcontractors • Access to systems (case) • Hardware requirements & performance • Legacy • Licensing • Known Unknowns • Database size (Tb case) • Integrations • Legacy code • … • Any doubts, comments, contradictions • Anything that can be interpreted wrongly (Tech upgrade case)
  • 14.
    14 Units • Effort vsDuration • Do you know you team? • Other projects / tasks • Man-hours, -days, -weeks • Billing, time-tracking • Schedule • Part-timers, calendars • Working hours vs effective hours? • How do you sell it? • Scheduling /8 /40 /160 • 2 + 2 + 2 = 6 = 8
  • 15.
    15 Executors • Estimate formiddle specialist • Do you know your team? • Do they have expertise? • Add to assumptions • Blends may vary in productivity by ~50% ! S M J M J J S M J M J M 1. 2.
  • 16.
    16 Who should estimate? •Trained seniors • With failure experience  • But must feel safe • Parkinson's law • Cognitive biases • Planning fallacy • Probability • Anchoring
  • 17.
    17 Estimate!  Per WBSitem • Development • BA/SA ~70% of development • QA ~50% • Task contingency 5 - 30% of task time  Outside of WBS • Meetings ~20% • Management ~15% • DevOps ~5%  Project buffer 5 - 15% of project time Tips • Use powers of two: 0.5, 1, 2, 4, 8, 16, (24), 32, (40) • Estimate all categories if you can • Optimistic in small, pessimistic in large % may vary depending on domain, technology, phase etc.
  • 18.
  • 19.
    19 What is development? •Read, understand and discuss task • Code • Build / Deploy • Dev test • Design and discuss solution • Research and prototype • Set up IDE, synch with VCS • Unit test • Create test data • Create test scenario • Fix bugs after dev test • Check for rules compliance • Commit / Review • Fix after review • Retest (!) • Commit • Fix bugs after QA • Document • Log status and time
  • 20.
    20 Formulae • Task effortis a random variable with its distribution! • Three-point estimation • a - the best case (Optimistic) • m - the most likely (Realistic) • b - the worst case (Pessimistic) • Best case - average amount if task is performed 100 times • Most likely case given normal problems and opportunities • Worst case - everything goes wrong • Weighted average (Expected time in PERT) E = (a + 4m + b) / 6 • Triangular E = (a + m + b) / 3 • Standard deviation S = (b – a) / 6
  • 21.
    21 O – R– P hrs # 1 0 0 2 0 0 3 1 3 4 3 12 5 8 40 6 5 30 7 3 21 8 2 16 9 0 0 10 2 20 11 0 0 12 1 12 25 154 6.16 https://goo.gl/ZUwwVa What is the “most likely” estimate? What is the most probable?
  • 22.
  • 23.
    23 Pros and Consof 3 points • Takes time and effort • Still ambiguous what is “Good, Bad, Real” • Allows to calculate standard deviation • Better use for risky tasks with high level of uncertainty • More or less same results with Realistic + Contingency • Less time  Agree on what you all mean by “Realistic”
  • 24.
    24 Central limit theorem Whenindependent random variables are added their properly normalized sum tends toward a normal distribution even if the original variables themselves are not normally distributed 20–30 tasks for this to be reasonable, and each of the estimates unbiased • Tasks are dependent… • Drucker's Law: "If one thing goes wrong, everything else will, and at the same time” • 6 sigma  Add (hide) project buffer!
  • 25.
    25 Tips • Be pessimisticfirst • “What could go wrong” exercise? • Use templates, checklists, samples, historical data • Agree about buffers • Beware of reverse engineering estimate • You have no DoD • Complexity is infinite
  • 26.
    26 Checklist – requirementsand management Check if you have not forgotten: • Planning, time tracking and budget control • Reporting • Clarifying and updating requirements and design • Translation to/from English • Onboarding and education • Any system-wide features • Demo data and environments, demo meetings • Delivery preparation • Design update after changes to code • The actual time spent of estimation (case)
  • 27.
    27 Checklist - development •Post-delivery support and warranty • Check SOW • Quality bar • Code quality • Unit tests • Certification • Environment maintenance and outage • Receiving access • Bugs in standard and 3rd-party products • Backlog from previous phases / projects
  • 28.
    28 Checklist - development •Data management and upgrade • Security development • Code review, refactoring and retest • Performance test and tuning • Integration with parallel teams or third parties • Interim upgrades • Risks of new technologies and frameworks
  • 29.
    29 Checklist - test •Test documentation • Performance test • Integration test • Regression test • SP / hotfix test • Test / Demo data setup • …
  • 30.
    30 Wrapping • Options • Visiblescope and assumptions • Show / hide details • Buffers / bug fixing… - client specific  Elite  Business  Economy
  • 31.
    31 Recap Define Scope Define Assumptionsand Risks Create WBS Estimate Development per task Define Contingency per task Add BA, Test per task Add Management, Meeting, DevOps, … Add Project buffer Check Pack and sell
  • 32.