Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
A Science, Not an Art?
ESTIMATION
2015
Andy Kucharski
2
• CEO & Founder
• Promet Source:
• Drupal Development
& Support
• Chicago, IL.
“The typical software organization is not struggling to improve its estimates from +-10% to +-5%
accuracy. The typical sof...
4
Agenda
What is the Problem with Software Estimation?1
2
3
Avoiding Risk
Estimation Techniques
What is the Problem with Software Estimation?
5
The greatest discrepancy between what the client thinks, an executive thinks, and
a developer thinks is often the definiti...
7
Estimates, Targets, and Commitments
1 Estimates
Prediction
• A Prediction of how
long a project will
take or how much it...
8
Estimates, Targets, and Commitments
2 3
Target
Commitment
Estimate
1
Cut From
Phase
Your best developer is not necessarily your best estimator
9
10
The Solution
Architect
About your unicorn:
• Knows the most about
Drupal
• Experience in
Ecommerce
Migration
Integratio...
What is the definition of a “GOOD” estimate?
11
90% confidence
12
After all, the process is called
Estimation, not Exactimation.
- Phillip Armour
What is the surface temp...
When given the option of a shorter average schedule with higher variability or a
longer average schedule with lower variab...
90% confidence
14
How confident is 90% confident?
The average number of correct
answers was 2.8. Only 2% of quiz
takers sc...
Software is not developed in a vacuum
15
16
EXTERNAL FACTORS ARE OFTEN UNACCOUNTED FOR
LESS EXPERIENCED
STAFF
STAFF NOT READY
REQUIREMENTS
REMOVED
STAFF DIVERTED T...
17
Events that happen during the project nearly always invalidate the assumptions
that were used to estimate the project i...
18
Project
Dynamics
The activities of a late or compressed
project create destructive “late-
project” dynamics that make t...
19
Results
Results
Timelines Budgets
20
“Software does not have a neutral estimation problem. The industry data shows
clearly that the software industry has an un...
22
Avoiding Risk
Major Areas of Risk
23
• What kind of content
type/field/thing
• Scope w/o specs
• Specific definitions
• Setup Costs
• Ma...
Consider your project’s current position within the cone of uncertainty, this will help
you gauge the level of accuracy in...
25
The Cone of
Uncertainty
Initial Product
Concept
“I want a website”
Product
Definition
“I know the scope of
services”
Re...
Leverage the power of well defined products to move your way through the cone of
uncertainty
26
27
Products
Training
Workshops
Drupal Concept Education as
an entry into the product
Discovery Process
Discovery
Technical...
Know when you are in a chaos project and how to reset
28
Chaos Projects
How does BAD go to TERRIBLE?
29
External
Factors
Re-estimation
or Planning
Poor Project
Dynamics
Pure Chaos...
NEVER, let your estimator forget (or your sales team remove) these important
activities
30
Omitted Activities
31
Management
• Stand ups, Contract review, Training, QA review
• Engagement Management, Staffing, Gove...
32
Estimation Techniques
Estimating “Things”
33
Lets assume that you are all
expert estimators and that all the
process and risk areas we have
spok...
34
APPROACHES TO ESTIMATING “THINGS”
The Hobbyist
“I’m guessing
75 people”
The Factor
Guy
“15 rows 5
per row”
The Percenta...
Count, Compute, Judge
35
Count if at all possible.
Compute when you can’t count.
Use Judgement alone only
as a last resort...
• Click to edit Master text styles
Items to Consider
• Size of Project 5-25 people, 25-50, etc.
• Sequential vs. Scrum
• S...
37
Common Items in these areas
1 2 3Count/Calibrate Compute Judge
Site Specs
Content types
Taxonomies
Menus
Views
Wirefram...
38
Count &
Calibrate
1) Determine an item that’s
highly correlated with the
size of the software
2) COUNT how many you
hav...
39
Compute
1) Determine a defined
value to base your
computation
2) Determine a multiplier
that is a statistically
meaning...
40
Judgement
1) JUDGE the rating of a
specific multiplier (factor)
2) Apply multipliers based
on rating of your estimate
3...
Overcoming Judgement
41
Judgement is the most
hazardous kind of estimation
due to subjectivity and
unfounded optimism.
Mag...
Wide Delphi Method
• Coordinator presents each expert with a specification and an estimation form.
• Coordinator calls a g...
Subtitle for slide
Guide for Standardized Estimation Procedure
• Focus on counting and computing when possible, rather tha...
To Learn More...
44
Software Estimation:
Demystifying the Black
Art
Steve McConnell
Thank You
45
CHICAGO, ILLINOIS
1802 W. Berteau Ave. / Suite 209 / Chicago IL, 60613
773-525-8255
D.O / Twitter
@akucharski...
46
47
Estimation Techniques
Calibration and Historical Data
T-Shirt Sizing
Wideband Delphi Technique
Results
Costs
48
Upcoming SlideShare
Loading in …5
×

Estimation - web software development estimation DrupalCon and DrupalCamp presentation

1,243 views

Published on

Project Estimation Presentation at DrupalCamps. Presentation discussing risks, methods and recommendations when estimating software estimation with a focus on web CMS (drupal)

Published in: Business
  • Be the first to comment

Estimation - web software development estimation DrupalCon and DrupalCamp presentation

  1. 1. A Science, Not an Art? ESTIMATION 2015
  2. 2. Andy Kucharski 2 • CEO & Founder • Promet Source: • Drupal Development & Support • Chicago, IL.
  3. 3. “The typical software organization is not struggling to improve its estimates from +-10% to +-5% accuracy. The typical software organization is struggling to avoid estimates that are incorrect by 100% or more.” -- Steve McConnell What is this presentation about? 3 What is the presentation NOT about? • Fundamentals (overlooked) • Big Risk factors • Selling your estimate (Team) • 20%-100% over budget • Specific estimates (content type) • Magical formulas • Fine tuning error from 20% to 5% • Selling your estimate (client)
  4. 4. 4 Agenda What is the Problem with Software Estimation?1 2 3 Avoiding Risk Estimation Techniques
  5. 5. What is the Problem with Software Estimation? 5
  6. 6. The greatest discrepancy between what the client thinks, an executive thinks, and a developer thinks is often the definition of ESTIMATE 6
  7. 7. 7 Estimates, Targets, and Commitments 1 Estimates Prediction • A Prediction of how long a project will take or how much it will cost 2 Targets Statement/Desire • A Statement of a desirable business objective 3 Commitments Promise • A Promise to deliver defined functionality at a specific level of quality and by a certain date
  8. 8. 8 Estimates, Targets, and Commitments 2 3 Target Commitment Estimate 1 Cut From Phase
  9. 9. Your best developer is not necessarily your best estimator 9
  10. 10. 10 The Solution Architect About your unicorn: • Knows the most about Drupal • Experience in Ecommerce Migration Integration Site building & Theming • Understands business value • Can magically debug anything Things that may be missing: How fast can a DEVELOPER do the work? How much time will be spent training, instructing, reviewing, and doing oversight? Avoiding unfounded optimism
  11. 11. What is the definition of a “GOOD” estimate? 11
  12. 12. 90% confidence 12 After all, the process is called Estimation, not Exactimation. - Phillip Armour What is the surface temperature of the Sun? (Give a range in Fahrenheit) Make your range wide enough that you feel you are 90% confident that your range includes the answer 10,000 Degrees F (9,941 F)
  13. 13. When given the option of a shorter average schedule with higher variability or a longer average schedule with lower variability, 8 out of 10 executives will choose the second option 13
  14. 14. 90% confidence 14 How confident is 90% confident? The average number of correct answers was 2.8. Only 2% of quiz takers scored 8 or more correctly. Most peoples intuitive sense of 90% confidence is in fact closer to 30% confidence.
  15. 15. Software is not developed in a vacuum 15
  16. 16. 16 EXTERNAL FACTORS ARE OFTEN UNACCOUNTED FOR LESS EXPERIENCED STAFF STAFF NOT READY REQUIREMENTS REMOVED STAFF DIVERTED TO SUPPORT OLD PROJECT UNSTABLE FUNCTIONALITY REMOVED SERVER NOT READY STAFF DIVERTED TO SUPPORT TRADE SHOW REQUIREMENTS ADDED
  17. 17. 17 Events that happen during the project nearly always invalidate the assumptions that were used to estimate the project in the first place. 1 Functional Assumptions 2 Staffing Assumptions 3 Priority Assumptions 4 Input Assumptions
  18. 18. 18 Project Dynamics The activities of a late or compressed project create destructive “late- project” dynamics that make the project even WORSE than nominal. More Status Meetings Frequent Re- estimation Apologizing to key customers Preparing interim releases More discussion on priority & requirements Fixing problems that arise from quick and dirty workarounds Going back to the contract
  19. 19. 19 Results
  20. 20. Results Timelines Budgets 20
  21. 21. “Software does not have a neutral estimation problem. The industry data shows clearly that the software industry has an underestimation problem.” -- Steve McConnell 21
  22. 22. 22 Avoiding Risk
  23. 23. Major Areas of Risk 23 • What kind of content type/field/thing • Scope w/o specs • Specific definitions • Setup Costs • Management Costs • Other (non-dev) Costs • 90% confidence • Best Estimator • Terminology • External Factors • Lack of end-user involvement or production owner • Timeline • Poor wires/design
  24. 24. Consider your project’s current position within the cone of uncertainty, this will help you gauge the level of accuracy in your estimate. 24
  25. 25. 25 The Cone of Uncertainty Initial Product Concept “I want a website” Product Definition “I know the scope of services” Requirements Complete “I know the specs of the project” User Interface Design Complete “I have wireframes” Detailed Design Complete “I have full documented specs and design files”
  26. 26. Leverage the power of well defined products to move your way through the cone of uncertainty 26
  27. 27. 27 Products Training Workshops Drupal Concept Education as an entry into the product Discovery Process Discovery Technical and UI/UX planning for a large scale development projects Evaluation of the inner workings of a current Drupal site, for upgrade and migration planning Audits
  28. 28. Know when you are in a chaos project and how to reset 28
  29. 29. Chaos Projects How does BAD go to TERRIBLE? 29 External Factors Re-estimation or Planning Poor Project Dynamics Pure Chaos “Chaos projects introduce variability for each following step. The answer is not to continue to re-estimate but instead address each issue through project control.”
  30. 30. NEVER, let your estimator forget (or your sales team remove) these important activities 30
  31. 31. Omitted Activities 31 Management • Stand ups, Contract review, Training, QA review • Engagement Management, Staffing, Governance Maintenance • Updates, Upgrades, Deployments, Performance • Configuration, Automation, Reviews Setup • Setup, Installation, Downloads, Builds • Ramp-up, Kickoff, Onboarding, Accounting Non-software • Vacation, Holiday, Sick, Drupalcon • Outages, Troubleshooting, Workstations
  32. 32. 32 Estimation Techniques
  33. 33. Estimating “Things” 33 Lets assume that you are all expert estimators and that all the process and risk areas we have spoken today you know PERFECTLY how to avoid. How many people are in this room right now?
  34. 34. 34 APPROACHES TO ESTIMATING “THINGS” The Hobbyist “I’m guessing 75 people” The Factor Guy “15 rows 5 per row” The Percentage Guy “150 person capacity and its half full” The Data Guy “The last 3 sessions have had XX” Judgement Count Compute Calibration
  35. 35. Count, Compute, Judge 35 Count if at all possible. Compute when you can’t count. Use Judgement alone only as a last resort. COUNT (& Calibrate) Find something to count that has a direct correlation to what you are building1 COMPUTE Multiply by a statistically meaningful (proven) average2 JUDGE Only make assumptions as a final option (document)3
  36. 36. • Click to edit Master text styles Items to Consider • Size of Project 5-25 people, 25-50, etc. • Sequential vs. Scrum • Stage of project 36
  37. 37. 37 Common Items in these areas 1 2 3Count/Calibrate Compute Judge Site Specs Content types Taxonomies Menus Views Wireframes Designs Migration “features” Non-build items QA Deployment Project Management Training Governance Engagement Management Adjustment Factors Risk Multipliers Contingency Unknown
  38. 38. 38 Count & Calibrate 1) Determine an item that’s highly correlated with the size of the software 2) COUNT how many you have 3) CALIBRATE your count with data from - Industry Data - Historical Data - Project Data Factor Factor Count Calibration Gross Estimate # Content Types 8 4 32 How many hours for feature “X”?
  39. 39. 39 Compute 1) Determine a defined value to base your computation 2) Determine a multiplier that is a statistically meaningful average 3) COMPUTE subtotal for your line item Development Subtotal 1200 Hours Subtotal PM % PM SubTotal 1200 20% 240 How many hours for Project Management?
  40. 40. 40 Judgement 1) JUDGE the rating of a specific multiplier (factor) 2) Apply multipliers based on rating of your estimate 3) Determine factors influence on the total project 4) COMPUTE Site Build PM % Gross Estimate Rating Influence Net Estimate 1200 240 1440 Low = 1.25 1.1 1980 Site Building Definition 2.0 1.25 1.0 .95 .85 .75 1.1
  41. 41. Overcoming Judgement 41 Judgement is the most hazardous kind of estimation due to subjectivity and unfounded optimism. Magnitude of Relative Error MRE = (Actual Result - Estimated) / (Actual) Expected Case EC = (Best Case + (3 x Most Likely Case) + (2 x Worst Case)) / 6
  42. 42. Wide Delphi Method • Coordinator presents each expert with a specification and an estimation form. • Coordinator calls a group meeting in which the experts discuss estimation issues with the coordinator and each other. • Experts fill out forms anonymously. • Coordinator prepares and distributes a summary of the estimates • Coordinator calls a group meeting, specifically focusing on having the experts discuss points where their estimates vary widely • Experts fill out forms, again anonymously, and steps 4 to 6 are iterated for as many rounds as appropriate. 42
  43. 43. Subtitle for slide Guide for Standardized Estimation Procedure • Focus on counting and computing when possible, rather than using judgment - create tools to facilitate this and make it easy • Encourage multiple estimation approaches and comparison of results • Define toll gates and encourage re-estimation over the course of a project • Contains a clear description of an estimate’s inaccuracy • Defines when an estimate can be used as the basis for a project budget • Defines when an estimate can be used as the basis for internal and external commitments • Calls for archiving estimation data and reviewing effectiveness of the procedure • McConnell, Steve (2006-02-22). Software Estimation: Demystifying the Black Art (Developer Best Practices) (Kindle Locations 5142-5149). Pearson Education. Kindle Edition. 43
  44. 44. To Learn More... 44 Software Estimation: Demystifying the Black Art Steve McConnell
  45. 45. Thank You 45 CHICAGO, ILLINOIS 1802 W. Berteau Ave. / Suite 209 / Chicago IL, 60613 773-525-8255 D.O / Twitter @akucharski or @prometsource Email: andy@prometsource.com
  46. 46. 46
  47. 47. 47 Estimation Techniques Calibration and Historical Data T-Shirt Sizing Wideband Delphi Technique
  48. 48. Results Costs 48

×