Combining Estimates with Planning Poker  - An Empirical Study Kjetil Moløkken-Østvold Nils-Christian Haugen   April 2007
Agenda <ul><li>Combining expert estimates  </li></ul><ul><li>Planning Poker </li></ul><ul><li>Study results </li></ul><ul>...
Background <ul><li>Warning! Much of the software engineering literature misinterprets psychological research </li></ul><ul...
An overview of some methods for combining estimates Limited Yes No None Unstructured group  Limited Yes No Light Planning ...
Planning Poker <ul><li>James W. Grenning (www.objectmentor.com) </li></ul><ul><li>Process </li></ul><ul><ul><li>Task prese...
Study background <ul><li>Twofold purpose of study </li></ul><ul><ul><li>Investigate possible reduction of optimism in the ...
Team and methodology <ul><li>Consultants from same company </li></ul><ul><ul><li>4-6 people total </li></ul></ul><ul><ul><...
Results from the tasks estimated with planning poker <ul><li>Compared with the mechanical combination of individual estima...
Results PP vs. Individual Measure: BRE-bias =  (actual - estimate) /  min(actual, estimate) PP Ind. Comment Median 0,00 0,...
Results from analysis of code <ul><li>Planning poker tasks had on average: </li></ul><ul><ul><li>Twice as many deleted con...
Possible benefits of Planning Poker <ul><li>Includes several perspectives </li></ul><ul><li>Participants gets more ownersh...
Possible hazards of Planning Poker <ul><li>Decision may be vulnerable to peer-pressure </li></ul><ul><li>Often not easy to...
Summary <ul><li>Combination of estimates may increase accuracy, but can have certain side-effects </li></ul><ul><ul><li>Do...
Questions?
Upcoming SlideShare
Loading in …5
×

Combining Estimates with Planning Poker *- An Empirical Study

1,223
-1

Published on

Combination of expert opinion is frequently used to
produce estimates in software projects. However, if,
when and how to combine expert estimates, is poorly
understood. In order to study the effects of a
combination technique called planning poker, the
technique was introduced in a software project for half
of the tasks. The tasks estimated with planning poker
provided: 1) group consensus estimates that were less
optimistic than the mechanical combination of
individual estimates for the same tasks, and 2) group
consensus estimates that were more accurate than the
mechanical combination of individual estimates for the
same tasks. The set of control tasks in the same project,
estimated by individual experts, achieved similar
estimation accuracy as the planning poker tasks.
However, for both planning poker and the control
group, measures of the median estimation bias
indicated that both groups had unbiased estimates, as
the typical estimated task was perfectly on target.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,223
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
20
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Combining Estimates with Planning Poker *- An Empirical Study

  1. 1. Combining Estimates with Planning Poker - An Empirical Study Kjetil Moløkken-Østvold Nils-Christian Haugen April 2007
  2. 2. Agenda <ul><li>Combining expert estimates </li></ul><ul><li>Planning Poker </li></ul><ul><li>Study results </li></ul><ul><li>Summary </li></ul><ul><li>Q&A </li></ul>
  3. 3. Background <ul><li>Warning! Much of the software engineering literature misinterprets psychological research </li></ul><ul><li>Combination of expert estimates might reduce over-optimism </li></ul><ul><li>Lack of empirical research </li></ul><ul><ul><li>Properties of various methods used to combine estimates </li></ul></ul><ul><ul><li>Project issues (customer, priorities, management) </li></ul></ul><ul><ul><li>Types of experts involved </li></ul></ul>
  4. 4. An overview of some methods for combining estimates Limited Yes No None Unstructured group Limited Yes No Light Planning Poker Moderate Limited Limited Moderate Wideband Delphi Major No Yes Heavy Delphi Overhead Interaction Anonymity Structure Method
  5. 5. Planning Poker <ul><li>James W. Grenning (www.objectmentor.com) </li></ul><ul><li>Process </li></ul><ul><ul><li>Task presented </li></ul></ul><ul><ul><li>Task is discussed (defines the tasks before estimating) </li></ul></ul><ul><ul><li>Individual estimates derived (combines knowledge from several sources) </li></ul></ul><ul><ul><li>Estimates revealed simultaneously (reduces social comparison concerns) </li></ul></ul><ul><ul><li>High/low estimator justifies </li></ul></ul><ul><ul><li>Consensus sought </li></ul></ul>
  6. 6. Study background <ul><li>Twofold purpose of study </li></ul><ul><ul><li>Investigate possible reduction of optimism in the PP tasks </li></ul></ul><ul><ul><li>Compare PP tasks with individual estimates </li></ul></ul><ul><li>Planning poker estimates performed in sprint planning </li></ul><ul><ul><li>14-day sprints </li></ul></ul><ul><ul><li>Data from 4 sprints </li></ul></ul><ul><ul><li>50% of tasks re-estimated using planning poker </li></ul></ul><ul><li>Most likely estimates in hours (not predefined units) </li></ul>
  7. 7. Team and methodology <ul><li>Consultants from same company </li></ul><ul><ul><li>4-6 people total </li></ul></ul><ul><ul><li>All developers </li></ul></ul><ul><li>Scrum </li></ul><ul><ul><li>Solo programming </li></ul></ul><ul><ul><li>Tasks tested and QAed by another person on the team </li></ul></ul><ul><ul><li>Tasks kept in issue tracking system (Jira) </li></ul></ul><ul><ul><li>Daily scrum </li></ul></ul>
  8. 8. Results from the tasks estimated with planning poker <ul><li>Compared with the mechanical combination of individual estimates, the consensus estimates were </li></ul><ul><ul><li>less optimistic after discussion </li></ul></ul><ul><ul><li>more accurate after discussion </li></ul></ul><ul><li>This is opposite of what is found in most studies from other areas </li></ul><ul><li>Can be caused by different perspectives on a task and/or identification of sub-problems </li></ul>
  9. 9. Results PP vs. Individual Measure: BRE-bias = (actual - estimate) / min(actual, estimate) PP Ind. Comment Median 0,00 0,00 Typical case on target for both groups Mean 0,33 -0,04 Some PP tasks were underestimated
  10. 10. Results from analysis of code <ul><li>Planning poker tasks had on average: </li></ul><ul><ul><li>Twice as many deleted control statements (indicates that effort was spent to reduce complexity) </li></ul></ul><ul><ul><li>Twice as many out-of-class references deleted (indicates that effort was spent to reduce coupling) </li></ul></ul><ul><li>Extra time spent on restructuring and simplifying code can explain why there were some overruns in the planning poker tasks </li></ul><ul><li>Question: Did planning poker (group discussion) lead to increased focus on quality? </li></ul>
  11. 11. Possible benefits of Planning Poker <ul><li>Includes several perspectives </li></ul><ul><li>Participants gets more ownership of estimates </li></ul><ul><li>Uncertainty related to the implementation can be discussed and handled at an early stage </li></ul><ul><li>Cost-efficient </li></ul>
  12. 12. Possible hazards of Planning Poker <ul><li>Decision may be vulnerable to peer-pressure </li></ul><ul><li>Often not easy to deviate from original estimate(s) </li></ul><ul><li>An unstructured discussion might have side-effects, such as increasing number of sub-tasks </li></ul>
  13. 13. Summary <ul><li>Combination of estimates may increase accuracy, but can have certain side-effects </li></ul><ul><ul><li>Does planning poker lead to more focus on quality? </li></ul></ul><ul><li>Planning poker was popular among developers </li></ul><ul><ul><li>Found in Norway and previously in the UK </li></ul></ul><ul><ul><li>Rated as fun and easy to implement </li></ul></ul><ul><li>Choice of combination method should depend on project characteristics </li></ul>
  14. 14. Questions?

×