This document provides information on various classical test estimation approaches, including factors that influence test estimation, test execution estimation steps, the Wideband Delphi and Three Point estimation techniques, and how to convert estimates to confidence levels. It also briefly discusses work breakdown structures, top-down vs bottom-up testing, and Planning Poker estimation.

1. Classical Approaches in
Test Estimation
Anton Muzhailo
QA Open, Mykolaiv 2019

2. • ISTQB Certified Test Manager
• Provide ISTQB Certified Trainings in CodeSpace
• 400+ full course attendees during 5 years
/in/muzhailo
Anton Muzhailo
Lead Automation Engineer,
GlobalLogic
In other words I’m basically doing:
• Test Automation
• Performance Testing
• Test Management
• Consulting / Assessments
• Teaching & Mentoring
About me

3. Cost of quality
Costs of prevention, e.g.,
training developers to write
more maintainable or secure
code
Costs of detection, e.g., writing
test cases, configuring test
environments, and reviewing
requirements
Costs of internal failure, e.g.,
fixing defects detected during
testing or reviews, prior to
delivery
Costs of external failure, e.g.,
support costs associated with
defective software delivered to
customers
Cost of quality

4. (Test) Estimation
Test Estimation: The calculated approximation of a result related to various
aspects of testing (e.g., effort spent, completion date, costs involved, number of
test cases, etc.) which is usable even if input data may be incomplete, uncertain,
or noisy.
Estimation, as a management activity, is the creation of an approximate target
for costs and completion dates associated with the activities involved in a
particular operation or project.
• Assumptions made during estimation should always be documented as part
of the estimation.

5. Test Execution Estimation main steps
Estimate the number of tests
based on the size of the system.
Use historical data.
Factor in the quality of the
software. Using historical data
such as the average number of
defects per developer-hour on
similar past projects, estimate
the number of defects you’ll
find.

6. Test Estimation factors
Factors that can influence estimation 1/3:
Required level of quality of the system
Size of the system to be tested
Historical data from testing for previous test projects which
may be augmented with industry data or benchmark data
from other organizations
Process factors, including the test strategy, development or
maintenance lifecycle and process maturity, and the accuracy
of the project estimate

7. Test Estimation factors
Factors that can influence estimation 2/3:
Material factors, including test automation and tools, test
environment(s), test data, development environment(s), project
documentation (e.g., requirements, designs, etc.), and reusable
test work products
People factors, including managers and technical leaders,
executive and senior management commitment and
expectations, skills, experience, and attitudes in the project
team, stability of the project team, project team relationships,
test and debugging environment support, availability of skilled
contractors and consultants, and domain knowledge
Complexity of the process, technology, organization, number of
testing stakeholders, composition and location of sub teams
Significant ramp up, training, and orientation needs

8. Test Estimation factors
Factors that can influence estimation 3/3:
Assimilation or development of new tools, technology,
processes, techniques, custom hardware, or a large quantity of
testware
Requirements for a high degree of detailed test specification,
especially to comply with an unfamiliar standard of
documentation
Complex timing of component arrival, especially for
integration testing and test development
Fragile test data (e.g., data that is time sensitive)

9. Top-down & bottom-up testing
Bottom-up Testing: An incremental approach to integration testing where the
lowest level components are tested first, and then used to facilitate the testing of
higher level components. This process is repeated until the component at the
top of the hierarchy is tested.
Top-down Testing: An incremental approach to integration testing where the
component at the top of the component hierarchy is tested first, with lower level
components being simulated by stubs. Tested components are then used to test
lower level components. The process is repeated until the lowest level
components have been tested.

10. Work Breakdown Structures
WBS can be represented as a hierarchical list of project’s work activities. There
are two formats of WBS −
• Outline View (Indented Format)
• Tree Structure View (Organizational Chart)
And there are two types of WBS −
• Functional WBS − In functional WBS, the system is broken based on the
functions in the application to be developed. This is useful in estimating the
size of the system.
• Activity WBS − In activity WBS, the system is broken based on the activities
in the system. The activities are further broken into tasks. This is useful in
estimating effort and schedule in the system.

11. Outlined view WBS Example
• Software Development
o Scope
• Determine project scope
• Secure project sponsorship
• Define preliminary resources
• Secure core resources
• Scope complete
o Analysis/Software Requirements
• Conduct needs analysis
• Draft preliminary software specifications
• Develop preliminary budget
• Review software specifications/budget with the team
• Incorporate feedback on software specifications
• Develop delivery timeline
• Obtain approvals to proceed (concept, timeline, and budget)
• Secure required resources
• Analysis complete
o Design
• Review preliminary software specifications
• Develop functional specifications
• Obtain approval to proceed
• Design complete
o BlahBlahBlah

12. The steps
1. Gather the WBS nodes
2. Use appropriate estimation techniques (Wide-Band Delphi or Three Point
Analysis)
3. Define the internal dependencies between the tasks (Finish-to-Start and
Finish-to-Finish dependencies)
4. Define the concurrence of tasks execution
5. Use Critical Path Method to define the critical path for the project
1. Example (from previous slide, critical path is Scope – Analysis/Software Requirements – Design – BlahBlah)
6. Create a Gantt Chart and use it to drive your project

13. Wideband Delphi
Wideband Delphi: An expert-based test estimation technique that aims at
making an accurate estimation using the collective wisdom of the team
members.
Choose
Estimation
Team
Kick-off
Meeting
Individual
Preparation
Estimation
Meeting
Evaluation

14. Kick-off Meeting
Choose
Estimation
Team
Kick-off
Meeting
Individual
Preparation
Estimation
Meeting
Evaluation
1. The moderator conducts the kickoff meeting, in which the team is presented with the problem
specification and a high level task list, any assumptions or project constraints.
2. The team discusses on the problem and estimation issues, if any. They also decide on the units of
estimation.
3. The moderator guides the entire discussion, monitors time and after the kickoff meeting, prepares
a structured document containing problem specification, high level task list, assumptions, and the
units of estimation that are decided.

15. Individual Preparation
Choose
Estimation
Team
Kick-off
Meeting
Individual
Preparation
Estimation
Meeting
Evaluation
Each Estimation team member then individually generates a detailed WBS, estimates each task in the
WBS, and documents the assumptions made.
The template could be different, for example =>>

16. Wide-Band Delphi Estimation Tpl
Project <Project Name> Estimation Units Eng/h
Estimation Team Member <Name> Date MM/DD/YY
Task Initial
Estimate
Change1 Change2 Change3 Change4 Final
Task 1 E1
Task 2 E2
Task 3 E3
Task 4 E4
Task 5 E5
Net Change
Total Sum(E)

17. Estimation Meeting
Choose
Estimation
Team
Kick-off
Meeting
Individual
Preparation
Estimation
Meeting
Evaluation
1. Moderator collects all initial estimations from all Team members.
2. He plots each member’s total project estimate as an X on the Round 1 line, without disclosing the
corresponding names. The Estimation team gets an idea of the range of estimates, which initially
may be large.
3. Each team member reads detailed task list, identifying any assumptions made and raising any
questions or issues. The task estimates are not disclosed.
4. The team then discusses any doubt/problem they have about the tasks they have arrived at,
assumptions made, and estimation issues
5. Each team member then revisits his/her task list and assumptions, and makes changes if
necessary 🡺 STEP 2

18. Estimation Plot
Round 1
Round 2
Round 3
Round 4
0 20 40 60 80 100

19. New Wide-Band Delphi sheet
Project <Project Name> Estimation Units Eng/h
Estimation Team Member <Name> Date MM/DD/YY
Task Initial
Estimate
Change1 Change2 Change3 Change4 Final
Task 1 E1 -2
Task 2 E2 0
Task 3 E3 +1
Task 4 E4 -4
Task 5 E5 +2
Net Change
Total Sum(E)

20. Estimation Plot. Round 2
Round 1
Round 2
Round 3
Round 4
0 20 40 60 80 100

21. Three Point Estimation
Three-point Estimation looks at three values
• the most optimistic estimate (O),
• a most likely estimate (M), and
• a pessimistic estimate (least likely estimate (L)).
Estimation = (O + M + L) / 3
Standard Deviation = √ [((O − E)2
+ (M − E)2
+ (L − E)2
) / 2]
Take into account that Project Evaluation and Review Technique (PERT) is a
different technique. As sometimes those both can be understood as same.

22. Convert the Project Estimates to
Confidence Levels
The Three-point Estimate (E) and the Standard Deviation (SD) thus calculated
are used to convert the project estimates to “Confidence Levels”.
The conversion is based such that −
• Confidence Level in E +/– SD is approximately 68%.
• Confidence Level in E value +/– 1.645 × SD is approximately 90%.
• Confidence Level in E value +/– 2 × SD is approximately 95%.
• Confidence Level in E value +/– 3 × SD is approximately 99.7%.
• Commonly, the 95% Confidence Level, i.e., E Value + 2 × SD, is used for all
project and task estimates.

23. Planning Poker
Planning Poker Estimation Technique, estimates for the user stories are derived
by playing planning poker. The entire Scrum team is involved and it results in
quick but reliable estimates.
• Planning Poker is played with a deck of cards. As Fibonacci sequence is used,
the cards have numbers - 1, 2, 3, 5, 8, 13, 21, 34, etc. These numbers represent
the “Story Points”.
The story point represents the complexity. One Story Point can cost the simplest
story.

24. Thanks!
Questions?
in/muzhailo/