Volodymyr Prymakov and Vlada Benyukh Detailed manual estimation approach for Pre-sale phase

1. Detailed Manual Estimation
Approach for Pre-sale phase
Vlada Benyukh, Volodymyr Prymakov

2. Speaker info
Vlada Benyukh, Ukraine
QA Consultant/QA Lead at Ciklum, TCoE
• 7 years in QA
• Manual QA Engineer, QA Lead
• Certified: SAFe, ICAgile, ISTQB
/in/vlada-benyukh

3. Speaker info
/in/vladimirprimakov/
Volodymyr Prymakov, Ukraine
Senior QA Manager at Ciklum, TCoE
Head of Performance QA Unit at TCoE
•14 years in QA
•45 projects experience
•Certified: ISTQB Advanced Test Analyst & Manager,
ICAgile CP, SAFE

4. Agenda
1. What is an Estimate? Why it is important?
2. Which basis and factors should be considered in estimation process?
3. What main and additional activities should be included in QA Estimation
approach?
4. How Regression testing should be calculated?
5. How QA Estimation can be adjusted by risks multipliers?
6. Demo of real-life example

5. Why we need to estimate?

6. How to estimate these tasks?

7. Assumption
s
We assume that:
1. Project is planned to work based on Scrum
2. Development team size: up to 30 people
3. QA Engineer Competency is Middle Level
4. Non-functional requirements are estimated separately

8. Test Basis
Such test basis should be considered as below:
1. Functional requirements in scope (WBS) are defined
2. Test environments and their priorities are determined while pre-sale phase
3. Project duration (number of sprints) is considered before estimation
process

9. What factors?
1. Domain specific knowledge and technologies used
2. Integration-related problems/risks (within internal and external systems)
3. Complexity of the application (e.g. calculations verifications)
4. Compatibility testing (e.g. file compatibility, plugin computability etc)

10. What main activities?
1. Detailed Requirements Review*
2. Test Design
3. Test Execution
4. Bug investigation and submission. Bug Retest
5. Environments/Platforms Multipliers

11. How to estimate time?
Time calculation for main test activities =
(test execution + bugs submission and retest) *environments/platforms multiplier
+ test design
+ detailed requirements review*

12. How to calculate env multiplier?
Environments/platforms factors to select:
1. Identify customer region and target audience where the application will be
used
2. Select top used environments/platforms based on the customer’s
region/audience

13. Example to calculate env multiplier?
Web-browser e.g. Test Type Coefficien
t
The highest priority web-browser Chrome 57
Full Test (Positive,
negative flows) 1
High priority web-browser IE 11, FF54 Only positive flows tests 0,5
Medium priority web-browser Edge14 Explorartory testing 0.25
Low priority web-browser Chrome 56
Smoke test (at least 1
positive flow test) 0.1
EXAMPLE of multiplier calculation: 1 + 0.5 + 0.5 + 0.25 + 0.1 = 2.35

14. Example of estimation basis

15. What typical preparation activities?
1. Detailed evaluation of the product/project. Full Master Test Plan creation
2. Learning and getting acquainted with the product's documentation and/or
functionality. Preliminary requirements review and feedback
3. Checklist/Suite creation (with empty test cases)
4. Test Environment and Infrastructure Preparation & Access

16. Example of preparation activities?

17. What typical repetitive activities?
1. Sprint Grooming & Planning, Retrospective
2. Daily Communication Meetings
3. Demo Meetings
4. Global Release Readiness Analysis and Reporting
5. Regular Product Deploy and Configuration, Test Environment Configuration,
Test Data Preparation
* If estimated by PM, it should NOT be estimated here and it should be set to 0h

18. Example of typical repetitive activities

19. How to calculate Regression?
1. Number of sprints: (e.g. 12)
2. Number of regression testing cycles: (e.g. 6 or 3)
3. Pure New functionality testing time ⇒ (without time working with bugs and
test environment multiplier)= (e.g. 300h)
4. Percentage of regression-tested functionality: (e.g. 60%)
5. Test environments Multiplier: (e.g. 2)

20. How to calculate Regression?
Pure new functionality testing time (if 0 bugs)

21. How to calculate Regression?
Regression cycles: 6; regressed functionality: 60%

22. How to calculate Regression?
Regression testing time * Test environments Multiplier: 2

23. How to calculate Regression?
If number of regression cycles is 3

24. How to calculate Regression?
Calculation:
1) Time for last regression cycle = Pure New functionality testing time (0 bugs) * % of
regressed functionality * Test environments Multiplier
2) Regression increment = Time for last regression cycle *
number of regression cycles
3) Time for regression testing = Regression Increment * (1+2+… number of regression cycles)

25. Example of Regression calculations

26. What basis for Smoke Testing?
1. Number of smoke testing cycles in sprint
2. Number of sprints/iterations expected
3. Approximate time for smoke
4. Test environments Multiplier

27. Example of Smoke calculations

28. What Risks should be included?
1. Changing/Not Stable Requirements, Increased scope (10%)
2. Documentation and requirements clarification problems (7%)
3. Risk of functionality integration/interdependence and regression
problems (10%)
4. Buggy application risk, because of developers competence (5%)
5. Technology risks (5%)
6. Risk of not mature processes and time spent for improving them (5%)
7. Infrastructure & environment Security & Accessibility Risks (5%)
8. Cross project communication necessity/dependency risk (5%)*

29. Example of Risks basis

30. Example of Summary calculations

31. Demo of real-life example

32. Demo of real-life example

33. Statistics
1. General amount of estimated projects: 15
2. General amount of the project where QA estimation should be adjusted, because of
scope and deadlines changes: 5
3. Duration of the projects: 1 -12 months
4. Team size: 5-30

34. Questions