The Value-Adding Tester


Published on

A presentation about how testers add value to a project or organization, and the costs and value-detractors associated with testing.

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The Value-Adding Tester

  1. 1. The Value-Adding Tester How a Tester Adds Value to an Organization or Project 1 2013-12-13 PA1 Confidential
  2. 2. Introduction ▪ The Value-Adding Tester adds value to the project or organization ▪ The Value-Adding Tester understand and reduces costs ▪ The Value-Adding Tester does not detract value ▪ To be a value-adding tester it is required to continuously evaluate value and cost ▪ How does a tester add and detract value, and what are the costs? 2 2013-12-13 PA1 Confidential
  3. 3. How to Add Value - Overview Mandatory Tests 3 2013-12-13 PA1 Fixed Defects Information to Stakeholders Development Support Confidential
  4. 4. Fixed Defects ▪ One of the major ways a tester can add value is by finding defects in the product which are actually fixed at some point during the product life cycle ▪ The value of a defect found can be quantified by estimating how the defect would impact sales, and if the customer returns the product because of the defect 4 2013-12-13 PA1 Confidential
  5. 5. Information to Stakeholders ▪ Another way to generate value for a tester is to provide stakeholders with information and material for tollgates, milestones and decision points ▪ The value is harder to quantify but can be divided into two parts: ▪ Helping stakeholders taking informed decisions ▪ Allowing stakeholders to feel more confident in their decisions 5 2013-12-13 PA1 Confidential
  6. 6. Mandatory Tests ▪ In many cases different governing bodies require that certain criteria are met for a product to be allowed to be sold ▪ Meeting these criteria often require mandatory tests that must be executed according to strict requirements ▪ Specific customers can also have mandatory requirements that must be tested to be able to sell to those customers ▪ The value a tester provides by executing these tests can be quantified by comparing how much it would cost to outsource this to an accredited outsourcing partner 6 2013-12-13 PA1 Confidential
  7. 7. Development Support ▪ A tester can also provide value by increasing the efficiency of the development team ▪ This can be done by providing a robust, easy-to-use test framework, or by creating and maintaining automated integration and regression test suites which the developers can use ▪ The tester can also offer support by helping with unit test plans and strategies ▪ Yet another example could be pushing for and educating in testability, which can also drive efficiency 7 2013-12-13 PA1 Confidential
  8. 8. How to Detract Value - Overview Low Quality Reports 8 2013-12-13 PA1 Irrelevant Defects Irrelevant Information Inefficient Test Tools Confidential
  9. 9. Low Quality Reports ▪ Testers should not create defect reports that ▪ Are hard to understand because of bad language ▪ Are duplicates of already existing defect reports ▪ Lack information that is critical to understand the defect ▪ Lack critical attachments such as log files or screen shots ▪ Etc. ▪ This will lead to increased report handling and analysis effort for developers ▪ If testers create reports which cannot be understood due to improper structure, language, formatting, etc., this also costs additional analysis effort for stakeholders 9 2013-12-13 PA1 Confidential
  10. 10. Irrelevant Defects ▪ When a tester reports a defect this has to be analyzed, prioritized and handled in different ways ▪ If the defect is not fixed, and the information stored in the defect is not used by stakeholders, submitting the defect actually detracts value instead of adding ▪ Many irrelevant defects could however together provide valuable data by revealing trends or problem areas ▪ The value detraction can be reduced by handling less relevant defects differently than higher priority issues 10 2013-12-13 PA1 Confidential
  11. 11. Irrelevant Information ▪ When testers provide reports to stakeholders, the information presented must be beneficial and help the stakeholders to take required decisions ▪ Presenting data that is not relevant or even misleading is not only not valuable, but can be very costly ▪ Burying key information in a mountain of data ▪ A report that states “99 % Pass Rate” as the main conclusion, when the remaining 1% represents critical quality issues in the product, will not help stakeholders make the correct decisions 11 2013-12-13 PA1 Confidential
  12. 12. Inefficient Test Tools ▪ The cost that testers impose on stakeholders with inefficient tools ▪ When testers use tools that impact others, such as developers, project managers, or other stakeholders, in a negative way, this detracts value ▪ One example could be inefficient test frameworks or automation tools which cause developers to write inefficient tests ▪ Another example could be a reporting tool which is difficult for stakeholders to extract reports from, or the tool generates reports which take time to analyze and understand 12 2013-12-13 PA1 Confidential
  13. 13. Costs - Overview Test Design & Execution Test Planning Test Tools & Frameworks 13 2013-12-13 PA1 Test Reporting Administrative Overhead Confidential
  14. 14. Test Design & Execution ▪ Designing, executing, maintaining and porting tests ▪ Execution Effort Exploratory Test Scripted Test Automated Test Scripted Test Exploratory Test ▪ Design Cost Automated Test 14 2013-12-13 PA1 Confidential
  15. 15. Test Planning ▪ Planning test activities and creating corresponding test artifacts such as test plans ▪ Setting scope for different test activities ▪ Risk analysis as impact to scope selection ▪ Probability of failure ▪ Technical risk analysis ▪ Impact of failure ▪ Business risk analysis 15 2013-12-13 PA1 Confidential
  16. 16. Test Reporting ▪ Creating test reports ▪ Test result metrics ▪ Qualitative summaries/product stories ▪ Creating defect reports ▪ Analysis of automated test results 16 2013-12-13 PA1 Confidential
  17. 17. Test Tools & Frameworks ▪ Creating new tools and test frameworks ▪ Cost of buying test tools and frameworks ▪ Integrating new tools ▪ Maintaining tools and frameworks 17 2013-12-13 PA1 Confidential
  18. 18. Administrative Overhead ▪ How much time does the tester actually spent on design, analysis, execution, tools and reporting, and how much time does the tester loose to administrative overhead? ▪ Getting the right software artifacts for test, and understand what to actually test ▪ Coordinating between testers, developers, and organizations ▪ Non-value adding meetings, etc. 18 2013-12-13 PA1 Confidential
  19. 19. Net Value of a Tester Net Value 19 2013-12-13 PA1 Value Cost Value Detraction Confidential
  20. 20. How do we increase value and decrease cost and value detraction? ▪ We want to maximize value gain, and minimize costs and value detraction ▪ How we do this depends heavily on context ▪ However we can still give some general guidelines ▪ Of course there are many other ways to reduce costs and add value, but these are some suggestions 20 2013-12-13 PA1 Confidential
  21. 21. How do we increase value and decrease cost and value detraction? ▪ Evaluate test artifacts and remove the non-value adding ones ▪ 10-minute Test Plan – don’t create extensive test plans that no one actually uses or updates ▪ Very detailed scripted test cases cost more than they add value most of the time ▪ Test Strategy documents that no one reads or uses should not be created at all ▪ However the discussions which lead to the creation of the documents are still valuable and important to have 21 2013-12-13 PA1 Confidential
  22. 22. How do we increase value and decrease cost and value detraction? ▪ Evaluate tools among all different stakeholders to secure that they are efficient ▪ Even if a tool is easy to use for some purposes, it could impose large costs for other stakeholders to use ▪ A test administration tool that is easy to record data in, may be very costly to generate reports from ▪ Evaluate the actual value gain of the automated test framework ▪ Don’t just calculate the execution effort saved ▪ Many different costs: Design, Analysis, Maintenance, Porting ▪ Actual gains: Defects Found, Mandatory Tests, Information and Support to Developers 22 2013-12-13 PA1 Confidential
  23. 23. How do we increase value and decrease cost and value detraction? ▪ Secure that the information and defects testers report is what stakeholders actually need ▪ Defects must be fixed ▪ Information must be used ▪ Make sure that testers actually work with testing, and not spend their time on everything else ▪ Let testers list what they spend their time on, and try to reduce administrative overhead and let the testers actually work with test 23 2013-12-13 PA1 Confidential
  24. 24. How do we increase value and decrease cost and value detraction? ▪ Proper risk analysis to support scope selection ▪ Risk-based testing ▪ Should be used when the reduction in cost for test execution is larger than the increase in cost for test planning that the risk analysis adds ▪ Efficiency gain of risk-based testing is very dependant on the complexity of the system under test ▪ Show the cost that the system complexity adds to testing – this can drive actions to reduce the complexity of the system 24 2013-12-13 PA1 Confidential
  25. 25. Conclusion ▪ The Value-Adding Tester continuously evaluates what value is added, what value is detracted, and the costs for providing this value ▪ There are many ways to add value, but there are also many ways to detract value and either way there is an associated cost ▪ Understanding the values and costs is critical to become efficient ▪ Understanding must be supported by actual metrics, and not only gut feelings 25 2013-12-13 PA1 Confidential
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.