Software Testing Foundations Part 7 - Basics of Test Management
Software Testing Foundations #7Basics of Test ManagementNikita Knyshnknysh@gmail.comhttp://www.facebook.com/groups/istqb/
Agenda• Test independence, Testing Roles• Prioritization of Tests, Test Exit Criteria• Cost of Defects, Cost of Testing, Test Estimation• Test Strategy and Planning• Approaches to Test Design and Test• Risk Management• Test Activity Management• Incident Management• More on Test Management
Options for Independent Testing• Developer tests code by another developer (component, integration test)• Testers in the development team (component, integration test of small projects)• Testing teams within the project team (integration test of large projects, system test)• Independent testers for specific testing tasks (performance, security, etc.) (integration test of large projects, system test)• Separate organization is responsible for testing (integration test of large projects, system test)
Testing Roles• Test manager (test leader)• Test designer (test analyst) – analyze requirements, write test specs, prepare & acquire test data• Test automator• Test administrator – maintain test environment• Tester – review & execute tests, log & evaluate results
Prioritizing Test CasesCriteria for prioritizing:• Usage frequency (probability of failure)• Risk of failure (failure probability and severity)• Visibility of failure to end user• Priority of (functional) requirements• Importance of (non-functional) quality characteristics• Importance of system components• Complexity of individual components• Project risk (try to find failures that are hard to fix first)Defect-prone test objects: where many faults have beenfound before, more are present.
Test Exit Criteria• Test coverage (requirement covered, test cases run successfully)• Product quality (number and breakdown of faults found)• Residual risk: not executed tests, not fixed bugs, incomplete coverage• Economic constraints
Cost of Defects• Direct cost ▫ Arises to the customer due to failures during operation• Indirect costs ▫ Arises to vednor in form of penalties, loss of reputation, sales, etc.• Defect correction cost. With every test level, the correction costs for a fault double with respect to the previous level.
Testing Cost Factors• Maturity of development process• Quality & testability of software• Test infrastructure (env, tools, standards/procedures)• Qualification of employees• Requirements to quality• Test strategy (the only one TM can influence & control in short term)
Test Effort Estimation Approaches• By test task owner or expert• Based on experience & typical valuesIf no data: test tasks (on all levels) take 50% of overallproject resources.
Test StrategyDefines the projects testing objectives and means toachieve them.The goal is to choose a test approach that optimizesthe relation between costs of testing and costs ofdefects.
Approaches to Test Planning• Preventive approach (like use of V-model) is way better that reactive approach where testers are involved (too) late.• Analytical approach to test planning uses data and (mathematical) analysis; heuristic approach relies on experience of experts & rules of thumb (used when data not available or modeling is too complicated).
Types of Approaches to Test Design & Test• Model-based• Risk-based• Process- or standard-compliant• Reuse-oriented• Checklist-based (methodical)• Expert-oriented
Risk ManagementOne of the best criteria when selecting and prioritizingtest objectives is risk. Risk = probability (or frequency) * damage.• Project risks threaten project’s capability to deliver product.• Product risks result from problems with delivered product.Risk management: assess, prioritize and mitigate (orfight) risks.
Test Activity Management• Test cycle planning (development status, test results, resources). Result: (regression) test plan for upcoming lifecycle.• Test cycle monitoring. Metric types: fault- and failure-based, test case-based, test object-based, cost-based. Result: test status report (test progress within the whole project) in the end of each cycle.• Test (cycle) control – if testing is delayed test manager takes actions like add resources, extend test duration or adjust test plan.
Incident Management• Test log analysis or documentation review -> registering an incident if needed.• Incident DB should have reporting and analysis tools.• Incident report structure should be defined.Incident classification: severity – impact to system, priority– order of correctionChange control board accepts or rejects incident reportsand change requests in larger projects. Usually consists ofproduct management, project management, testmanagement, customer.
Configuration ManagementA test process cannot be properly executed withoutreliable configuration management!Requirements to configuration management:• Version management and configuration identification in particular version• Incident status and change status accounting. IEEE 828 Standard for Software Configuration Management Plans.
Sources of Standards• Company standards• Best practices• Quality management standards• Standards for particular industrial sectors• Software test standards.