1) The document discusses software product line testing and optimizing test scope selection for similar software systems. 2) It identifies that directly executing the same test scope on all systems is inefficient, and that test scope should be based on identifying variation points and their impact, as well as the current risk space. 3) By understanding variation points, their impact on scope, and triggers that change the risk space, targeted test scopes can be created for each software system rather than duplicating tests across all systems.

1. Confidential
Software Product Line Testing in
Practice
Test Scope based on Risk
Rev PA1 2011-10-26 1

2. Introduction
 Software product lines, or software product line
development, refers to software engineering methods, tools and
techniques for creating a collection of similar software systems from
a shared set of software assets using a common means of
production. [1]
 It is obvious that executing the same test scope on all these similar
software systems will not be very efficient
 How is the scope selection for the different software systems
optimized?

3. Overview
Set
Scope
Identify Identify Scope
Impact of
Variation Risk based on
Variation
Points Space current
Points
Risk

4. Identify Variation Points
 The first step is to identify the variation points between the different
software systems
 Which components differ between system?
 What dependencies are specific to the software system compared to
the rest of the product line?
 Do the system resources differ in the product line?

5. Scope Impact of Variation Points
 What is the impact of less memory in one software system
compared to the rest of the product line to the test scope?
 What is the impact of a customer variant of a software system with
different UI, but otherwise identical system?
 What is the impact of including a certain feature in one software
system that is not included in the rest of the product line?
 The answers are of course context specific, but these are questions
that should be asked early – during the design of the different
software systems

6. Identify Risk Space
 Risks are completely context dependant, even though it is possible to have
check lists as support
 It is critical to identify the consequence severity and probability of each risk
for the software systems and take these into account during scope selection
High
Probability High Risk
Low
Low Risk
Probability
Low Impact High Impact

7. Risk Space Change Triggers (Example)
Risk Space Change Triggers
Bug fix Integration
Feature Integration
SW Environment Changes
Previous test results
New Quality Criteria
New requirements
Known issues
Time passed

8. Set Scope based on current Risk
Variation Points
Affect New System
Change in Risk Trigger Scope Selection Create Specific Test
Space
Scope

9. Without variation points
Not software system specific!
Change in Risk New Product Line
Trigger Scope Selection Create
Space Test Scope
Create
Waste and
Overlap in Scope

10. Solution?
 Tool support for identifying variation points
 Tool support for identifying dependencies in the software
 Risk identification process in place
 Standardized risk lists
 Risk matrixes
 Test automation to reduce risk at low cost by executing test cases to
explore and reveal the risk space

11. Conclusion
 Risk-based testing [2] is critical for software product line testing
 Identifying the variation points is the core of the problem
 Identifying the impact of the variation points on test scope is often even
more difficult to do correctly
 Identifying the risk space and what should trigger a new scope selection
adds even more complexity
 Software Product Line testing is not trivial and introduces risk if we
cannot accept a large overlap in test scopes between software systems
 Tool support is critical to manage these complexities

12. Reference
[1] Software Product Line [2011-11-08]
http://en.wikipedia.org/wiki/Software_product_line
[2] Risk Based Testing [2011-11-08]
http://en.wikipedia.org/wiki/Risk-based_testing