The document discusses different types and levels of software testing throughout the software development lifecycle. It describes the V-model which pairs testing activities with development activities. Component and integration testing occur at the lower levels, while system and acceptance testing occur at the higher levels. Regression testing is needed for new product versions and changes. Functional testing checks system behavior while non-functional testing examines qualities like performance, security and usability. Testing also targets the software structure and changes made between versions.

1. Software Testing Foundations #2
Testing in the Software Lifecycle
Nikita Knysh
nknysh@gmail.com
http://www.facebook.com/groups/istqb/

2. Agenda
• Levels of Test: The V-model
• Testing New Product Versions
• Types of Testing
• Testing Related to Changes

3. Development Activities
• Req specification: needs & reqs of the
customer, desired characteristics and features of the
system.
• Func system design: reqs are mapped onto system
functions & dialogs.
• Technical system design: implementation of system
is designed (architecture, interfaces, etc.).
• Component specification: task, behavior, inner
structure & interfaces are defined for each
subsystem.
• Programming.

4. Levels of Test: The V-model
Requirements Acceptance
Definition Test
Functional
System Test
System Design
Technical Integration
System Design Test
Component Component
Specification Test
Implementation &
Integration
Validation
Programming
Verification

5. V-model: Test Activities #1
• Component test: test functionality, robustness, efficiency
and maintainability of components. Techniques: white
box, black box, test-driven development.
• Integration test: test of collaboration between
components after assembling to expose faults in
interfaces and interaction between components. Test
drivers, stubs and monitors are used. Integration in the
large – integration of several software systems.
▫ Integration strategy has to be designed by test manager
taking into account system architecture, project plan and
test plan constraints. Top-down, bottom-up, ad-
hoc, backbone.

6. V-model: Test Activities #2
• System test: check if the integrated product meets the
specified requirements. System is examined from the
point of view of customer and future user. Environment
should be as close to the operation environment as
possible, but this should NOT be the customer’s
operational environment.
• Acceptance test: “Formal testing with respect to user
needs, requirements, and business processes conducted
to determine whether or not a system satisfies the
acceptance criteria and to enable the user, customers or
other authorized entity to determine whether or not to
accept the system.” [After IEEE 610]. Customer must be
involved.

7. V-model: Acceptance Test on Lower Levels
• A commercial off-the-shelf software product (COTS)
may be acceptance tested when it is installed or
integrated.
• Acceptance testing of new functionality may come
before system testing (using a prototype).

8. V-model: Forms of Acceptance Test
• According to contract
• User acceptance testing
• Operational testing
• Field testing
▫ alpha – at the producer’s site
▫ beta – at the customer’s site
Extent of acceptance test depends on application risk.

9. Testing New Product Versions
• Software maintenance:
▫ testing after maintenance (anything new or changed is
tested),
▫ testing after change of environment,
▫ testing for retirement (data archiving or data migration
into future system).
• Release: full regression test (if possible) to find side
effects of changes to the software.

10. Testing New Product Versions #2
• Testing in incremental development: several V-
models after each other, where every next "V"
reuses existing test material and adds the tests for
everything new, or for higher reliability
require-ments.

11. Types of Testing: Overview
• Functional
• Non-functional
• Testing of Software Structure
• Testing related to Changes

12. Types of Testing: Functional
Test what the system does. Tests are based on
functional specs, check system’s input-output behavior
and use black-box methods.
• Requirements-based testing focuses on single
system functions.
• Business-process-based testing focuses on whole
process consisting of many steps.

13. Types of Testing: Non-functional #1
Test how well the (partial) system carries out its
functions.
[ISO 9126]: reliability, usability, efficiency,
maintainability, portability.

14. Types of Testing: Non-functional #2
• Load test: measuring system behavior when working with the
number of users, number of transaction at a time.
• Performance test: measuring system processing speed and
response time (depending on increasing load).
• Volume test: observation of system behavior on large amount of
data (e.g. processing very large files).
• Stress test: observation of system behavior when it is
overloaded.
• Security test: against unauthorized access, denial of service
attacks, etc.
• Stability (reliability) test during permanent operation, e.g. time
between failures or failure rate within a given user profile.

15. Types of Testing: Non-functional #3
• Robustness test for response to operating errors, wrong
programming hardware failure, etc., as well as examination of
exception handling and recovery.
• Testing of compatibility (to other systems) and data conversion
(import / export).
• Testing of different configurations of the system, e.g. OS
version, UI language, hardware platform, etc.
• Usability test: ease of learning the system, ease and efficiency
of operation, etc. with respect to a specific group of users.
• Checking of documentation for compliance with the system
behavior (e.g. user manual and GUI).
• Checking of maintainability: is documentation understandable
and up-to-date? Does the system have modular structure, etc.

16. Types of Testing: Testing Software
Structure
Using white box techniques to test object’s internal
code or architecture.
• Goal is to cover all structure items.
• Most used in component and integration testing.

17. Testing Related to Changes
• Re-test of changed (modified or repaired) parts.
• Regression test of whole program (part) after
modification to ensure that no faults are introduced
as side effects of changes.

18. Volume of Regression Test
• Defect retest (confirmation test): rerunning of tests
that detected faults that have been then fixed.
• Testing of altered functionality: testing of all
program parts that were changed or corrected.
• Testing of new functionality: testing of all newly
integrated program parts.
• Complete regression test: testing of the whole
system.

19. Shrinking Regression Test Effort
Deciding which test cases we can omit:
• Only high-priority test cases (according to test plan)
are repeated.
• In the functional test, certain variations (test cases)
are omitted.
• Restriction of the test to certain configurations only
(on one OS only, in one language only, etc.)
• Restriction of the test to certain subsystems or test
levels.

20. Thank you!
http://www.facebook.com/groups/istqb/