EuroSTAR Software Testing Conference 2008 presentation on ISO 29119: The New International Software Testing Standard by Stuart Reid. See more at conferences.eurostarsoftwaretesting.com/past-presentations/

1. ISO/IEC 29119:
The New International
Software Testing Standard
Stuart Reid
sreid@testing-solutions.com
Testing Solutions Group
www.testing-solutions.com
EuroSTAR 2008 Workshop

2. Workshop Goals
• Introduce ISO 29119
• Delegate groups to:
– Review the overall scope & structure
– Review and debug the test processes
– Brainstorm the test techniques (part 4)
• What techniques do we want in it?
– Define the ongoing review process

3. Who Needs Standards?

4. What are standards?
• “Guideline documentation that reflects
agreements on products, practices, or
operations by nationally or internationally
recognized industrial, professional, trade
associations or governmental bodies”
- ISO
– Guidelines documents as they are not
compulsory unless mandated by an individual
or an organization
– Agreements because they often reflect a
certain level of consensus

5. Motivation for ISO 29119
• Lacking in current standards:
– Organizational Testing not covered
• e.g. Test Policy and Organizational Test Strategy
– no Project Test Management
– BS 7925 only covers unit testing
– common functional techniques missing
– poor coverage of non-functional testing
• Conflicts in definitions, processes & procedures
– plethora of standards to be replaced by one
• e.g. IEEE 829, IEEE 1008, BS 7925-1/-2, IEEE 1028
– practitioners do not know which standard to follow

6. Documentation
Part 3
Testing
Techniques
Part 4
Processes
Part 2
Concepts & Vocabulary
Part 1
BS 7925-2 IEEE 829
IEEE 1008
BS 7925-1
BS 7925-2
ISO 29119 – Scope & Structure

7. May
08
May
09
May
10
May
11
May
12
…
ISO 29119 - Timeline
Working Draft (WD)
Committee Draft (CD)
Final Committee Draft (FCD)
Final Draft International Standard (FDIS)
Final International Standard (FIS)
Parts 2 & 3
Parts 1 & 4
WD
CD
FCD
FDIS
FIS
WD
CD
FCD
FDIS
FIS

8. Organizational Test Strategy Process
Project Test Management Process
Test Level Process
Organizational Test Policy Process
Test Execution Cycle
Organizational Test Policy
Organizational Test Strategy
Strategise, Plan and Manage Test Project
Strategise, Plan and Manage Test Level
Design & Execute Tests
Manage Test Environment
Report Defects

9. Organizational Test Processes
Organizational Test Strategy Process
Create/Maintain
Test Strategy
Test Policy
IT Project
Management Policy
Test Strategy
Organizational Test Policy Process
Create/Maintain
Test PolicyIT Policy
Management Policy
Test Policy
Test Policy Test Policy Updates
Project Test Management Process
Organizational
Test Strategy
Comments on Organizational
Test Strategy usage

10. Manage
Conformance
with Test Policy
(TP4)
Organizational
Test Policy
[Test Policy
not working
effectively]
[Test Policy
working effectively]
Create/Update
Test Policy
(TP1)
Organizational Test Policy
Process
Organizational
Test Policy
Issue Log
Organizational
Test Policy
Update Log
Gain
Consensus
on Test
Policy
(TP2)
Approved
Organizational
Test Policy
Publish
Test Policy
(TP3)
Review
Test
Policy
(TP5)
[major revision required]
•Views of
primary
stakeholders;
•Knowledge of
the current test
practice within
the
organization;
and, where
available:
• Organization’s
Mission
Statement;
• IT Policy;
• IT Project
Management
Policy;
• Quality Policy;
• Current Test
Policy;
• Test Policy
Update Log. [Suitable period has elapsed:
• since Test Policy was published; or
• since Test Policy was last reviewed; or
• major organizational change]
Comments on
Organizational
Test Policy
usage
Draft
Organizational
Test Policy
[Test Policy changes not required]

11. Project Test Management Process
Comments on
Organizational
Test Strategy usage
Project
Test
Completion
Report
Create/Maintain
Project Test
Plan
(TM1)
Monitor and
Control
(TM2)
Report
Test Project
Completion
(TM3)
Project
Test Plan
Project
Test
Status
Reports
Project Test Plan Updates
Project Test
Management
Process
Test
Strategy
Organizational Test Strategy Process
Project Management Plan,
Product Documentation,
Software Development Plan,
Regulatory Standards
Level Test Plan,
Level Test Status Reports
Project
Test Plan
Level Test
Completion
Report
Control
Directives
Test Level Process

12. • Organizational Test Policy
• Organizational Test Strategy
• Regulatory Standards
• Project Management Plan
• Applicable product documentation
(e.g. System Requirements, Test Item Specifications)
• Software Development Plan
• Project and Product Risks
• Project Test Plan Updates
Create/Maintain
Project Test Plan
Plan
Creation of
Project
Test Plan
(TM1.2) Identify &
Analyze
Risks
(TM1.3)
Design
Project Test
Strategy
(TM1.5)Determine
Staffing
and
Scheduling
(TM1.6)Document
Project
Test Plan
(TM1.7)
Schedule,
Staffing Profile
Project Test
Strategy
Analyzed
Risks
Project
scope
Identify
Risk
Mitigation
Approaches
(TM1.4)
Gain
Consensus
on Project
Test Plan
(TM1.8)
Approved Project
Test Plan
Draft
Project
Test Plan
Project
Test
Plan Publish
Project
Test Plan
(TM1.9)
Understand
Project
Context
(TM1.1)
Mitigation
Approaches

13. Fundamental Test Process
Project
Test
Plan
Test
Level
Status
Report
Level Test
Completion
Reports
Control
Directives
Project Test Management Process
Report
Anomalies
(TL6)
Report
Test Level
Completion
(TL8)
Level Test
Plan
Test
Specification
Test
Results
[Issue
noticed]
[No issues
noticed]
Anomaly
Report
Test Level Process
Set-up &
Maintain Test
Environment
(TL4)
Plan Testing
(TL1)
Run
Tests
(TL5)
Test Environment
Readiness Report
Specific Test
Env’t Requirements
Level Test
Completion
Report
Design &
Build
Tests
(TL3)
Test Execution Cycle
Close Down
Testing
(TL7)
Control
Directives
Monitor and
Control
(TL2)
Test
Process
Info
Asset
Info
Measures
Level Test
Plan Updates
Project Management Plan,
Product Documentation,
Software Development Plan,
Regulatory Standards

14. Test Activity Descriptions
Example – Design & build tests
• Each testing activity is described using a standard template
(following ISO 24774), for example:
Purpose
The purpose of Design and Build Tests is to design and document test cases that will be executed during testing.
Entry Criteria
•the Test Plan has been agreed
•the Test Item Specification(s) is available
Inputs
•Test Plan
•Test Item Specification(s)
•System Requirements
Roles
•Test Designer
Tasks
1.Analyse the test item specification(s) to identify specification items;
2.Determine the test conditions by identifying which specification items to test using the selection criteria from the Test Plan;
3.Derive test coverage items to be exercised by applying the test case design technique(s) to the test conditions;
4.Derive test cases by selecting input values to exercise the test coverage items and corresponding expected results;
5.Create a test set by rationalising the test cases (i.e. removing redundancy, duplication, etc.);
6.Derive a test script by ordering the test cases within the test set using their pre- and post-conditions;
7.Identify and document any specific test environment requirements that are not already included in the Test Plan.
Outputs
•Test Specification
•Specific Test Environment Requirements
Outcomes
•the Test Specification has been documented.
Exit Criteria
•the Test Specification has been agreed.

15. Part 4: Techniques
• Test case design techniques
– Static testing techniques
• Reviews - inspections & walkthroughs, etc.
– Dynamic testing techniques
• Black-box, white-box, etc.
– Non-functional testing techniques
• Security, performance, usability, etc.
• Test measurement techniques (e.g. coverage)
• Appendices
– Examples of each technique, at each level of testing
– Test technique effectiveness
– Mappings to existing standards

16. Your turn....
• Session 1
– Choose the 1st topic you wish to ‘review’
– Create a group around the topic
– Review the topic
– Report back to the other groups
• Break
• Session 2
– Choose your 2nd topic
• it can be the same as your 1st topic
– Create a group around the topic
– Review the topic
– Report back

17. Topics
1. Review the overall scope & structure
2. Four-tier process model
3. Organizational testing (policy & strategy)
4. Project Test Management
5. Creating (and Maintaining) a Test Plan
6. Fundamental Test Process
7. Test techniques (part 4)
8. The ongoing review process

18. Conclusions
• We will consider the workshop feedback
• Do you want to be involved?
– Join the working group
• through your national standards body
• 6 day meetings, every 6 months
• contribute between meetings
– Contribute materials
– Review drafts
– May 2008 meeting - Berlin
• 18 participants representing 12 countries
– November 2008 (Seoul) – last week...
– May 2009 (India)

19. For more information
• sreid@testing-solutions.com – if interested
in trialling the standard on a project,
reviewing drafts or writing examples
• http://www.jtc1-sc7.org/ – access to
official documents released by WG 26