The document discusses test management and adopting the ISO 29119 standard. It begins with an anecdotal story about issues with various software applications to illustrate the need for improved quality assurance practices. It then provides an overview of ISO 29119, describing its multi-layer model of organizational, test management, and dynamic test processes. Key aspects of ISO 29119-2 on test processes are explained, including the relationships between the different process levels and specific sub-processes. The presentation concludes by offering a test planning toolkit and listing additional resources for learning more about applying ISO standards.

1. TEST MANAGEMENT
WITH ISO 29119
TEST MANAGEMENT
ADOPTING SKILLS TO BUILD EFFICIENT TEST PROCESS
Yulia Zavertailo, Senior Test Manager

2. About me
• 11 years of work experience in Quality
Assurance field.
• Itera’s employee since 2009, consultant since
2014.
• Worked in nearshore, offshore, in different
environments.
• One of key competences is establishment of
entire test organization (Itera), QMS (KLP) and
implementation of ISO quality standards.
• IT project manager/Test leader at Telenor.
03.04.2017 / 4

3. Agenda
• A real IT story
• Introduction to ISO and ISO 29119
• Digging into ISO 29119-2:2013 test process
• Tips/toolkit
03.04.2017 / 5

4. First of all…
03.04.2017 / 6
A STORY of…
NOT Glory

5. Survey
03.04.2017 / 7

6. Ruter.no
03.04.2017 / 8

7. Gjensidige.no
03.04.2017 / 9
o Submit claim error was sent over to Gjensidige in August,
fixed, though the same issue appeared again in December …
o In addition, following bugs were also discovered:
o My claims to disappear from the list of my claims
o Couldn’t open pdf report for a claim (eventually service
desk emailed it to me…)
o Travel cards disappear from my mobile app

8. Iphone 6. IOS 9. Lock screen keyboard
03.04.2017 / 10

9. Skiforeningens mobile app
03.04.2017 / 11
I …hate Sinober!

10. Norwegian.no mob app
03.04.2017 / 12

11. Lessons learnt
• It is not enough just “to test”/quality is still low:
– > IT often fail to assure no bugs on key functions in production which cost
customers time, money and patience.
• IT solutions become insecure to use:
– >many end users cant trust software features and content.
• Value of IT digitalization is doubtful:
– Does it make sense to automate online sale of tickets if user still has to
trust paper version?
– Do our users receive relevant notifications AND at the right time?
03.04.2017 / 13

12. Quality Assurance and Quality Control
• Quality assurance is: all the planned and systematic
activities implemented within the quality system that can
be demonstrated to provide confidence that a product or
service will fulfill requirements for quality.
• Quality control is: the operational techniques and
activities used to fulfill requirements for quality.
03.04.2017 / 14
Good test process!
And across the
organization.

13. ISO STANDARDS
03.04.2017 / 15

14. What is ISO/general
• ISO is the world largest standards developing organization.
• Accepted and applied all around the world.
• NOT ONLY IT (Between 1947 and the present day, ISO has published more than 16 500 International
Standards, ranging from standards for activities such as agriculture and construction, through mechanical
engineering, to medical devices, to the newest information technology developments).
03.04.2017 / 16

15. ISO standards in nutshell
 make the development, manufacturing and supply of products
and services more efficient, safer and cleaner
 facilitate trade between countries and make it fairer
 provide governments with a technical base for health, safety and
environmental legislation, and conformity assessment
 share technological advances and good management practice
 disseminate innovation
 safeguard consumers, and users in general, of products and
services
 make life simpler by providing solutions to common problems.
03.04.2017 / 17

16. Survey
• Automotive software?
• Aircraft construction?
• Telenor?
• Itera?
03.04.2017 / 18
Quality (and safety)

17. ISO/IEC/IEE 29119
03.04.2017 / 19

18. ISO 29119. Overview
• It is a serie of standards, not just 1 standard:
– ISO/IEC 29119-1: Concepts & Definitions
– ISO/IEC 29119-2: Test Processes
– ISO/IEC 29119-3: Test Documentation
– ISO/IEC 29119-4: Test Techniques
– ISO/IEC 29119-5: Keyword Driven Testing
• Revised every 5 years
• Last revision is done 2013/09
03.04.2017 / 20

19. ISO 29119. Purpose
• The purpose of the ISO/IEC/IEEE 29119 series of software testing standards
is to define an internationally-agreed set of standards for software
testing that can be used by any organization when performing any form of
software testing.
• By implementing these standards, you will be adopting the only
internationally-recognised and agreed standards for software testing, which
will provide your organisation with a high-quality approach to testing
that can be communicated throughout the world.
• It is not an alternative to ISTQB process, since ISTQB is not a proven
standard.
03.04.2017 / 21

20. Any kind of ISO in IT in Norway...
Why we don’t use it?
My theories….
• Local companies, are happy in our traditional environment (with bugs).
• Small companies, don’t want to invest (same as above).
• Lack of ISO knowledge as such (easier with ISTQb…).
• Lazy to find out what is ISO/how to apply/use best practices. Run after juicy
modern IT stuff instead.
– Last but not least….Chip top, top bop, hop pop…and in production! Fast,
efficient, with bugs.
03.04.2017 / 22

21. ISO/IEC 29119-2: TEST PROCESSES
03.04.2017 / 23

22. Mulit-layer model
• Organizational test process:
– Processes on organizational level. F.ex test policy, test
strategy for entire organization.
• Test Management processes:
– processes related to test management of entire
project/phase/test type within a project.
• Dynamic test processes:
– Processes which go in particular phase of testing f.eks SIT,
UAT etc or certain type of testing (ex. Performance, security,
etc)
03.04.2017 / 24
Quite often
missing!

23. Mulit-layer model
03.04.2017 / 25
Organizational test processes
Dynamic test processes
Test management processes
Test planning
Test design and
implementation
Test
environment
set-up
Test
execution
Test monitoring
and control
Test compeltion
Test
incident
reporting

24. Organizational test process
03.04.2017 / 26
Organizational test process
(Applied to test policy)
Organizational test process
(Applied to test strategy)
Test Management process
(Applied to project test management)
Organizationaltestpolicy
FeedbackonOrganizationaltestpolicy
Organizational
test policy
Organizational
test strategy
Feedack on
Organizational
test policy
Feecback on
Organizational
test strategy

25. Test management processes
03.04.2017 / 27
Organizational test
processes
Dynamic test processes
Test management processes
Test planning
Test design and
implementation
Test
environment
set-up
Test
execution
Test monitoring
and control
Test compeltion
Test
incident
reporting
Test
resultsTest plan
Test
completion
report
Organizational test policy
and test strategy
Feedback on organizational
test policy and test strategy
Test plan Test
Measures
Test Status reports, test
completion report
Test management processes

26. Dynamic test processes
03.04.2017 / 28
03.04.2017 / 28
Dynamic test processes
Test design and
implementation
Test
environment
set-up
Test
execution
Test
incident
reporting
Test plan
Test
Measures
Control directives
Test management processes
Test specification
Test environment
requirements
Test environment
readiness report
Test results No issues
Issues

27. Yulia’s Toolkit
• See Excel example for test planning.
03.04.2017 / 29

28. Summary
ISTQB sub-processes ISO 29119-2:2013
• Test planning and control
• Test analysis and design
• Test implementation and execution
• Evaluation exit criteria and reporting
• Test closure activities
• Organizational test process
• Test management processes:
– Test planning process
– Test monitoring and control
process
– Test completion process
• Dynamic test processes
– Test design and implementation
– Test environment setup and
maintenance
– Test execution process
– Test incident reporting process
03.04.2017 / 30

29. Sources
• iso.org
• Google
• Life-experience/practice
• Online courses at udemy (lots of)
• Books…
03.04.2017 / 31

30. THANK YOU
03.04.2017 / 32