Inside ISO 29119 Exposed Webinar with Jon D. Hagar - XBOSoft

XBOSoft, Inc. All Rights Reserved. 1
XBOSOFT
Inside ISO 29119 Exposed
with Jon Hagar,
Senior Consultant
#ISO29119
#InsideISO

XBOSoft
Dedicated to software quality improvement
Founded in 2006
We speed products to market with our expert:
• Software QA consulting
• Software testing
Global team with offices in San Francisco,Beijing
and Amsterdam
“Thorough, accurate and fast”

House Rules
 Participants other than the speakers are muted
 Questions via the GoToWebinar control on the right side of
your screen or through Twitter @XBOSoft
 Questions may be asked throughout the webinar - we’ll try
to answer them at the end
 You will receive info on recording after the webinar
 #ISO29119
 #InsideISO

Meet Our Speakers
Philip Lew
CEO and Founder , XBOSoft
• Relevant specialties and
passions
o Software quality process,
evaluation, measurement
and improvement
o Software quality in use /
UX design
o Mobile User Experience
and usability
o Cycling and travel
Karla Smith
Marketing Manager, XBOSoft
• Emails you about our
industry hot topic
webinars
• Invites you to download
our latest trends in
software testing
whitepapers
XBOSoft Team

Jon Hagar
• More than thirty years’ experience in software
and testing
• Systems-software tester consultant and trainer
• Supports software product integrity, testing,
verification, and validation with a specialization
in embedded, IoT, and mobile software systems
• Projects supported include control system (avionics and automotive), mobile-smart
devices, and ground systems (IT and PC).
• Publishes and writes regularly including his book: Software Test Attacks to Break
Mobile and Embedded Devices, CRC press.
• Lead editor/author on numerous international standards including ISO 291119
software testing.

Copyright 2013 Jon Hagar Grand Software Testing GST
Jon D. Hagar, Senior Consultant
embedded@ecentral.com
Author: Software Test Attacks to Break
Mobile and Embedded Devices
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
6

Current Situation
 Controversy
(https://www.youtube.com/watch?v=RCl8lGxY0W
U)
 Many people do NOT know much about it
 Starting to go into use in places in the world

Scope
 Purpose of standards
 Motivation for ISO 29119 standards
 Overview of ISO29119
 Applicability
 Thoughts
 Q&A

Some Preliminaries
 Under disclosure, I am the IEEE Project Editor for
ISO/IEEE/IEC 29119 Software Testing Standard
 For many organizations:
 Standards are a fact of life for many industry professionals
not just testers
 Standards form part of the business for many companies ,
customers, and countries
 We end up in a mixed relationship with standards
 Standards should be viewed as a baseline

What are standards?
 Guideline documents—not compulsory unless
mandated
 Agreements because they often reflect a certain level of
consensus within a community
“Guideline documentation that reflects
agreements on products, practices, or
operations by nationally or internationally
recognized industrial, professional, trade
associations or governmental bodies”
- ISO

How can a standard “help?”
 Standards are a basis for:
 Communication – common terminology (trading language)
 Professionalism and criticism
 Conformance/compliance schemes
 Benchmark of ‘industry practice’ (not best, may not be good
for all, etc.)
 Contracts and regulations
 Interoperability and consistency. . .
Standards describe a current ‘body of
knowledge’ that provides the basis for a
professional discipline

Poll
 Would you consider using a standard if it solved problems
your team might have and could be tailored to better fit
your needs?
 Yes
 No

Motivation for ISO29119
 Conflicts in definitions, processes and procedures
 “One ring to rule them all” — old testing standards to be replaced by
ISO 29119
 e.g., IEEE 829, IEEE 1008, BS7925-1/-2, IEEE 1028
 Lacking in current standards
 Organizational areas
 e.g., Test Policy and Organizational Test Strategy
 Project Test Management
 BS7925 only covers unit testing
 General processes
 Common functional techniques missing
 Coverage of non-functional testing
 Part of the larger ISO/IEEE family of systems and software
standard, e.g. ISO 12207, 15288, IEEE1012

Standardization Bodies

ISO/IEC/IEEE 29119 –Structure and History
BS7925-1
BS7925-2 IEEE 829
Concepts & Vocabulary
Part 1
Process
Assessment
Testing
Techniques
Part 4
Documentation
Part 3Part 2
Processes
Keyword-Driven
Testing
Part 5 ISO/IEC 33063
ISO 12207
ISO 15288
Directives
IEEE 1008

Part 1: Concepts & Vocabulary
SOFTWARE TESTING CONCEPTS
Scope, Conformance, Normative References
TESTING IN DIFFERENT LIFE CYCLE MODELS
ROLES AND RESPONSIBILITIES IN TESTING
ANNEXES – Metrics, Examples, Bibliography
DEFINITIONS
Test:Approach,Basis,Methods
-RiskBasedTesting

Part 2: Testing Processes
TEST MANAGEMENT PROCESSES
ORGANIZATIONAL TEST PROCESS
DYNAMIC TEST PROCESSES

Instantiating Testing Processes
Ref: S. Reid

Organizational
Test
Specification
Develop
Organizational
Test
Specification
(OT1)
Monitor and
Control Use of
Organizational
Test
Specification
(OT2)
Updated
Organizational
Test Specification
Update
Organizational
Test
Specification
(OT3)
Controlled
Organizational
Test
Specification
Organizational Test Processes

TEST
PLANNING
TEST
MONITORING &
CONTROL
TEST
COMPLETION
ORGANIZATIONAL
TEST
DOCUMENTATION
FEEDBACK ON
ORGANIZATIONAL TEST
DOCUMENTATION
TEST PLAN UPDATES
TEST
PLAN
TEST
COMPLETION
REPORT
DYNAMIC TEST
PROCESSES
TEST
MANAGEMENT
PROCESSES
TEST PLAN,
TEST COMPLETION
REPORT,
TEST MEASURES
TEST
MEASURES
TEST PLAN,
CONTROL
DIRECTIVES
TEST PLAN,
CONTROL
DIRECTIVES
Test Management Processes

Organize
Test Plan
Development
Identify &
Estimate Risks
Design Test
Strategy
Determine
Staffing and
Scheduling
Document
Test Plan
Schedule,
Staffing
Profile
Test
Strategy
Analyzed
Risks
Scope
Identify Risk
Treatment
Approaches
Gain
Consensus on
Test Plan
Approved
Test Plan
Draft
Test Plan
Test
Plan Publish
Test Plan
Understand
Context
Treatment
Approaches
Test Planning Processes

Part 2 Standard Primarily has a Risk-based Process Flavor
 Risk based testing is a primarily “good” practice
 Other bases/approaches (Agile, exploratory, model-based, math-
based, etc.) are included
 All parts may need “tailoring” (few efforts would be “full” conformance)
 Use of the standard implies skills, thinking, dependence on
context (“it depends”), etc.

Test Monitoring & Control Process
Monitor
Report
measures
ControlSet-Up
[testing
complete]
[testing incomplete]
control
directives
test
progress
info
test progress
info
test
plan
test
measures
test status report
test
control
info
...TEST PROCESSES...
Dynamic/Management

TEST
SPECIFICATION
[NO ISSUES
NOTICED]
TEST
ENVIRONMENT
READINESS
REPORT
TEST
ENVIRONMENT
REQUIREMENTS
INCIDENT
REPORT
TEST DESIGN &
IMPLEMENTATION
TEST
ENVIRONMENT
SET-UP
TEST
EXECUTION
TEST
INCIDENT
REPORTING
(LEVEL) TEST MANAGEMENT PROCESS
[ISSUE NOTICED
or
RETEST RESULT]
TEST
RESULTS
CONTROL
DIRECTIVES TEST
MEASURES
(LEVEL)
TEST PLAN
Dynamic Test Processes

Part 3 – Test Documentation
TEST DOCUMENTATION
ANNEXES - EXAMPLES
Scope, Conformance,
Normative References
Selectasubsetofdocs

Part 3: Test Documentation
 Organizational test documentation
 Test policy
 Test strategy
 Project test documentation
 Project test plan
 Test project completion report
 Test Level documentation
 Test plan
 Test specification
 Test results
 Anomaly reports
 Level test status report
 Test environment report
 Test level completion report
 Appendices
 Examples of documents at each level of testing
Not all need
to be
produced
when
tailoring is
used-
Most will
NOT
produce all
of these

Part 4 – Test Techniques
(possible publish in 2015)
TEST COVERAGE MEASUREMENT
Scope, Conformance, Normative References
ANNEXE – TESTING OF QUALITY CHARACTERISTICS
ANNEXE – SELECTION OF TECHNIQUES
ANNEXE – TEST TECHNIQUE EFFECTIVENESS
TEST DESIGN TECHNIQUES
Functional Structural

Part 5- Keyword-Driven Testing
(possible publish in 2016-17)
 Part 5 addresses:
 Concept
 Applicability
 Interfaces
 Approach
 Part 5 in review and ballot

Applicability of ISO/IEEE 29119
 It is intended to be a generic standard to support different
environments and domains of testing when selected
 Some domains are more standards oriented such as:
 Safety-related
 Telecoms
 International business and large companies
 Some domains are more regulated world-wide such as:
 Financial – banks, stock markets, etc.
 Medical
 Can support various lifecycle models
 Tailored to sequential, iterative and agile (highly tailored)
 Applicability to new as well as maintenance projects

Organization where the Standard Might Be Used
 Providing users a common baseline of testing concepts, terms, and ideas
 Trading Language
 Government organizations and regulators needing an industry
approved starting baseline
 Purchasers and procurers of software products to assess third parties
 Developers of software, e.g. to support unit level testing.
 Training course syllabus developers and providers
 University lecturers wishing to teach from an ISO standards
 University researchers wishing to conduct research
 Software testing tool vendors
 Organizations and regulators using the standard for accreditation
 International business and organization who use other ISO standards

Do Testers Need Standards? –
Not all, but some may
 Standards support common communication within the topic
 Common reference points
 Starting point for usage
 Maturity of the standard is an issue but a baseline serves as
sounding board and common reference point for “Scientific”
method
 An international benchmark
 Thinkers and researchers can prove/disprove benchmark(s)
 Part of being in a profession (but only part)
 Standards are not for everyone

ISO29119 Standard - My Viewpoint
 Any software standard is based on limited history (rear
looking) and developed based on consensus of many voices
 Represent only some of viewpoints of industry
 Will expand to include more testing ideas
 Not perfect
 Most using projects/professionals will use “tailoring”

My Thoughts on how to tailor
 Thinking testers with a context-driven/based viewpoint
 Keep “good” (worthwhile)
 Remove or change what is not worthwhile
 Use standards as a professional reference work, but
know when and how to break the “rules” via tailoring
 Get stakeholder buy in – Fight to Educate

Summary
 First attempt at a comprehensive software test
standard
 Even in the presence of standards, a thinking
tester is always needed
 Get it read it, tailor it (as needed), and form your
own opinions by “testing it” yourself
 Send me your thoughts and changes
 Not universal, best, or comprehensive

For more information
 jon.d.hagar@gmail.com
 For more information and details
 http://softwaretestingstandard.org/
 WG26 website
 http://www.jtc1-sc7.org/
 Access to official documents released by WG 26

Stuart Reid
Testing Solutions Group
117 Houndsditch
London EC3
UK
Tel: 0207 469 1500
Fax: 0207 623 8459
www.testing-solutions.com
Major Reused Reference Point:

Post your questions on Twitter and we'll answer them @XBOSoft
Join us to keep updated on all our webinars, reports and whitepapers:
facebook.com/xbosoft
+xbosoft
linkedin.com/company/xbosoft
We post regularly on our blog – check us out! http://xbosoft.com/software-quality-blog/
Download our free whitepapers, available here:
http://xbosoft.com/knowledge-center/
You will receive an email with information on today’s webinar slides and recording.
Any further queries regarding our services or ideas for future webinars, please email us!
Services@xbosoft.com
Q+A
www.xbosoft.com

Inside ISO 29119 Exposed Webinar with Jon D. Hagar - XBOSoft

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Inside ISO 29119 Exposed Webinar with Jon D. Hagar - XBOSoft

Similar to Inside ISO 29119 Exposed Webinar with Jon D. Hagar - XBOSoft (20)

More from XBOSoft

More from XBOSoft (20)

Recently uploaded

Recently uploaded (20)

Inside ISO 29119 Exposed Webinar with Jon D. Hagar - XBOSoft