iSQI Certification Days ISTQB Advanced Axel Rennoch

NEW SYLLABI ON
QUALITY ENGINEERING
©MatthiasHeyde/FraunhoferFOKUS
Ina Schieferdecker, Axel Rennoch
iSQI Certification days, November 9, 2017

2
© Fraunhofer FOKUS
OVERVIEW ON ISTQB SYLLABI

3
CERTIFIED TESTER FOUNDATION LEVEL 2011

4
1. Model-based testing, test automation, IoT testing and others are
(still) current topics in QA research …
2. … but not yet in industrial application
3. Often, they are not yet established on the market on a wider scale
and are not marketing hypes like agile testing, mobile testing, et al
Syllabi and people certifications
Increase industrial acceptance and deployment through growing
networks of practitioners and industrial applicants
Educate, train and certify industrial people and researcher
STATUS OF ADVANCED TESTING IN ENGINEERING

5
© Fraunhofer FOKUS
1. Test Automation Engineer
2. Model-Based Testing
3. Quality Engineer for IoT
OUTLINE

6
© Fraunhofer FOKUS
OUTLINE

7
Why a Model-Based Tester certification at
foundation level?
1. Test automation is an advanced test engineering approach to improve effectivity and
efficiency most often of the test execution process but not only.
2. The syllabus is about the tasks of a test automation engineer (TAE) in designing,
developing, and maintaining test automation solutions. It focuses on the concepts,
methods, tools, and processes for automating dynamic functional tests and the
relationship of those tests to test management, configuration management, defect
management, software development processes and quality assurance.
3. Methods described are generally applicable across variety of software lifecycle
approaches (e.g., agile, sequential, incremental, iterative), types of software systems
(e.g., embedded, distributed, mobile) and test types (functional and non-functional
testing).
TAE CERTIFICATION AT ADVANCED LEVEL – IN SCOPE

8
foundation level?
The ISTQB glossary of software testing terms defines Test
Automation as »The use of software toperformor support
test activities, e.g., test management, test design, test
executionandresultschecking. «
1. Test automation is an advanced test engineering approach to improve effectivity and
efficiency most often of the test execution process but not only.
2. The syllabus is about the tasks of a test automation engineer (TAE) in designing,
developing, and maintaining test automation solutions. It focuses on the concepts,
methods, tools, and processes for automating dynamic functional tests and the
relationship of those tests to test management, configuration management, defect
management, software development processes and quality assurance.
3. Methods described are generally applicable across variety of software lifecycle
approaches (e.g., agile, sequential, incremental, iterative), types of software systems
(e.g., embedded, distributed, mobile) and test types (functional and non-functional
testing).
TAE CERTIFICATION AT ADVANCED LEVEL – IN SCOPE

9
foundation level?
1. Test management, automated creation of test specifications and
automated test generation.
2. Tasks of test automation manager (TAM) in planning, supervising and
adjusting the development and evolution of test automation solutions.
3. Specifics of automating non-functional tests (e.g., performance).
4. Automation of static analysis (e.g., vulnerability analysis) and static test
tools.
5. Teaching of software engineering methods and programming (e.g., which
standards to use and which skills to have for realizing a test automation
solution).
6. Teaching of software technologies (e.g., which scripting techniques to use
for implementing a test automation solution).
7. Selection of software testing products and services (e.g., which products
and services to use for a test automation solution).
TAE CERTIFICATION AT ADVANCED LEVEL – OUT OF SCOPE

10
1. A Test Automation Engineer is one who
– has broad knowledge of testing in general, and
– an in-depth understanding in the special area of test automation
2. An in-depth understanding is defined as having
– sufficient knowledge of test automation theory and practice
– to be able to influence the direction that an organization and/or
project takes
– when designing, developing and maintaining test automation
solutions for functional tests.
TAE COURSE OBJECTIVES

11
1. Contribute to the development of a plan to integrate automated testing within the
testing process
2. Evaluate tools and technology for automation best fit to each project and
organization.
3. Create an approach and methodology for building a test automation architecture
(TAA).
4. Design and develop (new or modified) test automation solutions that meet the
business needs.
5. Enable the transition of testing from a manual to an automated approach.
6. Create automated test reporting and metrics collection.
7. Manage and optimize testing assets to facilitate maintainability and address
evolving (test) systems.
TAE SYLLABUS BUSINESS OBJECTIVES

13
• The introduction of test automation are ONLY reasonable and
even only possible with existing, well-established testing
processes
– “Automating chaos just gives faster chaos!”
– “A fool with a tool is just a fool!”
• Note: The introduction of test automation is often connected to short
term productivity loss.
– Test automation is therefore no solution for short term staff
shortages.
TEST AUTOMATION AND THE FUNDAMENTAL TEST PROCESS

14
TEST AUTOMATION AND FUNDAMENTAL TEST ACTIVITIES
Harder to automate
Identify and priortise
test conditions
Design test cases
(generate them automatically
e.g. with MBT)
Implement test cases
(automatically e.g.
with TTCN-3)
Rather single
design
Identification
Design
Implementation

15
Easy to automate
(automatically e.g.
with TTCN-3)
Execute test cases
(automatically e.g. with
TTCN-3 or JUnit)
Rather many
repetitions
Evaluate
test case executions
(automatically e.g.
with TTCN-3)
Implementation
Execution
Evaluation

16
Harder to automate
Easy to automate
Identify and priortise
test conditions
Design test cases
(generate them automatically
e.g. with MBT)
(automatically e.g.
with TTCN-3)
Execute test cases
(automatically e.g. with
TTCN-3 or JUnit)
Rather single
design
Rather many
repetitions
Evaluate
test case executions
(automatically e.g.
with TTCN-3)
Identification
Design
Implementation
Execution
Evaluation
Automating activities of the
fundamental test process

17
 TTCN-3 is the Testing and Test Control Notation
 Internationally standardized testing language for formally
defining test scenarios. Designed purely for testing
EXCURSUS: CHALLENGE TEST AUTOMATION
testcase Hello_Bob () {
p.send(“How do you do?“);
alt {
[]p.receive(“Fine!“);
{setverdict( pass )};
[else]
{setverdict( inconc )} //Bob asleep!
}
}

18
TTCN-3 HISTORY
 TTCN (1992)
 published as ISO standard
 “Tree and Tabular Combined Notation”
 used for protocol tests:
GSM, N-ISDN, B-ISDN
 TTCN-2/2++ (1997)
 enhancements by ETSI MTS
 module concept, concurrency
 used for conformance tests
TTCN-2
TTCN-2++
1997
1984
1994
1992
TTCN

19
TTCN-3 HISTORY (CONT.)
 TTCN-3 (2000)
 further development by ETSI MTS
 Testing and Test Control Notation
 standardised test specifications:
 SIP, SCTP, M3UA, IPv6
 HiperLan, HiperAccess, WiMAX
 3GPP LTE, NB-IoT
 OMA
 TETRA
 MOST, AUTOSAR
 EUROCONTROL
 oneM2M
2000
1998
2017
u

20
 One test technology for different tests
 Distributed, platform-independent testing
 Integrated graphical test development,
documentation and analysis
 Adaptable, open test environment
 Areas of Testing
 Regression testing
 Conformance and functional testing
 Interoperability and integration testing
 Real-time, performance, load and stress testing
 Security testing
 Used for system and product qualification and certification,
e.g. for GCF/PTCRB certification of handsets (mobile phones)
DESIGN PRINCIPLES OF TTCN-3

21
PROCESS OF TEST AUTOMATION IMPLEMENTATION
Automated test process design and
TAS development
TAS pilots and review of
pilot experiences
Refinement of automated test
process and adoption
User training and broad-scale
deployment
Accompanying TAS
maintenance and extension
including user coaching

22
PROCESS OF TEST AUTOMATION IMPLEMENTATION
Automated test process design and
TAS development
TAS pilots and review of
pilot experiences
Refinement of automated test
process and adoption
User training and broad-scale
deployment
Accompanying TAS
maintenance and extension
including user coaching
Any test automation is done by software which
has an architecture and may use software tools

23
WHY A GENERIC TEST AUTOMATION ARCHITECTURE
 Different types of software
 Different programming paradigms
 Different programming technologies
However, all share common concepts
Software …
 Provides services
 Implements algorithms
 Uses data
 Is accessible via interfaces

24
 Different types of software
 Different programming paradigms
 Different programming technologies
However, all share common concepts
Software …
 Provides services
 Implements algorithms
 Uses data
 Is accessible via interfaces
So does testing software …
● Provides testing services
● Implements test cases
● Uses test data
● Is accessible via test interfaces

25
Defines generic solution for tasks being similar in any test automation
project
Defines re-occurring concepts, steps, and approaches in automating testing
Test Automation
Solution TAS
(Specific) Test
Automation
Architecture TAA
Generic Test
Automation
Architecture
gTAA

26
Defines generic solution for tasks being similar in any test automation
project
Defines re-occurring concepts, steps, and approaches in automating testing
Test Automation
Solution TAS
(Specific) Test
Automation
Architecture TAA
Generic Test
Automation
Architecture
gTAA
1. Defining the concepts, layers,
services, and interfaces of a
TAS
2. Supporting simplified
components for the effective
and efficient development of test
automation
3. Re-using test automation
components for different or
evolving TASs
4. Easing the maintenance and
evolution of TASs
5. Defining the essential
features for a user of a TAS

27
• Single responsibility: Every TAS component must have a single
responsibility, and that responsibility must be encapsulated entirely in the
component.
• Extension (see e.g., open/closed principle by B. Myer): Every TAS
component must be open for extension, but closed for modification.
• Replacement (see e.g., substitution principle by B. Liskov): Every TAS
component must be replaceable without affecting the overall behaviour
of the TAS.
• Component segregation (see e.g., interfaces segregation principle by
R.C. Martin): It is better to have more specific components than a
general, multi-purpose component.
• Dependency inversion: The components of a TAS must depend on
abstractions rather than on low-level details.
DESIGN PRINCIPLES FOR A TAS

GTAA OVERVIEW 1 (4)
 Test automation to be
interconnected with
• Project management
• Configuration management
• Test management
 Often realized by use of a Test
Automation Framework such
as
• JUnit
• TTCN-3

GTAA OVERVIEW 2 (4)
 Four distinct layers:
• Test Generation Layer
• Manual or automated design of test cases
• Means for designing test cases
• Test Definition Layer
• Definition and implementation of test suites and/or
test cases
• Separates the test definition from the SUT and/or test
system technologies and tools
• Test Execution Layer
• Execution of test cases and test logging
• Test Adaptation Layer
• Adapt the automated tests for the various components
or interfaces of the SUT
• Provides different adaptors for connecting to the SUT
via APIs, protocols, services, and others

GTAA OVERVIEW 3 (4)
 Further details to the layers follow
 Important: A TAS may contain some, but not all layers
• A test execution TAS will have test execution and test adaption layers
• A test generation TAS will have test generation and test definition layers, and may also
have test execution and test adaptation layers

GTAA OVERVIEW 4 (4)
• The interfaces between the gTAA
layers and their components are
typically specific and, therefore,
not further elaborated
• Most often, one would start with the
implementation of a TAS from
bottom to top
• It is advised to implement the TAS
in incremental steps (e.g., in
sprints) in order to use the TAS as
soon as possible and to prove the
added value of the TAS
• Proofs of concept are
recommended as part of test
automation project

32
• 40 multiple choice questions, with a pass mark grade of 65% to be
completed within 90 minutes.
• Participants that take the exam not in their spoken language, will
have a total of 113 min.
• The typical duration of the Advanced Level Test Automation Engineer
Training offered by an Accredited Training Provider is 3 days.
ADVANCED LEVEL TEST AUTOMATION ENGINEER EXAM
STRUCTURE

34
foundation level?
1. Model-based testing is an innovative test approach to improve
effectivity and efficiency of the test generation process.
2. A model-based tester on a project uses models to drive test
analysis and design, and keeps advantage of the models for other
testing activities such as test implementation and reporting.
3. ISTQB® Model-Based Tester certification complements the core
foundation level as a specialist module.
4. It provides a practical and easy entry to the MBT approach.
WHY MBT CERTIFICATION AT FOUNDATION LEVEL?

35
foundation level?
The ISTQB glossary of software testing termsdefines
Model-Based Testing as »Testingbasedonor
involvingmodels«
1. Model-based testing is an innovative test approach to improve
effectivity and efficiency of the test generation process.
2. A model-based tester on a project uses models to drive test
analysis and design, and keeps advantage of the models for other
testing activities such as test implementation and reporting.
3. ISTQB® Model-Based Tester certification complements the core
foundation level as a specialist module.
4. It provides a practical and easy entry to the MBT approach.
WHY MBT CERTIFICATION AT FOUNDATION LEVEL?

36
1. In general, an ISTQB Certified Tester Model-Based Testing
Essentials has acquired the necessary skills to successfully
contribute to MBT projects in a given context.
– Practical modeling approach (excercises, …)
– On a foundation level
– On a technical level
2. Target audience – practitioners rather than enablers
– Main target: test analysts
– but also: test and project managers, analysts, quality engineers,
et al.
MBT COURSE OBJECTIVES

37
1. Collaborate in a model-based testing team using standard
terminology and established MBT concepts, processes and
techniques.
2. Apply and integrate model-based testing in a test process.
3. Effectively create and maintain MBT models using established
techniques and best practices of model-based testing.
4. Select, create and maintain test artifacts from MBT models
considering risk and value of the features tested.
5. Support the organization to improve its quality assurance
process to be more constructive and efficient.
MBT SYLLABUS BUSINESS OBJECTIVES

38
MBT SYLLABUS OVERVIEW: LEARNING OBJECTIVES

39
MBT is not self-propelling!
– Requires access to and acceptance from MBT and testing
communities
– Requires providers with specific industrial MBT expertise and
tooling
– Requires a network of providers with the objective to foster MBT
as part of their core business
OUTLOOK
A GOOD MBT TESTER HAS
MORE THAN TECHNICAL
KNOWHOW, HE/SHE HAS
ADEQUATE TESTING SKILLS
Training
Communication

40
• 40 multiple choice questions, with a pass mark grade of 65% to be
completed within 60 minutes.
• The typical duration of the Model-Based Tester Certification Training
offered by an Accredited Training Provider is 2 days.
• Online Syllabus, sample exam questions with
answers/justifications.
FOUNDATION LEVEL CERTIFIED MODEL-BASED TESTER

41
OUTLINE

42
• ASQF/GTB Certified Professional for IoT Foundation Level
• New syllabus for 3-day lectures (starts 2018)
QUALITY ENGINEERING FOR THE INTERNET OF THINKS
Business Outcomes
 Awareness raising
 Standards overview
 Competence landscape
 Processes for quality engineering in organisations
 Personal qualification
NEW FOUNDATION LEVEL: QE FOR IOT

43
1. Motivation and challenges
2. Constructive QE – quality characteristics
3. Constructive QE – IoT architecture
4. Constructive QE – processes and and methods
5. Analytic QE incl. testing
QE4IOT SYLLABUS CONTENTS

44
1. Motivation and challenges
2. Constructive QE – quality characteristics
3. Constructive QE – IoT architecture
4. Constructive QE – processes and and methods
5. Analytic QE incl. testing
6. IoT Lifecycle
QE4IOT SYLLABUS CONTENTS
source:[ISO/IECCD30141]

45
 Education and training gain importance
 Certified courses are also a mean for entering research fields
 Researchers may actively drive new initiatives and syllabi
 Also PhD students should be taught
 Recent developments are
 Model-Based Testing
 Test Automation Engineering
 Quality Engineering for IoT
 Security Testing
 Data science
 But not yet
 Evolutionary software engineering
 Model-driven engineering
 General artifical intelligence
 …
SUMMARY

46
CONTACT
Fraunhofer-Institut für
Offene Kommunikationssysteme
FOKUS
Kaiserin-Augusta-Allee 31
10589 Berlin, Germany
info@fokus.fraunhofer.de
www.fokus.fraunhofer.de
Director
Prof. Dr. Ina Schieferdecker
Tel. +49 (30) 34 63 -7241
ina.schieferdecker@fokus.fraunhofer.de
Axel Rennoch
Tel. +49 (30) 34 63 -7344
axel.rennoch@fokus.fraunhofer.de

iSQI Certification Days ISTQB Advanced Axel Rennoch

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to iSQI Certification Days ISTQB Advanced Axel Rennoch

Similar to iSQI Certification Days ISTQB Advanced Axel Rennoch (20)

More from Ievgenii Katsan

More from Ievgenii Katsan (20)

Recently uploaded

Recently uploaded (20)

iSQI Certification Days ISTQB Advanced Axel Rennoch