SlideShare a Scribd company logo
© 2021 Sogeti Nederland B.V.. All rights reserved.
Academy
TMAP quality engineering
16 April 2021
Workshop
© 2021 Sogeti Nederland B.V.. All rights reserved.
Academy
TMAP: Quality engineering
- Built-in quality & VOICE model
- Quality engineering & cross-functional teams
- Unit testing, mutation testing and path testing
Three-hour workshop based on the TMAP certification scheme
Presented by: Rik Marselis – Principal Quality Consultant
Friday 16 April 2021 14:00 – 17:15 (including 2 short breaks)
TMAP: the body of knowledge for
quality engineering in IT delivery
3
© 2021 Sogeti Nederland B.V.. All rights reserved.
This workshop is based on training material
from the TMAP certification scheme.
Created by
Exam provider
There are local training providers in multiple countries.
Agenda:
- built-in quality, the VOICE model and quality engineering topics
- code coverage types for unit testing and mutation testing
- path testing
+ 17:15 end (we will have breaks at logical moments in the workshop)
At the end 3 people will win a free download of the ePub of our book➔
Welcome to this workshop!!
Bookshop:
www.ict-books.com
(or on your own bookshop e.g.
amazon.nl or bol.com)
4
© 2021 Sogeti Nederland B.V.. All rights reserved.
Amstelveen
Where are you from? ➔ put your country & city in the chatbox
5
© 2021 Sogeti Nederland B.V.. All rights reserved.
Rik Marselis
Principal Quality Consultant
2007 2012 2012
2008 2014
2009
2018
1980 2020
6
© 2021 Sogeti Nederland B.V.. All rights reserved.
Built-in quality and VOICE model
7
© 2021 Sogeti Nederland B.V.. All rights reserved.
Challenges of today’s high-performance IT delivery
The business demands:
• Deliver business value
• Deliver quality at speed
The team challenges are:
• Quality engineering is everyone's responsibility
• QA & testing is integrated in process and people
The focus is:
• Organize high-performing cross-functional teams (you build it, you run it!)
• Automate everything
Using
AI ?
8
© 2021 Sogeti Nederland B.V.. All rights reserved.
Waterfall V-model Scrum DevOps Demand-Supply SAFe®
© 2021 Sogeti Nederland B.V.. All rights reserved.
9
© 2021 Sogeti Nederland B.V.. All rights reserved.
High-performance IT delivery
Definition:
High-performance IT delivery is an approach that enables cross-functional teams to continuously
improve the products, processes and people that are required to deliver value to the end users.
To achieve this continuous improvement quality must be built-in in not only the product but also
in the process (activities) and the people (individuals and the team as a whole).
Quality
Product People
Process
10
© 2021 Sogeti Nederland B.V.. All rights reserved.
Built-in quality
Built-in quality is one of the key principles in the Lean approach.
Built-in quality, continuously improved, leads to right-first-time, and
where the outcome of the process meets the expectation: fit-for-purpose.
Quality of the product is strongly linked to the quality of the delivery
process and the people.
Any deviation of the expected product quality should lead to
improvement,
not just fixing the fault
but also changing the process and people
to prevent such faults from occurring again.
Source of picture: book “The PointZERO vision” 2012
11
© 2021 Sogeti Nederland B.V.. All rights reserved.
Continuous quality engineering is a team responsibility
Quality engineering needs many activities to take place.
TMAP defines the QA & Testing topics to group all quality engineering activities.
These topics should be applied by all cross-functional (such as Agile or DevOps) team members.
Definition:
Quality Engineering is about team members and their stakeholders taking joint responsibility to
continuously deliver IT systems with the right quality at the right moment to the businesspeople
and their customers. It is a principle of software engineering concerned with applying quality
measures to assure built-in quality.
12
© 2021 Sogeti Nederland B.V.. All rights reserved.
Quality engineering topics
Aimed at: orchestrating, arranging, planning,
preparing and controlling
Focus of multiple teams.
Aimed at: the operational QA & testing activities
Mainly focused on the scope of one team. Note: The division in topics is not black-and-white, some
topics may be relevant for both groups, but the emphasize
is on one or the other.
13
© 2021 Sogeti Nederland B.V.. All rights reserved.
Put your ideas in the chatbox
Fundamental
DevOps
activities
How would you map the Quality engineering topics to DevOps?
Organizing
topics
Performing
topics
14
© 2021 Sogeti Nederland B.V.. All rights reserved.
Business value
Organizations want to deliver business value to their customers.
Today’s organizations expect that their IT systems will enable them to generate business value.
So first the organization must be clear what business value they pursue.
This may be financial value but also value in any other quantitative or qualitative way.
The focus on business value helps people remember that IT delivery itself is not the goal.
15
© 2021 Sogeti Nederland B.V.. All rights reserved.
Objectives
To support the business delivery there is IT delivery.
How the pursued value can be achieved is detailed in objectives that drive IT delivery.
16
© 2021 Sogeti Nederland B.V.. All rights reserved.
Indicators
Indicators define a way to get insight into specific objectives.
Measuring these indicators is done by testing and other quality and value measuring activities.
17
© 2021 Sogeti Nederland B.V.. All rights reserved.
Confidence in pursued value
Measuring the indicators results in information based on which the people involved can form an
opinion and determine to what extent they are confident that the pursued value can be achieved.
18
© 2021 Sogeti Nederland B.V.. All rights reserved.
Experience the real value
When the people involved have enough confidence, they can decide to start using the (new or
changed) (parts of) the IT system.
The organization (the users and/or their customers) can now experience the real value.
19
© 2021 Sogeti Nederland B.V.. All rights reserved.
Value improvement loop
Based on the experience of the real value the people involved will get new ideas to further
improve the business value which results in the value improvement loop of the VOICE model.
20
© 2021 Sogeti Nederland B.V.. All rights reserved.
Supply information
based on
measuring indicators
The VOICE model for delivering business value
Measuring indicators
=
(mainly) Testing
21
© 2021 Sogeti Nederland B.V.. All rights reserved.
The VOICE model relates to any IT delivery model.
The VOICE model is used to structure the work that
the team must do.
Quality engineering is one of the many task-groups
that need to be structured.
The use of the VOICE model is not limited to
structuring quality engineering, but is it not fully
worked out to all other task-groups.
22
© 2021 Sogeti Nederland B.V.. All rights reserved.
Translating Business Value to IT delivery objectives
The defined business value is the starting point for quality assurance and testing. It's important to
understand the purpose of an IT system and how to create maintain it. Therefor there must be
quantifiable objectives with an adequate level of detail.
high quality is not necessarily high value
23
© 2021 Sogeti Nederland B.V.. All rights reserved.
Exercise
What indicator(s) would you use in your current project/organization
to measure if you are reaching the IT-objectives
for the pursued Business-Value?
Put your answer in the chatbox
24
© 2021 Sogeti Nederland B.V.. All rights reserved.
This is section 4.1 of the book “Quality for DevOps teams”
25
© 2021 Sogeti Nederland B.V.. All rights reserved.
Unit testing and mutation testing
26
© 2021 Sogeti Nederland B.V.. All rights reserved.
Code coverage for unit testing
27
© 2021 Sogeti Nederland B.V.. All rights reserved.
Code coverage
Code coverage indicates what part of the code is being tested
Code coverage is measured while running tests (manual check = labour-intensive!)
Degree of code coverage depends on the type of coverage you want to achieve:
1. Line Coverage every line of code is executed
2. Statement Coverage every statement in the code is executed
3. Decision Coverage all possible conditional branches are tested
4. Branch coverage all possible branches (conditional and unconditional) are tested
5. Path coverage every path or path combination in a flow is tested
Worst coverage
Best coverage
28
© 2021 Sogeti Nederland B.V.. All rights reserved.
Code coverage – Line coverage
Process-oriented
• Every line is exercised
• Not every statement is necessarily exercised
Testdata
nop = 1
lop = 250
Result
100% line coverage
Although MOVE lop TO lsp is not exercised the line counts
Application code
DECLARE nop NUMERIC
DECLARE lop NUMERIC
DECLARE lsp NUMERIC
READ nop FROM input-screen
MOVE 250 TO lsp
DO nop TIMES
READ lop FROM input-screen
IF lop < lsp THEN MOVE lop TO lsp ENDIF
ENDDO
29
© 2021 Sogeti Nederland B.V.. All rights reserved.
Code coverage – Statement coverage
Process-oriented
• Every line is exercised
• Every statement is exercised
• Not every outcome is necessarily tested
Testdata
nop = 1
lop = 110
Result
100% statement coverage
An IF-statement has a true and false outcome.
But only the true is exercised
Application code
DECLARE nop NUMERIC
DECLARE lop NUMERIC
DECLARE lsp NUMERIC
READ nop FROM input-screen
MOVE 250 TO lsp
DO nop TIMES
READ lop FROM input-screen
IF lop < lsp THEN MOVE lop TO lsp ENDIF
ENDDO
30
© 2021 Sogeti Nederland B.V.. All rights reserved.
Code coverage – Decision/Branch coverage
Condition-oriented
The two are very similar, Branch coverage is typically for low level languages
• Every line is exercised
• Every statement is exercised
• Every outcome is tested
• Not every path is necessarily tested
Testdata
nop = 2
lop = 110 (test case 1)
lop = 150 (test case 2)
Result
100% decision/branch coverage by running 2 test cases
Application code
DECLARE nop NUMERIC
DECLARE lop NUMERIC
DECLARE lsp NUMERIC
READ nop FROM input-screen
MOVE 250 TO lsp
DO nop TIMES
READ lop FROM input-screen
IF lop < lsp THEN MOVE lop TO lsp ENDIF
ENDDO
31
© 2021 Sogeti Nederland B.V.. All rights reserved.
Code coverage – Path coverage
Process-oriented
• Every line is exercised
• Every statement is exercised
• Every outcome is tested
• Every path is tested
Every path or combination of paths in the flow are tested.
Path coverage with test depth level-1 all decision-outcomes are tested at least once.
Path coverage with test depth level-2 some decision-outcomes are tested multiple times.
Path coverage gives the best coverage!
(and therefore later in this workshop we will practice with path testing ☺)
32
© 2021 Sogeti Nederland B.V.. All rights reserved.
Code coverage – Final remarks
Advanced coverage types
• Conditional Decision Coverage
• Modified Conditional Decision Coverage
• Multiple Condition Coverage
➔ More information on tmap.net
High code coverage ≠ meaningful testing!
Always check if actual result meets expected result, otherwise a test is useless!
33
© 2021 Sogeti Nederland B.V.. All rights reserved.
How many test cases would you need as a minimum to test this piece of pseudo-code?
Give:
• LC (line coverage)
• SC (statement coverage)
• DC (decision coverage)
As a bonus question:
Give the test case(s) you would use.
Put your answers in the chatbox
Exercise code coverage
READ VAR_A(numeric)
READ VAR_B(numeric)
IF VAR_B > 100 PRINT “B is big” ENDIF
UNTIL VAR_A > VAR_B
DO
ADD 1 TO VAR_A
END-DO
END-UNTIL
34
© 2021 Sogeti Nederland B.V.. All rights reserved.
LC (line coverage)
Can be done with 1 test case:
A = 10
B = 10
The IF is executed (the are statements are not)
Until is executed
Do is executed
Add is executed
End-do is executed
End-until is executed
ALL lines are executed, but not all statements.
Solution line coverage
READ VAR_A(numeric)
READ VAR_B(numeric)
IF VAR_B > 100 PRINT “B is big” ENDIF
UNTIL VAR_A > VAR_B
DO
ADD 1 TO VAR_A
END-DO
END-UNTIL
35
© 2021 Sogeti Nederland B.V.. All rights reserved.
SC (statement coverage)
Can be done with 1 test case:
A = 110
B = 110
The IF, Print & Endif statements are executed
Until is executed
Do is executed
Add is executed
End-do is executed
End-until is executed
ALL lines are executed, but not all statements.
Solution statement coverage
READ VAR_A(numeric)
READ VAR_B(numeric)
IF VAR_B > 100 PRINT “B is big” ENDIF
UNTIL VAR_A > VAR_B
DO
ADD 1 TO VAR_A
END-DO
END-UNTIL
36
© 2021 Sogeti Nederland B.V.. All rights reserved.
DC (decision coverage)
Can be done with the 2 previously shown test
cases
A = 10 and A = 110
B = 10 and B = 110
These test cases execute both
outcomes of the decision.
See further explanation on
the previous slides.
Solution decision coverage
READ VAR_A(numeric)
READ VAR_B(numeric)
IF VAR_B > 100 PRINT “B is big” ENDIF
UNTIL VAR_A > VAR_B
DO
ADD 1 TO VAR_A
END-DO
END-UNTIL
37
© 2021 Sogeti Nederland B.V.. All rights reserved.
Mutation testing - tests the tests
38
© 2021 Sogeti Nederland B.V.. All rights reserved.
Mutation testing
At its core, mutation testing is the practice of executing a test, or a set of tests,
over many versions of the software under test—the so-called mutants.
Mutants are automated source code changes only during the test phase of development.
Mutation testing is a useful way to check if common faults would be detected by the test set and
this way helps to improve a test set in case the injected faults would not be detected.
Examples of Mutation testing tools are:
pitest (https://pitest.org/) for java
stryker (https://stryker-mutator.io/) for javascript
39
© 2021 Sogeti Nederland B.V.. All rights reserved.
Examples of possible mutations of source code
Examples of mutations (this is not an exhaustive list)
<  <= +  -
>  >= /  *
&  | !=  ==
|  &
Advanced mutation testing tools can also change return values,
or remove conditions in conditional statements.
if(length > 120 & age > 18) {
accessAllowed();
} else if (length <= 140 | age == 20) {
extraValidation();
} else {
noAccess();
}
Example mutated code 2
if(length >= 120 & age > 18) {
accessAllowed();
} else if (length <= 140 | age == 20) {
extraValidation();
} else {
noAccess();
}
Example original code
if(length >= 120 | age > 18) {
accessAllowed();
} else if (length <= 140 | age == 20) {
extraValidation();
} else {
noAccess();
}
Example mutated code 1
if(length >= 120 & age > 18) {
accessAllowed();
} else if (length <= 140 | age != 20) {
extraValidation();
} else {
noAccess();
}
Example mutated code 3
40
© 2021 Sogeti Nederland B.V.. All rights reserved.
Mutation testing in Action
Original code
IF length >= 120
THEN MOVE “admitted” TO status
ELSE MOVE ”not admitted” TO status
ENDIF
Testcases – Using boundary value analysis
TC1: length = 119 -> Expected outcome “not admitted”
TC2: length = 120 -> Expected outcome “admitted”
Code - Mutation 1
IF length > 120
THEN MOVE “admitted” TO status
ELSE MOVE ”not admitted” TO status
ENDIF
Testcases
TC1: length = 119 -> Expected outcome “not admitted”
TC2: length = 120 -> Expected outcome “admitted” -> FAIL
Code – Mutation 2
IF length = 120
THEN MOVE “admitted” TO status
ELSE MOVE ”not admitted” TO status
ENDIF
Testcases – Using boundary value analysis
TC1: length = 119 -> Expected outcome “not admitted”
TC2: length = 120 -> Expected outcome “admitted”
Which test case would catch the
mutant?
TC3: length = 121 -> Expected outcome “admitted”
41
© 2021 Sogeti Nederland B.V.. All rights reserved.
Mutation testing in Action
Original code
IF length >= 120
THEN MOVE “admitted” TO status
ELSE MOVE ”not admitted” TO status
ENDIF
Testcases – Using boundary value analysis
TC1: length = 119 -> Expected outcome “not admitted”
TC2: length = 120 -> Expected outcome “admitted”
TC3: length = 121 -> Expected outcome “admitted”
Code - Mutation 3
IF length < 120
THEN MOVE “admitted” TO status
ELSE MOVE ”not admitted” TO status
ENDIF
Testcases
TC1: length = 119 -> Expected outcome “not admitted” -> FAIL
TC2: length = 120 -> Expected outcome “admitted” -> FAIL
TC3: length = 121 -> Expected outcome “admitted” -> FAIL
Code – Mutation 4
IF length != 119
THEN MOVE “admitted” TO status
ELSE MOVE ”not admitted” TO status
ENDIF
Testcases – Using boundary value analysis
TC1: length = 119 -> Expected outcome “not admitted”
TC2: length = 120 -> Expected outcome “admitted”
TC3: length = 121 -> Expected outcome “admitted”
Which test case would catch the
mutant?
TC4: length = 118 -> Expected outcome “not admitted”
42
© 2021 Sogeti Nederland B.V.. All rights reserved.
For this piece of pseudo-code we have the following test cases:
B = 100, expected result “no print”
B = 101, expected result “B is big”
What mutation can you do in the code
that would not be found with these 2 testcases?
Put your answers in the chatbox
Exercise Mutation testing
READ VAR_B(numeric)
IF VAR_B > 100
PRINT “B is big”
ENDIF
43
© 2021 Sogeti Nederland B.V.. All rights reserved.
For this piece of pseudo-code we have the following test cases:
B = 100, expected result “no print”
B = 101, expected result “B is big”
We need the extra test case:
B = 99, expected result “no print”
Solution: Mutant 1
READ VAR_B(numeric)
IF VAR_B != 100
PRINT “B is big”
ENDIF
44
© 2021 Sogeti Nederland B.V.. All rights reserved.
For this piece of pseudo-code we have the following test cases:
B = 99, expected result “no print”
B = 100, expected result “no print”
B = 101, expected result “B is big”
We need the extra test case:
B = 102, expected result “B is big”
The total of 4 test cases is known as “4-value Boundary Value Analysis”
Solution: Mutant 2
READ VAR_B(numeric)
IF VAR_B = 101
PRINT “B is big”
ENDIF
45
© 2021 Sogeti Nederland B.V.. All rights reserved.
Test design and Path testing
46
© 2021 Sogeti Nederland B.V.. All rights reserved.
Test design – Introduction
47
© 2021 Sogeti Nederland B.V.. All rights reserved.
Test approaches: experience- and coverage-based
48
© 2021 Sogeti Nederland B.V.. All rights reserved.
Experience-based testing approaches
Contribution of a large group of people with
many different skills, insights, insights and
experiences.
Testing based on a charter, using test ideas and log.
Is done in pairs (2 people) or mob (whole team)
A largely intuitive and ad hoc approach to testing → unstructured
Relevant experience brought together.
Can be used separately or as a tool in another approach.
Checklists
Error guessing
Exploratory
testing
e.g. mob testing
Crowd
testing
Experience-
based
testing
Checklist-based testing
Crowd testing
Error guessing
Exploratory testing
49
© 2021 Sogeti Nederland B.V.. All rights reserved.
Experience-based testing approaches work iteratively
50
© 2021 Sogeti Nederland B.V.. All rights reserved.
Coverage-based testing ➔ up-front structured test design
This figure shows how test design is related to the test basis (pursued value, requirements and
risks) and that it results in test situations and/or test cases and test data which is combined in
test scenarios and test scripts.
51
© 2021 Sogeti Nederland B.V.. All rights reserved.
Coverage-based testing with test design techniques
A test design technique is a standardized method of deriving test cases from a specific test basis
that will achieve a certain coverage. Applying a test design technique results in test situations,
logical test cases and/or physical test cases.
Our investigation has shown that there are about 25 to 30 test design techniques in literature.
All test design techniques can be categorized in the four coverage groups mentioned below.
In this training course we will practice with at least one technique for each group.
Keep in mind that you don’t have to
know all existing test design
techniques.
Knowing a few techniques for each
coverage group makes that with just
about 8 techniques you can handle
any testing challenge.
52
© 2021 Sogeti Nederland B.V.. All rights reserved.
Condition-oriented test design – overview of test design techniques
Testing of a program flow
Test individual decision points with multiple conditions
Test multiple decision points that together form a process
(this brings higher coverage than path testing)
Test all combinations of all conditions (regardsless of
decision points)
53
© 2021 Sogeti Nederland B.V.. All rights reserved.
Condition-oriented test design
These coverage types all
can be applied to each of
the test design techniques
on the left.
In this training course we
will use the Decision Table
test design technique
54
© 2021 Sogeti Nederland B.V.. All rights reserved.
Data-oriented test design – overview of test design techniques
Uses the classes of a data item (= parameter) to identify
what to test.
Uses the boundaries between classes of data items to
identify what to test.
Combines values for multiple data items in test cases.
Uses the Creation, Retrieval, Update and Deletion (CRUD)
of data items to create tests.
Uses the way data flows through an IT system to create
tests.
55
© 2021 Sogeti Nederland B.V.. All rights reserved.
Appearance-oriented test design – overview of test design techniques
Testing the syntax of e.g. user interface
Performance testing based on a model of the load
Test multiple versions of software on different
audiences
Specifically focus on how a system can be
misused (by mistake) or abused (on purpose)
There are many non-functional quality
characteristics for which techniques exist.
56
© 2021 Sogeti Nederland B.V.. All rights reserved.
Process-oriented test design
➔ Path testing
57
© 2021 Sogeti Nederland B.V.. All rights reserved.
Process-oriented testing – overview of test design techniques
Tests transitions between states.
Tests the path-combinations in a process flow or
algorithm (program flow)
Tests the flow in a computer program.
58
© 2021 Sogeti Nederland B.V.. All rights reserved.
Link between code coverage types and the coverage groups
59
© 2021 Sogeti Nederland B.V.. All rights reserved.
What coverage does path-testing give?
Path-testing ensures a certain level of coverage by measuring the
number of consecutive paths that has been covered.
Test-depth-level-1 means: all individual paths are covered by one or
more test cases.
Test-depth-level-2 means: all combinations of 2 consecutive paths have
been covered by one or more test cases.
Test-depth-level-n means: all combinations of n consecutive paths have
been covered by one or more test cases.
TDL-1 & TDL-2 are most used, we will explain these on the next slides.
60
© 2021 Sogeti Nederland B.V.. All rights reserved.
What exactly is a path?
How many paths do you see?
How many test cases do you need
for 100% coverage?
Put your answer in the chatbox
Start
Do you like
testing?
End
Go to the testing theatre Go visit the magic boat ride
No
Yes
You are in
TestLand
61
© 2021 Sogeti Nederland B.V.. All rights reserved.
What exactly is a path?
A path is a line between one decision point and
another decision point,
or a line from the start to the first decision point,
or a line form the last decision point to the end.
To determine the paths,
first simplify the graph,
get rid of everything other than
decision points, lines, start and end.
Start
Do you like
testing?
End
Go to the testing theatre Go visit the magic boat ride
No
Yes
You are in
TestLand
62
© 2021 Sogeti Nederland B.V.. All rights reserved.
What exactly is a path?
So this flowchart has the following paths:
Start
Do you like
testing?
End
No
Yes
1
2
3
63
© 2021 Sogeti Nederland B.V.. All rights reserved.
Testcases
To cover this simple flowchart 100%
We need two test cases:
TC1: path-1 – path-2
TC2: path-1 – path-3
Start
Do you like
testing?
End
No
Yes
1
2
3
64
© 2021 Sogeti Nederland B.V.. All rights reserved.
Example: Entering the mini-scooter-bumper-cars (for age < 10 years)
Start
How many
drivers?
End
Enter age driver 1
Enter age driver 1
Enter age driver 2
Is anyone
over 10
years?
No
Yes Find another driver that’s the
right age
Have fun riding the
mini-scooter-bumper-cars
one
two
65
© 2021 Sogeti Nederland B.V.. All rights reserved.
Example: Entering the mini-scooter-bumper-cars (for age < 10 years)
How many test cases
would you need for
100% coverage of
this process?
(put answer in chatbox)
Different possibilities
for 100% coverage
Depending on the
“Test Depth Level”
Start
How many
drivers?
End
Enter age driver 1
Enter age driver 1
Enter age driver 2
Is anyone
over 10
years?
No
Yes Find another driver that’s the
right age
Have fun riding the
mini-scooter-bumper-cars
one
two
66
© 2021 Sogeti Nederland B.V.. All rights reserved.
Step 1: determine the paths
How many paths are
in this process flow?
Put your answer in
the chatbox
Start
How many
drivers?
End
Enter age driver 1
Enter age driver 1
Enter age driver 2
Is anyone
over 10
years?
No
Yes Find another driver that’s the
right age
Have fun riding the
mini-scooter-bumper-cars
one
two
67
© 2021 Sogeti Nederland B.V.. All rights reserved.
Step 1: determine the paths, simplify the chart
Start
How many
drivers?
End
Is anyone
over 10
years?
No
Yes
Let’s first simplify
the process flow,
only start, end, lines
and decision points.
How many paths are
in this process flow?
one
two
68
© 2021 Sogeti Nederland B.V.. All rights reserved.
Step 1: determine the paths, number them
And letter the decision points
A path starts at a decision point and
goes to the next decision point.
Or a path starts at a startpoint.
Or a path ends at an endpoint.
Start
A
End
B
No
Yes
Path
2
Path
4
Path
1
Path
3
Path
5
one
two
Note: Step 1 applies to TDL-1 and TDL-2
69
© 2021 Sogeti Nederland B.V.. All rights reserved.
Step 2: determine the test situations
TDL-1: every path is a test situation.
First define the paths from start and
from each decision point:
Start: 1
A: 2, 3
B: 4, 5
Start
A
End
B
No
Yes
Path
2
Path
4
Path
1
Path
3
Path
5
one
two
Test
situations
Test
situations
Test
situations
70
© 2021 Sogeti Nederland B.V.. All rights reserved.
Step 3: create test cases
TDL-1: every path is a test situation.
Create test cases by combining paths
from start and from each decision
point:
Start: 1
A: 2, 3
B: 4, 5
TC1: 1, 2, 4
TC2: 1, 3, 5
Start
A
End
B
No
Yes
Path
2
Path
4
Path
1
Path
3
Path
5
one
two
71
© 2021 Sogeti Nederland B.V.. All rights reserved.
Introduction of the excel template
1
2, 3
4, 5
1, 2, 4
1, 3, 5
72
© 2021 Sogeti Nederland B.V.. All rights reserved.
Step 2: determine the test situations
TDL-2: every combination of
incoming and outgoing path
is a test situation.
Make a table with 3 columns:
Dec.Pt. IN OUT Path-combinations
A: 1 2, 3 1-2, 1-3
B: 2, 3 4, 5 2-4, 2-5, 3-4, 3-5
Start
A
End
B
No
Yes
Path
2
Path
4
Path
1
Path
3
Path
5
one
two
Test
situations
73
© 2021 Sogeti Nederland B.V.. All rights reserved.
Step 3: determine the test cases
TDL-2: Combine path-combinations
into test cases.
Dec.Pt. IN OUT Path-combinations
A: 1 2, 3 1-2 1-3
B: 2, 3 4, 5 2-4, 2-5, 3-4, 3-5
TC1: 1, 2, 4
TC2: 1, 2, 5
TC3: 1, 3, 4
TC4: 1, 3, 5
Start
A
End
B
No
Yes
Path
2
Path
4
Path
1
Path
3
Path
5
one
two
74
© 2021 Sogeti Nederland B.V.. All rights reserved.
Introduction of the excel template
1 2, 3
4, 5
1, 2, 4
1, 3, 4
2, 3
1-2, 1-3
2-4,2-5,3-4,3-5
1, 2, 5
1, 3, 5
75
© 2021 Sogeti Nederland B.V.. All rights reserved.
So: The the number of test cases depends on “Test Depth Level”
(= the number of consecutive paths that are tested.)
TDL-1:
TC1: 1, 2, 4
TC2: 1, 3, 5
TDL-2:
TC1: 1, 2, 4
TC2: 1, 2, 5
TC3: 1, 3, 4
TC4: 1, 3, 5
Start
How many
drivers?
End
Enter age driver 1
Enter age driver 1
Enter age driver 2
Is anyone
over 10
years?
No
Yes Find another driver that’s the
right age
Have fun riding the
mini-scooter-bumper-cars
one
two
76
© 2021 Sogeti Nederland B.V.. All rights reserved.
One coverage type
Two test design techniques
So we are talking about
“Path coverage”.
If you are testing a business
process, the technique used is the
Process Cycle Test.
Start
How many
drivers?
End
Enter age driver 1
Enter age driver 1
Enter age driver 2
Is anyone
over 10
years?
No
Yes Find another driver that’s the
right age
Have fun riding the
mini-scooter-bumper-cars
one
two
77
© 2021 Sogeti Nederland B.V.. All rights reserved.
One coverage type
Two test design techniques
So we are talking about
“Path coverage”.
If you are testing a business
process, the technique used is the
Process Cycle Test.
If you are testing a program flow,
the technique used is the
Algorithm Test.
But actually the technique is
exactly the same, the only
difference is the test basis.
Start
IF NR-PEOPLE
= 1
End
True
False
READ AGE-1
READ AGE-1
READ AGE-2
IF AGE-1 >
10 OR AGE-2
> 10
False
True
CALL subroutine-other-drivers
ENTER THE RIDE
78
© 2021 Sogeti Nederland B.V.. All rights reserved.
Recap of the Process Cycle Test and Algorithm Test techniques
Step 1: Determine the paths
Simplify the chart by removing everything else than decision points, start, end and lines.
Number the paths and letter the decision points
Step 2: Determine the test situations
For TDL-1 ➔ list the paths and where they originate
For TDL-2 ➔ list the decision points, the INcoming and OUTgoing paths,
create path-combinations
Step 3: Combine the test situations into test cases
From start to end combine test situations into test cases
Tips:
Process Cycle Test is very well suited for preparing acceptance tests and end-to-end tests.
Algorithm Test is very well suited for preparing unit tests, TDL-1 also assures Decision Coverage.
79
© 2021 Sogeti Nederland B.V.. All rights reserved.
Exercise path testing
➔ use the excel template
Determine the paths
and the test cases
for TDL-1 and TDL-2
Start
Decision A
End
Action 2
Action 3
Action 4
Decision B
No
Yes
Action 5
Action 6
two
three
Action 1
one
80
© 2021 Sogeti Nederland B.V.. All rights reserved.
Solution path testing
Path numbers
1
4
2
3
5
6
81
© 2021 Sogeti Nederland B.V.. All rights reserved.
Solution path testing TDL-1
Solution with
excel template
TDL-1
1
4
2
3
5
6
82
© 2021 Sogeti Nederland B.V.. All rights reserved.
Solution path testing TDL-2
1
4
2
3
5
6
83
© 2021 Sogeti Nederland B.V.. All rights reserved.
Now we add a loop in the flowchart
Step 1: determine the paths
How many paths are
in this process flow?
Start
How many
drivers?
End
Enter age driver 1
Enter age driver 1
Enter age driver 2
Is anyone
over 10
years?
No
Yes
Have fun riding the
mini-scooter-bumper-cars
one
two
84
© 2021 Sogeti Nederland B.V.. All rights reserved.
Step 1: determine the paths, simplify the chart Start
How many
drivers?
End
Is anyone
over 10
years?
No
Yes
Let’s first simplify
the process flow,
only start, end and
decision points.
How many paths are
in this process flow?
one
two
85
© 2021 Sogeti Nederland B.V.. All rights reserved.
Step 1: determine the paths Start
How many
drivers?
End
Is anyone
over 10
years?
No
Yes
There are 5 paths in
this process flow
How many test cases
do you need for 100%
coverage
TDL-1 ?
And TDL-2?
one
two
Path
2
Path
4
Path
1
Path
3
Path
5
86
© 2021 Sogeti Nederland B.V.. All rights reserved.
Step 2 and 3:
determine test situations and test cases
Start
How many
drivers?
End
Is anyone
over 10
years?
No
Yes
one
two
Path
2
Path
4
Path
1
Path
3
Path
5
TDL-1: every path = test situation
First define the paths from start
and from each decision point:
Start: 1
A: 2, 3
B: 4, 5
The paths can be combined in one
test case:
TC1: 1,2,4,3,5
87
© 2021 Sogeti Nederland B.V.. All rights reserved.
Step 2:
determine test situations
Start
How many
drivers?
End
Is anyone
over 10
years?
No
Yes
one
two
Path
2
Path
4
Path
1
Path
3
Path
5
TDL-2: every combination of
incoming and outgoing path
is a test situation.
Make a table with 3 columns:
Dec.Pt. IN OUT Path-combinations
A: 1,4 2,3 1-2, 1-3, 4-2, 4-3
B: 2,3 4,5 2-4, 2-5, 3-4, 3-5
88
© 2021 Sogeti Nederland B.V.. All rights reserved.
Step 3:
determine test cases
Start
How many
drivers?
End
Is anyone
over 10
years?
No
Yes
one
two
Path
2
Path
4
Path
1
Path
3
Path
5
TDL-2: Combine path-combinations
into test cases.
Dec.Pt. IN OUT Path-combinations
A: 1,4 2,3 1-2, 1-3, 4-2, 4-3
B: 2,3 4,5 2-4, 2-5, 3-4, 3-5
TC1: 1, 2, 5 (zero times through the loop)
TC2: 1, 3, 4, 2, 5 (one time through the loop)
Now we need to cover 2-4, 4-3 and 3-5
TC3: 1, 2, 4, 3, 5 (one time through the loop)
To improve your coverage (and confidence) do this:
TC3’: 1, 2, 4, 3, 4, 3, 5 (two times through the loop)
89
© 2021 Sogeti Nederland B.V.. All rights reserved.
Recap of the Path testing
Step 1: Determine the paths
Simplify the chart by removing everything else than decision points and start & end lines.
Number the paths and letter the decision points
Step 2: Determine the test situations
For TDL-1 ➔ list the paths and where they originate
For TDL-2 ➔ list the decision points, the INcoming and OUTgoing paths,
create path-combinations
Step 3: Combine the test situations into test cases
From start to end combine test situations into test cases
When there’s a loop, for good confidence create test cases that go through it:
zero times (= not through the loop)
one time through the loop
multiple times (= 2 or more times through the loop)
For progression testing: create many different short test cases (easier investigation of anomalies)
For regression testing: create fewer longer test cases (easier execution)
90
© 2021 Sogeti Nederland B.V.. All rights reserved.
Exercise path testing with loop
➔ use the excel template
Determine the paths
and the test cases
for TDL-1 and TDL-2
Start
Decision A
End
Action 2
Decision B
No
Yes
Action 3
two
one
Action 1
91
© 2021 Sogeti Nederland B.V.. All rights reserved.
Solution path testing with loop
Path numbers
Start
Decision A
End
Decision B
No
Yes
two
one
1
2
3
4
5
92
© 2021 Sogeti Nederland B.V.. All rights reserved.
Solution path testing with loop TDL-1
Start
Decision A
End
Decision B
No
Yes
two
one
1
2
3
4
5
93
© 2021 Sogeti Nederland B.V.. All rights reserved.
Solution path testing with loop TDL-2
Start
Decision A
End
Decision B
No
Yes
two
one
1
2
3
4
5
94
© 2021 Sogeti Nederland B.V.. All rights reserved.
Closing
95
© 2021 Sogeti Nederland B.V.. All rights reserved.
TMAP training courses and certification scheme of
Exams provided by
TMAP: Quality
for cross-
functional teams
TMAP: High-
performance
quality
engineering
TMAP:
Organizing built-
in quality
at scale
Performing
people
3-day training course
including
exam of 30 questions
3-day training courses
with separate
exams of 40 questions
All people
involved in
cross-
functional
teams
Organizing
people
Do you want to learn more about TMAP?
https://pages.isqi.org/tmap-2020/
https://tmap.net/page/tmap-devops-teams
https://academy.sogeti.nl/2020-tmap
96
© 2021 Sogeti Nederland B.V.. All rights reserved.
And visit the renewed
body of knowledge
on www.tmap.net
Who wins the ePub of my TMAP book?
and if you don’t win, find it on www.ict-books.com
or download the free preview-version on:
https://tmap.net/book/quality-devops-teams
Find the excel template for path testing also
on www.TMAP.net :
https://tmap.net/page/download-templates
97
© 2021 Sogeti Nederland B.V.. All rights reserved.
About Sogeti
Learn more about us at
www.sogeti.com
This message contains information that may be privileged or
confidential and is the property of the Capgemini Group.
Copyright© 2018 Sogeti. All rights reserved.
Sogeti is a leading provider of technology and engineering services. Sogeti
delivers solutions that enable digital transformation and offers cutting-edge
expertise in Cloud, Cybersecurity, Digital Manufacturing, Digital Assurance
& Testing, and emerging technologies. Sogeti combines agility and speed of
implementation with strong technology supplier partnerships, world class
methodologies and its global delivery model, Rightshore®. Sogeti brings
together more than 25,000 professionals in 15 countries, based in over 100
locations in Europe, USA and India. Sogeti is a wholly-owned subsidiary of
Capgemini SE, listed on the Paris Stock Exchange.
Good luck with quality engineering !!!
Thank you for your participation
Contact me on: rik.Marselis@sogeti.com
Or on twitter: @rikmarselis

More Related Content

What's hot

TestOps and Shift Left
TestOps and Shift LeftTestOps and Shift Left
TestOps and Shift Left
Gervais Johnson, Advisor
 
TCoE
TCoETCoE
QA Center Of Excellence (TCoE)
QA Center Of Excellence (TCoE)QA Center Of Excellence (TCoE)
QA Center Of Excellence (TCoE)
JAX Chamber IT Council
 
Introduction to Software Test Automation
Introduction to Software Test AutomationIntroduction to Software Test Automation
Introduction to Software Test Automation
Amr Ali (ISTQB CTAL Full, CSM, ITIL Foundation)
 
Sap test center of excellence
Sap test center of excellenceSap test center of excellence
Sap test center of excellence
Infosys
 
Emerging QA COE Practice by Mukund Wangikar
Emerging QA COE Practice by Mukund WangikarEmerging QA COE Practice by Mukund Wangikar
Emerging QA COE Practice by Mukund Wangikar
Agile Testing Alliance
 
Testing & Quality Assurance
Testing & Quality AssuranceTesting & Quality Assurance
Testing & Quality Assurance
Anand Subramaniam
 
Agile Testing by Example
Agile Testing by ExampleAgile Testing by Example
Agile Testing by Example
Mikalai Alimenkou
 
Software Quality Gate.pptx
Software Quality Gate.pptxSoftware Quality Gate.pptx
Software Quality Gate.pptx
ssuser702665
 
Testing in Agile Projects
Testing in Agile ProjectsTesting in Agile Projects
Testing in Agile Projects
sriks7
 
How to Design a Successful Test Automation Strategy
How to Design a Successful Test Automation Strategy How to Design a Successful Test Automation Strategy
How to Design a Successful Test Automation Strategy
Impetus Technologies
 
QA Best Practices in Agile World_new
QA Best Practices in Agile World_newQA Best Practices in Agile World_new
QA Best Practices in Agile World_new
Praveen Dosapati (CSM)
 
What is (tcoe) testing center of excellence
What is (tcoe) testing center of excellenceWhat is (tcoe) testing center of excellence
What is (tcoe) testing center of excellence
Maveric Systems
 
Manual testing
Manual testingManual testing
Manual testing
vigneshasromio
 
Automation Concepts
Automation ConceptsAutomation Concepts
Automation Concepts
Nishant Worah
 
Agile QA and Testing process
Agile QA and Testing processAgile QA and Testing process
Agile QA and Testing process
Gloria Stoilova
 
Software test automation
Software test automationSoftware test automation
Software test automation
Oksana Demediuk
 
Tcoe team
Tcoe teamTcoe team
Tcoe team
Ankita Mohanty
 
Testing Centre of Excellence Model 2016
Testing Centre of Excellence Model 2016Testing Centre of Excellence Model 2016
Testing Centre of Excellence Model 2016
Tony Barber
 
Functional to Visual: AI-powered UI Testing from Testim and Applitools
Functional to Visual: AI-powered UI Testing from Testim and ApplitoolsFunctional to Visual: AI-powered UI Testing from Testim and Applitools
Functional to Visual: AI-powered UI Testing from Testim and Applitools
Applitools
 

What's hot (20)

TestOps and Shift Left
TestOps and Shift LeftTestOps and Shift Left
TestOps and Shift Left
 
TCoE
TCoETCoE
TCoE
 
QA Center Of Excellence (TCoE)
QA Center Of Excellence (TCoE)QA Center Of Excellence (TCoE)
QA Center Of Excellence (TCoE)
 
Introduction to Software Test Automation
Introduction to Software Test AutomationIntroduction to Software Test Automation
Introduction to Software Test Automation
 
Sap test center of excellence
Sap test center of excellenceSap test center of excellence
Sap test center of excellence
 
Emerging QA COE Practice by Mukund Wangikar
Emerging QA COE Practice by Mukund WangikarEmerging QA COE Practice by Mukund Wangikar
Emerging QA COE Practice by Mukund Wangikar
 
Testing & Quality Assurance
Testing & Quality AssuranceTesting & Quality Assurance
Testing & Quality Assurance
 
Agile Testing by Example
Agile Testing by ExampleAgile Testing by Example
Agile Testing by Example
 
Software Quality Gate.pptx
Software Quality Gate.pptxSoftware Quality Gate.pptx
Software Quality Gate.pptx
 
Testing in Agile Projects
Testing in Agile ProjectsTesting in Agile Projects
Testing in Agile Projects
 
How to Design a Successful Test Automation Strategy
How to Design a Successful Test Automation Strategy How to Design a Successful Test Automation Strategy
How to Design a Successful Test Automation Strategy
 
QA Best Practices in Agile World_new
QA Best Practices in Agile World_newQA Best Practices in Agile World_new
QA Best Practices in Agile World_new
 
What is (tcoe) testing center of excellence
What is (tcoe) testing center of excellenceWhat is (tcoe) testing center of excellence
What is (tcoe) testing center of excellence
 
Manual testing
Manual testingManual testing
Manual testing
 
Automation Concepts
Automation ConceptsAutomation Concepts
Automation Concepts
 
Agile QA and Testing process
Agile QA and Testing processAgile QA and Testing process
Agile QA and Testing process
 
Software test automation
Software test automationSoftware test automation
Software test automation
 
Tcoe team
Tcoe teamTcoe team
Tcoe team
 
Testing Centre of Excellence Model 2016
Testing Centre of Excellence Model 2016Testing Centre of Excellence Model 2016
Testing Centre of Excellence Model 2016
 
Functional to Visual: AI-powered UI Testing from Testim and Applitools
Functional to Visual: AI-powered UI Testing from Testim and ApplitoolsFunctional to Visual: AI-powered UI Testing from Testim and Applitools
Functional to Visual: AI-powered UI Testing from Testim and Applitools
 

Similar to TMAP Quality Engineering workshop on A4Q congress by Rik Marselis

Quality engineering & testing in DevOps IT delivery with TMAP
Quality engineering & testing in DevOps IT delivery with TMAPQuality engineering & testing in DevOps IT delivery with TMAP
Quality engineering & testing in DevOps IT delivery with TMAP
Rik Marselis
 
Quality for DevOps teams - Quality engineering in the DevOps culture
Quality for DevOps teams - Quality engineering in the DevOps cultureQuality for DevOps teams - Quality engineering in the DevOps culture
Quality for DevOps teams - Quality engineering in the DevOps culture
Rik Marselis
 
Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Mars...
Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Mars...Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Mars...
Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Mars...
Rik Marselis
 
Introduction of TMAP to representatives of ISTQB boards in the GA week in Mar...
Introduction of TMAP to representatives of ISTQB boards in the GA week in Mar...Introduction of TMAP to representatives of ISTQB boards in the GA week in Mar...
Introduction of TMAP to representatives of ISTQB boards in the GA week in Mar...
Rik Marselis
 
Quality Engineering in today's cross-functTeams with TMAP
Quality Engineering in today's cross-functTeams with TMAPQuality Engineering in today's cross-functTeams with TMAP
Quality Engineering in today's cross-functTeams with TMAP
Rik Marselis
 
DevOps with Chef
DevOps with ChefDevOps with Chef
DevOps with Chef
Kellton Tech Solutions Ltd
 
Quality engineering instead of testing... Why? How?
Quality engineering instead of testing... Why? How?Quality engineering instead of testing... Why? How?
Quality engineering instead of testing... Why? How?
Rik Marselis
 
Goken Corporate Presentation - Interiors & Seating
Goken Corporate Presentation - Interiors & SeatingGoken Corporate Presentation - Interiors & Seating
Goken Corporate Presentation - Interiors & Seating
AshleyTaylor66055
 
VishalSrivastava_NewV1.0
VishalSrivastava_NewV1.0VishalSrivastava_NewV1.0
VishalSrivastava_NewV1.0
Vishal Srivastava
 
GTR-The End Of Testing As We Know It
GTR-The End Of Testing As We Know ItGTR-The End Of Testing As We Know It
GTR-The End Of Testing As We Know It
Rik Marselis
 
Rohit Oza_CV_2015
Rohit Oza_CV_2015Rohit Oza_CV_2015
Rohit Oza_CV_2015
Rohit Oza
 
DevOps Transformation Solution Recommendation.pptx
DevOps Transformation Solution Recommendation.pptxDevOps Transformation Solution Recommendation.pptx
DevOps Transformation Solution Recommendation.pptx
PrasannaKumarN8
 
Advanced sap cybersecurity webinar
Advanced sap cybersecurity webinarAdvanced sap cybersecurity webinar
Advanced sap cybersecurity webinar
David Milano
 
FutureLink | Strategic Tooling Decisions in ALM Engineering: Migrate or Coexi...
FutureLink | Strategic Tooling Decisions in ALM Engineering: Migrate or Coexi...FutureLink | Strategic Tooling Decisions in ALM Engineering: Migrate or Coexi...
FutureLink | Strategic Tooling Decisions in ALM Engineering: Migrate or Coexi...
Intland Software GmbH
 
FAQs’ for Unit, Integration and Functional Testing Services
FAQs’ for Unit, Integration and Functional Testing ServicesFAQs’ for Unit, Integration and Functional Testing Services
FAQs’ for Unit, Integration and Functional Testing Services
Embitel Technologies (I) PVT LTD
 
Telelogic
TelelogicTelelogic
Telelogic
Umesh
 
DevOps - Orientation Session
DevOps - Orientation SessionDevOps - Orientation Session
DevOps - Orientation Session
M M Nair
 
Code campiasi scm-project-gabriel-cristescu-ditech
Code campiasi scm-project-gabriel-cristescu-ditechCode campiasi scm-project-gabriel-cristescu-ditech
Code campiasi scm-project-gabriel-cristescu-ditech
Codecamp Romania
 
Ravindra Prasad
Ravindra PrasadRavindra Prasad
Ravindra Prasad
Ravindra Prasad
 
Consulting
ConsultingConsulting
Consulting
Markus Voelter
 

Similar to TMAP Quality Engineering workshop on A4Q congress by Rik Marselis (20)

Quality engineering & testing in DevOps IT delivery with TMAP
Quality engineering & testing in DevOps IT delivery with TMAPQuality engineering & testing in DevOps IT delivery with TMAP
Quality engineering & testing in DevOps IT delivery with TMAP
 
Quality for DevOps teams - Quality engineering in the DevOps culture
Quality for DevOps teams - Quality engineering in the DevOps cultureQuality for DevOps teams - Quality engineering in the DevOps culture
Quality for DevOps teams - Quality engineering in the DevOps culture
 
Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Mars...
Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Mars...Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Mars...
Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Mars...
 
Introduction of TMAP to representatives of ISTQB boards in the GA week in Mar...
Introduction of TMAP to representatives of ISTQB boards in the GA week in Mar...Introduction of TMAP to representatives of ISTQB boards in the GA week in Mar...
Introduction of TMAP to representatives of ISTQB boards in the GA week in Mar...
 
Quality Engineering in today's cross-functTeams with TMAP
Quality Engineering in today's cross-functTeams with TMAPQuality Engineering in today's cross-functTeams with TMAP
Quality Engineering in today's cross-functTeams with TMAP
 
DevOps with Chef
DevOps with ChefDevOps with Chef
DevOps with Chef
 
Quality engineering instead of testing... Why? How?
Quality engineering instead of testing... Why? How?Quality engineering instead of testing... Why? How?
Quality engineering instead of testing... Why? How?
 
Goken Corporate Presentation - Interiors & Seating
Goken Corporate Presentation - Interiors & SeatingGoken Corporate Presentation - Interiors & Seating
Goken Corporate Presentation - Interiors & Seating
 
VishalSrivastava_NewV1.0
VishalSrivastava_NewV1.0VishalSrivastava_NewV1.0
VishalSrivastava_NewV1.0
 
GTR-The End Of Testing As We Know It
GTR-The End Of Testing As We Know ItGTR-The End Of Testing As We Know It
GTR-The End Of Testing As We Know It
 
Rohit Oza_CV_2015
Rohit Oza_CV_2015Rohit Oza_CV_2015
Rohit Oza_CV_2015
 
DevOps Transformation Solution Recommendation.pptx
DevOps Transformation Solution Recommendation.pptxDevOps Transformation Solution Recommendation.pptx
DevOps Transformation Solution Recommendation.pptx
 
Advanced sap cybersecurity webinar
Advanced sap cybersecurity webinarAdvanced sap cybersecurity webinar
Advanced sap cybersecurity webinar
 
FutureLink | Strategic Tooling Decisions in ALM Engineering: Migrate or Coexi...
FutureLink | Strategic Tooling Decisions in ALM Engineering: Migrate or Coexi...FutureLink | Strategic Tooling Decisions in ALM Engineering: Migrate or Coexi...
FutureLink | Strategic Tooling Decisions in ALM Engineering: Migrate or Coexi...
 
FAQs’ for Unit, Integration and Functional Testing Services
FAQs’ for Unit, Integration and Functional Testing ServicesFAQs’ for Unit, Integration and Functional Testing Services
FAQs’ for Unit, Integration and Functional Testing Services
 
Telelogic
TelelogicTelelogic
Telelogic
 
DevOps - Orientation Session
DevOps - Orientation SessionDevOps - Orientation Session
DevOps - Orientation Session
 
Code campiasi scm-project-gabriel-cristescu-ditech
Code campiasi scm-project-gabriel-cristescu-ditechCode campiasi scm-project-gabriel-cristescu-ditech
Code campiasi scm-project-gabriel-cristescu-ditech
 
Ravindra Prasad
Ravindra PrasadRavindra Prasad
Ravindra Prasad
 
Consulting
ConsultingConsulting
Consulting
 

More from Rik Marselis

The End Of Testing As We Know It (TestCon - Rik Marselis).pdf
The End Of Testing As We Know It (TestCon - Rik Marselis).pdfThe End Of Testing As We Know It (TestCon - Rik Marselis).pdf
The End Of Testing As We Know It (TestCon - Rik Marselis).pdf
Rik Marselis
 
TestExpo Quality Engineering & Sustainability
TestExpo Quality Engineering & Sustainability TestExpo Quality Engineering & Sustainability
TestExpo Quality Engineering & Sustainability
Rik Marselis
 
How to write a proposal to speak at a conference? Hints & tips
How to write a proposal to speak at a conference? Hints & tipsHow to write a proposal to speak at a conference? Hints & tips
How to write a proposal to speak at a conference? Hints & tips
Rik Marselis
 
Testen ISTQB, TMap, ISO25010 en meer. Gastcollege Windesheim Rik Marselis
Testen ISTQB, TMap, ISO25010 en meer. Gastcollege Windesheim Rik MarselisTesten ISTQB, TMap, ISO25010 en meer. Gastcollege Windesheim Rik Marselis
Testen ISTQB, TMap, ISO25010 en meer. Gastcollege Windesheim Rik Marselis
Rik Marselis
 
Over testen gesproken TestNet najaar 2019 openingskeynote Rik Marselis
Over testen gesproken TestNet najaar 2019 openingskeynote Rik MarselisOver testen gesproken TestNet najaar 2019 openingskeynote Rik Marselis
Over testen gesproken TestNet najaar 2019 openingskeynote Rik Marselis
Rik Marselis
 
EuroSTAR 2018 tutorial Rik Marselis Testing Intelligent Machines
EuroSTAR 2018 tutorial Rik Marselis Testing Intelligent MachinesEuroSTAR 2018 tutorial Rik Marselis Testing Intelligent Machines
EuroSTAR 2018 tutorial Rik Marselis Testing Intelligent Machines
Rik Marselis
 
Testing intelligent machines; approaches and techniques (QA&Test Bilbao 2018)
Testing intelligent machines; approaches and techniques (QA&Test Bilbao 2018)Testing intelligent machines; approaches and techniques (QA&Test Bilbao 2018)
Testing intelligent machines; approaches and techniques (QA&Test Bilbao 2018)
Rik Marselis
 
Qx day2018 digital_quality_strategy_rik_marselis
Qx day2018 digital_quality_strategy_rik_marselisQx day2018 digital_quality_strategy_rik_marselis
Qx day2018 digital_quality_strategy_rik_marselis
Rik Marselis
 
Testing in the digital age; AI makes the difference! Booklaunch Rik Marselis
Testing in the digital age; AI makes the difference! Booklaunch Rik MarselisTesting in the digital age; AI makes the difference! Booklaunch Rik Marselis
Testing in the digital age; AI makes the difference! Booklaunch Rik Marselis
Rik Marselis
 
Testing in the digital age; international booklaunch at German Testing Day
Testing in the digital age; international booklaunch at German Testing DayTesting in the digital age; international booklaunch at German Testing Day
Testing in the digital age; international booklaunch at German Testing Day
Rik Marselis
 
Testing of artificial intelligence; AI quality engineering skils - an introdu...
Testing of artificial intelligence; AI quality engineering skils - an introdu...Testing of artificial intelligence; AI quality engineering skils - an introdu...
Testing of artificial intelligence; AI quality engineering skils - an introdu...
Rik Marselis
 
Webinar trends in testen 2017-12-13 door Rik Marselis
Webinar trends in testen 2017-12-13 door Rik MarselisWebinar trends in testen 2017-12-13 door Rik Marselis
Webinar trends in testen 2017-12-13 door Rik Marselis
Rik Marselis
 
How to test when robots become part of your process? Workshop robotesting agi...
How to test when robots become part of your process? Workshop robotesting agi...How to test when robots become part of your process? Workshop robotesting agi...
How to test when robots become part of your process? Workshop robotesting agi...
Rik Marselis
 
Robotesting at Tricentis Accelerate 2017 by Rik Marselis
Robotesting at Tricentis Accelerate 2017 by Rik MarselisRobotesting at Tricentis Accelerate 2017 by Rik Marselis
Robotesting at Tricentis Accelerate 2017 by Rik Marselis
Rik Marselis
 
Ketenregie in Agile / DevOps: Noodzaak? (oplossing: Gilde, SAFe, combi ...)
Ketenregie in Agile / DevOps: Noodzaak? (oplossing: Gilde, SAFe, combi ...)Ketenregie in Agile / DevOps: Noodzaak? (oplossing: Gilde, SAFe, combi ...)
Ketenregie in Agile / DevOps: Noodzaak? (oplossing: Gilde, SAFe, combi ...)
Rik Marselis
 
Robotesting, are you ready for that yet? 7WCSQ Rik Marselis
Robotesting, are you ready for that yet? 7WCSQ Rik MarselisRobotesting, are you ready for that yet? 7WCSQ Rik Marselis
Robotesting, are you ready for that yet? 7WCSQ Rik Marselis
Rik Marselis
 
Webinar trends in testing 2017 03 08 (in dutch)
Webinar trends in testing 2017 03 08 (in dutch)Webinar trends in testing 2017 03 08 (in dutch)
Webinar trends in testing 2017 03 08 (in dutch)
Rik Marselis
 
Past het testvak nog in de nieuwe IT-wereld?
Past het testvak nog in de nieuwe IT-wereld?Past het testvak nog in de nieuwe IT-wereld?
Past het testvak nog in de nieuwe IT-wereld?
Rik Marselis
 
Testing OF robots and testing WITH robots. Netherlands testing day 2016 Rik M...
Testing OF robots and testing WITH robots. Netherlands testing day 2016 Rik M...Testing OF robots and testing WITH robots. Netherlands testing day 2016 Rik M...
Testing OF robots and testing WITH robots. Netherlands testing day 2016 Rik M...
Rik Marselis
 
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik MarselisTesting machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
Rik Marselis
 

More from Rik Marselis (20)

The End Of Testing As We Know It (TestCon - Rik Marselis).pdf
The End Of Testing As We Know It (TestCon - Rik Marselis).pdfThe End Of Testing As We Know It (TestCon - Rik Marselis).pdf
The End Of Testing As We Know It (TestCon - Rik Marselis).pdf
 
TestExpo Quality Engineering & Sustainability
TestExpo Quality Engineering & Sustainability TestExpo Quality Engineering & Sustainability
TestExpo Quality Engineering & Sustainability
 
How to write a proposal to speak at a conference? Hints & tips
How to write a proposal to speak at a conference? Hints & tipsHow to write a proposal to speak at a conference? Hints & tips
How to write a proposal to speak at a conference? Hints & tips
 
Testen ISTQB, TMap, ISO25010 en meer. Gastcollege Windesheim Rik Marselis
Testen ISTQB, TMap, ISO25010 en meer. Gastcollege Windesheim Rik MarselisTesten ISTQB, TMap, ISO25010 en meer. Gastcollege Windesheim Rik Marselis
Testen ISTQB, TMap, ISO25010 en meer. Gastcollege Windesheim Rik Marselis
 
Over testen gesproken TestNet najaar 2019 openingskeynote Rik Marselis
Over testen gesproken TestNet najaar 2019 openingskeynote Rik MarselisOver testen gesproken TestNet najaar 2019 openingskeynote Rik Marselis
Over testen gesproken TestNet najaar 2019 openingskeynote Rik Marselis
 
EuroSTAR 2018 tutorial Rik Marselis Testing Intelligent Machines
EuroSTAR 2018 tutorial Rik Marselis Testing Intelligent MachinesEuroSTAR 2018 tutorial Rik Marselis Testing Intelligent Machines
EuroSTAR 2018 tutorial Rik Marselis Testing Intelligent Machines
 
Testing intelligent machines; approaches and techniques (QA&Test Bilbao 2018)
Testing intelligent machines; approaches and techniques (QA&Test Bilbao 2018)Testing intelligent machines; approaches and techniques (QA&Test Bilbao 2018)
Testing intelligent machines; approaches and techniques (QA&Test Bilbao 2018)
 
Qx day2018 digital_quality_strategy_rik_marselis
Qx day2018 digital_quality_strategy_rik_marselisQx day2018 digital_quality_strategy_rik_marselis
Qx day2018 digital_quality_strategy_rik_marselis
 
Testing in the digital age; AI makes the difference! Booklaunch Rik Marselis
Testing in the digital age; AI makes the difference! Booklaunch Rik MarselisTesting in the digital age; AI makes the difference! Booklaunch Rik Marselis
Testing in the digital age; AI makes the difference! Booklaunch Rik Marselis
 
Testing in the digital age; international booklaunch at German Testing Day
Testing in the digital age; international booklaunch at German Testing DayTesting in the digital age; international booklaunch at German Testing Day
Testing in the digital age; international booklaunch at German Testing Day
 
Testing of artificial intelligence; AI quality engineering skils - an introdu...
Testing of artificial intelligence; AI quality engineering skils - an introdu...Testing of artificial intelligence; AI quality engineering skils - an introdu...
Testing of artificial intelligence; AI quality engineering skils - an introdu...
 
Webinar trends in testen 2017-12-13 door Rik Marselis
Webinar trends in testen 2017-12-13 door Rik MarselisWebinar trends in testen 2017-12-13 door Rik Marselis
Webinar trends in testen 2017-12-13 door Rik Marselis
 
How to test when robots become part of your process? Workshop robotesting agi...
How to test when robots become part of your process? Workshop robotesting agi...How to test when robots become part of your process? Workshop robotesting agi...
How to test when robots become part of your process? Workshop robotesting agi...
 
Robotesting at Tricentis Accelerate 2017 by Rik Marselis
Robotesting at Tricentis Accelerate 2017 by Rik MarselisRobotesting at Tricentis Accelerate 2017 by Rik Marselis
Robotesting at Tricentis Accelerate 2017 by Rik Marselis
 
Ketenregie in Agile / DevOps: Noodzaak? (oplossing: Gilde, SAFe, combi ...)
Ketenregie in Agile / DevOps: Noodzaak? (oplossing: Gilde, SAFe, combi ...)Ketenregie in Agile / DevOps: Noodzaak? (oplossing: Gilde, SAFe, combi ...)
Ketenregie in Agile / DevOps: Noodzaak? (oplossing: Gilde, SAFe, combi ...)
 
Robotesting, are you ready for that yet? 7WCSQ Rik Marselis
Robotesting, are you ready for that yet? 7WCSQ Rik MarselisRobotesting, are you ready for that yet? 7WCSQ Rik Marselis
Robotesting, are you ready for that yet? 7WCSQ Rik Marselis
 
Webinar trends in testing 2017 03 08 (in dutch)
Webinar trends in testing 2017 03 08 (in dutch)Webinar trends in testing 2017 03 08 (in dutch)
Webinar trends in testing 2017 03 08 (in dutch)
 
Past het testvak nog in de nieuwe IT-wereld?
Past het testvak nog in de nieuwe IT-wereld?Past het testvak nog in de nieuwe IT-wereld?
Past het testvak nog in de nieuwe IT-wereld?
 
Testing OF robots and testing WITH robots. Netherlands testing day 2016 Rik M...
Testing OF robots and testing WITH robots. Netherlands testing day 2016 Rik M...Testing OF robots and testing WITH robots. Netherlands testing day 2016 Rik M...
Testing OF robots and testing WITH robots. Netherlands testing day 2016 Rik M...
 
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik MarselisTesting machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
 

Recently uploaded

"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
Fwdays
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
 
Christine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptxChristine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptx
christinelarrosa
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
Fwdays
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
Safe Software
 
Session 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdfSession 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdf
UiPathCommunity
 
What is an RPA CoE? Session 2 – CoE Roles
What is an RPA CoE?  Session 2 – CoE RolesWhat is an RPA CoE?  Session 2 – CoE Roles
What is an RPA CoE? Session 2 – CoE Roles
DianaGray10
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
LizaNolte
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ivo Velitchkov
 
A Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's ArchitectureA Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's Architecture
ScyllaDB
 
From Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMsFrom Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMs
Sease
 
High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024
Vadym Kazulkin
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
Fwdays
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptxPRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
christinelarrosa
 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
Fwdays
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Neo4j
 

Recently uploaded (20)

"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
 
Christine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptxChristine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptx
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
 
Session 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdfSession 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdf
 
What is an RPA CoE? Session 2 – CoE Roles
What is an RPA CoE?  Session 2 – CoE RolesWhat is an RPA CoE?  Session 2 – CoE Roles
What is an RPA CoE? Session 2 – CoE Roles
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
 
A Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's ArchitectureA Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's Architecture
 
From Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMsFrom Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMs
 
High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptxPRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
 

TMAP Quality Engineering workshop on A4Q congress by Rik Marselis

  • 1. © 2021 Sogeti Nederland B.V.. All rights reserved. Academy TMAP quality engineering 16 April 2021 Workshop
  • 2. © 2021 Sogeti Nederland B.V.. All rights reserved. Academy TMAP: Quality engineering - Built-in quality & VOICE model - Quality engineering & cross-functional teams - Unit testing, mutation testing and path testing Three-hour workshop based on the TMAP certification scheme Presented by: Rik Marselis – Principal Quality Consultant Friday 16 April 2021 14:00 – 17:15 (including 2 short breaks) TMAP: the body of knowledge for quality engineering in IT delivery
  • 3. 3 © 2021 Sogeti Nederland B.V.. All rights reserved. This workshop is based on training material from the TMAP certification scheme. Created by Exam provider There are local training providers in multiple countries. Agenda: - built-in quality, the VOICE model and quality engineering topics - code coverage types for unit testing and mutation testing - path testing + 17:15 end (we will have breaks at logical moments in the workshop) At the end 3 people will win a free download of the ePub of our book➔ Welcome to this workshop!! Bookshop: www.ict-books.com (or on your own bookshop e.g. amazon.nl or bol.com)
  • 4. 4 © 2021 Sogeti Nederland B.V.. All rights reserved. Amstelveen Where are you from? ➔ put your country & city in the chatbox
  • 5. 5 © 2021 Sogeti Nederland B.V.. All rights reserved. Rik Marselis Principal Quality Consultant 2007 2012 2012 2008 2014 2009 2018 1980 2020
  • 6. 6 © 2021 Sogeti Nederland B.V.. All rights reserved. Built-in quality and VOICE model
  • 7. 7 © 2021 Sogeti Nederland B.V.. All rights reserved. Challenges of today’s high-performance IT delivery The business demands: • Deliver business value • Deliver quality at speed The team challenges are: • Quality engineering is everyone's responsibility • QA & testing is integrated in process and people The focus is: • Organize high-performing cross-functional teams (you build it, you run it!) • Automate everything Using AI ?
  • 8. 8 © 2021 Sogeti Nederland B.V.. All rights reserved. Waterfall V-model Scrum DevOps Demand-Supply SAFe® © 2021 Sogeti Nederland B.V.. All rights reserved.
  • 9. 9 © 2021 Sogeti Nederland B.V.. All rights reserved. High-performance IT delivery Definition: High-performance IT delivery is an approach that enables cross-functional teams to continuously improve the products, processes and people that are required to deliver value to the end users. To achieve this continuous improvement quality must be built-in in not only the product but also in the process (activities) and the people (individuals and the team as a whole). Quality Product People Process
  • 10. 10 © 2021 Sogeti Nederland B.V.. All rights reserved. Built-in quality Built-in quality is one of the key principles in the Lean approach. Built-in quality, continuously improved, leads to right-first-time, and where the outcome of the process meets the expectation: fit-for-purpose. Quality of the product is strongly linked to the quality of the delivery process and the people. Any deviation of the expected product quality should lead to improvement, not just fixing the fault but also changing the process and people to prevent such faults from occurring again. Source of picture: book “The PointZERO vision” 2012
  • 11. 11 © 2021 Sogeti Nederland B.V.. All rights reserved. Continuous quality engineering is a team responsibility Quality engineering needs many activities to take place. TMAP defines the QA & Testing topics to group all quality engineering activities. These topics should be applied by all cross-functional (such as Agile or DevOps) team members. Definition: Quality Engineering is about team members and their stakeholders taking joint responsibility to continuously deliver IT systems with the right quality at the right moment to the businesspeople and their customers. It is a principle of software engineering concerned with applying quality measures to assure built-in quality.
  • 12. 12 © 2021 Sogeti Nederland B.V.. All rights reserved. Quality engineering topics Aimed at: orchestrating, arranging, planning, preparing and controlling Focus of multiple teams. Aimed at: the operational QA & testing activities Mainly focused on the scope of one team. Note: The division in topics is not black-and-white, some topics may be relevant for both groups, but the emphasize is on one or the other.
  • 13. 13 © 2021 Sogeti Nederland B.V.. All rights reserved. Put your ideas in the chatbox Fundamental DevOps activities How would you map the Quality engineering topics to DevOps? Organizing topics Performing topics
  • 14. 14 © 2021 Sogeti Nederland B.V.. All rights reserved. Business value Organizations want to deliver business value to their customers. Today’s organizations expect that their IT systems will enable them to generate business value. So first the organization must be clear what business value they pursue. This may be financial value but also value in any other quantitative or qualitative way. The focus on business value helps people remember that IT delivery itself is not the goal.
  • 15. 15 © 2021 Sogeti Nederland B.V.. All rights reserved. Objectives To support the business delivery there is IT delivery. How the pursued value can be achieved is detailed in objectives that drive IT delivery.
  • 16. 16 © 2021 Sogeti Nederland B.V.. All rights reserved. Indicators Indicators define a way to get insight into specific objectives. Measuring these indicators is done by testing and other quality and value measuring activities.
  • 17. 17 © 2021 Sogeti Nederland B.V.. All rights reserved. Confidence in pursued value Measuring the indicators results in information based on which the people involved can form an opinion and determine to what extent they are confident that the pursued value can be achieved.
  • 18. 18 © 2021 Sogeti Nederland B.V.. All rights reserved. Experience the real value When the people involved have enough confidence, they can decide to start using the (new or changed) (parts of) the IT system. The organization (the users and/or their customers) can now experience the real value.
  • 19. 19 © 2021 Sogeti Nederland B.V.. All rights reserved. Value improvement loop Based on the experience of the real value the people involved will get new ideas to further improve the business value which results in the value improvement loop of the VOICE model.
  • 20. 20 © 2021 Sogeti Nederland B.V.. All rights reserved. Supply information based on measuring indicators The VOICE model for delivering business value Measuring indicators = (mainly) Testing
  • 21. 21 © 2021 Sogeti Nederland B.V.. All rights reserved. The VOICE model relates to any IT delivery model. The VOICE model is used to structure the work that the team must do. Quality engineering is one of the many task-groups that need to be structured. The use of the VOICE model is not limited to structuring quality engineering, but is it not fully worked out to all other task-groups.
  • 22. 22 © 2021 Sogeti Nederland B.V.. All rights reserved. Translating Business Value to IT delivery objectives The defined business value is the starting point for quality assurance and testing. It's important to understand the purpose of an IT system and how to create maintain it. Therefor there must be quantifiable objectives with an adequate level of detail. high quality is not necessarily high value
  • 23. 23 © 2021 Sogeti Nederland B.V.. All rights reserved. Exercise What indicator(s) would you use in your current project/organization to measure if you are reaching the IT-objectives for the pursued Business-Value? Put your answer in the chatbox
  • 24. 24 © 2021 Sogeti Nederland B.V.. All rights reserved. This is section 4.1 of the book “Quality for DevOps teams”
  • 25. 25 © 2021 Sogeti Nederland B.V.. All rights reserved. Unit testing and mutation testing
  • 26. 26 © 2021 Sogeti Nederland B.V.. All rights reserved. Code coverage for unit testing
  • 27. 27 © 2021 Sogeti Nederland B.V.. All rights reserved. Code coverage Code coverage indicates what part of the code is being tested Code coverage is measured while running tests (manual check = labour-intensive!) Degree of code coverage depends on the type of coverage you want to achieve: 1. Line Coverage every line of code is executed 2. Statement Coverage every statement in the code is executed 3. Decision Coverage all possible conditional branches are tested 4. Branch coverage all possible branches (conditional and unconditional) are tested 5. Path coverage every path or path combination in a flow is tested Worst coverage Best coverage
  • 28. 28 © 2021 Sogeti Nederland B.V.. All rights reserved. Code coverage – Line coverage Process-oriented • Every line is exercised • Not every statement is necessarily exercised Testdata nop = 1 lop = 250 Result 100% line coverage Although MOVE lop TO lsp is not exercised the line counts Application code DECLARE nop NUMERIC DECLARE lop NUMERIC DECLARE lsp NUMERIC READ nop FROM input-screen MOVE 250 TO lsp DO nop TIMES READ lop FROM input-screen IF lop < lsp THEN MOVE lop TO lsp ENDIF ENDDO
  • 29. 29 © 2021 Sogeti Nederland B.V.. All rights reserved. Code coverage – Statement coverage Process-oriented • Every line is exercised • Every statement is exercised • Not every outcome is necessarily tested Testdata nop = 1 lop = 110 Result 100% statement coverage An IF-statement has a true and false outcome. But only the true is exercised Application code DECLARE nop NUMERIC DECLARE lop NUMERIC DECLARE lsp NUMERIC READ nop FROM input-screen MOVE 250 TO lsp DO nop TIMES READ lop FROM input-screen IF lop < lsp THEN MOVE lop TO lsp ENDIF ENDDO
  • 30. 30 © 2021 Sogeti Nederland B.V.. All rights reserved. Code coverage – Decision/Branch coverage Condition-oriented The two are very similar, Branch coverage is typically for low level languages • Every line is exercised • Every statement is exercised • Every outcome is tested • Not every path is necessarily tested Testdata nop = 2 lop = 110 (test case 1) lop = 150 (test case 2) Result 100% decision/branch coverage by running 2 test cases Application code DECLARE nop NUMERIC DECLARE lop NUMERIC DECLARE lsp NUMERIC READ nop FROM input-screen MOVE 250 TO lsp DO nop TIMES READ lop FROM input-screen IF lop < lsp THEN MOVE lop TO lsp ENDIF ENDDO
  • 31. 31 © 2021 Sogeti Nederland B.V.. All rights reserved. Code coverage – Path coverage Process-oriented • Every line is exercised • Every statement is exercised • Every outcome is tested • Every path is tested Every path or combination of paths in the flow are tested. Path coverage with test depth level-1 all decision-outcomes are tested at least once. Path coverage with test depth level-2 some decision-outcomes are tested multiple times. Path coverage gives the best coverage! (and therefore later in this workshop we will practice with path testing ☺)
  • 32. 32 © 2021 Sogeti Nederland B.V.. All rights reserved. Code coverage – Final remarks Advanced coverage types • Conditional Decision Coverage • Modified Conditional Decision Coverage • Multiple Condition Coverage ➔ More information on tmap.net High code coverage ≠ meaningful testing! Always check if actual result meets expected result, otherwise a test is useless!
  • 33. 33 © 2021 Sogeti Nederland B.V.. All rights reserved. How many test cases would you need as a minimum to test this piece of pseudo-code? Give: • LC (line coverage) • SC (statement coverage) • DC (decision coverage) As a bonus question: Give the test case(s) you would use. Put your answers in the chatbox Exercise code coverage READ VAR_A(numeric) READ VAR_B(numeric) IF VAR_B > 100 PRINT “B is big” ENDIF UNTIL VAR_A > VAR_B DO ADD 1 TO VAR_A END-DO END-UNTIL
  • 34. 34 © 2021 Sogeti Nederland B.V.. All rights reserved. LC (line coverage) Can be done with 1 test case: A = 10 B = 10 The IF is executed (the are statements are not) Until is executed Do is executed Add is executed End-do is executed End-until is executed ALL lines are executed, but not all statements. Solution line coverage READ VAR_A(numeric) READ VAR_B(numeric) IF VAR_B > 100 PRINT “B is big” ENDIF UNTIL VAR_A > VAR_B DO ADD 1 TO VAR_A END-DO END-UNTIL
  • 35. 35 © 2021 Sogeti Nederland B.V.. All rights reserved. SC (statement coverage) Can be done with 1 test case: A = 110 B = 110 The IF, Print & Endif statements are executed Until is executed Do is executed Add is executed End-do is executed End-until is executed ALL lines are executed, but not all statements. Solution statement coverage READ VAR_A(numeric) READ VAR_B(numeric) IF VAR_B > 100 PRINT “B is big” ENDIF UNTIL VAR_A > VAR_B DO ADD 1 TO VAR_A END-DO END-UNTIL
  • 36. 36 © 2021 Sogeti Nederland B.V.. All rights reserved. DC (decision coverage) Can be done with the 2 previously shown test cases A = 10 and A = 110 B = 10 and B = 110 These test cases execute both outcomes of the decision. See further explanation on the previous slides. Solution decision coverage READ VAR_A(numeric) READ VAR_B(numeric) IF VAR_B > 100 PRINT “B is big” ENDIF UNTIL VAR_A > VAR_B DO ADD 1 TO VAR_A END-DO END-UNTIL
  • 37. 37 © 2021 Sogeti Nederland B.V.. All rights reserved. Mutation testing - tests the tests
  • 38. 38 © 2021 Sogeti Nederland B.V.. All rights reserved. Mutation testing At its core, mutation testing is the practice of executing a test, or a set of tests, over many versions of the software under test—the so-called mutants. Mutants are automated source code changes only during the test phase of development. Mutation testing is a useful way to check if common faults would be detected by the test set and this way helps to improve a test set in case the injected faults would not be detected. Examples of Mutation testing tools are: pitest (https://pitest.org/) for java stryker (https://stryker-mutator.io/) for javascript
  • 39. 39 © 2021 Sogeti Nederland B.V.. All rights reserved. Examples of possible mutations of source code Examples of mutations (this is not an exhaustive list) <  <= +  - >  >= /  * &  | !=  == |  & Advanced mutation testing tools can also change return values, or remove conditions in conditional statements. if(length > 120 & age > 18) { accessAllowed(); } else if (length <= 140 | age == 20) { extraValidation(); } else { noAccess(); } Example mutated code 2 if(length >= 120 & age > 18) { accessAllowed(); } else if (length <= 140 | age == 20) { extraValidation(); } else { noAccess(); } Example original code if(length >= 120 | age > 18) { accessAllowed(); } else if (length <= 140 | age == 20) { extraValidation(); } else { noAccess(); } Example mutated code 1 if(length >= 120 & age > 18) { accessAllowed(); } else if (length <= 140 | age != 20) { extraValidation(); } else { noAccess(); } Example mutated code 3
  • 40. 40 © 2021 Sogeti Nederland B.V.. All rights reserved. Mutation testing in Action Original code IF length >= 120 THEN MOVE “admitted” TO status ELSE MOVE ”not admitted” TO status ENDIF Testcases – Using boundary value analysis TC1: length = 119 -> Expected outcome “not admitted” TC2: length = 120 -> Expected outcome “admitted” Code - Mutation 1 IF length > 120 THEN MOVE “admitted” TO status ELSE MOVE ”not admitted” TO status ENDIF Testcases TC1: length = 119 -> Expected outcome “not admitted” TC2: length = 120 -> Expected outcome “admitted” -> FAIL Code – Mutation 2 IF length = 120 THEN MOVE “admitted” TO status ELSE MOVE ”not admitted” TO status ENDIF Testcases – Using boundary value analysis TC1: length = 119 -> Expected outcome “not admitted” TC2: length = 120 -> Expected outcome “admitted” Which test case would catch the mutant? TC3: length = 121 -> Expected outcome “admitted”
  • 41. 41 © 2021 Sogeti Nederland B.V.. All rights reserved. Mutation testing in Action Original code IF length >= 120 THEN MOVE “admitted” TO status ELSE MOVE ”not admitted” TO status ENDIF Testcases – Using boundary value analysis TC1: length = 119 -> Expected outcome “not admitted” TC2: length = 120 -> Expected outcome “admitted” TC3: length = 121 -> Expected outcome “admitted” Code - Mutation 3 IF length < 120 THEN MOVE “admitted” TO status ELSE MOVE ”not admitted” TO status ENDIF Testcases TC1: length = 119 -> Expected outcome “not admitted” -> FAIL TC2: length = 120 -> Expected outcome “admitted” -> FAIL TC3: length = 121 -> Expected outcome “admitted” -> FAIL Code – Mutation 4 IF length != 119 THEN MOVE “admitted” TO status ELSE MOVE ”not admitted” TO status ENDIF Testcases – Using boundary value analysis TC1: length = 119 -> Expected outcome “not admitted” TC2: length = 120 -> Expected outcome “admitted” TC3: length = 121 -> Expected outcome “admitted” Which test case would catch the mutant? TC4: length = 118 -> Expected outcome “not admitted”
  • 42. 42 © 2021 Sogeti Nederland B.V.. All rights reserved. For this piece of pseudo-code we have the following test cases: B = 100, expected result “no print” B = 101, expected result “B is big” What mutation can you do in the code that would not be found with these 2 testcases? Put your answers in the chatbox Exercise Mutation testing READ VAR_B(numeric) IF VAR_B > 100 PRINT “B is big” ENDIF
  • 43. 43 © 2021 Sogeti Nederland B.V.. All rights reserved. For this piece of pseudo-code we have the following test cases: B = 100, expected result “no print” B = 101, expected result “B is big” We need the extra test case: B = 99, expected result “no print” Solution: Mutant 1 READ VAR_B(numeric) IF VAR_B != 100 PRINT “B is big” ENDIF
  • 44. 44 © 2021 Sogeti Nederland B.V.. All rights reserved. For this piece of pseudo-code we have the following test cases: B = 99, expected result “no print” B = 100, expected result “no print” B = 101, expected result “B is big” We need the extra test case: B = 102, expected result “B is big” The total of 4 test cases is known as “4-value Boundary Value Analysis” Solution: Mutant 2 READ VAR_B(numeric) IF VAR_B = 101 PRINT “B is big” ENDIF
  • 45. 45 © 2021 Sogeti Nederland B.V.. All rights reserved. Test design and Path testing
  • 46. 46 © 2021 Sogeti Nederland B.V.. All rights reserved. Test design – Introduction
  • 47. 47 © 2021 Sogeti Nederland B.V.. All rights reserved. Test approaches: experience- and coverage-based
  • 48. 48 © 2021 Sogeti Nederland B.V.. All rights reserved. Experience-based testing approaches Contribution of a large group of people with many different skills, insights, insights and experiences. Testing based on a charter, using test ideas and log. Is done in pairs (2 people) or mob (whole team) A largely intuitive and ad hoc approach to testing → unstructured Relevant experience brought together. Can be used separately or as a tool in another approach. Checklists Error guessing Exploratory testing e.g. mob testing Crowd testing Experience- based testing Checklist-based testing Crowd testing Error guessing Exploratory testing
  • 49. 49 © 2021 Sogeti Nederland B.V.. All rights reserved. Experience-based testing approaches work iteratively
  • 50. 50 © 2021 Sogeti Nederland B.V.. All rights reserved. Coverage-based testing ➔ up-front structured test design This figure shows how test design is related to the test basis (pursued value, requirements and risks) and that it results in test situations and/or test cases and test data which is combined in test scenarios and test scripts.
  • 51. 51 © 2021 Sogeti Nederland B.V.. All rights reserved. Coverage-based testing with test design techniques A test design technique is a standardized method of deriving test cases from a specific test basis that will achieve a certain coverage. Applying a test design technique results in test situations, logical test cases and/or physical test cases. Our investigation has shown that there are about 25 to 30 test design techniques in literature. All test design techniques can be categorized in the four coverage groups mentioned below. In this training course we will practice with at least one technique for each group. Keep in mind that you don’t have to know all existing test design techniques. Knowing a few techniques for each coverage group makes that with just about 8 techniques you can handle any testing challenge.
  • 52. 52 © 2021 Sogeti Nederland B.V.. All rights reserved. Condition-oriented test design – overview of test design techniques Testing of a program flow Test individual decision points with multiple conditions Test multiple decision points that together form a process (this brings higher coverage than path testing) Test all combinations of all conditions (regardsless of decision points)
  • 53. 53 © 2021 Sogeti Nederland B.V.. All rights reserved. Condition-oriented test design These coverage types all can be applied to each of the test design techniques on the left. In this training course we will use the Decision Table test design technique
  • 54. 54 © 2021 Sogeti Nederland B.V.. All rights reserved. Data-oriented test design – overview of test design techniques Uses the classes of a data item (= parameter) to identify what to test. Uses the boundaries between classes of data items to identify what to test. Combines values for multiple data items in test cases. Uses the Creation, Retrieval, Update and Deletion (CRUD) of data items to create tests. Uses the way data flows through an IT system to create tests.
  • 55. 55 © 2021 Sogeti Nederland B.V.. All rights reserved. Appearance-oriented test design – overview of test design techniques Testing the syntax of e.g. user interface Performance testing based on a model of the load Test multiple versions of software on different audiences Specifically focus on how a system can be misused (by mistake) or abused (on purpose) There are many non-functional quality characteristics for which techniques exist.
  • 56. 56 © 2021 Sogeti Nederland B.V.. All rights reserved. Process-oriented test design ➔ Path testing
  • 57. 57 © 2021 Sogeti Nederland B.V.. All rights reserved. Process-oriented testing – overview of test design techniques Tests transitions between states. Tests the path-combinations in a process flow or algorithm (program flow) Tests the flow in a computer program.
  • 58. 58 © 2021 Sogeti Nederland B.V.. All rights reserved. Link between code coverage types and the coverage groups
  • 59. 59 © 2021 Sogeti Nederland B.V.. All rights reserved. What coverage does path-testing give? Path-testing ensures a certain level of coverage by measuring the number of consecutive paths that has been covered. Test-depth-level-1 means: all individual paths are covered by one or more test cases. Test-depth-level-2 means: all combinations of 2 consecutive paths have been covered by one or more test cases. Test-depth-level-n means: all combinations of n consecutive paths have been covered by one or more test cases. TDL-1 & TDL-2 are most used, we will explain these on the next slides.
  • 60. 60 © 2021 Sogeti Nederland B.V.. All rights reserved. What exactly is a path? How many paths do you see? How many test cases do you need for 100% coverage? Put your answer in the chatbox Start Do you like testing? End Go to the testing theatre Go visit the magic boat ride No Yes You are in TestLand
  • 61. 61 © 2021 Sogeti Nederland B.V.. All rights reserved. What exactly is a path? A path is a line between one decision point and another decision point, or a line from the start to the first decision point, or a line form the last decision point to the end. To determine the paths, first simplify the graph, get rid of everything other than decision points, lines, start and end. Start Do you like testing? End Go to the testing theatre Go visit the magic boat ride No Yes You are in TestLand
  • 62. 62 © 2021 Sogeti Nederland B.V.. All rights reserved. What exactly is a path? So this flowchart has the following paths: Start Do you like testing? End No Yes 1 2 3
  • 63. 63 © 2021 Sogeti Nederland B.V.. All rights reserved. Testcases To cover this simple flowchart 100% We need two test cases: TC1: path-1 – path-2 TC2: path-1 – path-3 Start Do you like testing? End No Yes 1 2 3
  • 64. 64 © 2021 Sogeti Nederland B.V.. All rights reserved. Example: Entering the mini-scooter-bumper-cars (for age < 10 years) Start How many drivers? End Enter age driver 1 Enter age driver 1 Enter age driver 2 Is anyone over 10 years? No Yes Find another driver that’s the right age Have fun riding the mini-scooter-bumper-cars one two
  • 65. 65 © 2021 Sogeti Nederland B.V.. All rights reserved. Example: Entering the mini-scooter-bumper-cars (for age < 10 years) How many test cases would you need for 100% coverage of this process? (put answer in chatbox) Different possibilities for 100% coverage Depending on the “Test Depth Level” Start How many drivers? End Enter age driver 1 Enter age driver 1 Enter age driver 2 Is anyone over 10 years? No Yes Find another driver that’s the right age Have fun riding the mini-scooter-bumper-cars one two
  • 66. 66 © 2021 Sogeti Nederland B.V.. All rights reserved. Step 1: determine the paths How many paths are in this process flow? Put your answer in the chatbox Start How many drivers? End Enter age driver 1 Enter age driver 1 Enter age driver 2 Is anyone over 10 years? No Yes Find another driver that’s the right age Have fun riding the mini-scooter-bumper-cars one two
  • 67. 67 © 2021 Sogeti Nederland B.V.. All rights reserved. Step 1: determine the paths, simplify the chart Start How many drivers? End Is anyone over 10 years? No Yes Let’s first simplify the process flow, only start, end, lines and decision points. How many paths are in this process flow? one two
  • 68. 68 © 2021 Sogeti Nederland B.V.. All rights reserved. Step 1: determine the paths, number them And letter the decision points A path starts at a decision point and goes to the next decision point. Or a path starts at a startpoint. Or a path ends at an endpoint. Start A End B No Yes Path 2 Path 4 Path 1 Path 3 Path 5 one two Note: Step 1 applies to TDL-1 and TDL-2
  • 69. 69 © 2021 Sogeti Nederland B.V.. All rights reserved. Step 2: determine the test situations TDL-1: every path is a test situation. First define the paths from start and from each decision point: Start: 1 A: 2, 3 B: 4, 5 Start A End B No Yes Path 2 Path 4 Path 1 Path 3 Path 5 one two Test situations Test situations Test situations
  • 70. 70 © 2021 Sogeti Nederland B.V.. All rights reserved. Step 3: create test cases TDL-1: every path is a test situation. Create test cases by combining paths from start and from each decision point: Start: 1 A: 2, 3 B: 4, 5 TC1: 1, 2, 4 TC2: 1, 3, 5 Start A End B No Yes Path 2 Path 4 Path 1 Path 3 Path 5 one two
  • 71. 71 © 2021 Sogeti Nederland B.V.. All rights reserved. Introduction of the excel template 1 2, 3 4, 5 1, 2, 4 1, 3, 5
  • 72. 72 © 2021 Sogeti Nederland B.V.. All rights reserved. Step 2: determine the test situations TDL-2: every combination of incoming and outgoing path is a test situation. Make a table with 3 columns: Dec.Pt. IN OUT Path-combinations A: 1 2, 3 1-2, 1-3 B: 2, 3 4, 5 2-4, 2-5, 3-4, 3-5 Start A End B No Yes Path 2 Path 4 Path 1 Path 3 Path 5 one two Test situations
  • 73. 73 © 2021 Sogeti Nederland B.V.. All rights reserved. Step 3: determine the test cases TDL-2: Combine path-combinations into test cases. Dec.Pt. IN OUT Path-combinations A: 1 2, 3 1-2 1-3 B: 2, 3 4, 5 2-4, 2-5, 3-4, 3-5 TC1: 1, 2, 4 TC2: 1, 2, 5 TC3: 1, 3, 4 TC4: 1, 3, 5 Start A End B No Yes Path 2 Path 4 Path 1 Path 3 Path 5 one two
  • 74. 74 © 2021 Sogeti Nederland B.V.. All rights reserved. Introduction of the excel template 1 2, 3 4, 5 1, 2, 4 1, 3, 4 2, 3 1-2, 1-3 2-4,2-5,3-4,3-5 1, 2, 5 1, 3, 5
  • 75. 75 © 2021 Sogeti Nederland B.V.. All rights reserved. So: The the number of test cases depends on “Test Depth Level” (= the number of consecutive paths that are tested.) TDL-1: TC1: 1, 2, 4 TC2: 1, 3, 5 TDL-2: TC1: 1, 2, 4 TC2: 1, 2, 5 TC3: 1, 3, 4 TC4: 1, 3, 5 Start How many drivers? End Enter age driver 1 Enter age driver 1 Enter age driver 2 Is anyone over 10 years? No Yes Find another driver that’s the right age Have fun riding the mini-scooter-bumper-cars one two
  • 76. 76 © 2021 Sogeti Nederland B.V.. All rights reserved. One coverage type Two test design techniques So we are talking about “Path coverage”. If you are testing a business process, the technique used is the Process Cycle Test. Start How many drivers? End Enter age driver 1 Enter age driver 1 Enter age driver 2 Is anyone over 10 years? No Yes Find another driver that’s the right age Have fun riding the mini-scooter-bumper-cars one two
  • 77. 77 © 2021 Sogeti Nederland B.V.. All rights reserved. One coverage type Two test design techniques So we are talking about “Path coverage”. If you are testing a business process, the technique used is the Process Cycle Test. If you are testing a program flow, the technique used is the Algorithm Test. But actually the technique is exactly the same, the only difference is the test basis. Start IF NR-PEOPLE = 1 End True False READ AGE-1 READ AGE-1 READ AGE-2 IF AGE-1 > 10 OR AGE-2 > 10 False True CALL subroutine-other-drivers ENTER THE RIDE
  • 78. 78 © 2021 Sogeti Nederland B.V.. All rights reserved. Recap of the Process Cycle Test and Algorithm Test techniques Step 1: Determine the paths Simplify the chart by removing everything else than decision points, start, end and lines. Number the paths and letter the decision points Step 2: Determine the test situations For TDL-1 ➔ list the paths and where they originate For TDL-2 ➔ list the decision points, the INcoming and OUTgoing paths, create path-combinations Step 3: Combine the test situations into test cases From start to end combine test situations into test cases Tips: Process Cycle Test is very well suited for preparing acceptance tests and end-to-end tests. Algorithm Test is very well suited for preparing unit tests, TDL-1 also assures Decision Coverage.
  • 79. 79 © 2021 Sogeti Nederland B.V.. All rights reserved. Exercise path testing ➔ use the excel template Determine the paths and the test cases for TDL-1 and TDL-2 Start Decision A End Action 2 Action 3 Action 4 Decision B No Yes Action 5 Action 6 two three Action 1 one
  • 80. 80 © 2021 Sogeti Nederland B.V.. All rights reserved. Solution path testing Path numbers 1 4 2 3 5 6
  • 81. 81 © 2021 Sogeti Nederland B.V.. All rights reserved. Solution path testing TDL-1 Solution with excel template TDL-1 1 4 2 3 5 6
  • 82. 82 © 2021 Sogeti Nederland B.V.. All rights reserved. Solution path testing TDL-2 1 4 2 3 5 6
  • 83. 83 © 2021 Sogeti Nederland B.V.. All rights reserved. Now we add a loop in the flowchart Step 1: determine the paths How many paths are in this process flow? Start How many drivers? End Enter age driver 1 Enter age driver 1 Enter age driver 2 Is anyone over 10 years? No Yes Have fun riding the mini-scooter-bumper-cars one two
  • 84. 84 © 2021 Sogeti Nederland B.V.. All rights reserved. Step 1: determine the paths, simplify the chart Start How many drivers? End Is anyone over 10 years? No Yes Let’s first simplify the process flow, only start, end and decision points. How many paths are in this process flow? one two
  • 85. 85 © 2021 Sogeti Nederland B.V.. All rights reserved. Step 1: determine the paths Start How many drivers? End Is anyone over 10 years? No Yes There are 5 paths in this process flow How many test cases do you need for 100% coverage TDL-1 ? And TDL-2? one two Path 2 Path 4 Path 1 Path 3 Path 5
  • 86. 86 © 2021 Sogeti Nederland B.V.. All rights reserved. Step 2 and 3: determine test situations and test cases Start How many drivers? End Is anyone over 10 years? No Yes one two Path 2 Path 4 Path 1 Path 3 Path 5 TDL-1: every path = test situation First define the paths from start and from each decision point: Start: 1 A: 2, 3 B: 4, 5 The paths can be combined in one test case: TC1: 1,2,4,3,5
  • 87. 87 © 2021 Sogeti Nederland B.V.. All rights reserved. Step 2: determine test situations Start How many drivers? End Is anyone over 10 years? No Yes one two Path 2 Path 4 Path 1 Path 3 Path 5 TDL-2: every combination of incoming and outgoing path is a test situation. Make a table with 3 columns: Dec.Pt. IN OUT Path-combinations A: 1,4 2,3 1-2, 1-3, 4-2, 4-3 B: 2,3 4,5 2-4, 2-5, 3-4, 3-5
  • 88. 88 © 2021 Sogeti Nederland B.V.. All rights reserved. Step 3: determine test cases Start How many drivers? End Is anyone over 10 years? No Yes one two Path 2 Path 4 Path 1 Path 3 Path 5 TDL-2: Combine path-combinations into test cases. Dec.Pt. IN OUT Path-combinations A: 1,4 2,3 1-2, 1-3, 4-2, 4-3 B: 2,3 4,5 2-4, 2-5, 3-4, 3-5 TC1: 1, 2, 5 (zero times through the loop) TC2: 1, 3, 4, 2, 5 (one time through the loop) Now we need to cover 2-4, 4-3 and 3-5 TC3: 1, 2, 4, 3, 5 (one time through the loop) To improve your coverage (and confidence) do this: TC3’: 1, 2, 4, 3, 4, 3, 5 (two times through the loop)
  • 89. 89 © 2021 Sogeti Nederland B.V.. All rights reserved. Recap of the Path testing Step 1: Determine the paths Simplify the chart by removing everything else than decision points and start & end lines. Number the paths and letter the decision points Step 2: Determine the test situations For TDL-1 ➔ list the paths and where they originate For TDL-2 ➔ list the decision points, the INcoming and OUTgoing paths, create path-combinations Step 3: Combine the test situations into test cases From start to end combine test situations into test cases When there’s a loop, for good confidence create test cases that go through it: zero times (= not through the loop) one time through the loop multiple times (= 2 or more times through the loop) For progression testing: create many different short test cases (easier investigation of anomalies) For regression testing: create fewer longer test cases (easier execution)
  • 90. 90 © 2021 Sogeti Nederland B.V.. All rights reserved. Exercise path testing with loop ➔ use the excel template Determine the paths and the test cases for TDL-1 and TDL-2 Start Decision A End Action 2 Decision B No Yes Action 3 two one Action 1
  • 91. 91 © 2021 Sogeti Nederland B.V.. All rights reserved. Solution path testing with loop Path numbers Start Decision A End Decision B No Yes two one 1 2 3 4 5
  • 92. 92 © 2021 Sogeti Nederland B.V.. All rights reserved. Solution path testing with loop TDL-1 Start Decision A End Decision B No Yes two one 1 2 3 4 5
  • 93. 93 © 2021 Sogeti Nederland B.V.. All rights reserved. Solution path testing with loop TDL-2 Start Decision A End Decision B No Yes two one 1 2 3 4 5
  • 94. 94 © 2021 Sogeti Nederland B.V.. All rights reserved. Closing
  • 95. 95 © 2021 Sogeti Nederland B.V.. All rights reserved. TMAP training courses and certification scheme of Exams provided by TMAP: Quality for cross- functional teams TMAP: High- performance quality engineering TMAP: Organizing built- in quality at scale Performing people 3-day training course including exam of 30 questions 3-day training courses with separate exams of 40 questions All people involved in cross- functional teams Organizing people Do you want to learn more about TMAP? https://pages.isqi.org/tmap-2020/ https://tmap.net/page/tmap-devops-teams https://academy.sogeti.nl/2020-tmap
  • 96. 96 © 2021 Sogeti Nederland B.V.. All rights reserved. And visit the renewed body of knowledge on www.tmap.net Who wins the ePub of my TMAP book? and if you don’t win, find it on www.ict-books.com or download the free preview-version on: https://tmap.net/book/quality-devops-teams Find the excel template for path testing also on www.TMAP.net : https://tmap.net/page/download-templates
  • 97. 97 © 2021 Sogeti Nederland B.V.. All rights reserved. About Sogeti Learn more about us at www.sogeti.com This message contains information that may be privileged or confidential and is the property of the Capgemini Group. Copyright© 2018 Sogeti. All rights reserved. Sogeti is a leading provider of technology and engineering services. Sogeti delivers solutions that enable digital transformation and offers cutting-edge expertise in Cloud, Cybersecurity, Digital Manufacturing, Digital Assurance & Testing, and emerging technologies. Sogeti combines agility and speed of implementation with strong technology supplier partnerships, world class methodologies and its global delivery model, Rightshore®. Sogeti brings together more than 25,000 professionals in 15 countries, based in over 100 locations in Europe, USA and India. Sogeti is a wholly-owned subsidiary of Capgemini SE, listed on the Paris Stock Exchange. Good luck with quality engineering !!! Thank you for your participation Contact me on: rik.Marselis@sogeti.com Or on twitter: @rikmarselis