Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Marselis)

Quality Engineering
in the digital age…
Why? How?
Rik Marselis
ASQF NRW
15 April 2021
TMAP: the body of knowledge for
quality engineering in IT delivery
You can win a copy
of this book!
(the ePub version)

2
© 2021 Sogeti. All rights reserved.
Quality engineering in the digital age… Why? How?

3
Challenges of today’s high-performance IT delivery
The business demands:
• Deliver business value
• Deliver quality at speed
The team challenges are:
• Quality engineering is everyone's responsibility
• QA & testing is integrated in process and people
The focus is:
• Organize high-performing cross-functional teams (you build it, you run it!)
• Automate everything
Using
AI

4
Businesspeople expect:
Deliver business value
with the right quality - at speed…

5
The VOICE model for delivering business value

6
Supply information
based on
measuring indicators
The VOICE model for delivering business value
Measuring indicators
=
(mainly) Testing

7
Remember the
meaning of IT:
INFORMATION
technology

8
What stakeholders need: a monitor for
“confidence in pursued business value”
This monitor is created by combining information
from multiple measured indicators, both “hard
indicatiors” (such as measured values) and “soft
indicators” (such as opinions of involved people).

9

10
Evolution of the reasons for testing and quality engineering
1979:
Find all bugs
and fix them
(so you’ll get perfect
software)
1995:
Create
structured
test cases
2006:
Supply info on
Quality & Risk
2014:
Confidence
in quality
2018:
Quality
forecasting
2020:
Confidence in
Business value

11
Built-in quality ➔ Quality engineering
Quality Engineering is about team members and their stakeholders
taking joint responsibility to continuously deliver IT systems
with the right quality at the right moment to the businesspeople and their customers.
It is a principle of software engineering concerned with applying quality measures to
assure the quality of IT systems.
It is all about quality of:
• Product
• Process
• People

12
Quality engineering

13
Quality measures
Quality was, is and remains a challenge within the IT industry.
The high-performance cross-functional team must actively work on quality engineering.
Quality engineering consists of a great number of possible activities, the so-called
quality measures.
After all, quality must be built in, not tested in!
Examples of quality measures:
• Specification and Example (for common understanding of what needs to be created)
• Test Driven Development and Pair Programming (for fast creation of high quality software)
• Feature Toggles and Monitoring (for efficient and effective operations)
and…
• Testing (to gather information about quality and risks, based on indicators)

14
Definition of testing as part of quality engineering
Testing consists of verification, validation and exploration activities
that provide information about the quality and the related risks,
to establish the level of confidence
that a test object will be able to deliver the pursued business value.

15

16
Automate everything (as long as it is useful)
Team
scope
Business
process
scope

17
Automate everything ➔ use machine intelligence:
from descriptive analytics to predictive analytics to…
Descriptive
analytics
Predictive
analytics

18
… quality forecasting
Descriptive
analytics
Predictive
analytics
Prescrictive
analytics
Test
environment
Live
environment
Future
situation

AI based analytics and forecasting
Machine intelligence is based on:
• Descriptive analytics
• Predictive analytics
• Prescriptive analytics
A major benefit is…
Forecasting

20
A practical example of forecasting

AI based analytics and forecasting
• The car predicted a crash…
• and started breaking
• Already before the crash actually happened!!
21
A brilliant example of
prescriptive analytics

22
Machine intelligence ➔ Artificial Intelligence & Machine Learning
Machine learning (how it looked 30 years ago…)
In the 1980’s there was no WorldWideWeb so knowledge had to come from books…
From the film “Short circuit” 1986

23
Machine Learning (as the most common way of implementing AI)
▪ Machine Learning is one of the ways to
achieve artificial intelligence.
▪ It contains different algorithms – each with
its own strengths and weaknesses.
▪ These algorithms are often grouped into
three categories:
• Supervised learning
• Unsupervised machine learning
• Reinforcement learning
See the article of
Vijay Raghani
on SogetiLabs.com

Machine Learning categories
▪ Supervised learning is the machine-learning task of learning a
function that maps an input to an output based on example input-
output pairs. It infers a function from labeled training data consisting
of a set of training examples.
▪ Unsupervised learning is the machine-learning task of inferring a
function to describe hidden structure from "unlabeled" data (a
classification or categorization is not included in the observations).
▪ Reinforcement learning is a variety of machine learning that
determines various options to find the option that maximizes some
notion of cumulative reward. Reinforcement learning differs from
standard supervised learning in that correct input/output pairs are
never presented, nor sub-optimal actions explicitly corrected. Instead
the focus is on performance.

25

A machine learning test set is not part of software testing
(for testing the result of learning, use the validation set)
Training data
Used for
the actual
machine
learning
Test data
Validation data
Used to check the
results of the
learning process
The training data and test
data are extracted multiple
times randomly, so various
combinations of the data are
used to train the model in
multiple iterations.
© Sogeti 2019 26
Beware of confusing terminology between data scientists and testers ☺

This is also called
“Feature
engineering”
(the variables are
called features)
Machine learning is a complex process
which is not fully automated
Select
data
Preprocess
data
Transform
data
Train
model
Preparation of the data Actual
machine
learning
Largely manual!! Mainly automatic

28
The difference in descriptive analytics before and today
Traditional approach
sampling
Today’s possibility
full analysis

Multi-model predictions
▪ Predictive analytics uses multiple models
▪ The available data from test execution and
live monitoring is fed into multiple models.
▪ Each model calculates the expected
evolution of quality.
▪ The results are shown in a “plume”. Today
▪ The digital quality engineer determines, based
on the relevant parameters, which is the most
likely future situation. If the quality is at risk a
“code red” can be issued.
▪ Based on this forecast the team can decide
what actions need to be taken to ensure
quality.
Near future
▪ Prescriptive analytics determine the most
likely situation and if quality is at risk
immediately triggers corrective action.

Descriptive and predictive analysis can be used for reporting
“confidence in pursued business value”

Waterfall V-model Scrum DevOps Demand-Supply SAFe®
© 2021 Sogeti Nederland B.V.. All rights reserved.

DevOps – principles and definition
The six DevOps principles:
1. Customer-centric action
2. Create with the end in mind
3. End-to-end responsibility
4. Cross-functional autonomous teams
5. Continuous improvement
6. Automate everything you can
DevOps is a cross-functional systems engineering culture that aims at unifying
systems development (Dev) and systems operations (Ops) with the ability to create and
deliver fast, cheap, flexible and with adequate quality, whereby the team as a whole is
responsible for the quality. Other areas of expertise, such as business analysis and quality
assurance (including testing) are usually integrated in the team. A DevOps culture has an
Agile mindset that can be supported/implemented by, e.g., the Scrum framework.
Source: the DevOps handbook

34
High-performance IT delivery
Definition:
High-performance IT delivery is an approach that enables cross-functional teams to continuously
improve the products, processes and people that are required to deliver value to the end users.
To achieve this continuous improvement quality must be built-in in not only the product but also
in the process (activities) and the people (individuals and the team as a whole).
Quality
Product People
Process

35
Built-in quality
Built-in quality is one of the key principles in the Lean approach.
Built-in quality, continuously improved, leads to right-first-time, and
where the outcome of the process meets the expectation: fit-for-purpose.
Quality of the product is strongly linked to the quality of the delivery
process and the people.
Any deviation of the expected product quality should lead to
improvement,
not just fixing the fault
but also changing the process and people
to prevent such faults from occurring again.
Source of picture: book “The PointZERO vision” 2012

36
Built-in quality and SAFe®
Test-first approach (TDD & BDD) &
continuous delivery pipeline
Determining future needs, design for
quality ➔ good testability
TDD & automated unit testing, paired
working, collective ownership (across
teams), coding standards
BDD ➔
alignment and shared understanding ➔
reduce rework and delays
Scalable definition of done (DoD) aligned
with goals of teams and organization
Note: TDD = Test Driven Development
BDD = Behavior Driven Development

37
TEAM:
Together
Everyone
Achieves
More
The starting point for DevOps:
Cross-functional teams
Aligned with the DevOps culture of IT delivery – where roles are
important and functions are not – keep in mind that if in TMAP you see
a term such as developer or tester, we refer to the role of a team
member at a specific moment in time, not to a function.

38
A team should be cross-functional, not multi-disciplinary
Collaboration is the key to success in high-performance IT delivery !!

39
What activities should the team do?
The TMAP QA & testing topics are
11 Organizing topics
9 Performing topics
That describe all quality engineering
activities that any team or organization
should consider.

40
Continuous quality engineering is a team responsibility
Quality engineering needs many activities to take place.
TMAP defines the QA & Testing topics to group all quality engineering activities.
These topics should be applied by all cross-functional (such as Scrum, Agile or DevOps)
team members.
Definition:
Quality Engineering is about team members and their stakeholders taking joint responsibility to
continuously deliver IT systems with the right quality at the right moment to the businesspeople
and their customers. It is a principle of software engineering concerned with applying quality
measures to assure built-in quality.

41
Conclusion
• Quality engineering for quality at speed:
Automation in your pipeline
• Measure indicators to support confidence
• From descriptive analytics
(use AI to analyze information)
• To predictive analytics
(quality forecasting)
• Towards prescriptive analytics
(fixing problems before anyone has noticed them)
• This way quality engineering supports that the team will
Deliver Business Value to the stakeholders

42
© 2021 Sogeti. All rights reserved. 42
The real value of Artificial Intelligence is…
that people learn
completely new ways of thinking
and therefore develop
a higher level of intelligence!!
Source: “AI first; learning from the machine” by ViNT/SogetiLabs, 2018

The elevator that can look into the future.
Source: https://www.slideshare.net/RikMarselis/an-intelligent-elevator
An anecdote on forecasting…

44
TMAP training courses and certification scheme of
Exams provided by
TMAP: Quality
for cross-
functional teams
TMAP: High-
performance
quality
engineering
TMAP:
Organizing built-
in quality
at scale
Performing
people
3-day training course
including
exam of 30 questions
3-day training courses
with separate
exams of 40 questions
All people
involved in
cross-
functional
teams
Organizing
people
Do you want to learn more about TMAP and quality engineering?
https://pages.isqi.org/tmap-2020/
https://tmap.net/page/tmap-devops-teams
https://academy.sogeti.nl/2020-tmap

Download an information package using this link:
https://fts.capgemini.com/pubpwd/365156930079/ASQF_NRW_keynote_QualityEngineeringInTheDigitalAge_RikMarselis.zip
username: ajdfdhfsj Password: SCZ7WhRkAJ
It contains: preview-versions of both books, the article on the elevator
that looks into the future and info on the TMAP certifications scheme.
And visit the renewed
TMAP body of knowledge
on www.tmap.net
2018
2020
You can buy a book on
www.ict-books.com

About Sogeti
Learn more about us at
www.sogeti.com
This message contains information that may be privileged or
confidential and is the property of the Capgemini Group.
Copyright© 2018 Sogeti. All rights reserved.
Sogeti is a leading provider of technology and engineering services. Sogeti
delivers solutions that enable digital transformation and offers cutting-edge
expertise in Cloud, Cybersecurity, Digital Manufacturing, Digital Assurance
& Testing, and emerging technologies. Sogeti combines agility and speed of
implementation with strong technology supplier partnerships, world class
methodologies and its global delivery model, Rightshore®. Sogeti brings
together more than 25,000 professionals in 15 countries, based in over 100
locations in Europe, USA and India. Sogeti is a wholly-owned subsidiary of
Capgemini SE, listed on the Paris Stock Exchange.
Good luck with quality engineering !!!
Thank you for your participation
Contact me on: rik.Marselis@sogeti.com
Or on twitter: @rikmarselis

47
Rik.Marselis@sogeti.com
Principal Quality Consultant
2007 2012 2012
2008 2014
2009
2018
1980 2020

Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Marselis)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Marselis)

Similar to Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Marselis) (20)

More from Rik Marselis

More from Rik Marselis (20)

Recently uploaded

Recently uploaded (20)

Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Marselis)