SlideShare a Scribd company logo
1 of 34
Download to read offline
Testing…
in the digital age
AI makes the difference
Booklaunch
German Testing Days, Frankfurt, 8 June 2018
Rik Marselis
|
Who has a robot?
© Sogeti 2018 2
Who has a robot? In 10 years all of you will !!
1980
Rik Marselis@rikmarselis
2007 2012 20122008
Workgroup
-member
20142009
Management consultant
Digital assurance & testing
|
The future of testing: quality forecasting
© Sogeti 2018 4
|
Would you like to work with a robot?
© Sogeti 2018 5
|
Cobotics: collaborating with robots
© Sogeti 2018 6
|
Exoskeleton: robot gives extra muscle-power
© Sogeti 2018 7
Ruben de Sain crowdfunded his
exoskeleton to get out of his
wheelchair.
|
What are your fears about intelligent machines?
© Sogeti 2018 8
|
The Frankenstein factor: fear of AI
© Sogeti 2018 9Source: SogetiLabs “The Frankenstein Factor”
|© Sogeti 2018 10
Super intelligence & super stupidity
Foto uit de film “Ex Machina” 2015
|
Two more general worries
Loss of culture and defeating nature
© Sogeti 2018 11
|
Let’s cope with these fears in a scientific way…
© Sogeti 2018 12
|
Six angles of quality
© Sogeti 2018 13
Existing knowledge New knowledge
|
Mechanical, Electrical, Information processing
© Sogeti 2018 14
No. 2001 is seen on the rollers at the Vitry test plant
Gresley Society Collection, 1934
Testing an electric stove,
1970’s © handyman
The 7 generic testing activities of
TMap® 1995/2006
Existing knowledge
But still very relevant,
also for testing
intelligent machines.
|
Testing intelligent machines is different…
© Sogeti 2018 15
Traditional testing compares
expected and actual outcome.
For intelligent machines predicting
outcome is difficult; todays
correct result may differ from
tomorrows correct result, so:
- Define tolerances
- Control input
- Test the machine in its
intended environment
- Use new quality characteristics
|
ISO25010 – Product Quality characteristics
© Sogeti 2018 16
|
ISO25010 extended quality characteristics
© Sogeti 2018 17
|
The new quality characteristics
© Sogeti 2018 18
|
Machine learning – ability to learn
© Sogeti 2017 19
<<< vervangen door filmclip >>>
This is what machine learning looked like ☺ Scene from the film “Short circuit” (1986)
|
Ability to learn and to unlearn
Machine learning
• Continued learning
• Frozen
Examination
• One-time
• Continuous
Un-learning
© Sogeti 2018 20
Video:
Unlearning and
relearning to
ride a bicycle
|
How to unlearn AI’s ?
Rule-based AI:
• Change the programmed rules
Reinformement learning AI:
• Change goal(s)
• Control input
• Feed new learning data
© Sogeti 2018 21
U N
|Morality: Ethics & Privacy
Scene from the film “Robot and Frank” 2012
|
Scene from documentary
“I am Alice” 2015
Morality: Ethics & Privacy
|
Personality - mood
© Sogeti 2018 24
Personality: mood & humor
|
now let’s look at
testing WITH them
Well, this was all about
testing OF
intelligent machines,
25© Sogeti 2018
|
Robot automatically performs regression tests
© Sogeti 26
Video that shows how Sogeti does regression testing of a cockpit at Airbus
|
Robot automatically performs regression tests
© Sogeti 2017 27
Low-cost
(€300,-)
robot-arm
tests mobile
devices.
Connected
to test ma-
nagement
tool.
Test cases
check if
the quality
goals are
met and
risks are
covered.
|
Use a digital twin to virtually test a product…
© Sogeti 2018 28
|
More testing in less time
© Sogeti 2018 29
|
The future of testing: quality forecasting
© Sogeti 2018 30
|
Forecasting using multiple models
© Sogeti 2018 31
The tester will interpret the outcome of multiple
models and create the advice for the stakeholders
|
Cognitive QA – for example: smart dashboards
© Sogeti 2018 32
|Sogeti PowerPoint Referentie 2014 33
Thank you.
This book was introduced on 1 June 2018
Humayun Shaukat Tom van de Ven Rik Marselis
© Sogeti 2018

More Related Content

Similar to Testing in the digital age; international booklaunch at German Testing Day

Magic of Automation and Everyday Chores.pptx
Magic of Automation and Everyday Chores.pptxMagic of Automation and Everyday Chores.pptx
Magic of Automation and Everyday Chores.pptxJamie Coleman
 
CWIN17 Toulouse / How robotics as a service improves your industrial perform...
CWIN17 Toulouse / How robotics as a service improves  your industrial perform...CWIN17 Toulouse / How robotics as a service improves  your industrial perform...
CWIN17 Toulouse / How robotics as a service improves your industrial perform...Capgemini
 
OneBot: A Comprehensive Case Study on Enterprise Digital Assistants
OneBot: A Comprehensive Case Study on Enterprise Digital AssistantsOneBot: A Comprehensive Case Study on Enterprise Digital Assistants
OneBot: A Comprehensive Case Study on Enterprise Digital AssistantsSoham Dasgupta
 
“Automotive Vision – What’s Growing, What’s Not, and Why?,” a Presentation fr...
“Automotive Vision – What’s Growing, What’s Not, and Why?,” a Presentation fr...“Automotive Vision – What’s Growing, What’s Not, and Why?,” a Presentation fr...
“Automotive Vision – What’s Growing, What’s Not, and Why?,” a Presentation fr...Edge AI and Vision Alliance
 
Accelerating Machine Learning Adoption in the Automotive Industry
Accelerating Machine Learning Adoption in the Automotive IndustryAccelerating Machine Learning Adoption in the Automotive Industry
Accelerating Machine Learning Adoption in the Automotive IndustryBigML, Inc
 
Digital transformation continues to drive IT strategy, How is QA and testing ...
Digital transformation continues to drive IT strategy, How is QA and testing ...Digital transformation continues to drive IT strategy, How is QA and testing ...
Digital transformation continues to drive IT strategy, How is QA and testing ...QA or the Highway
 
Sensors for robotic vehicles 2018 report by yole developpement i-micronews
Sensors for robotic vehicles 2018 report by yole developpement i-micronewsSensors for robotic vehicles 2018 report by yole developpement i-micronews
Sensors for robotic vehicles 2018 report by yole developpement i-micronewsYole Developpement
 
Collaborative edge intelligence-mihir punjabi and sunil kumar jang bahadur_cw...
Collaborative edge intelligence-mihir punjabi and sunil kumar jang bahadur_cw...Collaborative edge intelligence-mihir punjabi and sunil kumar jang bahadur_cw...
Collaborative edge intelligence-mihir punjabi and sunil kumar jang bahadur_cw...Capgemini
 
GAIBT Pune - Future of AI.pptx
GAIBT Pune - Future of AI.pptxGAIBT Pune - Future of AI.pptx
GAIBT Pune - Future of AI.pptxLuis Beltran
 
Cwin16 tls cnes-realite_augmentee_eng_v1 2
Cwin16 tls cnes-realite_augmentee_eng_v1 2Cwin16 tls cnes-realite_augmentee_eng_v1 2
Cwin16 tls cnes-realite_augmentee_eng_v1 2Capgemini
 
Experiment Management for the Enterprise
Experiment Management for the EnterpriseExperiment Management for the Enterprise
Experiment Management for the EnterpriseSigOpt
 
CWIN17 Toulouse / Virtual reality to improve the training path of soldiers m....
CWIN17 Toulouse / Virtual reality to improve the training path of soldiers m....CWIN17 Toulouse / Virtual reality to improve the training path of soldiers m....
CWIN17 Toulouse / Virtual reality to improve the training path of soldiers m....Capgemini
 
Ac ford innovation day-fina lv2
Ac ford innovation day-fina lv2Ac ford innovation day-fina lv2
Ac ford innovation day-fina lv2Atakan Cetinsoy
 
Introduction to CW (Cambridge Wireless) Future Devices & Technologies Group (...
Introduction to CW (Cambridge Wireless) Future Devices & Technologies Group (...Introduction to CW (Cambridge Wireless) Future Devices & Technologies Group (...
Introduction to CW (Cambridge Wireless) Future Devices & Technologies Group (...3G4G
 
My failed projects & lessons learned
My failed projects & lessons learnedMy failed projects & lessons learned
My failed projects & lessons learnedLaihioh Yeh
 
Pragmatic use of artificial intelligence in smart cities 03262018
Pragmatic use of artificial intelligence in smart cities 03262018Pragmatic use of artificial intelligence in smart cities 03262018
Pragmatic use of artificial intelligence in smart cities 03262018ThomasCook16
 
The Smartest Smart Cities Will Benefit from the Confluence of AI and Big Data...
The Smartest Smart Cities Will Benefit from the Confluence of AI and Big Data...The Smartest Smart Cities Will Benefit from the Confluence of AI and Big Data...
The Smartest Smart Cities Will Benefit from the Confluence of AI and Big Data...Echelon Corporation
 
Data & Analytics trends STKI summit 2016
Data & Analytics trends STKI summit 2016Data & Analytics trends STKI summit 2016
Data & Analytics trends STKI summit 2016Einat Shimoni
 
Automation revolution AI ML RPAs 2019
Automation revolution   AI ML RPAs 2019Automation revolution   AI ML RPAs 2019
Automation revolution AI ML RPAs 2019Galit Fein
 

Similar to Testing in the digital age; international booklaunch at German Testing Day (20)

Magic of Automation and Everyday Chores.pptx
Magic of Automation and Everyday Chores.pptxMagic of Automation and Everyday Chores.pptx
Magic of Automation and Everyday Chores.pptx
 
CWIN17 Toulouse / How robotics as a service improves your industrial perform...
CWIN17 Toulouse / How robotics as a service improves  your industrial perform...CWIN17 Toulouse / How robotics as a service improves  your industrial perform...
CWIN17 Toulouse / How robotics as a service improves your industrial perform...
 
OneBot: A Comprehensive Case Study on Enterprise Digital Assistants
OneBot: A Comprehensive Case Study on Enterprise Digital AssistantsOneBot: A Comprehensive Case Study on Enterprise Digital Assistants
OneBot: A Comprehensive Case Study on Enterprise Digital Assistants
 
“Automotive Vision – What’s Growing, What’s Not, and Why?,” a Presentation fr...
“Automotive Vision – What’s Growing, What’s Not, and Why?,” a Presentation fr...“Automotive Vision – What’s Growing, What’s Not, and Why?,” a Presentation fr...
“Automotive Vision – What’s Growing, What’s Not, and Why?,” a Presentation fr...
 
Accelerating Machine Learning Adoption in the Automotive Industry
Accelerating Machine Learning Adoption in the Automotive IndustryAccelerating Machine Learning Adoption in the Automotive Industry
Accelerating Machine Learning Adoption in the Automotive Industry
 
Digital transformation continues to drive IT strategy, How is QA and testing ...
Digital transformation continues to drive IT strategy, How is QA and testing ...Digital transformation continues to drive IT strategy, How is QA and testing ...
Digital transformation continues to drive IT strategy, How is QA and testing ...
 
Sensors for robotic vehicles 2018 report by yole developpement i-micronews
Sensors for robotic vehicles 2018 report by yole developpement i-micronewsSensors for robotic vehicles 2018 report by yole developpement i-micronews
Sensors for robotic vehicles 2018 report by yole developpement i-micronews
 
Collaborative edge intelligence-mihir punjabi and sunil kumar jang bahadur_cw...
Collaborative edge intelligence-mihir punjabi and sunil kumar jang bahadur_cw...Collaborative edge intelligence-mihir punjabi and sunil kumar jang bahadur_cw...
Collaborative edge intelligence-mihir punjabi and sunil kumar jang bahadur_cw...
 
GAIBT Pune - Future of AI.pptx
GAIBT Pune - Future of AI.pptxGAIBT Pune - Future of AI.pptx
GAIBT Pune - Future of AI.pptx
 
Cwin16 tls cnes-realite_augmentee_eng_v1 2
Cwin16 tls cnes-realite_augmentee_eng_v1 2Cwin16 tls cnes-realite_augmentee_eng_v1 2
Cwin16 tls cnes-realite_augmentee_eng_v1 2
 
Experiment Management for the Enterprise
Experiment Management for the EnterpriseExperiment Management for the Enterprise
Experiment Management for the Enterprise
 
Agile Adventures: Developers vs. Testers
Agile Adventures: Developers vs. TestersAgile Adventures: Developers vs. Testers
Agile Adventures: Developers vs. Testers
 
CWIN17 Toulouse / Virtual reality to improve the training path of soldiers m....
CWIN17 Toulouse / Virtual reality to improve the training path of soldiers m....CWIN17 Toulouse / Virtual reality to improve the training path of soldiers m....
CWIN17 Toulouse / Virtual reality to improve the training path of soldiers m....
 
Ac ford innovation day-fina lv2
Ac ford innovation day-fina lv2Ac ford innovation day-fina lv2
Ac ford innovation day-fina lv2
 
Introduction to CW (Cambridge Wireless) Future Devices & Technologies Group (...
Introduction to CW (Cambridge Wireless) Future Devices & Technologies Group (...Introduction to CW (Cambridge Wireless) Future Devices & Technologies Group (...
Introduction to CW (Cambridge Wireless) Future Devices & Technologies Group (...
 
My failed projects & lessons learned
My failed projects & lessons learnedMy failed projects & lessons learned
My failed projects & lessons learned
 
Pragmatic use of artificial intelligence in smart cities 03262018
Pragmatic use of artificial intelligence in smart cities 03262018Pragmatic use of artificial intelligence in smart cities 03262018
Pragmatic use of artificial intelligence in smart cities 03262018
 
The Smartest Smart Cities Will Benefit from the Confluence of AI and Big Data...
The Smartest Smart Cities Will Benefit from the Confluence of AI and Big Data...The Smartest Smart Cities Will Benefit from the Confluence of AI and Big Data...
The Smartest Smart Cities Will Benefit from the Confluence of AI and Big Data...
 
Data & Analytics trends STKI summit 2016
Data & Analytics trends STKI summit 2016Data & Analytics trends STKI summit 2016
Data & Analytics trends STKI summit 2016
 
Automation revolution AI ML RPAs 2019
Automation revolution   AI ML RPAs 2019Automation revolution   AI ML RPAs 2019
Automation revolution AI ML RPAs 2019
 

More from Rik Marselis

The End Of Testing As We Know It (TestCon - Rik Marselis).pdf
The End Of Testing As We Know It (TestCon - Rik Marselis).pdfThe End Of Testing As We Know It (TestCon - Rik Marselis).pdf
The End Of Testing As We Know It (TestCon - Rik Marselis).pdfRik Marselis
 
TestExpo Quality Engineering & Sustainability
TestExpo Quality Engineering & Sustainability TestExpo Quality Engineering & Sustainability
TestExpo Quality Engineering & Sustainability Rik Marselis
 
GTR-The End Of Testing As We Know It
GTR-The End Of Testing As We Know ItGTR-The End Of Testing As We Know It
GTR-The End Of Testing As We Know ItRik Marselis
 
Introduction of TMAP to representatives of ISTQB boards in the GA week in Mar...
Introduction of TMAP to representatives of ISTQB boards in the GA week in Mar...Introduction of TMAP to representatives of ISTQB boards in the GA week in Mar...
Introduction of TMAP to representatives of ISTQB boards in the GA week in Mar...Rik Marselis
 
Quality Engineering in today's cross-functTeams with TMAP
Quality Engineering in today's cross-functTeams with TMAPQuality Engineering in today's cross-functTeams with TMAP
Quality Engineering in today's cross-functTeams with TMAPRik Marselis
 
Quality engineering instead of testing... Why? How?
Quality engineering instead of testing... Why? How?Quality engineering instead of testing... Why? How?
Quality engineering instead of testing... Why? How?Rik Marselis
 
Quality Engineering and Testing with TMAP in DevOps IT delivery
Quality Engineering and Testing with TMAP in DevOps IT deliveryQuality Engineering and Testing with TMAP in DevOps IT delivery
Quality Engineering and Testing with TMAP in DevOps IT deliveryRik Marselis
 
Quality engineering & testing in DevOps IT delivery with TMAP
Quality engineering & testing in DevOps IT delivery with TMAPQuality engineering & testing in DevOps IT delivery with TMAP
Quality engineering & testing in DevOps IT delivery with TMAPRik Marselis
 
Quality engineering in DevOps... Why? How? (TestBusters Day&Night))
Quality engineering in DevOps... Why? How? (TestBusters Day&Night))Quality engineering in DevOps... Why? How? (TestBusters Day&Night))
Quality engineering in DevOps... Why? How? (TestBusters Day&Night))Rik Marselis
 
TMAP Quality Engineering workshop on A4Q congress by Rik Marselis
TMAP Quality Engineering workshop on A4Q congress by Rik Marselis TMAP Quality Engineering workshop on A4Q congress by Rik Marselis
TMAP Quality Engineering workshop on A4Q congress by Rik Marselis Rik Marselis
 
Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Mars...
Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Mars...Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Mars...
Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Mars...Rik Marselis
 
Quality for DevOps teams - Quality engineering in the DevOps culture
Quality for DevOps teams - Quality engineering in the DevOps cultureQuality for DevOps teams - Quality engineering in the DevOps culture
Quality for DevOps teams - Quality engineering in the DevOps cultureRik Marselis
 
How to write a proposal to speak at a conference? Hints & tips
How to write a proposal to speak at a conference? Hints & tipsHow to write a proposal to speak at a conference? Hints & tips
How to write a proposal to speak at a conference? Hints & tipsRik Marselis
 
Testen ISTQB, TMap, ISO25010 en meer. Gastcollege Windesheim Rik Marselis
Testen ISTQB, TMap, ISO25010 en meer. Gastcollege Windesheim Rik MarselisTesten ISTQB, TMap, ISO25010 en meer. Gastcollege Windesheim Rik Marselis
Testen ISTQB, TMap, ISO25010 en meer. Gastcollege Windesheim Rik MarselisRik Marselis
 
Over testen gesproken TestNet najaar 2019 openingskeynote Rik Marselis
Over testen gesproken TestNet najaar 2019 openingskeynote Rik MarselisOver testen gesproken TestNet najaar 2019 openingskeynote Rik Marselis
Over testen gesproken TestNet najaar 2019 openingskeynote Rik MarselisRik Marselis
 
Testing of artificial intelligence; AI quality engineering skils - an introdu...
Testing of artificial intelligence; AI quality engineering skils - an introdu...Testing of artificial intelligence; AI quality engineering skils - an introdu...
Testing of artificial intelligence; AI quality engineering skils - an introdu...Rik Marselis
 
Webinar trends in testen 2017-12-13 door Rik Marselis
Webinar trends in testen 2017-12-13 door Rik MarselisWebinar trends in testen 2017-12-13 door Rik Marselis
Webinar trends in testen 2017-12-13 door Rik MarselisRik Marselis
 
Ketenregie in Agile / DevOps: Noodzaak? (oplossing: Gilde, SAFe, combi ...)
Ketenregie in Agile / DevOps: Noodzaak? (oplossing: Gilde, SAFe, combi ...)Ketenregie in Agile / DevOps: Noodzaak? (oplossing: Gilde, SAFe, combi ...)
Ketenregie in Agile / DevOps: Noodzaak? (oplossing: Gilde, SAFe, combi ...)Rik Marselis
 
Webinar trends in testing 2017 03 08 (in dutch)
Webinar trends in testing 2017 03 08 (in dutch)Webinar trends in testing 2017 03 08 (in dutch)
Webinar trends in testing 2017 03 08 (in dutch)Rik Marselis
 
Past het testvak nog in de nieuwe IT-wereld?
Past het testvak nog in de nieuwe IT-wereld?Past het testvak nog in de nieuwe IT-wereld?
Past het testvak nog in de nieuwe IT-wereld?Rik Marselis
 

More from Rik Marselis (20)

The End Of Testing As We Know It (TestCon - Rik Marselis).pdf
The End Of Testing As We Know It (TestCon - Rik Marselis).pdfThe End Of Testing As We Know It (TestCon - Rik Marselis).pdf
The End Of Testing As We Know It (TestCon - Rik Marselis).pdf
 
TestExpo Quality Engineering & Sustainability
TestExpo Quality Engineering & Sustainability TestExpo Quality Engineering & Sustainability
TestExpo Quality Engineering & Sustainability
 
GTR-The End Of Testing As We Know It
GTR-The End Of Testing As We Know ItGTR-The End Of Testing As We Know It
GTR-The End Of Testing As We Know It
 
Introduction of TMAP to representatives of ISTQB boards in the GA week in Mar...
Introduction of TMAP to representatives of ISTQB boards in the GA week in Mar...Introduction of TMAP to representatives of ISTQB boards in the GA week in Mar...
Introduction of TMAP to representatives of ISTQB boards in the GA week in Mar...
 
Quality Engineering in today's cross-functTeams with TMAP
Quality Engineering in today's cross-functTeams with TMAPQuality Engineering in today's cross-functTeams with TMAP
Quality Engineering in today's cross-functTeams with TMAP
 
Quality engineering instead of testing... Why? How?
Quality engineering instead of testing... Why? How?Quality engineering instead of testing... Why? How?
Quality engineering instead of testing... Why? How?
 
Quality Engineering and Testing with TMAP in DevOps IT delivery
Quality Engineering and Testing with TMAP in DevOps IT deliveryQuality Engineering and Testing with TMAP in DevOps IT delivery
Quality Engineering and Testing with TMAP in DevOps IT delivery
 
Quality engineering & testing in DevOps IT delivery with TMAP
Quality engineering & testing in DevOps IT delivery with TMAPQuality engineering & testing in DevOps IT delivery with TMAP
Quality engineering & testing in DevOps IT delivery with TMAP
 
Quality engineering in DevOps... Why? How? (TestBusters Day&Night))
Quality engineering in DevOps... Why? How? (TestBusters Day&Night))Quality engineering in DevOps... Why? How? (TestBusters Day&Night))
Quality engineering in DevOps... Why? How? (TestBusters Day&Night))
 
TMAP Quality Engineering workshop on A4Q congress by Rik Marselis
TMAP Quality Engineering workshop on A4Q congress by Rik Marselis TMAP Quality Engineering workshop on A4Q congress by Rik Marselis
TMAP Quality Engineering workshop on A4Q congress by Rik Marselis
 
Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Mars...
Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Mars...Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Mars...
Quality engineering in the digital age... Why? How? (ASQF Keynote by Rik Mars...
 
Quality for DevOps teams - Quality engineering in the DevOps culture
Quality for DevOps teams - Quality engineering in the DevOps cultureQuality for DevOps teams - Quality engineering in the DevOps culture
Quality for DevOps teams - Quality engineering in the DevOps culture
 
How to write a proposal to speak at a conference? Hints & tips
How to write a proposal to speak at a conference? Hints & tipsHow to write a proposal to speak at a conference? Hints & tips
How to write a proposal to speak at a conference? Hints & tips
 
Testen ISTQB, TMap, ISO25010 en meer. Gastcollege Windesheim Rik Marselis
Testen ISTQB, TMap, ISO25010 en meer. Gastcollege Windesheim Rik MarselisTesten ISTQB, TMap, ISO25010 en meer. Gastcollege Windesheim Rik Marselis
Testen ISTQB, TMap, ISO25010 en meer. Gastcollege Windesheim Rik Marselis
 
Over testen gesproken TestNet najaar 2019 openingskeynote Rik Marselis
Over testen gesproken TestNet najaar 2019 openingskeynote Rik MarselisOver testen gesproken TestNet najaar 2019 openingskeynote Rik Marselis
Over testen gesproken TestNet najaar 2019 openingskeynote Rik Marselis
 
Testing of artificial intelligence; AI quality engineering skils - an introdu...
Testing of artificial intelligence; AI quality engineering skils - an introdu...Testing of artificial intelligence; AI quality engineering skils - an introdu...
Testing of artificial intelligence; AI quality engineering skils - an introdu...
 
Webinar trends in testen 2017-12-13 door Rik Marselis
Webinar trends in testen 2017-12-13 door Rik MarselisWebinar trends in testen 2017-12-13 door Rik Marselis
Webinar trends in testen 2017-12-13 door Rik Marselis
 
Ketenregie in Agile / DevOps: Noodzaak? (oplossing: Gilde, SAFe, combi ...)
Ketenregie in Agile / DevOps: Noodzaak? (oplossing: Gilde, SAFe, combi ...)Ketenregie in Agile / DevOps: Noodzaak? (oplossing: Gilde, SAFe, combi ...)
Ketenregie in Agile / DevOps: Noodzaak? (oplossing: Gilde, SAFe, combi ...)
 
Webinar trends in testing 2017 03 08 (in dutch)
Webinar trends in testing 2017 03 08 (in dutch)Webinar trends in testing 2017 03 08 (in dutch)
Webinar trends in testing 2017 03 08 (in dutch)
 
Past het testvak nog in de nieuwe IT-wereld?
Past het testvak nog in de nieuwe IT-wereld?Past het testvak nog in de nieuwe IT-wereld?
Past het testvak nog in de nieuwe IT-wereld?
 

Recently uploaded

➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men 🔝malwa🔝 Escorts Ser...
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men  🔝malwa🔝   Escorts Ser...➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men  🔝malwa🔝   Escorts Ser...
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men 🔝malwa🔝 Escorts Ser...amitlee9823
 
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...amitlee9823
 
Call Girls Indiranagar Just Call 👗 9155563397 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 9155563397 👗 Top Class Call Girl Service B...Call Girls Indiranagar Just Call 👗 9155563397 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 9155563397 👗 Top Class Call Girl Service B...only4webmaster01
 
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteedamy56318795
 
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night StandCall Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Standamitlee9823
 
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men 🔝Dindigul🔝 Escor...
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men  🔝Dindigul🔝   Escor...➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men  🔝Dindigul🔝   Escor...
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men 🔝Dindigul🔝 Escor...amitlee9823
 
Discover Why Less is More in B2B Research
Discover Why Less is More in B2B ResearchDiscover Why Less is More in B2B Research
Discover Why Less is More in B2B Researchmichael115558
 
➥🔝 7737669865 🔝▻ Sambalpur Call-girls in Women Seeking Men 🔝Sambalpur🔝 Esc...
➥🔝 7737669865 🔝▻ Sambalpur Call-girls in Women Seeking Men  🔝Sambalpur🔝   Esc...➥🔝 7737669865 🔝▻ Sambalpur Call-girls in Women Seeking Men  🔝Sambalpur🔝   Esc...
➥🔝 7737669865 🔝▻ Sambalpur Call-girls in Women Seeking Men 🔝Sambalpur🔝 Esc...amitlee9823
 
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night StandCall Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Standamitlee9823
 
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...amitlee9823
 
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...amitlee9823
 
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...amitlee9823
 
Just Call Vip call girls roorkee Escorts ☎️9352988975 Two shot with one girl ...
Just Call Vip call girls roorkee Escorts ☎️9352988975 Two shot with one girl ...Just Call Vip call girls roorkee Escorts ☎️9352988975 Two shot with one girl ...
Just Call Vip call girls roorkee Escorts ☎️9352988975 Two shot with one girl ...gajnagarg
 
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -Pooja Nehwal
 
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service BangaloreCall Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangaloreamitlee9823
 
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...amitlee9823
 
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men 🔝Bangalore🔝 Esc...
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men  🔝Bangalore🔝   Esc...➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men  🔝Bangalore🔝   Esc...
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men 🔝Bangalore🔝 Esc...amitlee9823
 

Recently uploaded (20)

Anomaly detection and data imputation within time series
Anomaly detection and data imputation within time seriesAnomaly detection and data imputation within time series
Anomaly detection and data imputation within time series
 
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men 🔝malwa🔝 Escorts Ser...
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men  🔝malwa🔝   Escorts Ser...➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men  🔝malwa🔝   Escorts Ser...
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men 🔝malwa🔝 Escorts Ser...
 
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
 
Call Girls Indiranagar Just Call 👗 9155563397 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 9155563397 👗 Top Class Call Girl Service B...Call Girls Indiranagar Just Call 👗 9155563397 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 9155563397 👗 Top Class Call Girl Service B...
 
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
 
CHEAP Call Girls in Rabindra Nagar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Rabindra Nagar  (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Rabindra Nagar  (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Rabindra Nagar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
 
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night StandCall Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
 
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men 🔝Dindigul🔝 Escor...
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men  🔝Dindigul🔝   Escor...➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men  🔝Dindigul🔝   Escor...
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men 🔝Dindigul🔝 Escor...
 
Discover Why Less is More in B2B Research
Discover Why Less is More in B2B ResearchDiscover Why Less is More in B2B Research
Discover Why Less is More in B2B Research
 
➥🔝 7737669865 🔝▻ Sambalpur Call-girls in Women Seeking Men 🔝Sambalpur🔝 Esc...
➥🔝 7737669865 🔝▻ Sambalpur Call-girls in Women Seeking Men  🔝Sambalpur🔝   Esc...➥🔝 7737669865 🔝▻ Sambalpur Call-girls in Women Seeking Men  🔝Sambalpur🔝   Esc...
➥🔝 7737669865 🔝▻ Sambalpur Call-girls in Women Seeking Men 🔝Sambalpur🔝 Esc...
 
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night StandCall Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand
 
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
 
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
 
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
 
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
 
Just Call Vip call girls roorkee Escorts ☎️9352988975 Two shot with one girl ...
Just Call Vip call girls roorkee Escorts ☎️9352988975 Two shot with one girl ...Just Call Vip call girls roorkee Escorts ☎️9352988975 Two shot with one girl ...
Just Call Vip call girls roorkee Escorts ☎️9352988975 Two shot with one girl ...
 
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -
 
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service BangaloreCall Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
 
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
 
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men 🔝Bangalore🔝 Esc...
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men  🔝Bangalore🔝   Esc...➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men  🔝Bangalore🔝   Esc...
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men 🔝Bangalore🔝 Esc...
 

Testing in the digital age; international booklaunch at German Testing Day

  • 1. Testing… in the digital age AI makes the difference Booklaunch German Testing Days, Frankfurt, 8 June 2018 Rik Marselis
  • 2. | Who has a robot? © Sogeti 2018 2 Who has a robot? In 10 years all of you will !!
  • 3. 1980 Rik Marselis@rikmarselis 2007 2012 20122008 Workgroup -member 20142009 Management consultant Digital assurance & testing
  • 4. | The future of testing: quality forecasting © Sogeti 2018 4
  • 5. | Would you like to work with a robot? © Sogeti 2018 5
  • 6. | Cobotics: collaborating with robots © Sogeti 2018 6
  • 7. | Exoskeleton: robot gives extra muscle-power © Sogeti 2018 7 Ruben de Sain crowdfunded his exoskeleton to get out of his wheelchair.
  • 8. | What are your fears about intelligent machines? © Sogeti 2018 8
  • 9. | The Frankenstein factor: fear of AI © Sogeti 2018 9Source: SogetiLabs “The Frankenstein Factor”
  • 10. |© Sogeti 2018 10 Super intelligence & super stupidity Foto uit de film “Ex Machina” 2015
  • 11. | Two more general worries Loss of culture and defeating nature © Sogeti 2018 11
  • 12. | Let’s cope with these fears in a scientific way… © Sogeti 2018 12
  • 13. | Six angles of quality © Sogeti 2018 13 Existing knowledge New knowledge
  • 14. | Mechanical, Electrical, Information processing © Sogeti 2018 14 No. 2001 is seen on the rollers at the Vitry test plant Gresley Society Collection, 1934 Testing an electric stove, 1970’s © handyman The 7 generic testing activities of TMap® 1995/2006 Existing knowledge But still very relevant, also for testing intelligent machines.
  • 15. | Testing intelligent machines is different… © Sogeti 2018 15 Traditional testing compares expected and actual outcome. For intelligent machines predicting outcome is difficult; todays correct result may differ from tomorrows correct result, so: - Define tolerances - Control input - Test the machine in its intended environment - Use new quality characteristics
  • 16. | ISO25010 – Product Quality characteristics © Sogeti 2018 16
  • 17. | ISO25010 extended quality characteristics © Sogeti 2018 17
  • 18. | The new quality characteristics © Sogeti 2018 18
  • 19. | Machine learning – ability to learn © Sogeti 2017 19 <<< vervangen door filmclip >>> This is what machine learning looked like ☺ Scene from the film “Short circuit” (1986)
  • 20. | Ability to learn and to unlearn Machine learning • Continued learning • Frozen Examination • One-time • Continuous Un-learning © Sogeti 2018 20 Video: Unlearning and relearning to ride a bicycle
  • 21. | How to unlearn AI’s ? Rule-based AI: • Change the programmed rules Reinformement learning AI: • Change goal(s) • Control input • Feed new learning data © Sogeti 2018 21 U N
  • 22. |Morality: Ethics & Privacy Scene from the film “Robot and Frank” 2012
  • 23. | Scene from documentary “I am Alice” 2015 Morality: Ethics & Privacy
  • 24. | Personality - mood © Sogeti 2018 24 Personality: mood & humor
  • 25. | now let’s look at testing WITH them Well, this was all about testing OF intelligent machines, 25© Sogeti 2018
  • 26. | Robot automatically performs regression tests © Sogeti 26 Video that shows how Sogeti does regression testing of a cockpit at Airbus
  • 27. | Robot automatically performs regression tests © Sogeti 2017 27 Low-cost (€300,-) robot-arm tests mobile devices. Connected to test ma- nagement tool. Test cases check if the quality goals are met and risks are covered.
  • 28. | Use a digital twin to virtually test a product… © Sogeti 2018 28
  • 29. | More testing in less time © Sogeti 2018 29
  • 30. | The future of testing: quality forecasting © Sogeti 2018 30
  • 31. | Forecasting using multiple models © Sogeti 2018 31 The tester will interpret the outcome of multiple models and create the advice for the stakeholders
  • 32. | Cognitive QA – for example: smart dashboards © Sogeti 2018 32
  • 34. Thank you. This book was introduced on 1 June 2018 Humayun Shaukat Tom van de Ven Rik Marselis © Sogeti 2018