Ingo Philipp Distinguished Evangelist – Tricentis Ingo Philipp champions the methodologies and technologies at the core of the company’s continuous testing solution. In his previous position as a senior product manager, he orchestrated product development and product marketing.

1. Artificial Intelligence
The Best & Worst Uses of AI in Software Testing
Ingo Philipp

2. Nothing has moved as fast as artificial
intelligence is moving right now in the
enterprise
© Paul Daugherty
The New Paradigm
André Mendes
The Next Digital Frontier
McKinsey Institute
The Next Disruptive Force
Bloomberg
The New Electricity
Andrew Ng
The New Black
MIT Technology Review

3. We are just a baby step
away from eliminating the
need for human thinking
in software testing
Hasty Conclusion
The New Paradigm
André Mendes
The Next Digital Frontier
McKinsey Institute
The Next Disruptive Force
Bloomberg
The New Electricity
Andrew Ng
The New Black
MIT Technology Review

4. The New Paradigm
André Mendes
The Next Digital Frontier
McKinsey Institute
The Next Disruptive Force
Bloomberg
The New Electricity
Andrew Ng
The New Black
MIT Technology Review

5. I won't sell you
artificial intelligence in
software testing in the way miracle
weight loss programs or anti-aging face
creams (with micro beads!)
are being sold
© James Bach

6. I won't sell you
artificial intelligence in
software testing in the way miracle
weight loss programs or anti-aging face
creams (with micro beads!)
are being sold
© James Bach © Robert Sternberg
Viewed narrowly, there seem to be
almost as many definitions of intelligence
as the number of experts asked to define it
Artificial
Intelligence

7. © Robert Sternberg
Viewed narrowly, there seem to be
almost as many definitions of intelligence
as the number of experts asked to define it
Artificial
Intelligence

8. Emotional
Thinking
Critical
Thinking
Problem
Solving
Logical
Thinking
Communication
Perception
Modelling
Planning
Abstraction
Learning
Imagination
Creativity
Memory
Experience
Understanding
Knowledge
Judgement
Analysis
Environment
Manipulation
Environment
Adaptation
Strategic
Goal Setting
Instinctive
Judgement

9. Emotional
Thinking
Critical
Thinking
Problem
Solving
Logical
Thinking
Communication
Perception
Modelling
Planning
Abstraction
Learning
Imagination
Creativity
Memory
Experience
Understanding
Knowledge
Judgement
Analysis
Environment
Manipulation
Environment
Adaptation
Strategic
Goal Setting
Instinctive
Judgement
• The Turing Test
• The Reverse Turing Test
• The Visual Turing Test
• The Lovelace Test
• The Lovelace 2.0 Test
• The Winograd Schema Challenge
• The Ex Machina Test
• The Tokyo Test
• The AIQ Test
• The DeepMind Test
• The Marcus Test
• The IKEA Challenge
• The NCC Test
Intelligence is what is measured by
intelligence tests
Edwin Boring

10. Emotional
Thinking
Critical
Thinking
Problem
Solving
Logical
Thinking
Communication
Perception
Modelling
Planning
Abstraction
Learning
Imagination
Creativity
Memory
Experience
Understanding
Knowledge
Judgement
Analysis
Environment
Manipulation
Environment
Adaptation
Strategic
Goal Setting
Instinctive
Judgement
Intelligence measures an agent’s ability
to achieve goals in a wide range of
environments
Shane Legg
Artificial intelligence is anything
machines can't yet do
Chris Bishop
Intelligence is what is measured by
intelligence tests
Edwin Boring

11. Emotional
Thinking
Critical
Thinking
Problem
Solving
Logical
Thinking
Communication
Perception
Modelling
Planning
Abstraction
Learning
Imagination
Creativity
Memory
Experience
Understanding
Knowledge
Judgement
Analysis
Environment
Manipulation
Environment
Adaptation
Strategic
Goal Setting
Instinctive
Judgement
General• Artificial Intelligence •
A machine with the ability to apply
intelligence to any problem

12. General• Artificial Intelligence •
A machine with the ability to apply
intelligence to any problem
Narrow• Artificial Intelligence •
A machine with the ability to apply
intelligence to a specific problem

13. Narrow• Artificial Intelligence •
A machine with the ability to apply
intelligence to a specific problem
Software Testing

14. Narrow• Artificial Intelligence •
A machine with the ability to apply
intelligence to a specific problem
© Dominik Weissböck

15. Narrow• Artificial Intelligence •
A machine with the ability to apply
intelligence to a specific problem
Our product
is a solution to someone's
problem.
If the problem isn't
solved, the product
doesn't work.
© Dominik Weissböck

16. Our highest priority is
satisfying our customers
…except when it's hard
…or unprofitable
…or when we're busy
© dilbert.com
Narrow• Artificial Intelligence •
A machine with the ability to apply
intelligence to a specific problem

17. A machine with the ability to apply
intelligence to a specific problem
Narrow• Artificial Intelligence •How can AI assist the human
tester in specific testing use
cases?
5400+ Responses; 720+ Customers

18. A machine with the ability to apply
intelligence to a specific problem
Narrow• Artificial Intelligence •
Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best

19. AI uses production data to
prioritize features, to define
what to test, what to automate,
and even what to build
Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best
Test Strategy Optimization

20. AI interprets requirements
(e.g. user stories) and generates
the minimal number of test cases from the
requirements to
maximize risk coverage
Automated Test Design
Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best

21. AI eliminates and
prevents redundancies in
test case portfolios to achieve
the same results in terms
of business risk coverage
but with less effort
Redundancy Prevention
Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best

22. AI interacts with the
application, builds a model of it, discovers
relevant functionality, reveals defects, and
extracts test cases to reduce test effort
Automated Exploratory Testing
Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best

23. AI reduces the effort
required for results analysis by indicating
whether a failed test
case actually detected a defect in the
application, or just broke due
to technical issues with
the test case itself
False-Positive Detection
Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best

24. AI proposes potential
reasons that caused a test
case to fail to help development reduce the
time it takes to
analyze the root cause
of a defect
Automated Defect Diagnosi
Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best

25. AI monitors and
interprets user emotions
during exploratory testing and
links its findings back to the
related application component
to increase the precision
of UX analysis
User Experience Analysi
Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best

26. AI tracks flaky test cases,
unused test cases, test cases
not linked to requirements,
untested requirements, etc.
to indicate weak spots
in test portfolios
Portfolio Inspection
Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best

27. Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best
© David Colwell ‹› Director Innovation Lab

28. If you can define concrete
steps to get to a solution, don't
use machine learning
© David Colwell ‹› Director Innovation Lab
Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best

29. Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best
If you can define concrete
steps to get to a solution, don't
use machine learning
© David Colwell ‹› Director Innovation Lab

30. Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best
If you can define concrete
steps to get to a solution, don't
use machine learning
© David Colwell ‹› Director Innovation Lab

31. If you can define concrete
steps to get to a solution, don't
use machine learning
© David Colwell ‹› Director Innovation Lab
How to make test automation
more robust against changes?
How to teach a machine
doing exploratory testing?
Value
Effort » Uncertainty » Complexity » Risks » Costs
Best Uses
Worst Uses

32. If you can define concrete
steps to get to a solution, don't
use machine learning
© David Colwell ‹› Director Innovation Lab
Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best

33. Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best
Explicit
Design
Explicit programming; fixed knowledge; no
learning capabilities; fake and rigid intelligence

34. Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best
We do have 16 hours.
What's the best possible
we can achieve?
?

35. We do have 16 hours.
What's the best possible
we can achieve?
?
Coverage
Test Cases
Resources
Defects
Costs
Time
16 hours

36. We do have 16 hours.
What's the best possible
we can achieve?
Defects
Resources
Time
Costs
Coverage
Test Cases
?16 hours
#1576
67%
$3000
4/10 testers; 8/10 machines
6 critical defects (12%)

37. We do have 6 hours and
5 testers. What's the best
possible we can achieve?
Defects
Resources
Time
Costs
Coverage
Test Cases
?6 hours
#180
35%
$2000
5 testers
8 critical defects (46%)

38. We want at least
60% risk coverage.
What does it cost?
Defects
Resources
Time
Costs
Coverage
Test Cases
?3 hours
#3476
60%
$800
7/12 machines
2 critical defects (36%)

39. We want at least
60% risk coverage.
What does it cost?
?
The science of getting computers to
act by being explicitly programmed
Explicit
Design

40. The science of getting computers to
act by being explicitly programmed
Explicit
Design
Machine
Learning
The science of getting computers to act
without being explicitly programmed
© Andrew Ng

41. Machine
Learning
The science of getting computers to act
without being explicitly programmed
© Andrew Ng
The science of getting people to test
without being explicitly told to do so
Human
Learning
© Ingo Philipp

42. Machine
Learning
The science of getting computers to act
without being explicitly programmed
© Andrew Ng
Approach Research Use
Test Strategy
Optimization
Rule-Based
System
No Worst
Automated
Test Design
Learning
System
No Worst
Redundancy
Prevention
Rule-Based
System
Yes Best
Risk Coverage
Optimization
Rule-Based
System
Yes Best
Automated
Exploratory Testing
Learning
System
Yes Worst
Resilient
Automation
Learning
System
Yes Best
False-Positive
Detection
Rule-Based
System
Yes Best
Automated
Defect Diagnosis
Rule-Based
System
No Worst
User Experience
Analysis
Learning
System
Yes Worst
Portfolio
Inspection
Rule-Based
System
Yes Best

43. Machine
Learning
The science of getting computers to act
without being explicitly programmed
© Andrew Ng
Low-Code UI
Automation
« Model-Based Test Automation »
No-Code UI
Automation
« Model-Free Test Automation »
P
r
e
s
e
n
t
F
u
t
u
r
e

44. Machine
Learning
The science of getting computers to act
without being explicitly programmed
© Andrew Ng
Application
« User's Perspective »
Next >>
Make Audi
Engine Performance [kW] 200
Number of Seats 5
Fuel Petrol
Year of Construction 2016
List Price [$] 35.000
License Plate Number B-CD 123
Annual Mileage [mi] 10.000
Usage Private

45. Next >>
Make Audi
Engine Performance [kW] 200
Number of Seats 5
Fuel Petrol
Year of Construction 2016
List Price [$] 35.000
License Plate Number B-CD 123
Annual Mileage [mi] 10.000
Usage Private
Application
« User's Perspective »
Modules
« Machine's Perspective »
-
-
1;2;3;4;5;6;7;8;9
Petrol;Diesel;Gas;Other
-
-
{CLICK};
{RIGHTCLICK}
Private;Commercial
2016;2015;2014;2013
-
Insurant Data
Quote Details
Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
{CLIC
K}
01/03/2016;01/04/2016
3Mio;7Mio;10Mio;15Mio
Yearly;Quarterly;Monthly
Product Data
Next
Start Date
Insurance Sum [$]
Payment Option

46. Test Case
« Tester's Perspective »
Modules
« Machine's Perspective »
-
-
1;2;3;4;5;6;7;8;9
Petrol;Diesel;Gas;Other
-
-
{CLICK};
{RIGHTCLICK}
Private;Commercial
2016;2015;2014;2013
-
Insurant Data
Quote Details
Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
{CLIC
K}
01/03/2016;01/04/2016
3Mio;7Mio;10Mio;15Mio
Yearly;Quarterly;Monthly
Product Data
Next
Start Date
Insurance Sum [$]
Payment Option
Audi
200
Petrol
35.000
10.000
{CLIC
K}
Private
{Year
}
B-CD 123
5
{CLIC
K}
01/03/2016
7Mio
Yearly
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Enter Insurant Data
Next
Start Date
Insurance Sum [$]
Payment Option
Enter Product Data
Verify Quote Details
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Create Vehicle Insurance

47. Business
« Test & Test Data Logic »
Technical
« Automation Logic »
-
-
1;2;3;4;5;6;7;8;9
Petrol;Diesel;Gas;Other
-
-
{CLICK};
{RIGHTCLICK}
Private;Commercial
2016;2015;2014;2013
-
Insurant Data
Quote Details
Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
{CLIC
K}
01/03/2016;01/04/2016
3Mio;7Mio;10Mio;15Mio
Yearly;Quarterly;Monthly
Product Data
Next
Start Date
Insurance Sum [$]
Payment Option
Audi
200
Petrol
35.000
10.000
{CLIC
K}
Private
{Year
}
B-CD 123
5
{CLIC
K}
01/03/2016
7Mio
Yearly
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Enter Insurant Data
Next
Start Date
Insurance Sum [$]
Payment Option
Enter Product Data
Verify Quote Details
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Create Vehicle Insurance

48. Insurant Data
Quote Details
Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Product Data
Next
Start Date
Insurance Sum [$]
Payment Option
Audi
200
Petrol
35.000
10.000
{CLIC
K}
Private
{Year
}
B-CD 123
5
{CLIC
K}
01/03/2016
7Mio
Yearly
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Enter Insurant Data
Next
Start Date
Insurance Sum [$]
Payment Option
Enter Product Data
Verify Quote Details
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Create Vehicle Insurance
Identify By
Properties
Identify By
Image
Identify By
Anchor
Identify By
Index
Business
« Test & Test Data Logic »
Technical
« Automation Logic »

49. Insurant Data
Quote Details
Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Product Data
Next
Start Date
Insurance Sum [$]
Payment Option
Audi
200
Petrol
35.000
10.000
{CLIC
K}
Private
{Year
}
B-CD 123
5
{CLIC
K}
01/03/2016
7Mio
Yearly
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Enter Insurant Data
Next
Start Date
Insurance Sum [$]
Payment Option
Enter Product Data
Verify Quote Details
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Create Vehicle Insurance
Identify By
Properties
Identify By
Image
Identify By
Anchor
Identify By
Index
Business
« Test & Test Data Logic »
Technical
« Automation Logic »
Technical Changes

50. Insurant Data
Quote Details
Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Product Data
Next
Start Date
Insurance Sum [$]
Payment Option
Audi
200
Petrol
35.000
10.000
{CLIC
K}
Private
{Year
}
B-CD 123
5
{CLIC
K}
01/03/2016
7Mio
Yearly
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Enter Insurant Data
Next
Start Date
Insurance Sum [$]
Payment Option
Enter Product Data
Verify Quote Details
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Create Vehicle Insurance
Identify By
Properties
Identify By
Image
Identify By
Anchor
Identify By
Index
Business
« Human Territory »
Technical
« Human Territory »
Human Territory Human Territory

51. Insurant Data
Quote Details
Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Product Data
Next
Start Date
Insurance Sum [$]
Payment Option
Audi
200
Petrol
35.000
10.000
{CLIC
K}
Private
{Year
}
B-CD 123
5
{CLIC
K}
01/03/2016
7Mio
Yearly
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Enter Insurant Data
Next
Start Date
Insurance Sum [$]
Payment Option
Enter Product Data
Verify Quote Details
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Create Vehicle Insurance
Identify By
Properties
Identify By
Image
Identify By
Anchor
Identify By
Index
Business
« Human Territory »
Technical
« Machine Territory »
Human Territory Machine Territory

52. Business
« Human Territory »
Audi
200
Petrol
35.000
10.000
{CLIC
K}
Private
{Year
}
B-CD 123
5
{CLIC
K}
01/03/2016
7Mio
Yearly
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Enter Insurant Data
Next
Start Date
Insurance Sum [$]
Payment Option
Enter Product Data
Verify Quote Details
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Create Vehicle Insurance
Human Territory
Visual
Recognition
Technical
Recognition
Deep Convolutional
Neural Network
Deep Regression
Neural Network
Identification Execution
From Static to Dynamic Identificatio

53. Visual
Recognition
Technical
Recognition
Deep Convolutional
Neural Network
Deep Regression
Neural Network
Identification Execution
From Static to Dynamic Identificatio

54. 1
1
1
1
1
1
1
1
4
4
4
4
4
4
4
4
5
6 6 6 6 6
6 6
3
2
2
3
3
2
2
2
5
6 6
6 1
6 6 6 6
6 6
Visual
Recognition
Technical
Recognition
Deep Convolutional
Neural Network
Deep Regression
Neural Network
Identification Execution
From Static to Dynamic Identificatio

55. <div class="field">
<label class="main">Make</label>
<select id="make" name="Make">
<option value="Audi">Audi</option>
<option value="BMW">BMW</option>
<option value="Ford">Ford</option>
...
</select>
<span class="error"></span>
</div>
<div class="field">
<label class="main">List Price [$]</label>
<input id="price" name="price" type="text">
<span class="error"></span>
</div>
Visual
Recognition
Technical
Recognition
Deep Convolutional
Neural Network
Deep Regression
Neural Network
Identification Execution
From Static to Dynamic Identificatio

56. Test Case. Create Automobile Quote
Step. Enter Vehicle Data
Enter Audi into the edit-box Make
Enter 200 into the edit-box Engine Performance
Enter 01/03/2015 into the edit-box Year of Construction
Select 5 in the combo-box Number of Seats
Select Petrol in the combo-box Fuel
Enter $35000 in the edit-box List Price
Enter W975633 in the edit-box License Plate Number
Select Private in the combo-box Usage
Enter 10.000 in the edit-box Annual Mileage
Click the button Next
Step. Enter Insurant Data
Step. Enter Product Data
Select 09/10/2018 in the date-picker Start Date
Enter 7000000 in the combo-box Insurance Sum
Select Yearly in the combo-box Payment Option
Click the button Next
Step. Verify Quote Details
In the table Price List verify $792 in the column
Gross Premium and in the row Premium Tax
Technical Definition
Create Truck Quote
Automated Test Cases
Create Motorcycle Quote
Create Trailer Quote
Automated Test Cases
Create Automobile Quote
Semantic Definition
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
27
28
29
30
31
32
33
34
35
Semantic Test Case Definition
Visual
Recognition
Technical
Recognition
Deep Convolutional
Neural Network
Deep Regression
Neural Network
Identification Execution
From Static to Dynamic Identificatio

57. Test Case. Create Automobile Quote
Step. Enter Vehicle Data
Enter Audi into the edit-box Make
Enter 200 into the edit-box Engine Performance
Enter 01/03/2015 into the edit-box Year of Construction
Select 5 in the combo-box Number of Seats
Select Petrol in the combo-box Fuel
Enter $35000 in the edit-box List Price
Enter W975633 in the edit-box License Plate Number
Select Private in the combo-box Usage
Enter 10.000 in the edit-box Annual Mileage
Click the button Next
Step. Enter Insurant Data
Step. Enter Product Data
Select 09/10/2018 in the date-picker Start Date
Enter 7000000 in the combo-box Insurance Sum
Select Yearly in the combo-box Payment Option
Click the button Next
Step. Verify Quote Details
In the table Price List verify $792 in the column
Gross Premium and in the row Premium Tax
Technical Definition
Create Truck Quote
Automated Test Cases
Create Motorcycle Quote
Create Trailer Quote
Automated Test Cases
Create Automobile Quote
Semantic Definition
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
27
28
29
30
31
32
33
34
35
Semantic Test Case Definition

58. The number one testing tool is not the
computer, it is still the human brain
Jerry Weinberg
The most important intelligence isn't artificial

59. Don't expect AI to solve all your problems soon.
Do something about natural stupidity in testing now
Ingo Philipp

60. Questions
The show is over. It’s your turn.
?