@roesslerj1
Wie man KI ins Testen bringt
Dr. Jeremias Rößler
https://retest.de
@roesslerj2
@roesslerj3
@roesslerj4
1 - 2 = ?
• -1 is mathematically correct
• 0 if I don’t want to payback gift certificates to costumers
• 23 if I’m working with hours
• 59 if I’m working with minutes or seconds
• Something between 27 and 30 if I’m working with days
• 11 if I am working with months
• exception if I am working with degrees Fahrenheit
@roesslerj5
@roesslerj6
@roesslerj7
Oracle Problem
@roesslerj8
How do you bring AIinto testing?
@roesslerj9
How do YOU do Testing?
@roesslerj10
Testing
Actual wrong
Bug
Comparison Expected vs Actual
Test Execution
Done
No Difference
@roesslerj11
Actual wrong
Bug
Comparison Expected vs Actual
Update Expected
Expected wrong
Test Execution
Done
No Difference Difference
Evaluate Difference
Testing
@roesslerj12
When is a bug a bug?
When it’s not a feature!
@roesslerj13
Is it a bug?
.
..
text.c
other.files
for (. . .) {
if (name[0] == '.') continue;
. . .
}
ls
@roesslerj14
Is it a bug?
@roesslerj15
Is it a bug?
@roesslerj16
What is a bug?
Without specification, there are no bugs
— only surprises.
Brian Kernighan
“
@roesslerj17
Why do we test?
After
implementation?
1 + 1 = 3
2 + 2 = 4
@roesslerj18
@roesslerj20
@roesslerj21
Why do we test?
After
implementation?
1 + 1 = 3
2 + 2 = 4
After
change?
1 + 1 = 3
1 + 1 = 5
@roesslerj
automated
regression test
≠
test
22
@roesslerj
automated
regression test
=
version control
23
@roesslerj24
Why do we test?
After
implementation?
1 + 1 = 3
2 + 2 = 4
After
change?
1 + 1 = 3
1 + 1 = 5
1 + 1 = 2
@roesslerj25
Actual wrong
Bug
Comparison Expected vs Actual
Update Expected
Expected wrong
Test Execution
Done
No Difference Difference
Evaluate Difference
Testing
@roesslerj26
Actual wrong
Bug
Comparison Expected vs Actual
Update Expected
Expected wrong
Test Execution
Done
No Difference Difference
Evaluate Difference
Manual and Automated Testing
effort
effort
@roesslerj27
@roesslerj28
@roesslerj29
@roesslerj30
@roesslerj31
@roesslerj32
@roesslerj33
@roesslerj34
@roesslerj35
@roesslerj36
@roesslerj37
@roesslerj38
@roesslerj39
@roesslerj40
Multilocators
@roesslerj41
@roesslerj42
https://github.com/retest/recheck-web
@roesslerj43
How do you bring AIinto testing?
@roesslerj44
Oracle Problem
@roesslerj45
@roesslerj46
@roesslerj47
Infinite-Monkey-Theorem:
A monkey hitting keys at random on a typewriter
for a long enough time
will type the complete works of William Shakespeare.
@roesslerj48
We replace the typewriter with a computer…
@roesslerj49
public static void main(String... args) throws Exception {
Robot robot = new Robot();
while (true) {
robot.mouseMove(random.nextInt(maxX), random.nextInt(maxY));
robot.mousePress(InputEvent.BUTTON1_DOWN_MASK);
robot.mouseRelease(InputEvent.BUTTON1_DOWN_MASK);
robot.delay(200);
for (char inputChar : randomString().toCharArray()) {
robot.keyPress((int) inputChar);
robot.keyRelease((int) inputChar);
robot.delay(10);
}
robot.keyPress(KeyEvent.VK_ENTER);
robot.keyRelease(KeyEvent.VK_ENTER);
}
}
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
@roesslerj50
DEMO
@roesslerj51
@roesslerj52
- gremlins.js
- NetFlix Chaos Monkey
- UI/Application Exerciser Monkey
Open Source Products
@roesslerj53
@roesslerj54
@roesslerj55
public static void main(String... args) throws Exception {
WebDriver driver = new FirefoxDriver();
driver.get("http://www.retest.de");
while (true) {
List<WebElement> links = driver.findElements(By.tagName("a"));
links.get(random.nextInt(links.size())).click();
Thread.sleep(500);
List<WebElement> fields =
driver.findElements(By.xpath("//input[@type='text']"));
WebElement field = fields.get(random.nextInt(fields.size()));
field.sendKeys(randomString());
Thread.sleep(500);
}
}
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
@roesslerj56
Intelligent Monkey
DEMO
@roesslerj57
@roesslerj58
@roesslerj59
@roesslerj60
@roesslerj61
Problem Space
Problem
Space
@roesslerj62
@roesslerj63
@roesslerj64
@roesslerj65
@roesslerj66
train
AI
Artificial
Neural Net
Genetic
Algorithm
AI
@roesslerj67
@roesslerj68
test
1.0
approve
1.1
diff
train
user
automate
@roesslerj69
@Test
public void test_login() throws Exception {
final WebDriver driver = new FirefoxDriver();
driver.get( "http://www.retest.de" );
re.check( driver, "launch" );
driver.findElement( By.id( "username" ) ).sendKeys( username );
driver.findElement( By.id( "pwd" ) ).sendKeys( password );
driver.findElement( By.className( "button" ) ).submit();
re.check( driver, "login");
}
@roesslerj
@roesslerjX
https://github.com/retest/recheck-web
@roesslerjX
train
AI
Artificial
Neural Net
Genetic
Algorithm
AI
70
Summary
Difference Testing
little maintenance
no wasteful testing
yet more complete
Monkey-Testing:
cheaper
faster
better
multiplies manual tests
@roesslerj
roessler@retest.de
@roesslerj
Questions?
https://www.facebook.com/retest.de
http://retest.rocks

Wie man KI ins Testing bringt