Assertions sind heutzutage der Standard-Prüfmechanismus im Unit Testing. Aber wenn man sie auf Schnittstellen anwendet – insbesondere GUIs –, dann werden diese dysfunktional. Typische Probleme sind hoher Erstellungs- und Pflegeaufwand, brüchige Tests, viele "false Positives". Es gibt jedoch Alternativen: Ob ApprovalsTest, Jest oder recheck – die Anzahl der Tools, die nach dem Ansatz des Golden Master Testing/Characterization Testing/Approval Testing funktionieren, wird immer größer. Dieser Ansatz verspricht robustere Tests bei weniger Erstellungs- und Pflegeaufwand. Wieso das so ist, was das genau bedeutet, welche verschiedenen Tools zur Auswahl stehen (Open Source), und wo die Stärken und Schwächen dieses Ansatzes grundsätzlich liegen, wird in diesem Vortrag genau erläutert. Verpassen Sie nicht den nächsten Trend in der Testautomatisierung.

1. @roesslerj1
Testautomatisierung
ohne Assertions
Dr. Jeremias Rößler
https://retest.de

2. @roesslerj2
What is an assertion?

3. @roesslerj3

4. @roesslerj4

5. @roesslerj5

6. @roesslerj6

7. @roesslerj7

8. @roesslerj8

9. @roesslerj9
Test Pyramid
UI
Service
Unit
Maintenance
Fragility
Cost

10. @roesslerj10

11. @roesslerj11
Binding test to test-data!

12. @roesslerj12
What does an assertion do?

13. @roesslerj13
Assert correct behavior

14. @roesslerj14
1 - 2 = ?
• -1 is mathematically correct
• 0 if I don’t want to payback gift certificates to costumers
• 23 if I’m working with hours
• 59 if I’m working with minutes or seconds
• Something between 27 and 30 if I’m working with days
• 11 if I am working with months
• exception if I am working with degrees Calvin
• …

15. @roesslerj15
Requirements
Waterfall
Design
Development
Testing
Deployment
no more change

16. @roesslerj16
Agile
Requirements Design
Development
Testing
Deployment
change
Requirements Design
Development
Testing
Deployment

17. @roesslerj17
“correct” changes

18. @roesslerj18
When is a bug a bug?
When it’s not a feature!

19. @roesslerj19
Is it a bug?
.
..
text.c
other.files
for (. . .) {
if (name[0] == '.') continue;
. . .
}
ls

20. @roesslerj20
Is it a bug?

21. @roesslerj21
Is it a bug?

22. @roesslerj22
What is a bug?
Without specification, there are no bugs
— only surprises.
Brian Kernighan
“

23. @roesslerj23
Assert correct behavior
Detect change

24. @roesslerj24
Assertions detect change

25. @roesslerj25
Why do we test?
After
implementation?
1 + 1 = 3
2 + 2 = 4

26. @roesslerj26

27. @roesslerj28

28. @roesslerj29
Why do we test?
After
implementation?
1 + 1 = 3
2 + 2 = 4
After
change?
1 + 1 = 3
1 + 1 = 5

29. @roesslerj
automated
regression test
≠
test
30

30. @roesslerj
automated
regression test
=
version control
31

31. @roesslerj32
Why do we test?
After
implementation?
1 + 1 = 3
2 + 2 = 4
After
change?
1 + 1 = 3
1 + 1 = 5
1 + 1 = 2

32. @roesslerj33

33. @roesslerj34

34. @roesslerj35

35. @roesslerj36
Die Lücke schließen:
automatische Tests

36. @roesslerj37
Drawbacks of assertions

37. @roesslerj38
Testing
Actual wrong
Bug
Comparison Expected vs Actual
Test Execution
Done
No Difference

38. @roesslerj39
Actual wrong
Bug
Comparison Expected vs Actual
Update Expected
Expected wrong
Test Execution
Done
No Difference Difference
Evaluate Difference
Testing

39. @roesslerj40
Actual wrong
Bug
Comparison Expected vs Actual
Update Expected
Expected wrong
Test Execution
Done
No Difference Difference
Evaluate Difference
Testing
effort
effort

40. @roesslerj41

41. @roesslerj42

42. @roesslerj43

43. @roesslerj44

44. @roesslerj45
passes

45. @roesslerj46
Golden Master to the rescue

46. @roesslerj47

47. @roesslerj48

48. @roesslerj49

49. @roesslerj50

50. @roesslerj51

51. @roesslerj52

52. @roesslerj53
Even better…

53. @roesslerj54

54. @roesslerj55
Multilocators

55. @roesslerj56

56. @roesslerj57
Tool support

57. @roesslerj58

58. @roesslerj59

59. @roesslerj60

60. @roesslerj61

61. @roesslerj62

62. @roesslerj63

63. @roesslerj64

64. @roesslerj65
@Test
public void test_login() throws Exception {
final WebDriver driver = new FirefoxDriver();
driver.get( "http://www.retest.de" );
recheck.check( driver );
driver.findElement( By.id( "username" ) ).sendKeys( username );
driver.findElement( By.id( "pwd" ) ).sendKeys( password );
driver.findElement( By.className( "button" ) ).submit();
recheck.check( driver );
}

65. @roesslerj66
@Test
public void test_login() throws Exception {
final WebDriver driver = new RecheckDriver(new FirefoxDriver());
driver.get( "http://www.retest.de" );
recheck.check( driver );
driver.findElement( By.id( "username" ) ).sendKeys( username );
driver.findElement( By.id( "pwd" ) ).sendKeys( password );
driver.findElement( By.className( "button" ) ).submit();
recheck.check( driver );
}

66. @roesslerj67
RecheckDriver
findElement(by.id())
FirefoxDriver
findElement(by.bestMatch())
GoldenMaster

67. @roesslerj68
@Test
public void test_login() throws Exception {
final WebDriver driver = new RecheckDriver(new FirefoxDriver());
driver.get( "http://www.retest.de" );
recheck.check( driver );
driver.findElement( By.retestId( "username" ) ).sendKeys( username );
driver.findElement( By.retestId( "pwd" ) ).sendKeys( password );
driver.findElement( By.retestId( "button" ) ).submit();
recheck.check( driver );
}

68. @roesslerj69
https://github.com/retest/recheck-web

69. @roesslerj70
DEMO

70. @roesslerj
@roesslerjX
@roesslerjX@roesslerjX
Actual wrong
Bug
Comparison Expected vs Actual
Update Expected
Expected wrong
Test Execution
Done
No Difference Difference
Evaluate Difference
Testing
effort
effort
@roesslerjX
71
Questions?

71. @roesslerj
roessler@retest.de
@roesslerj
Thank you!
https://www.facebook.com/retest.de
http://retest.rocks