GUI test automation is broken — because regression testing is not testing, but version control of the behavior of the software. Gradually, an insight is gaining ground in the testing world: Test automation without assertions works better! Whether Approvals Test, Jest (Facebook, Open Source) or recheck (retest, Open Source) - the number of tools that work according to the approach of Golden Master Testing / Characterization Testing / Approval Testing is constantly increasing. This approach promises more robust tests with less effort for creation and maintenance, while all the while testing much more completely. What this means, which different tools are available (mostly Open Source), and where the strengths and weaknesses of this approach lie, will be explained in this presentation. Don’t miss the next trend in test automation…

1. @roesslerj1
Dr. Jeremias Rößler
https://retest.de
Test Automation
without Assertions

2. @roesslerj2
Teaser-DEMO

3. @roesslerj3
expect(umbrellaOpens).toBe(true)
tests: 1 passed, 1 total
**all tests passed**

4. @roesslerj5
We do one thing different.

5. @roesslerj6
You don’t define the outcome,
you approve it.
First you ensure that everything is as wanted.
Then we make sure it doesn’t change.

6. @roesslerj7
automated tests
≠
tests

7. @roesslerj8
automated tests
=
version control

8. @roesslerj9
Where has the
Code changed?

9. @roesslerj10

10. @roesslerj11
Where has the
_Software_ changed?

11. @roesslerj12
Closing the gap:
automated tests

12. @roesslerj13
automated tests
=
version control

13. @roesslerj14

14. @roesslerj15

15. @roesslerj16
passes

16. @roesslerj17

17. @roesslerj18

18. @roesslerj19

19. @roesslerj20

20. @roesslerj21

21. @roesslerj22
automated tests
=
version control

22. @roesslerj23
*.log
*.class
target/.*
.gitignore

23. @roesslerj24

24. @roesslerj25

25. @roesslerj26

26. @roesslerj27

27. @roesslerj28
Git for the GUI

28. @roesslerj29
# These files are ignored by git
# based on their name:
*.log
*.class
target/.*
.gitignore

29. @roesslerj30
# Ignore file for recheck.
# Ignore attributes globally:
attribute=class
.retest/recheck.ignore

30. @roesslerj31
# Ignore file for recheck.
# Ignore attributes with regex:
attribute-regex=.*
.retest/recheck.ignore

31. @roesslerj32
# Ignore file for recheck.
# Ignore elements by tag:
matcher: type=a
.retest/recheck.ignore

32. @roesslerj33
# Ignore file for recheck.
# Ignore elements by attribute:
matcher: id=banner
.retest/recheck.ignore

33. @roesslerj34
# Ignore file for recheck.
# These can be combined:
matcher: id=div-b4f23, attribute-regex: font.*
.retest/recheck.ignore

34. @roesslerj35
# Ignore file for recheck.
# Ignore whole subtrees:
matcher: xpath=HTML[1]/BODY[1]/DIV[1]/DIV[1]/DIV[1]
.retest/recheck.ignore

35. @roesslerj36
ignore ≈ filter

36. @roesslerj37
Demo

37. @roesslerj38
Better than pixel diffing

38. @roesslerj39
Better than pixel diffing

39. @roesslerj40
Better than pixel diffing

40. @roesslerj41
Works with animations

41. @roesslerj42
+ rule-based deterministic ignore
+ open source
+ offline usage
+ unbreakable selenium
+ works for any technical interface
recheck benefits:

42. @roesslerj43
some Feedback
… I was thrilled by that
product and new
approach…
… let me congratulate you on your brilliant
idea of the unbreakable Selenium and the
web page difference checking solutions. I
think this is a very much needed solution in
the website testing field.
Ahmed Magdy
Michael Wowro
… the tool is very helpful, gives detailed
information and is easy to use …
Karolina Wierbol

43. @roesslerj44

44. @roesslerj45
new version
findElement(by.id(“sign-in”))
ID changed
use
new element
and log
find in
old version
old version
breaks
create 1-on-1
assignment
new version
best match

45. @roesslerj46

46. @roesslerj48

47. @roesslerj49

48. @roesslerj50

49. @roesslerj51
data-independent tests

50. @roesslerj52
https://github.com/retest/recheck-web

51. @roesslerj53
https://github.com/retest/recheck-web-chrome-extension

52. @roesslerj
new version
findElement(by.id(“sign-in”))
ID changed
use
new element
and log
find in
old version
old version
breaks
create 1-on-1
assignment
new version
best match
@roesslerjX
Better than pixel diffing
54
Summary
Difference Testing
whitelist testing
easy creation
easy maintenance
Unbreakable Selenium:
easy to use
define own ids
data-independent tests

53. @roesslerj55
roessler@retest.de
https://www.facebook.com/retest.de
http://retest.rocks
@roesslerj
Please star us on GitHub!