Automated end-to-end tests are often seen as a necessary evil. A common example is Selenium-based browser tests. This kind of testing has many drawbacks: 1) They take a long time to run. 2) They require complicated setup. 3) They are fragile. But it doesn't have to be like that! In this talk I describe how we can write automated end-to-end tests that are 1) Superfast. 2) As easy to setup as unit tests. 3) As robust as unit tests. This technique is leveraging existing unit testing techniques. Tests for lower layers (such as server code) is instrumented to record all requests and responses in a log file. Tests for higher layers (like client code) is extended with an mocking layer that automatically configures with the contents of that log file. It is then possible to run a almost a hundred end-to-end tests every second.

1. Super fast
end-to-end tests
Lars Thorup
ZeaLake Software Consulting
March, 2016

2. Lars Thorup
● Software developer/architect
● JavaScript, C#
● Test Driven Development
● Coach
● Agile engineering practices
and tools
● Founder
● BestBrains
● ZeaLake
● @larsthorup

3. Agenda
● What's so bad about end-to-end tests?
● Why are unit tests not sufficient?
● Getting the best of both worlds
● Demo!

4. What's so bad about end-to-end tests?
* Thanks to Randall Munroe: xkcd.com/303/

5. What's so bad about end-to-end tests?
● Not fast
● Not precise
● Not simple
● Not robust

6. End-to-end tests are slow
● Selenium tests: 0.1 / second
● Unit tests: 61 / second

7. End-to-end tests give poor feedback
● End-to-end tests: "something went wrong"
● Unit tests: the problem is "in this method in this class"

8. End-to-end tests are complicated
● Load specific set of test data into the database
● Run the application on a web server
● Start a browser
● Writing the test
● Depend on test data loaded elsewhere
● Ignore side-effects of other tests

9. End-to-end tests are fragile
● End-to-end tests fails sporadically:
● Unit tests fails for a reason:

10. Unit tests have many nice properties
● Fast
● Precise
● Simple
● Robust

11. Why are unit tests not sufficient?
● They test code in isolation
● Dependencies are mocked
● So what happens when a
dependency changes
interface?
● The application may break,
while all unit tests pass!
● Because...
● Mocks lie to you!

12. The problem with unit tests: mocks lie to you!
● All is well for now:

13. The problem with unit tests: mocks lie to you!
● Then watch what happens:

14. The problem with unit tests: mocks lie to you!
● How can mocks lie to us?
● ...because we hand-write them
● ...we copy assumptions that can change
● Can we automatically generate correct mocks?

15. Getting the best of both worlds

16. Demonstration of a sample implementation
● Node.JS
● Back-end: a REST-ful API
● Front-end: HTML + JavaScript
● Almost 100 end-to-end tests per second
● github.com/larsthorup/http-auto-mock-demo

17. End-to-end tests now also become
● Fast
● Precise
● Simple
● Robust

18. Questions!