Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Super fast
end-to-end tests
Lars Thorup
ZeaLake Software Consulting
March, 2016
Lars Thorup
● Software developer/architect
● JavaScript, C#
● Test Driven Development
● Coach
● Agile engineering practice...
Agenda
● What's so bad about end-to-end tests?
● Why are unit tests not sufficient?
● Getting the best of both worlds
● De...
What's so bad about end-to-end tests?
* Thanks to Randall Munroe: xkcd.com/303/
What's so bad about end-to-end tests?
● Not fast
● Not precise
● Not simple
● Not robust
End-to-end tests are slow
● Selenium tests: 0.1 / second
● Unit tests: 61 / second
End-to-end tests give poor feedback
● End-to-end tests: "something went wrong"
● Unit tests: the problem is "in this metho...
End-to-end tests are complicated
● Load specific set of test data into the database
● Run the application on a web server
...
End-to-end tests are fragile
● End-to-end tests fails sporadically:
● Unit tests fails for a reason:
Unit tests have many nice properties
● Fast
● Precise
● Simple
● Robust
Why are unit tests not sufficient?
● They test code in isolation
● Dependencies are mocked
● So what happens when a
depend...
The problem with unit tests: mocks lie to you!
● All is well for now:
The problem with unit tests: mocks lie to you!
● Then watch what happens:
The problem with unit tests: mocks lie to you!
● How can mocks lie to us?
● ...because we hand-write them
● ...we copy ass...
Getting the best of both worlds
Demonstration of a sample implementation
● Node.JS
● Back-end: a REST-ful API
● Front-end: HTML + JavaScript
● Almost 100 ...
End-to-end tests now also become
● Fast
● Precise
● Simple
● Robust
Questions!
Upcoming SlideShare
Loading in …5
×

Super fast end-to-end-tests

539 views

Published on

Automated end-to-end tests are often seen as a necessary evil. A common example is Selenium-based browser tests. This kind of testing has many drawbacks: 1) They take a long time to run. 2) They require complicated setup. 3) They are fragile. But it doesn't have to be like that!

In this talk I describe how we can write automated end-to-end tests that are 1) Superfast. 2) As easy to setup as unit tests. 3) As robust as unit tests.

This technique is leveraging existing unit testing techniques. Tests for lower layers (such as server code) is instrumented to record all requests and responses in a log file. Tests for higher layers (like client code) is extended with an mocking layer that automatically configures with the contents of that log file.

It is then possible to run a almost a hundred end-to-end tests every second.

Published in: Software
  • Be the first to comment

Super fast end-to-end-tests

  1. 1. Super fast end-to-end tests Lars Thorup ZeaLake Software Consulting March, 2016
  2. 2. Lars Thorup ● Software developer/architect ● JavaScript, C# ● Test Driven Development ● Coach ● Agile engineering practices and tools ● Founder ● BestBrains ● ZeaLake ● @larsthorup
  3. 3. Agenda ● What's so bad about end-to-end tests? ● Why are unit tests not sufficient? ● Getting the best of both worlds ● Demo!
  4. 4. What's so bad about end-to-end tests? * Thanks to Randall Munroe: xkcd.com/303/
  5. 5. What's so bad about end-to-end tests? ● Not fast ● Not precise ● Not simple ● Not robust
  6. 6. End-to-end tests are slow ● Selenium tests: 0.1 / second ● Unit tests: 61 / second
  7. 7. End-to-end tests give poor feedback ● End-to-end tests: "something went wrong" ● Unit tests: the problem is "in this method in this class"
  8. 8. End-to-end tests are complicated ● Load specific set of test data into the database ● Run the application on a web server ● Start a browser ● Writing the test ● Depend on test data loaded elsewhere ● Ignore side-effects of other tests
  9. 9. End-to-end tests are fragile ● End-to-end tests fails sporadically: ● Unit tests fails for a reason:
  10. 10. Unit tests have many nice properties ● Fast ● Precise ● Simple ● Robust
  11. 11. Why are unit tests not sufficient? ● They test code in isolation ● Dependencies are mocked ● So what happens when a dependency changes interface? ● The application may break, while all unit tests pass! ● Because... ● Mocks lie to you!
  12. 12. The problem with unit tests: mocks lie to you! ● All is well for now:
  13. 13. The problem with unit tests: mocks lie to you! ● Then watch what happens:
  14. 14. The problem with unit tests: mocks lie to you! ● How can mocks lie to us? ● ...because we hand-write them ● ...we copy assumptions that can change ● Can we automatically generate correct mocks?
  15. 15. Getting the best of both worlds
  16. 16. Demonstration of a sample implementation ● Node.JS ● Back-end: a REST-ful API ● Front-end: HTML + JavaScript ● Almost 100 end-to-end tests per second ● github.com/larsthorup/http-auto-mock-demo
  17. 17. End-to-end tests now also become ● Fast ● Precise ● Simple ● Robust
  18. 18. Questions!

×