Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Hermetic environment for your functional tests


Published on

What are the most common problems with testing environments?
- You are not the only one who is using it.
- Test failures are not repeatable.
- Test data can be easily messed up due to tests overlap.

Those problems are introducing flakiness in your tests, increase frustration level and decrease confidence in quality of a product you are building. Forcing your development team to have a testing queue increases delivery time dramatically. Creating zillions of environments does not sound as cheapest solution either.

At Spotify we experimented with different approaches on how testing environments can be configured: from shared environment to mocks, stubs and hermetic servers. During my presentation I will share the lessons we learned, what worked, what not and what is the direction we are pursuing in order to stabilise our testing suites.

Published in: Software
  • Login to see the comments

Hermetic environment for your functional tests

  1. 1. March 9, 2015 Andrii Dzynia Software Engineer in Quality @adzynia Hermetic environment for your functional tests
  2. 2. 2 Spotify brings you the right music for every moment! Over 20,000 songs added every day Started in 2006 (in Sweden) Now 1500+ employees, 600+ engineers 5(STO, GBG, NYC, BOS, SF) engineering offices Over 30 million songs available Over 60 million active users Over 15 million paying subscribers
  3. 3. 3 High level architectural view
  4. 4. Persistent Test Environment 4 you are not the only one who is using it test failures not always repeatable test data can be easily messed up
  5. 5. 5 synchronise testing efforts between teams create environment for every team/person test in production Hard way of solving test environment problem
  6. 6. There is an easier way. 6
  7. 7. Containers. 7
  8. 8. 8 Docker @ Spotify
  9. 9. 9 Helios Framework (c) Spotify Helios is a Docker orchestration platform for deploying and managing containers across an entire fleet of servers. Helios provides a HTTP API as well as a command line client to interact with servers running your containers. It also keeps history of events in your cluster including information such as deploys, restarts and version changes.
  10. 10. High level usage example 10
  11. 11. Code sample 11
  12. 12. DEMO 12
  13. 13. Not only for ‘selenium’ tests 13 hermetic environment for verifying back-end services  hermetic environment for verifying infrastructure jobs hermetic environment for verifying consumer clients* *proof of concept only
  14. 14. Pros vs Cons More deterministic test results Repeatable test failures Decreased system failure localisation time Increased test run time Data sets are smaller than in production 14
  15. 15. Alternatives 15 Stub implementations Mock servers
  16. 16. 16 Take Aways Consider using container technologies Create test environment on-the-fly Check out Helios Framework No silver bullets. Just an alternative to Mocks and Stubs.
  17. 17. 17 @adzynia Spotify Engineering Culture part 1 part 2
  18. 18. Check out or @Spotifyjobs for more information Want to join the band? Follow @SpotifyEng on Twitter.