Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

QCon NYC 2016: Reach Production Faster with Containers in Testing

1,719 views

Published on

https://qconnewyork.com/ny2016/presentation/reaching-production-faster-with-containers-in-testing

Spotify adopted container technology early on and built its own OSS framework for container orchestration called Helios. Not only do containers run many critical systems at Spotify, they also improve and accelerate development. We run containerized integration tests close to 400 times a day.

This talk covers how our Helios testing framework drives integration tests and spins up entirely self-contained environments during test runs. Developers can test services locally in an environment closely resembling the production stack; spin up dependent services like Cassandra, memcached, or even other containerized Spotify services; and even test their deployment and service discovery configurations.

Learn how this style of integration testing has increased our code quality and successful deployments.

Published in: Technology
  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

QCon NYC 2016: Reach Production Faster with Containers in Testing

  1. 1. Reach Production Faster with Containers in Testing David Xia
  2. 2. Spotify’s Scale • 150+ people in infrastructure • Thousands of hosts, 2000+ running containers • 1500+ deploys in past month, majority were containers
  3. 3. About David Xia • Work on deployment infrastructure • Work on open-source Docker orchestration tool Helios
  4. 4. Prerequisites for This Talk • You’re familiar with containers • You like tests • That’s it!
  5. 5. Prerequisites for This Talk You don’t need to use containers in production for this talk to be useful!
  6. 6. Three Problems, Three Ways to Solve with Containers How can I enable developers to: • catch container misconfiguration in tests? • easily install and start non-trivial test dependencies? • make their tests isolated and reproducible?
  7. 7. “Why did my service pass integration tests but fail when I deployed it as a container?” - Sad Developer 1 Problem 1: Container Misconfiguration
  8. 8. Problem 1: Container Misconfiguration
  9. 9. Solution 1: Container Misconfiguration
  10. 10. “I want to run my project's integration tests locally. The tests need a local Cassandra/other DB. How do I set everything up?” - Sad Developer 2 Problem 2: Non-trivial dependencies
  11. 11. Problem 2: Non-trivial dependencies
  12. 12. docker run --name foo -d cassandra - Happy Developer 2 Solution 2: Non-trivial dependencies
  13. 13. Solution 2: Non-trivial dependencies
  14. 14. “How can I easily restore my test dependencies to a clean state?” - Sad Developer 3 Problem 3: Reproducible Tests
  15. 15. docker stop <container ID> docker run --name foo -d cassandra - Happy Developer 3 Solution 3: Reproducible Tests
  16. 16. Key Takeaways - Using Containers in Tests Can Help You: • Test more of the stack in an env resembling production • Easily start real dependencies • Ensure tests are reproducible and isolated
  17. 17. Helios in a Nutshell
  18. 18. How helios-testing was born • multitenancy • docker • container orchestration (helios) • container testing framework (helios-testing)
  19. 19. What does helios-testing do? Let’s you write code to start and stop containers.
  20. 20. What does helios-testing do? Let’s you write code to start and stop containers.
  21. 21. What does helios-testing do? Let’s you write code to start and stop containers.
  22. 22. Demo! Solution 1: Container Configuration https://www.youtube.com/watch?v=iWtTFI9zDfk
  23. 23. Demo! Solution 2: Isolated Tests https://www.youtube.com/watch?v=GInAJSMd9cs
  24. 24. “Testing with a Cassandra container is the closest I can get to testing against Cassandra in reality.” Successes
  25. 25. “I especially like the fact that I can test my image in its final state and be confident that it will work in production.” Successes
  26. 26. “Using helios-testing to run datastores in containers has made the tests portable and setup free (by setup I mean no manual installation of the datastore on the test machine or locally).” Successes
  27. 27. Make sure your testing framework and infrastructure are fast and reliable. Lessons Learned
  28. 28. Make framework’s interface and implementation as simple as possible. Lessons Learned
  29. 29. Provide great test examples. Lessons Learned
  30. 30. Key Takeaways - Using Containers in Tests Can Help You: • Test more of the stack in an env resembling production • Easily start real dependencies • Ensure tests are reproducible and isolated
  31. 31. When Not to Use Containers in Tests • Don’t test functionality unrelated to containers that you can easily test separately • When your container-based tests overlaps a lot with regular integration tests
  32. 32. Acknowledgements Rohan Singh Matt Brown Staffan Gimåker Mats Linander Nic Cope
  33. 33. Q&A @davidxia_ github.com/davidxia github.com/spotify/helios github.com/davidxia/qcon-demo Demo videos: example 1 and 2 helios-testing

×