Your SlideShare is downloading. ×
0
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Docker at Spotify
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Docker at Spotify

6,195

Published on

Why we're using Docker at Spotify and what we're doing with it.

Why we're using Docker at Spotify and what we're doing with it.

Published in: Software, Technology, Business
0 Comments
27 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
6,195
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
63
Comments
0
Likes
27
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Docker at Spotify ! Rohan Singh @rohansingh
  • 2. Spotify “Music for every moment” ! 40+ million active users ! 20+ million songs available (we’re in 55+ countries, differs by market) ! Live in the US for 2 years 2
  • 3. What are we working with? ! 100+ distinct backend services ! 5000+ production servers in 4 sites worldwide ! Almost 300 servers per ops engineer 3
  • 4. Deployment at Spotify 4
  • 5. Build Debian packages ! Requisition new hardware ! Use combination of Puppet and internal tooling (deployify) 5
  • 6. old way: SSH to a bunch of machines, apt-get install || upgrade ! deployify: Uses fabric, which just SSH’s, runs apt-get install || upgrade 6
  • 7. What’s wrong with that? 7
  • 8. Requisitioning new hardware is slow ! Hardware utilization is low ! We’ve automated and improved, but still too slow 8
  • 9. Deployments often fail… ! …though usually only partially 9
  • 10. ! Machines get denormalized easily 10
  • 11. Debian packaging is its own special hell So many damn files! ! debhelper ! Distro & version specific ! apt-get update 11
  • 12. What do we want? 12
  • 13. Deployments that are: ! Repeatable ! Straightforward ! Fault-tolerant 13
  • 14. Why Docker? ! It gives us “repeatable”, right out of the box ! ! And it’s fairly straightforward, too! 14
  • 15. Repeatability? ! ! The code you test isn’t necessarily the code that runs in prod 15
  • 16. Repeatability — Docker ! Build an image ! Run tests against the image ! Run the tested image in prod 16
  • 17. Straightforward ! Just one Dockerfile ! Pretty simple syntax ! Immutable images — easy to reason about 17
  • 18. Fault-tolerance? Even if your code is perfect, deployments might fail ! Puppet or Debian repos are down ! Network issues ! The machine dies 18
  • 19. Fault-tolerance — Docker Just one thing has to work — pulling & running an image ! If it fails, just try again ! No machines left in weird states* ! ! *(in theory) 19
  • 20. What more do we need? 20
  • 21. Deploying Docker containers across our entire fleet of servers ! Some way to deal with Docker or container failures ! That’s pretty much it 21
  • 22. Looking around for a Docker platform Several really feature-full offerings ! Pretty opinionated and all-encompassing ! All we need is reliably deploying containers to a fleet of Docker hosts 22
  • 23. Helios 23
  • 24. Makes sure your containers are deployed & running exactly where you want them… ! …and that’s pretty much it ! ! github.com/spotify/helios 24
  • 25. What’s in the box? Helios master frontend that you talk to ! Helios agent runs alongside each Docker instance ! Zookeeper ! Straightforward CLI 25
  • 26. How does it work? Create a job — tell Helios which Docker image to deploy ! Deploy the job — tell Helios where to run the Docker containers 26
  • 27. See it in action 27
  • 28. What’s next? 28
  • 29. Helios is just the beginning ! Brought up plenty of Helios+Docker agents in our private cloud for our devs ! Integrated Helios with JUnit — write JUnit tests run against Docker containers 29
  • 30. Dockerized integration tests 30
  • 31. This week — first Helios+Docker-based service in production ! ! In the future — Containerizing more of our infrastructure 31
  • 32. Give it a go Open source: github.com/spotify/helios ! Try it out: git clone https://github.com/spotify/helios.git && cd helios && vagrant up ! ! ! Come play with us (spotify.com/jobs) 32

×