Like many other companies, SoundCloud migrated to a microservices architecture over the last couple of years. Today, there are several hundreds of services with thousands of container instances running in our datacenters. In this talk, I’ll give a brief overview of the current state of our infrastructure and how a typical service is deployed and can communicate with other services. To make it simple for teams to prototype, deploy and operate several services on their own, we built our own container runtime environment, called Bazooka. I’ll give an overview of Bazooka, its features and design decisions, but also the shortcomings and problems we faced over time: automated scheduling. resource management for services with different load profiles. monitoring of highly dynamic deploys or the requirements of stateful services. With the rise of Docker and a general shift towards container-based environments, SoundCloud started to build more and more of its development workflows based on these new solutions. When it became clear that our existing system needed an overhaul to support additional requirements and overcome its shortcomings, we started to look into other container management technologies and available open-source options. In the second part, I’ll present our evaluation process and some of the requirements we defined for a suitable candidate. The attendee will learn which features and properties of Kubernetes make it the ideal choice for us. Finally, I’ll talk about the current state of our Kubernetes migration, some challenges we need to solve to integrate Kubernetes in our existing infrastructure, and present some open issues we are working on in order to eventually deploy and run all our services with Kubernetes. KubeCon schedule link: http://sched.co/4Wd0

1. Tobias Schmidt
KubeCon 2015 - San Francisco
Moving to Kubernetes
Tales from SoundCloud

2. http://prometheus.io

3. Why containers?
What is Bazooka?
What have we learned?
Why Kubernetes?
What’s next?

4. Why did we build a container
runtime environment?

7. slow scaling

8. fragile deploys

9. inconsistent state

10. cumbersome new app setup

11. What is Bazooka?

12. $ bazooka init
$ bazooka push
$ bazooka scale ...
$ bazooka ps
...

13. 12-Factor applications
immutable revisions
declarative deploys

14. LXC
agent on each node
consensus store
CLI

15. What are the strengths
of Bazooka?

16. independent teams

17. quick
deploys / rollback / scaling

18. orchestration is not
in the critical path

19. What have we learned?

20. containers are the only way

21. Golang is a great fit

22. monitoring dynamically
deployed services is complex

23. choose the right data store

24. use a common
integration service

25. resource isolation is important

26. automate everything

27. Why moving on?

28. Bazooka doesn’t fit anymore

29. problem is too big for us alone

30. Docker et al.

31. Why Kubernetes?

32. simple domain objects
(container, pod, service, rc)

33. powerful networking
(security, auditing, performance)

34. pod scheduling

35. label system
(grouping, discoverability, resource constraints)

36. community + support

37. extends Bazooka concepts

38. What’s next?

39. cluster runs on bare-metal

40. continuous integration pipeline

41. Prometheus monitoring
integration

42. logging integration

43. service discovery integration

44. THANK YOU
@dagrobie
https://github.com/grobie