The Deis project, an open source PaaS platform, recently announced in our public roadmap meeting that we are transitioning our entire platform onto Kubernetes. We call it the Great Kubernetes Rebase. In the process, we’ve gained valuable insight into what it means to build a complex platform in the Kubernetes way. This presentation is a crisply paced walk through the tough lessons we've learned as we've tried (and sometimes failed) to re-think our application. Among the things this presentation will cover: Turtles all the way down: Running a VM in a container in a pod on a kubelet… in VirtualBox, just so that we can safely build Docker images. Namespaces in theory and practice. Running an HA Postgres cluster inside Kubernetes. Ditch OSX: Productive developers do it on Linux. Storage: The data’s gotta go somewhere! The secret to using secrets: They’re good for credentials, but they’re also good for a few other things. We take stability very seriously: Continuous testing inside the cluster. KubeCon schedule link: http://sched.co/4Uq6

1. Matt Butcher
Deis Architect
Author of Go in Practice & 7 other
books. Contributed to over 100 FOSS
projects. Philosopher (no, really).
Has serious coffee drinking problems.

2. The Great Kubernetes Rebase
Lessons the Deis Team is Learning

3. The Deis Project
• Containerized PaaS
• Easy as Heroku
• Run on your own infrastructure
• Pick what you like

4. Deis v1
CoreOS
Fleet Etcd
Docker
RouterController Registry
BuilderStorageLogger
Apps

5. Deis v2
???
Kubernetes
RouterController Registry
Slugbuilder
Object
Store
Logger
Apps
EtcdSlugrunnerGit GW

6. Five Lessons We Learned

7. 5. Turtles All the Way Down

8. The Things We Do For Security
http://is.gd/raLYgY
Pod + Container
Qemu
Docker
Image
Builder

9. Fun Fact: It’s really hard to find a photo with more than four turtles
stacked.

10. Well….
“Did it work?”

11. 4. The Bartleby Effect

12. Managing Manifests
• No “real world examples” outside of
the k8s codebase
• Examples and tutorials often broken
• Dearth of tools for working with
manifests
• Steep learning curve for newcomers
(and we were newcomers)
http://gastoncavalleri.com/2014/05/14/bartleby-existentialism-and-marxism-by-gaston-cavalleri/

13. At The Helm
We thought a lot about this problem.
We wanted something like Homebrew
(or maybe apt-get/yum) for
Kubernetes.
So we created Helm, a package
manager for Kubernetes.

14. Today we’re happy
to release Helm!
http://helm.sh

15. 3. The Key (Value) To Success

16. Etc, Etc, Etc(d).
• We needed mutable K/V storage
across arbitrary pods
• Relearning how to run a core
service
• Internal discovery
• Which leads us to…
http://is.gd/m4P35X

17. 2. The Chumbawamba Principle

18. –Tubthumping
“I get knocked down, but I get up again.”

19. Don’t Just Do “Images”
• No more “solipsistic” Docker images
• Kubernetes-aware images that give
us…
• Custom bootstrap
• Downward API
• Resilience (bounce-back)

20. 1. The Service is Part of the
Consumer

21. We Tend to Divide Services and Consumers Like
This
My App My DatabaseDB
Service

22. A Little Problem
My App My DatabaseDB
Service
Problem: App code (thus dev)
needs to know about services
at dev time.
But service is not part of
app code.

23. Service Belongs to Consumer
My App My DatabaseDB
Service
Relationship between svc
and app is closer.
Label query (still) handles this.

24. The End
Questions?
http://is.gd/hg0hh4