This document discusses container orchestration and provides an overview of different container orchestration technologies including Mesos, Kubernetes, CoreOS Fleet, and Docker libswarm. It explains the benefits of containers and orchestration, and covers concepts like schedulers, service discovery, monitoring, and clustering.

1. Container Orchestration
Timo Derstappen @teemow

2. Giant Swarm
● PaaS for Simple Service Orchestration
● Based in Cologne
● Hiring Devs and DevOps
http://giantswarm.io

4. The Beginning

5. ● Containers are not that new
● But Docker solved packaging and
distribution very well
● Standardization: libcontainer
● Open Source ecosystem is growing
Why Containers?

7. ● SOA / Microservice Architectures
● Polyglot Persistence
● Fault tolerance / High Availability
● Scalability
● Utilization
Why Orchestration?

9. Orchestration Layer
● Schedule Containers
● Configuration Management
● Service Discovery
● Logging / Monitoring / Alerting

10. Many Choices
● CoreOS
● Kubernetes (Google)
● Apache Mesos (Twitter, Airbnb)
● Consul (Hashicorp)
● Project Atomic (Redhat)

12. Docker libswarm
● Primitives of container orchestration
● With adaptors to actual implementations
● Docker Remote API abstraction
● Will be probably rewritten a view times
before it will be released

13. Datacenter

14. Static Partitioning

15. Static Partitioning
http://cto.vmware.com/wp-content/uploads/2012/10/mesos_workloads.png

16. One Single Computer

17. Schedulers (High level)
● Google Borg (Monolithic)
● Apache Mesos (Two Level)
● Hadoop on Demand (Two Level)
● Google Omega (Shared State)
Read: “Omega: flexible, scalable schedulers for
large compute clusters”
http://research.google.com/pubs/pub41684.html

18. Mesos
● Primitives for distributed scheduling
● Meta Scheduler
● Writing your own scheduler is “easy”
● Mesosphere announced Kubernetes on
Mesos

20. Mesos Apps

21. Schedulers
● Mesos => Marathon/Aurora/Chronos/...
● Kubernetes (not Omega)
● CoreOS => Fleet

22. Consistent Cluster State
● Etcd (Raft)
● Consul (Raft)
● Doozer (Paxos)
● Zookeeper (Paxos)
● Spanner (Paxos)
http://raftconsensus.github.io/

23. Service Discovery
● Ambassador Pattern
● Consul
● Etcd
● Haproxy / confd

24. Ambassador
Pattern

25. Plan 9

26. Docker libchan
● Link containers from everywhere
● Unix Socket
● HTTP2 / TLS
● Messaging / Raw TCP
● Like Go Channels over the network

27. ● Keep it outside of your containers
● Run it in separate but privileged containers
● Marathon /metrics
● cAdvisor
Monitoring / Logging

28. Mesos - Marathon
● Framework for long running processes
● Like init for a cluster
● Constraints
● Health Checks
● “Service Discovery” / Load balancing
● REST API

29. Demo

30. Kubernetes Design

31. Kubernetes Concepts
● Pods
○ A group of containers on the same host
○ They share the same network namespace
○ Eg. presence, ambassador, logging, monitoring,
caching, configuring sidekicks
● Labels
● Service
● Replication Controller

32. Kubernetes Master
● Apiserver
○ REST, Client kubecfg
● Controller Manager
● Scheduler
● Etcd

33. Kubernetes Node
● Docker
● Kubelet
○ Watches etcd or receives via HTTP a container
manifest
● cAdvisor
● Proxy

34. Demo

35. CoreOS
● Systemd / Fleet
● Etcd
● Locksmith
● Small Footprint
● Unix Philosophy

36. Fleet
● Systemd API for the cluster
● Simple scheduling via Metadata
● Health Check / Failover

37. Demo