The document discusses the evolution of container orchestration and compares various container platforms such as Kubernetes, DC/OS, and Docker. It outlines the capabilities and functionalities of different types of platforms, including IaaS, PaaS, and FaaS, emphasizing organizational considerations for choosing a platform. Notable quotes from industry experts are included to provide context and insight into the current landscape of cloud infrastructure and orchestration solutions.

1. © 2017 Mesosphere, Inc. All Rights Reserved. 1
CONTAINER
ORCHESTRATION
WARS
( 2017 Edition )

2. © 2017 Mesosphere, Inc. All Rights Reserved. 2
KARL WHO?
WHERE?
- Mesosphere
- Pivotal
WHAT?
- DC/OS
- Kubernetes
- CloudFoundry
- BOSH
- github.com/karlkfi
- twitter.com/karlkfi
- linkedin.com/in/karlkfi
- karl.isenberg.us

3. © 2017 Mesosphere, Inc. All Rights Reserved. 3
INFRASTRUCTURE EVOLUTION
PLATFORM SPECTRUM
CONTAINER PLATFORM LAYERS
CONTAINER PLATFORM CAPABILITIES
CONTAINER PLATFORM COMPARISON
HOW DO I CHOOSE?

4. © 2017 Mesosphere, Inc. All Rights Reserved. 4
INFRASTRUCTURE
EVOLUTION

5. © 2017 Mesosphere, Inc. All Rights Reserved. 5
MONOLITH
AT SCALE
ONLINE
LATENCY ROUTED
MULTI-REGION
LOAD BALANCED
MULTI-ZONE
REPLICATED
AUTO-SCALED
DATA REPLICATION
DATA SYNCHRONIZATION
AWS Simple Icons
By Amazon Web Services, Inc., CC BY-SA 3.0

6. © 2017 Mesosphere, Inc. All Rights Reserved. 6
MICROSERVICES
AT SCALE
Wheel of Doom
From A Journey into Microservices by Hailo

7. © 2017 Mesosphere, Inc. All Rights Reserved. 7
INFRASTRUCTURE
EVOLUTION
APPLICATION
PROVISIONING
ON
BARE METAL

8. © 2017 Mesosphere, Inc. All Rights Reserved. 8
“Ultimately, utility cloud providers have
exposed how difficult it is to properly operate
data centers — and reminded all of us that the
ability to expertly operate infrastructure is
what really fuels the consumption of open
source infrastructure.”
Brian Stein
(Rackspace VP - 2017)

9. © 2017 Mesosphere, Inc. All Rights Reserved. 9
INFRASTRUCTURE
EVOLUTION
APPLICATION
PROVISIONING
ON
VIRTUAL INFRASTRUCTURE
PLATFORM
(IaaS)

10. © 2017 Mesosphere, Inc. All Rights Reserved. 10
“The goal of Cloud Foundry is to put more of the
controls back in the hands of developers so they can
self-provision, so there aren’t a lot of roadblocks in
their way. But it gives a lot of guardrails.”
Chip Childers
(Cloud Foundry Foundation CTO - 2017)

11. © 2017 Mesosphere, Inc. All Rights Reserved. 11
INFRASTRUCTURE
EVOLUTION
APPLICATION
PLATFORM
(PaaS / aPaaS)
ON
INFRASTRUCTURE
PLATFORM
(IaaS)

12. © 2017 Mesosphere, Inc. All Rights Reserved. 12
“...traditional “PaaS” roles have now been
taken over by containers… The piece that is
left for PaaS is the part that was always the
most important part of PaaS in the first place,
and that’s the opinionated developer
experience.”
Brendan Burns
(Kubernetes Cofounder - 2017)

13. © 2017 Mesosphere, Inc. All Rights Reserved. 13
INFRASTRUCTURE
EVOLUTION
CONTAINER
ORCHESTRATION
ON
INFRASTRUCTURE
PLATFORM
(IaaS)

14. © 2017 Mesosphere, Inc. All Rights Reserved. 14
INFRASTRUCTURE
EVOLUTION
CONTAINER
PLATFORM
(CaaS)
ON
INFRASTRUCTURE
PLATFORM
(IaaS)

15. © 2017 Mesosphere, Inc. All Rights Reserved. 15
INFRASTRUCTURE
EVOLUTION
CONTAINER
PLATFORM
(CaaS)
ON
BARE METAL

16. © 2017 Mesosphere, Inc. All Rights Reserved. 16
“If your PaaS can efficiently start
instances in 20ms that run for half
a second, then call it serverless.”
Adrian Cockcroft
(AWS VP - 2016)

17. © 2017 Mesosphere, Inc. All Rights Reserved. 17
INFRASTRUCTURE
EVOLUTION
FUNCTION
PLATFORM
(FaaS)
ON
INFRASTRUCTURE
PLATFORM
(IaaS)

18. © 2017 Mesosphere, Inc. All Rights Reserved. 18
INFRASTRUCTURE
EVOLUTION
FUNCTION
PLATFORM
(FaaS)
ON
CONTAINER
PLATFORM
(CaaS)

19. © 2017 Mesosphere, Inc. All Rights Reserved. 19
INFRASTRUCTURE
EVOLUTION
FUNCTION PLATFORM
(FaaS)
ON
CONTAINER PLATFORM
(CaaS)
ON
INFRASTRUCTURE PLATFORM
(IaaS)

20. © 2017 Mesosphere, Inc. All Rights Reserved. 20
PLATFORM
SPECTRUM

21. © 2017 Mesosphere, Inc. All Rights Reserved. 21
PLATFORM SPECTRUM
INFRASTRUCTURE
PLATFORM
( IaaS )
Virtual Machines
Disks
Networks
Firewalls
SOFTWARE
PLATFORM
( SaaS )
Whatever
You
Want
( to pay for )
FUNCTION
PLATFORM
( FaaS )
Actions
/tmp
Triggers
Gateways
CONTAINER
PLATFORM
( CaaS )
Containers
Volumes
IPs & Ports
Load Balancers
APPLICATION
PLATFORM
( PaaS / aPaaS )
Apps
/tmp
80/443
Routes
Abstraction
Flexibility Velocity
Low Level High Level

22. © 2017 Mesosphere, Inc. All Rights Reserved. 22
PLATFORM EXAMPLES
INFRASTRUCTURE
PLATFORM
( IaaS )
OpenStack
vSphere
Azure Stack VMs
AWS EC2
GCE
Azure VMs
SOFTWARE
PLATFORM
( SaaS )
BYO
Salesforce
Oracle
SAP
FUNCTION
PLATFORM
( FaaS )
OpenWhisk
Fission
Iron.io
Lambda
GCF
Azure Functions
CONTAINER
PLATFORM
( CaaS )
Kubernetes
DC/OS
Docker Datacenter
GKE
ECS
ACS
APPLICATION
PLATFORM
( PaaS / aPaaS )
CloudFoundry
OpenShift
WaveMaker RAD
Heroku
PCF
Jelastic
HOSTED

23. © 2017 Mesosphere, Inc. All Rights Reserved. 23
CONTAINER
PLATFORM
LAYERS

24. © 2017 Mesosphere, Inc. All Rights Reserved. 24
CONTAINER ORCHESTRATION
Apps
Container Runtime
Container Orchestration
- User workloads
- Distributed container management
- Local container management
Infrastructure
ServicesJobs
- Container agnostic infrastructure

25. © 2017 Mesosphere, Inc. All Rights Reserved. 25
CONTAINER PLATFORM
Apps
Container Storage
Container Runtime
Container Orchestration
System Services
- User workloads
- System management & service enablement
- Distributed container management
- Local container management
- Container aware infrastructureContainer Networks
Infrastructure
ServicesJobs
- Container agnostic infrastructure

26. © 2017 Mesosphere, Inc. All Rights Reserved. 26
DISTRIBUTED OPERATING SYSTEM
Apps
Container Storage
Container Runtime
Container Orchestration
System Services
Container Networks
Infrastructure
ServicesJobs
Security
&
Multitenancy
Logging
&
Metrics
&
Monitoring

27. © 2017 Mesosphere, Inc. All Rights Reserved. 27
CONTAINER
PLATFORM
CAPABILITIES

28. © 2017 Mesosphere, Inc. All Rights Reserved. 28
CONTAINERS
- Resource Isolation
- Resource
Constraints
- Process Tree
- Environment
Isolation
- Shell / Exec
IMAGES
- Build
- Layers
- Download
- Cache
- Publish
- Prune
NETWORKS
- Container
- Bridge
- Host
- Virtual
- Overlay
- Remote
- User-defined
- Port Mapping
DATA VOLUMES
- Ephemeral
- Host
- Backup / Restore
- Copy In / Out
- Shared
RUNTIME CAPABILITIES

29. © 2017 Mesosphere, Inc. All Rights Reserved. 29
ORCHESTRATION CAPABILITIES
SCHEDULING
- Placement
- Replication/Scaling
- Readiness Checking
- Resurrection
- Rescheduling
- Rolling Updates
- Collocation
- Daemons
- Cron Jobs
RESOURCE MGMT
- Memory
- CPU
- GPU
- Ephemeral Volumes
- Remote Persistent Volumes
- Local Persistent Volumes
- Ports
- IPs (per container)
SERVICE MGMT
- Labels
- Groups/Namespaces
- Dependencies
- Load Balancing (L7)
- VIPs (L3/L4 LB)
- DNS
- DNS Proxy
- Secrets
- Config Mgmt

30. © 2017 Mesosphere, Inc. All Rights Reserved. 30
OPERATIONAL CAPABILITIES
ADMINISTRATION
- GUI
- CLI
- Metrics API
- Logs API
- Events API
- Rolling Upgrades
- Backups & Restores
MULTI-INFRASTRUCTURE
- Multi-cloud
- Multi-zone
- Multi-region
- Hybrid-cloud
- Federation
SYSTEM SERVICES
- Auto-Scaling
- Package Management
- Service Catalog
- Service Brokers
- Admin Proxy
- API Gateway

31. © 2017 Mesosphere, Inc. All Rights Reserved. 31
PLATFORM CAPABILITIES
CONTAINER NETWORKS
- Overlay
- Routing
- Network Address
Translation (NAT)
- Firewalls
- Access Control Lists
- Quality of Service
CONTAINER STORAGE
- Local Volumes
- Remote Volumes
- Block Storage
- File System Storage
- Object Storage
PLATFORM DATABASES
- Lock Service
- Key-Value Database
- Relational Database
- Time Series Database

32. © 2017 Mesosphere, Inc. All Rights Reserved. 32
SECURITY
- User Accounts
- Service Accounts
- System/User Space
- E2E Encryption
- Non-root User Workloads
- Audit Logging
- Public Key Infrastructure
- Certifications
MULTITENANCY
- User Groups
- Permissions
- RBAC
- ABAC
- Resource Sharing
- FIFO
- Fair
- Quotas
- Branding
- Quality of Service
PLATFORM CAPABILITIES

33. © 2017 Mesosphere, Inc. All Rights Reserved. 33
NON-FUNCTIONAL QUALITIES
SCALABILITY
Performance, Responsiveness, Efficiency
AVAILABILITY
Fault Tolerance, Robustness, Reliability, Resilience,
Disaster Recovery
FLEXIBILITY
Format Support, Interoperability, Extensibility,
Container Runtimes
USABILITY
Familiarity, Maintainability, Compatibility,
Debuggability
PORTABILITY
Host OS, Cloud, Bare-Metal, Hybrid
SECURITY
Encryption Quality, Vulnerability Process,
Fast Patching, Backporting

34. © 2017 Mesosphere, Inc. All Rights Reserved. 34
CONTAINER
PLATFORM
COMPARISON

35. © 2017 Mesosphere, Inc. All Rights Reserved. 35
Docker
Kontena
ECS
Nomad
Kubernetes
Rancher Cattle
DC/OS
OpenShift
TODAY’S CONTESTANTS

36. © 2017 Mesosphere, Inc. All Rights Reserved. 36
CloudFoundry GKE
Heroku
Tectonic
Juju
Triton
HONORABLE MENTIONS

37. © 2017 Mesosphere, Inc. All Rights Reserved.
Placement
Replication/Scaling
Readiness Checking
Resurrection
Rescheduling
Rolling Updates
Collocation
37
SCHEDULING
CONTAINER PLATFORM COMPARISON (06/2017)
Kubernetes DC/OS ECS Docker Nomad Cattle Kontena
Included New/External/Partial/Experimental
OpenShift
Daemons
Cron Jobs

38. © 2017 Mesosphere, Inc. All Rights Reserved. 38
RESOURCE MANAGEMENT
CONTAINER PLATFORM COMPARISON (06/2017)
Memory
Kubernetes DC/OS ECS Docker Nomad Cattle Kontena
Included New/External/Partial/Experimental
CPU
GPU
Disk Space
Ephemeral Volumes
Remote Persistent Vols
Local Persistent Vols
Ports
IPs (per container)
OpenShift

39. © 2017 Mesosphere, Inc. All Rights Reserved. 39
SERVICE MANAGEMENT
CONTAINER PLATFORM COMPARISON (06/2017)
Labels
Kubernetes DC/OS ECS Docker Nomad Cattle Kontena
Included New/External/Partial/Experimental
Groups/Namespaces
Dependencies
Load Balancing (L7)
VIPs (L3/L4 LB)
DNS
DNS Proxy
Secrets
Config Mgmt
OpenShift

40. © 2017 Mesosphere, Inc. All Rights Reserved. 40
HOW DO I CHOOSE?

41. © 2017 Mesosphere, Inc. All Rights Reserved. 41
KUBERNETES
- Huge community
- Solid API
- Some assembly required
- Multitude of vendors/installers
OPENSHIFT
- Application platform based on Kubernetes
- Always trailing Kubernetes releases
- No assembly required
- Open core, enterprise platform
DC/OS
- Runs native applications (non-Docker)
- Specialized in data services
- Ambitious scope (on-prem AWS)
- No assembly required
- Open core, enterprise platform
DOCKER
- Huge community
- Fast moving API
- Integrated orchestration and runtime
- Recent pivot from runtime to orchestration
- Open core, enterprise platform
HEAVY HITTERS

42. © 2017 Mesosphere, Inc. All Rights Reserved. 42
EC2 CONTAINER SERVICE (ECS)
- Hosted-only solution
- Tight integration with AWS services
- Closed platform
RANCHER CATTLE
- Gateway to Kubernetes, Mesos, and Docker
- Open platform, enterprise support
NOMAD
- Provisioner with orchestration features
- Runs native applications (non-Docker)
- Tight integration with Vault and Consul
- Some assembly required
- Open platform, enterprise support
KONTENA
- Simple to set up
- No assembly required
- Open core, enterprise platform
LITE ORCHESTRATION

43. © 2017 Mesosphere, Inc. All Rights Reserved. 43
CONSIDERATIONS
- Which is more important to you: velocity or flexibility?
- Do you want an opinionated application platform?
- Do you need to support Big Data initiatives and pipelines?
- Do you want a hosted solution?
- Are you willing to build out your own integrations?
- Do you need on-prem & hybrid capabilities?
- Do you want to avoid infrastructure lock-in?
- Are you already invested in a specific infrastructure?
- Are you already invested in a specific operating system?
- Do you need federation and multi-region support?
- Do you want multitenancy or is multi-instance good enough?
- How important are seamless automated rolling upgrades?
- How many nines do your customers need?
- How important is reverse compatibility & API stability?
- Do you need to support non-Docker workloads?
HOW DO I
CHOOSE?

44. © 2017 Mesosphere, Inc. All Rights Reserved. 44
KARL ISENBERG
karlkfi