State of Union - Containerz
---------------------
Shiva (narshiva@)
--------------------
 ^__^
 (oo)_______
(__) )/
||----w |
|| ||
TO BEGIN AT THE BEGINNING…
Let’s start, shall we?
Containerized Microservices
Dom 0
Instance Instance Instance
OS OS OS
Container Runtime Container Runtime Container Runtime
App App
Service Service
App App
Service Service
App App
Service Service
Container Orchestration
Dom 0
Instance Instance Instance
OS OS OS
Container Runtime Container Runtime Container Runtime
App Service App App Service Service
Container Orchestration
Container Orchestration
Dom 0
Instance/OS Instance/OS Instance/OS
App Service App App Service Service
Service Management
Scheduling
Resource Management
Orchestration
Service Management
§Labels
§Groups/Namespaces
§Dependencies
§Load Balancing
§Health Check
§Service Discovery
Container Orchestration
Dom 0
Instance/OS Instance/OS Instance/OS
App Service App App Service Service
Service Management
Scheduling
Resource Management
Orchestration
Scheduling
§Placement
§Replication/Scaling
§Resurrection
§Rescheduling
§Rolling deploys
§Upgrades
§Downgrades
§Colocation
Container Orchestration
Dom 0
Instance/OS Instance/OS Instance/OS
App Service App App Service Service
Service Management
Scheduling
Resource Management
Orchestration
Resource Management
§Memory
§CPU
§GPU
§Volumes
§Ports
§IPs
Non Functional Capabilities
Scalability
Performance, Responsiveness, Efficiency
Availability
Fault Tolerance, Reliability, DR
Flexibility
Extensibility, Portability, Interoperability
Usability
Familiarity, Debuggability, Maintainability
Portability
Container Runtime, Host OS, Cloud Provider, On-prem
Security
Isolation, Encryption, Secrets Management,
Auditability
Container Operations
Development Lifecycle
Source repo, CI-CD, Artefact repo
Container Orchestration
Scheduling, Resource Management, Service Management
BAU Operations
Monitoring and Metrics, Maintenance, Debugging
Did you hear that?
In no particular order…
[ ] Schedulers and Orchestration
[ ] Networking
[ ] Security
[ ] Operating Systems
[ ] PaaS
[ ] Storage
[ ] Monitoring
[ ] Container Integration and Container Deployment
[ ] Miscellaneous
In no particular order…
[ ] Schedulers and Orchestration
[ ] General Blurb
[ ] ECS
[ ] Kubernetes
[ ] Mesos
[ ] Docker Swarm
[ ] Orchestration Wars
Schedulers – General Blurb
Cluster
Machines
Cluster State
Information
Monolothic Two-Level Shared State
No Concurrency Pessimistic
Concurrency
(offers)
Optimistic
Concurrency
(transactions)
Scheduling Logic
Docker
Task
Container Instance
Amazon
ECS
Container
ECS Agent
ELB
Internet
ELB
User /
Scheduler
API
Cluster Management Engine
Task
Container
Docker
Task
Container Instance
Container
ECS Agent
Task
Container
Docker
Task
Container Instance
Container
ECS Agent
Task
Container
AZ 1 AZ 2
Key/Value Store
Agent Communication Service
ECS
Mesos
Master
Marathon
ZooKeeper
Mesos + Marathon
Mesos Slaves
Long Running
Tasks
Jobs
Coordination &
Configuration
Kubernetes
Replication
Controller
API Server
Kubernetes Master
Kubelet KubeProxy Docker
Container Container
Pod Pod
Kubelet KubeProxy Docker
Container Container
Pod Pod
Kubernetes Cluster
etcd
Docker Swarm
I hope we win
In no particular order…
[X] Schedulers and Orchestration
[ ] Networking
[ ] Security
[ ] Operating Systems
[ ] PaaS
[ ] Storage
[ ] Monitoring
[ ] Container Integration and Container Deployment
[ ] Miscellaneous
Container Networking
Dom 0
Instance Instance Instance
OS OS OS
Container Runtime Container Runtime Container Runtime
Container Container Container Container Container Container
Overlay all of the thingz
• Flannel
• Calico
• WeaveNet
• Swarm Mode
WeaveNet
Mode
Swarm Mode
Manager
Swarm Mode
Node
TLS CA
Load
Balancing
Service
Discovery
Distributed
Store
Docker Engine
Libnetwork
Volumes
Plugins
Container
Runtime
In no particular order…
[X] Schedulers and Orchestration
[X] Networking
[ ] Security
[ ] Operating Systems
[ ] PaaS
[ ] Storage
[ ] Monitoring
[ ] Container Integration and Container Deployment
[ ] Miscellaneous
Sekkirity is everybodys biznezz
Host Security
• Lock it down!
• Namespaces and cgroups are your friends
• Select few belong to docker UNIX group
• SELinux is also your friend
• Docker daemon runs as root!
Whale-say
"Only trusted users should be allowed to control
your Docker daemon"
Docker daemon security
• Do not run in privileged mode
• Lock down inter container comms –icc=false
• Secure APIs with TLS certificates
Whale-say
“If you run Docker on a server, it is recommended to run
exclusively Docker in the server, and move all other
services within containers controlled by Docker”
Container Image Security
• Use a small selection of trusted images
• Scan your images
• CoreOS’s Clair scans Quay.io,
• Docker Security Scanning works with Docker Trusted
Registry
• Red Hat has built a new scanner in Project Atomic for
its Atomic Registry.
• Other scanners are such as Aqua Peekr, Anchore,
and Twistlock Trust work independently of specific
registries
Lot more prescriptive advice here…
https://benchmarks.cisecurity.org/tools2/docker/CIS
_Docker_1.6_Benchmark_v1.0.0.pdf
In no particular order…
[X] Schedulers and Orchestration
[X] Networking
[X] Security
[ ] Operating Systems
[ ] PaaS
[ ] Storage
[ ] Monitoring
[ ] Container Integration and Container Deployment
[ ] Miscellaneous
Micro OS
• CoreOS
• RancherOS
• Ubuntu Snappy
• RedHat Atom
• VmWare Photon
• ECS Optimized Amazon Linux
RedHat
Atomic
VmWare
Photon
Ubuntu
Snappy
CoreOS RancherOS
395 MB
317 MB
215 MB
20 MB
150 MB
In no particular order…
[X] Schedulers and Orchestration
[X] Networking
[X] Security
[X] Operating Systems
[ ] PaaS
[ ] Storage
[ ] Monitoring
[ ] Container Integration and Container Deployment
[ ] Miscellaneous
Remind Empire
Twelve-Factor
Convox
$ convox apps create go-app
$ convox deploy
$ convox apps info go-app
$ convox build --app go-app –d "Hello Build”
$ convox releases promote RLYSUALSGCT
$ convox ps
$ convox scale main --count=2
Docker Data Center
Universal Control Plane (UCP)
Security Content Trust
Docker Trusted Registry
Orchestration Swarm
Container Runtime Engine
Operating System
Others
In no particular order…
[X] Schedulers and Orchestration
[X] Networking
[X] Security
[X] Operating Systems
[X] PaaS
[ ] Storage
[ ] Monitoring
[ ] Container Integration and Container Deployment
[ ] Miscellaneous
Are we there yet?
In no particular order…
[X] Schedulers and Orchestration
[X] Networking
[X] Security
[X] Operating Systems
[X] PaaS
[-] Storage
[-] Monitoring
[-] Container Integration and Container Deployment
[-] Miscellaneous
Demoz
• Marathon scheduler on ECS (Credit : Ryosuke-san)
• Convox
• Docker Swarm
• Weave Net and Weave Scope
• ECS (ALB, Task AutoScaling, Task IAM Role)
---------------------
T H A N K Y O U
--------------------
 ^__^
 (oo)_______
(__) )/
||----w |
|| ||

State of Union - Containerz

  • 1.
    State of Union- Containerz --------------------- Shiva (narshiva@) -------------------- ^__^ (oo)_______ (__) )/ ||----w | || ||
  • 2.
    TO BEGIN ATTHE BEGINNING… Let’s start, shall we?
  • 3.
    Containerized Microservices Dom 0 InstanceInstance Instance OS OS OS Container Runtime Container Runtime Container Runtime App App Service Service App App Service Service App App Service Service
  • 4.
    Container Orchestration Dom 0 InstanceInstance Instance OS OS OS Container Runtime Container Runtime Container Runtime App Service App App Service Service Container Orchestration
  • 5.
    Container Orchestration Dom 0 Instance/OSInstance/OS Instance/OS App Service App App Service Service Service Management Scheduling Resource Management Orchestration Service Management §Labels §Groups/Namespaces §Dependencies §Load Balancing §Health Check §Service Discovery
  • 6.
    Container Orchestration Dom 0 Instance/OSInstance/OS Instance/OS App Service App App Service Service Service Management Scheduling Resource Management Orchestration Scheduling §Placement §Replication/Scaling §Resurrection §Rescheduling §Rolling deploys §Upgrades §Downgrades §Colocation
  • 7.
    Container Orchestration Dom 0 Instance/OSInstance/OS Instance/OS App Service App App Service Service Service Management Scheduling Resource Management Orchestration Resource Management §Memory §CPU §GPU §Volumes §Ports §IPs
  • 8.
    Non Functional Capabilities Scalability Performance,Responsiveness, Efficiency Availability Fault Tolerance, Reliability, DR Flexibility Extensibility, Portability, Interoperability Usability Familiarity, Debuggability, Maintainability Portability Container Runtime, Host OS, Cloud Provider, On-prem Security Isolation, Encryption, Secrets Management, Auditability
  • 9.
    Container Operations Development Lifecycle Sourcerepo, CI-CD, Artefact repo Container Orchestration Scheduling, Resource Management, Service Management BAU Operations Monitoring and Metrics, Maintenance, Debugging Did you hear that?
  • 10.
    In no particularorder… [ ] Schedulers and Orchestration [ ] Networking [ ] Security [ ] Operating Systems [ ] PaaS [ ] Storage [ ] Monitoring [ ] Container Integration and Container Deployment [ ] Miscellaneous
  • 11.
    In no particularorder… [ ] Schedulers and Orchestration [ ] General Blurb [ ] ECS [ ] Kubernetes [ ] Mesos [ ] Docker Swarm [ ] Orchestration Wars
  • 12.
    Schedulers – GeneralBlurb Cluster Machines Cluster State Information Monolothic Two-Level Shared State No Concurrency Pessimistic Concurrency (offers) Optimistic Concurrency (transactions) Scheduling Logic
  • 13.
    Docker Task Container Instance Amazon ECS Container ECS Agent ELB Internet ELB User/ Scheduler API Cluster Management Engine Task Container Docker Task Container Instance Container ECS Agent Task Container Docker Task Container Instance Container ECS Agent Task Container AZ 1 AZ 2 Key/Value Store Agent Communication Service ECS
  • 14.
    Mesos Master Marathon ZooKeeper Mesos + Marathon MesosSlaves Long Running Tasks Jobs Coordination & Configuration
  • 15.
    Kubernetes Replication Controller API Server Kubernetes Master KubeletKubeProxy Docker Container Container Pod Pod Kubelet KubeProxy Docker Container Container Pod Pod Kubernetes Cluster etcd
  • 16.
  • 17.
  • 18.
    In no particularorder… [X] Schedulers and Orchestration [ ] Networking [ ] Security [ ] Operating Systems [ ] PaaS [ ] Storage [ ] Monitoring [ ] Container Integration and Container Deployment [ ] Miscellaneous
  • 19.
    Container Networking Dom 0 InstanceInstance Instance OS OS OS Container Runtime Container Runtime Container Runtime Container Container Container Container Container Container
  • 20.
    Overlay all ofthe thingz • Flannel • Calico • WeaveNet • Swarm Mode
  • 21.
  • 22.
    Mode Swarm Mode Manager Swarm Mode Node TLSCA Load Balancing Service Discovery Distributed Store Docker Engine Libnetwork Volumes Plugins Container Runtime
  • 23.
    In no particularorder… [X] Schedulers and Orchestration [X] Networking [ ] Security [ ] Operating Systems [ ] PaaS [ ] Storage [ ] Monitoring [ ] Container Integration and Container Deployment [ ] Miscellaneous
  • 24.
  • 25.
    Host Security • Lockit down! • Namespaces and cgroups are your friends • Select few belong to docker UNIX group • SELinux is also your friend • Docker daemon runs as root!
  • 26.
    Whale-say "Only trusted usersshould be allowed to control your Docker daemon"
  • 27.
    Docker daemon security •Do not run in privileged mode • Lock down inter container comms –icc=false • Secure APIs with TLS certificates
  • 28.
    Whale-say “If you runDocker on a server, it is recommended to run exclusively Docker in the server, and move all other services within containers controlled by Docker”
  • 29.
    Container Image Security •Use a small selection of trusted images • Scan your images • CoreOS’s Clair scans Quay.io, • Docker Security Scanning works with Docker Trusted Registry • Red Hat has built a new scanner in Project Atomic for its Atomic Registry. • Other scanners are such as Aqua Peekr, Anchore, and Twistlock Trust work independently of specific registries
  • 30.
    Lot more prescriptiveadvice here… https://benchmarks.cisecurity.org/tools2/docker/CIS _Docker_1.6_Benchmark_v1.0.0.pdf
  • 31.
    In no particularorder… [X] Schedulers and Orchestration [X] Networking [X] Security [ ] Operating Systems [ ] PaaS [ ] Storage [ ] Monitoring [ ] Container Integration and Container Deployment [ ] Miscellaneous
  • 32.
    Micro OS • CoreOS •RancherOS • Ubuntu Snappy • RedHat Atom • VmWare Photon • ECS Optimized Amazon Linux RedHat Atomic VmWare Photon Ubuntu Snappy CoreOS RancherOS 395 MB 317 MB 215 MB 20 MB 150 MB
  • 33.
    In no particularorder… [X] Schedulers and Orchestration [X] Networking [X] Security [X] Operating Systems [ ] PaaS [ ] Storage [ ] Monitoring [ ] Container Integration and Container Deployment [ ] Miscellaneous
  • 34.
  • 35.
    Convox $ convox appscreate go-app $ convox deploy $ convox apps info go-app $ convox build --app go-app –d "Hello Build” $ convox releases promote RLYSUALSGCT $ convox ps $ convox scale main --count=2
  • 36.
    Docker Data Center UniversalControl Plane (UCP) Security Content Trust Docker Trusted Registry Orchestration Swarm Container Runtime Engine Operating System
  • 37.
  • 38.
    In no particularorder… [X] Schedulers and Orchestration [X] Networking [X] Security [X] Operating Systems [X] PaaS [ ] Storage [ ] Monitoring [ ] Container Integration and Container Deployment [ ] Miscellaneous
  • 39.
  • 40.
    In no particularorder… [X] Schedulers and Orchestration [X] Networking [X] Security [X] Operating Systems [X] PaaS [-] Storage [-] Monitoring [-] Container Integration and Container Deployment [-] Miscellaneous
  • 41.
    Demoz • Marathon scheduleron ECS (Credit : Ryosuke-san) • Convox • Docker Swarm • Weave Net and Weave Scope • ECS (ALB, Task AutoScaling, Task IAM Role)
  • 42.
    --------------------- T H AN K Y O U -------------------- ^__^ (oo)_______ (__) )/ ||----w | || ||