Kubernetes in Docker
Tom Barlow
@tomwbarlow
About Me
• Tom Barlow (@tomwbarlow)
• Building enterprise products at
Docker, Inc. for the last 2.5 years
• Living and working in Galway
Agenda
1. Intro: the Docker Platform
2. Kubernetes in Docker
3. Demo: Kubernetes in Docker Desktop
4. General CE/EE Architectures
5. Demo: Kubernetes in Docker EE 2.0
6. EE: Topics on mixed workloads
Introduction
The Docker Platform
Traditional
Micro
services
ISV / COTS IoT
Big Data
ML
AI
...Serverless
Cloud VM Bare
Metal
Edge
Device
Docker Platform
Docker Momentum
Docker
Hosts
21.0M
Growth in Docker
job listings
77K%
Container
downloads
24B
Industry
Standards
Enterprise Momentum
Portability Agility Security
50% total cost savings
DEVELOPERS OPERATORS
Applications
Infrastructure
The Docker Platform in a nutshell
Docker Community Edition
Developers EnterpriseContainer Ecosystem
The Docker Innovation Model
Docker Enterprise Edition
9,149 Open Source Contributors 8800 PRs/Year
runc
Notary
Registry LibNetworkVPNKit
DataKit HyperKitCompose
Kubernetes in Docker
What is a container orchestrator?
Management of containers running in one or more container runtimes
Kubernetes
● Developed at Google
● Inspired by and resembles Borg & Omega
● The third container-management system built by Google
● v1.0 was released on July 21, 2015 (around since 2014)
● Commonly referred to as k8s or kube
● Greek for ‘Helmsman’ (pilot of a ship)
Kubernetes Primitives
Pod: Consists of one or more containers that run together
and share a process, network, filesystem namespace. Each
Pod has its own unique IP address on the cluster.
Controller: A reconciliation loop that ensures the system
matches the desired state by managing pods. (e.g. a
ReplicationController will ensure n pods of a given spec are
running)
Services: Targets a set of pods and provides a policy in
which they can be accessed over the network (e.g. for
external routing, load balancing etc.)
Kubernetes Components
Master:
● API Server: Allows access to query/change the cluster state
● Controller Manager: Controller reconciliation loops
● Scheduler: Makes decisions where Pods should run
● etcd: Stores cluster state
Worker:
● Kubelet: Manages Pod lifecycle running on a node
● Proxy: Routes incoming traffic to appropriate local
container using IP/Port
Docker Enterprise Edition
Docker Community Edition
containerd
1
2
3
4
The best container
development workflow
The best enterprise
container security and
management
Native Kubernetes
integration provides full
ecosystem
compatibility Industry-standard
container runtime
Docker with Swarm and Kubernetes
Test locally on Swarm
and Kubernetes
Develop with Docker
Community Edition
on your workstation
Deploy to production
in Swarm
Deploy to production
in Kubernetes
Docker Community Edition
All in one development for Swarm and Kubernetes
Demo: Kubernetes in
Docker Desktop
Linuxkit VM
Kubernetes CLI
Swarm Mode Kubernetes
etcd
Docker CLI
kubeadm
Kubernetes in Docker CE (Windows and Mac)
Compose
CRD
Single Docker Engine
vpnkitHost fs mounts hyperkit / hyperv
Docker EE 2.0: A conformant kubernetes distribution
Demo: Kubernetes in
Docker EE 2.0
Docker EE to include Kubernetes
Docker Enterprise Edition
Production Ready Windows and IBM P/Z Support
Pods, batch jobs, blue-green deployments,
horizontal pod auto-scaling
Docker Swarm Swarm-Mode Kubernetes
Private Image Registry
Secure Access and User
Management
App and Cluster Management
Image Security Scanning Content Trust and Verification
Policy Management
GUI
Universal Control Plane
Trusted Registry Kubernetes CLI
Docker Engine
Swarm-Mode
Docker Swarm Kubernetes
etcd
CA OIDC Provider
Docker CLI
Node Agent Reconciler
Kubernetes in Docker EE
Kubernetes Plugin Interfaces in Docker EE
● General:
○ Native API extensibility supported
○ Some apiserver/kubelet flags modifiable by users
● Networking:
○ Support for CNI plugin during install
○ Ingress
● Storage: Docker Volume Plugins supported via built-in flexvolume driver, CSI in future
● Metrics: Heapster Storage Backends or Prometheus
Docker EE Architectural Highlights
● Conformant Kubernetes components ran as Docker containers
● Swarm Managers are Kubernetes Masters
● Swarmkit node inventory is source of truth
● Cryptographic Node Identity and mTLS used throughout
- Easy High Availability provisioning
- Cryptographic node identity
Features Swarm Support
- Registry
- Content Trust
- Secure Scanning
- Clean upstream integration
- Full ecosystem compatibility
- Role Based Access Control
- Authorization, Authentication
- Node Segmentation
Secure Cluster Lifecycle
Secure Supply Chain
100% Interoperability
Secure Multi-tenancy
Management Dashboard
Kubernetes Support
Docker Enterprise Edition
Management for Swarm and Kubernetes
Uses of Kubernetes Plugin
Interfaces
Authentication
● X509 Client Certificates
○ Used for authentication of kubectl and the docker CLI via the “client bundle” feature
● OpenID Connect Identity Provider
○ GUI sessions use a custom identity provider and a token exchange service to
authenticate with the OIDC authentication plugin
Authorization
● All requests authorized via the Authorization Webhook Mode
● Custom RBAC system shared between Swarm and Kubernetes:
○ Users, Teams, Organizations, Service Accounts
○ Custom Roles
○ Hierarchical “Grants”
● No support for RBAC Mode (rbac.authorization.k8s.io) currently, future plans for API
translation
Resource Contention
● Allocatable Resources: The set of CPU and Memory resources available for scheduling by
an orchestrator on a single node
● Multiple orchestrators = Different definitions of allocatable resources
○ Docker Swarm: Respectful of CPU/Memory limits, but container cache may be stale
○ Docker Engine with Swarm-Mode: Only aware of its own reservations
○ Kubernetes: Effective handling of out-of-resource situations, but only for kubernetes
workloads
● When a node is at/near capacity:
○ All CPU shares throttled equally
○ The OS’s OOM killer kills processes
○ All orchestrators will reschedule on OOM, but potential workload interruption
Image Signing Policy Enforcement
● Enforces that all workloads deployed in the cluster have a fully qualified image reference
● Resolves image references to always include a digest
● Contacts the registry to ensure that the referenced image has been signed by an
authorized user.
In Summary...
● Docker EE and CE will include a conformant
Kubernetes distribution for Devs and Ops
Thank You!
@tomwbarlow
thomas.barlow@docker.com

Docker Dublin Meetup | 22 Feb 2018 | Docker + Kubernetes

  • 1.
    Kubernetes in Docker TomBarlow @tomwbarlow
  • 2.
    About Me • TomBarlow (@tomwbarlow) • Building enterprise products at Docker, Inc. for the last 2.5 years • Living and working in Galway
  • 3.
    Agenda 1. Intro: theDocker Platform 2. Kubernetes in Docker 3. Demo: Kubernetes in Docker Desktop 4. General CE/EE Architectures 5. Demo: Kubernetes in Docker EE 2.0 6. EE: Topics on mixed workloads
  • 4.
  • 5.
    Traditional Micro services ISV / COTSIoT Big Data ML AI ...Serverless Cloud VM Bare Metal Edge Device Docker Platform
  • 6.
    Docker Momentum Docker Hosts 21.0M Growth inDocker job listings 77K% Container downloads 24B Industry Standards
  • 7.
    Enterprise Momentum Portability AgilitySecurity 50% total cost savings
  • 8.
  • 9.
    Docker Community Edition DevelopersEnterpriseContainer Ecosystem The Docker Innovation Model Docker Enterprise Edition 9,149 Open Source Contributors 8800 PRs/Year
  • 10.
  • 11.
  • 12.
    What is acontainer orchestrator? Management of containers running in one or more container runtimes
  • 13.
    Kubernetes ● Developed atGoogle ● Inspired by and resembles Borg & Omega ● The third container-management system built by Google ● v1.0 was released on July 21, 2015 (around since 2014) ● Commonly referred to as k8s or kube ● Greek for ‘Helmsman’ (pilot of a ship)
  • 14.
    Kubernetes Primitives Pod: Consistsof one or more containers that run together and share a process, network, filesystem namespace. Each Pod has its own unique IP address on the cluster. Controller: A reconciliation loop that ensures the system matches the desired state by managing pods. (e.g. a ReplicationController will ensure n pods of a given spec are running) Services: Targets a set of pods and provides a policy in which they can be accessed over the network (e.g. for external routing, load balancing etc.)
  • 15.
    Kubernetes Components Master: ● APIServer: Allows access to query/change the cluster state ● Controller Manager: Controller reconciliation loops ● Scheduler: Makes decisions where Pods should run ● etcd: Stores cluster state Worker: ● Kubelet: Manages Pod lifecycle running on a node ● Proxy: Routes incoming traffic to appropriate local container using IP/Port
  • 17.
    Docker Enterprise Edition DockerCommunity Edition containerd 1 2 3 4 The best container development workflow The best enterprise container security and management Native Kubernetes integration provides full ecosystem compatibility Industry-standard container runtime Docker with Swarm and Kubernetes
  • 19.
    Test locally onSwarm and Kubernetes Develop with Docker Community Edition on your workstation Deploy to production in Swarm Deploy to production in Kubernetes Docker Community Edition All in one development for Swarm and Kubernetes
  • 20.
  • 21.
    Linuxkit VM Kubernetes CLI SwarmMode Kubernetes etcd Docker CLI kubeadm Kubernetes in Docker CE (Windows and Mac) Compose CRD Single Docker Engine vpnkitHost fs mounts hyperkit / hyperv
  • 22.
    Docker EE 2.0:A conformant kubernetes distribution
  • 23.
  • 24.
    Docker EE toinclude Kubernetes Docker Enterprise Edition Production Ready Windows and IBM P/Z Support Pods, batch jobs, blue-green deployments, horizontal pod auto-scaling Docker Swarm Swarm-Mode Kubernetes Private Image Registry Secure Access and User Management App and Cluster Management Image Security Scanning Content Trust and Verification Policy Management
  • 25.
    GUI Universal Control Plane TrustedRegistry Kubernetes CLI Docker Engine Swarm-Mode Docker Swarm Kubernetes etcd CA OIDC Provider Docker CLI Node Agent Reconciler Kubernetes in Docker EE
  • 26.
    Kubernetes Plugin Interfacesin Docker EE ● General: ○ Native API extensibility supported ○ Some apiserver/kubelet flags modifiable by users ● Networking: ○ Support for CNI plugin during install ○ Ingress ● Storage: Docker Volume Plugins supported via built-in flexvolume driver, CSI in future ● Metrics: Heapster Storage Backends or Prometheus
  • 27.
    Docker EE ArchitecturalHighlights ● Conformant Kubernetes components ran as Docker containers ● Swarm Managers are Kubernetes Masters ● Swarmkit node inventory is source of truth ● Cryptographic Node Identity and mTLS used throughout
  • 28.
    - Easy HighAvailability provisioning - Cryptographic node identity Features Swarm Support - Registry - Content Trust - Secure Scanning - Clean upstream integration - Full ecosystem compatibility - Role Based Access Control - Authorization, Authentication - Node Segmentation Secure Cluster Lifecycle Secure Supply Chain 100% Interoperability Secure Multi-tenancy Management Dashboard Kubernetes Support Docker Enterprise Edition Management for Swarm and Kubernetes
  • 29.
    Uses of KubernetesPlugin Interfaces
  • 30.
    Authentication ● X509 ClientCertificates ○ Used for authentication of kubectl and the docker CLI via the “client bundle” feature ● OpenID Connect Identity Provider ○ GUI sessions use a custom identity provider and a token exchange service to authenticate with the OIDC authentication plugin
  • 31.
    Authorization ● All requestsauthorized via the Authorization Webhook Mode ● Custom RBAC system shared between Swarm and Kubernetes: ○ Users, Teams, Organizations, Service Accounts ○ Custom Roles ○ Hierarchical “Grants” ● No support for RBAC Mode (rbac.authorization.k8s.io) currently, future plans for API translation
  • 32.
    Resource Contention ● AllocatableResources: The set of CPU and Memory resources available for scheduling by an orchestrator on a single node ● Multiple orchestrators = Different definitions of allocatable resources ○ Docker Swarm: Respectful of CPU/Memory limits, but container cache may be stale ○ Docker Engine with Swarm-Mode: Only aware of its own reservations ○ Kubernetes: Effective handling of out-of-resource situations, but only for kubernetes workloads ● When a node is at/near capacity: ○ All CPU shares throttled equally ○ The OS’s OOM killer kills processes ○ All orchestrators will reschedule on OOM, but potential workload interruption
  • 33.
    Image Signing PolicyEnforcement ● Enforces that all workloads deployed in the cluster have a fully qualified image reference ● Resolves image references to always include a digest ● Contacts the registry to ensure that the referenced image has been signed by an authorized user.
  • 34.
    In Summary... ● DockerEE and CE will include a conformant Kubernetes distribution for Devs and Ops
  • 35.