Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Moby KubeCon 2017


Published on

Moby is an open source project providing a "LEGO set" of dozens of components, the framework to assemble them into specialized container-based systems, and a place for all container enthusiasts to experiment and exchange ideas.
One of these assemblies is Docker CE, an open source product that lets you build, ship, and run containers.

This talk will explain how you can leverage the Moby project to assemble your own specialized container-based system, whether for IoT, cloud or bare metal scenarios.
We will cover Moby itself, the framework, and tooling around the project, as well as many of it’s components: LinuxKit, InfraKit, containerd, SwarmKit, Notary.
Then we will present a few use cases and demos of how different companies have leveraged Moby and some of the Moby components to create their own container-based systems.

Video at

Published in: Software
  • Be the first to comment

  • Be the first to like this

Moby KubeCon 2017

  1. 1. Patrick Chanezon, @chanezon Justin Cormack, @justincormack Stephen Day, @stevvooe Build your own container-based system with the December 2017
  2. 2. French Polyglot Platforms Software Plumber San Francisco Developer Relations @chanezon
  3. 3. Docker
  4. 4. Docker Enterprise Edition Container Runtime Orchestration Developer tools Management Services Docker Community Edition Swarm containerd
  5. 5. enterprise edition Ubuntu Fedora Mac Azure CentOS Windows 10 AWS Debian community edition Ubuntu Windows Server Azure CentOS Suse Red Hat AWS Oracle Linux
  6. 6. Image Registry CI/CD Security scan & sign Traditional Third Party Microservices docker store DEVELOPERS IT OPERATIONS Control Plane Docker EE Container Platform to Modernize Traditional Apps and beyond More Info:
  7. 7. Docker Enterprise Edition Docker Community Edition containerd The best container development workflow The best enterprise container security and management Docker: Now Powered by Swarm and Kubernetes Native Kubernetes integration provides full ecosystem compatibility Industry-standard container runtime
  8. 8. Docker Community Edition Developers EnterpriseContainer Ecosystem The Docker Innovation Model Docker Enterprise Edition 9,149 Open Source Contributors 8800 PRs/Year
  9. 9. runc Notary Registry LibNetworkVPNKit DataKit HyperKitCompose
  10. 10. November 2016 containerd 1.0 development starts at KubeCon March 2017 containerd contributed to CNCF at CloudNativeCon April 2017 LinuxKit + Kubernetes at DockerCon Kubernetes + cri-containerd at Moby Summit September 2017 libnetwork + CNI at Open Source Summit October 2017 Notary submitted to CNCF contribution Q4 2017 Beta of Docker with Kubernetes support An Open Source Integration 1 Year in the Making
  11. 11. containers and Notary are CNCF projects
  12. 12. What happens when the two largest open source container projects collaborate?
  14. 14. Governance: BDFL -> TSC
  15. 15. containerd An open and reliable container runtime
  16. 16. containerd 1.0 shipped yesterday!
  17. 17. Early History APRIL 2016 Containerd “0.2” announced, Docker 1.11 DECEMBER 2016Announce expansion of containerd OSS project, roadmap to 1.0 Management/Supervisor for the OCI runc executor Containerd 1.0: A core container runtime project for the industry
  18. 18. runc containerd Why Containerd 1.0? ▪ Continue projects spun out from monolithic Docker engine ▪ Expected use beyond Docker engine (Kubernetes CRI) ▪ Donation to foundation for broad industry collaboration ▫ Similar to runc/libcontainer and the OCI
  19. 19. Technical Goals/Intentions ▪ Clean gRPC-based API + client library ▪ Full OCI support (runtime and image spec) ▪ Stability and performance with tight, well- defined core of container function ▪ Decoupled systems (image, filesystem, runtime) for pluggability, reuse
  20. 20. Requirements - A la carte: use only what is required - Runtime agility: fits into different platforms - Pass-through container configuration (direct OCI) - Decoupled - Use known-good technology - OCI container runtime and images - gRPC for API - Prometheus for Metrics
  21. 21. Use cases - CURRENT - Docker (moby) - Kubernetes (cri- containerd) - SwarmKit (experimental) - LinuxKit - BuildKit - FUTURE/POTENTIAL - IBM Cloud/Bluemix - OpenFaaS - {your project here}
  22. 22. containerd 1.0 facts and figures • 1994 GitHub stars, 401 forks • 108 contributors • 8 maintainers from independents and and member companies alike including Docker, Google, IBM, ZTE and ZJU .
  23. 23. Resources To participate in containerd: • Getting Started with containerd: getting-started/ • Roadmap: • Scope table: • Architecture document:
  24. 24. LinuxKit A toolkit for building secure, portable and lean operating systems for containers
  25. 25. Get Started with LinuxKit
  26. 26. Moby An open framework to assemble specialized container systems without reinventing the wheel.
  27. 27. Moby and Docker
  28. 28. What it means for you Moby helps you innovate without tying you to Docker System BuildersDocker Users Docker will better leverage the ecosystem to innovate faster for you
  29. 29. Moby transforms multi-month R&D projects into weekend projects.
  30. 30. “RedisOS” Weekend project #4:
  31. 31. "RedisOS" for Windows "RedisOS" for Mac "RedisOS" for bare metal HyperKit bare metal
  32. 32. SSHD Kubernetes on the Mac Weekend project #6: HyperKit
  33. 33.
  34. 34. Getting Started - Blog - - Twitter @moby - Github moby/moby
  35. 35. InfraKit A toolkit for building declarative, self-healing infrastructure.
  36. 36. What is it? 47 • Launched at LinuxCon, Berlin in October, 2016. • Toolkit for building declarative, self-managing distributed applications • Active management with active controllers • scaling groups, rolling updates • monitoring / health checks • connecting nodes to L4 / ingress • Declarative infrastructure • Proposal to contribute to CNCF 6/20, too soon
  37. 37. What is InfraKit 48 • Toolkit for infrastructure automation • Provisioning and management services for higher-level systems • Focus on patterns and automation: • Convergence to declarative specification • Scaling groups, rolling updates • Infrastructure metadata, events • Immutable infrastructure Application Definition/ Development Orchestration & Management Runtime Provisioning Infrastructure (Bare Metal/Cloud)
  38. 38. InfraKit in a Cloud Native Ecosystem 49 • Immutable nodes + attached storage • OS Images - LinuxKit integration • Devops Deployment Tooling & Provisioning • Infrastructure Automation • Compute - rolling updates, scaling groups • Storage • Network Provisioning layer + infrastructure automation services
  39. 39. InfraKit Use Cases 50 • Day-0 (install), Day-1 (configure) of container orchestrators • Docker Swarm - Docker for GCP, AWS, Appcelerator/AMP • Kubernetes • Day-N automation of infrastructure - provisioning, rolling updates and capacity scaling. • A cloud provider for Kubernetes Cluster Autoscaler • GPU cluster provisioning • LinuxKit integration for building, deployment of custom OS on bare-metal or virtualized infrastructure (video).
  40. 40. InfraKit Architecture 51 Group Controller Metadata Exporter Instance Plugin (T3) Infrastructure API Manager Flavor Plugin (F2) Spec Store infrakit CLI Leadership Templates Playbooks Event Publisher Resource Controller Application / Orchestration API Dependency Graph Template Processor Node 1 (T1) Node 1 (T3) Node 1 (T1) Node 1 (T1) Node (T3) Instance Plugin (T2) Instance Plugin (T1) Flavor Plugin (F1) instance (T2) instance (T2) Volume (T2) Network (T1) Application Definition/ Development Orchestration & Management Runtime Provisioning Infrastructure (Bare Metal/Cloud) Metadata Plugin (M1) Event Plugin (E1) Event Plugin (E1) Metadata Plugin (M1) scale drain join provision/ configure destroy ● Active controllers ● Modular, plugin-based ● Defined SPI ● Customizable, contextual CLI
  41. 41. InfraKit Deployment 52 CLI API Control Plane • High availability, single leader • Can share leader election / spec storage with higher-level systems: • Docker swarm mode • etcd (k8s) • As Docker or containerd / oci containers • Typically “embedded” in control plane of higher systems as “system” containers (e.g. LinuxKit image)
  42. 42. InfraKit Community: active and growing • Made public at LinuxCon, Berlin in October, 2016 •1.5K Github stars, 140+ forks •16 infrastructure providers •4 maintainers, 4 companies (Docker, IBM, NTT, Axway) •25 contributors total, 200+ members on slack •460+ commits, 7 releases, ~50 commits / month •Meetups: Moby Project Summit, April 20, 2017; Next: June 19, 2017 53
  43. 43. InfraKit Community 54 source:
  44. 44. InfraKit - Why CNCF • Aligned with CNCF goals – Cloud-native: container packaged, micro-services oriented – Dynamic, self-healing for cloud-native, distributed services • Enhancing & complementary to CNCF projects – Common infrastructure provisioning and automation – Kubernetes: cluster autoscaler – Prometheus: infrastructure monitoring & automated remediation 55
  45. 45. Status September 2017
  46. 46. Infrakit Update - September, 2017 • Provision AWS spot instances (672 @YujiOshima) • Multi-Zone / Multi-Cloud / Multi-Tiered provisioning (652, 671 @chungers, 668 @YujiOshima) • Improved Kubernetes support (676 @YujiOshima) • Improved Terraform integration (651, 663, 670 @kaufers) • Docker Swarm Ingress controller (621 @chungers)
  47. 47. Example: build an autoscaling group ● Pick a plugin to create instances ● Add flavor plugin ● Embed config inside definition of a group. ID: group/workers Properties: Instance: Plugin: terraform Properties: // terraform config here Flavor: Plugin: kubernetes/worker Properties: // config add-on, etc. terraform kubernetes configs Group RPC API infrastructure API Client
  48. 48. … across zones / clouds ● Wrap instance plugins with Selector ● Selector selects plugin to provision, based on weights or spread evenly. ID: group/workers Properties: Instance: Plugin: selector/weighted Properties: aws-us-east/workers: gcp-us-central/workers: Options: - aws-us-east:80 - gcp-us-central:20 Flavor: Plugin: kubernetes/worker Properties: // config add-on, etc. aws-us-east kubernetes configs Group RPC API Client gcpaws gcp-us-central 80% 20%
  49. 49. … with provisioning priorities ● Tiered selector is just another Instance ● Selects one option after another until provisioning succeeds. ID: group/workers Properties: Instance: Plugin: selector/tiered Properties: Plugin: vsphere/on-prem-workers: Properties: // ... Plugin: aws/ec2-spot-instance: Properties: // spot price... Plugin: aws/ec2-instance: Properties: // on-demand… Flavor: Plugin: kubernetes/worker ... on-prem: vsphere kubernetes configs Group RPC API Client cloud: AWS spot cloud: AWS on-demand
  50. 50. Get involved #infrakit
  51. 51. Learn More - - - -
  52. 52. THANK YOU