Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Oscon 2017: Build your own container-based system with the Moby project

3,315 views

Published on

Build your own container-based system
with the Moby project

Docker Community Edition—an open source product that lets you build, ship, and run containers—is an assembly of modular components built from an upstream open source project called Moby. Moby provides a “Lego set” of dozens of components, the framework for assembling them into specialized container-based systems, and a place for all container enthusiasts to experiment and exchange ideas.

Patrick Chanezon and Mindy Preston explain how you can leverage the Moby project to assemble your own specialized container-based system, whether for IoT, cloud, or bare-metal scenarios. Patrick and Mindy explore Moby’s framework, components, and tooling, focusing on two components: LinuxKit, a toolkit to build container-based Linux subsystems that are secure, lean, and portable, and InfraKit, a toolkit for creating and managing declarative, self-healing infrastructure. Along the way, they demo how to use Moby, LinuxKit, InfraKit, and other components to quickly assemble full-blown container-based systems for several use cases and deploy them on various infrastructures.

Published in: Technology
  • Want to preview some of our plans? You can get 50 Woodworking Plans and a 440-Page "The Art of Woodworking" Book... Absolutely FREE ♥♥♥ http://tinyurl.com/y3hc8gpw
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Want to preview some of our plans? You can get 50 Woodworking Plans and a 440-Page "The Art of Woodworking" Book... Absolutely FREE ●●● http://tinyurl.com/yy9yh8fu
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Oscon 2017: Build your own container-based system with the Moby project

  1. 1. Patrick Chanezon, @chanezon David Chung, @dchungsf Mindy Preston, @mindypreston Build your own container-based system with the Moby project May 2017
  2. 2. French Polyglot Platforms Software Plumber San Francisco Developer Relations @chanezon
  3. 3. Docker
  4. 4. The world needs tools of mass innovation
  5. 5. A programmable Internet would be the ultimate tool of mass innovation
  6. 6. A commercial product, built on a development platform, built on infrastructure, built on standards. Docker is building a stack to program the Internet
  7. 7. Docker is building a stack to program the Internet CE EE
  8. 8. enterprise edition Ubuntu Fedora Mac Azure CentOS Windows 10 AWS Debian community edition Ubuntu Windows Server Azure CentOS Suse Red Hat AWS Oracle Linux
  9. 9. Orchestration Container Runtime OS Infrastructure Management Container Platform Layers Application Services
  10. 10. Docker is a platform made of components Raft Store Node Identity Secrets Routing Mesh Overlay Networking Swarm Orchestration Engine Application Services
  11. 11. 12,000,000,000 11,000,000,000 10,000,000,000 9,000,000,000 8,000,000,000 7,000,000,000 6,000,000,000 5,000,000,000 4,000,000,000 3,000,000,000 2,000,000,000 1,000,000,000 Notary runC containerd HyperKit , VPNKit, DataKit SwarmKit libcontainer libnetwork InfraKit 2013 2014 2015 2016 2017 1M 2014 PULLS 1B 2015 PULLS 6B 2016 PULLS 12B 2017 PULLS linuxKit
  12. 12. LinuxKit A toolkit for building secure, portable and lean operating systems for containers
  13. 13. Taking Docker multi-platform “I want Docker for X”
  14. 14. Desktop Server Cloud I want Docker for…
  15. 15. Not every platform provides a Linux subsystem
  16. 16. Not every platform provides a Linux subsystem Orchestration Container Runtime Linux Subsystem Infrastructure Management Application Services
  17. 17. The container movement needs a secure, lean, portable subsystem
  18. 18. The container movement needs a secure, lean, portable Linux subsystem. introducing
  19. 19. Only works with containers - Smaller attack surface - Immutable infrastructure - Sandboxed system services - Specialized patches and configuration Incubator for security innovations - Wireguard, Landlock, KSPP - MirageOS type safe system daemons Community-first security process - Linux is too big for any one company to secure it - Participate in existing Linux security efforts 1. LinuxKit: a SECURE Linux subsystem
  20. 20. - Minimal size, minimal boot time - All system services are containers - Everything can be removed or replaced 2. LinuxKit: a LEAN Linux subsystem
  21. 21. - Desktop, server, IoT, mainframe - Intel & ARM - Bare metal & virtualized 3. LinuxKit: a PORTABLE Linux subsystem
  22. 22. Docker and Microsoft collaborate to bring Linux containers to Windows + +
  23. 23. https://github.com/linuxkit/linuxkit Get Started with LinuxKit
  24. 24. Moby An open framework to assemble specialized container systems without reinventing the wheel.
  25. 25. Pioneers 2013 - 2014
  26. 26. Production Model: open-source!
  27. 27. Use case: cloud native apps on Linux server Early Adopters 2015 - 2016
  28. 28. Production Model: OPEN COMPONENTS
  29. 29. Mainstream 2017 - 2018 Containers are spreading to every category of computing: server, datacenter, cloud, IoT, desktop, mobile…
  30. 30. Case study: Specializing Docker for the mainstream Desktop Server Cloud
  31. 31. The open component model shows its limits…
  32. 32. The auto industry has solved this problem: COMMON ASSEMBLIES.
  33. 33. Scaling the Docker production model: share components AND ASSEMBLIES.
  34. 34. It’s time to take our ecosystem to the next level… By collaborating on components AND COMMON ASSEMBLIES.
  35. 35. – Library of 80+ components – Package your own components as containers – Reference assemblies deployed on millions of nodes – Create your own assemblies or start from an existing one A framework to assemble specialized container systems without reinventing the wheel.
  36. 36. Docker uses Moby for its open-source – Thousands of contributors, hundreds of patches/week – Component development – Specialized assembly development – Integration tests – Architecture design – Integration with other projects – Experimentation and bleeding edge features
  37. 37. Docker uses Moby for its open-source... and so can you! – Community-run – Open governance inspired by the Fedora project – Plays well with existing projects - no donation necessary!
  38. 38. Moby and Docker
  39. 39. What it means for you Moby helps you innovate without tying you to Docker System BuildersDocker Users Docker will better leverage the ecosystem to innovate faster for you
  40. 40. Moby transforms multi-month R&D projects into weekend projects.
  41. 41. locked-down Linux with remote attestation Weekend project #1: Notary
  42. 42. custom CI/CD stack Weekend project #2: Notary Registry Docker Builder +
  43. 43. custom CI/CD stack + Debian + Terraform Weekend project #3: Notary Docker Builder + Registry
  44. 44. “RedisOS” Weekend project #4:
  45. 45. "RedisOS" for Windows "RedisOS" for Mac "RedisOS" for bare metal HyperKit bare metal
  46. 46. Etcd clustering on Google Cloud Weekend project #5:
  47. 47. SSHD Kubernetes on the Mac Weekend project #6: HyperKit
  48. 48. Getting Started - Blog https://mobyproject.org/blog - Twitter @moby - Github moby/moby
  49. 49. Let’s take containers mainstream!
  50. 50. InfraKit A toolkit for building declarative, self-healing infrastructure.
  51. 51. What is it? 53 • Launched at LinuxCon, Berlin in October, 2016. • Toolkit for building declarative, self-managing distributed applications • Active management with active controllers • scaling groups, rolling updates • monitoring / health checks • connecting nodes to L4 / ingress • Declarative infrastructure
  52. 52. Architecture CLI API
  53. 53. container orchestration Where does it fit? 55 kubectl run nginx --image=nginx gcloud container node-pools list --zone us- central1-f --cluster MyWorkers aws autoscaling update-auto-scaling-group --auto-scaling-group-name MyWorkers docker create service nginx … infrakit group describe workers az vmss create --resource-group vmss- test-1 --name MyWorkers container orchestration infrastructure orchestrationinfrastructure orchestration list, err := group.Controller.Describe(“workers”)
  54. 54. App Opscontainer orchestrationApp Ops One console across environments 56 kubectl run nginx --image=nginx docker create service nginx … infrakit group describe workers container orchestration infrastructure orchestration list, err := group.Controller.Describe(“workers”) AWS RackHDAZ GCP OneVIEWMAASKVM VMW Cloud Ops Hardware OpsCluster Ops
  55. 55. Configuration Example config file (zk.conf): Group configuration = Instance + Flavor { "Properties": { /* raw configuration */ } } { "groups" : { "my_zookeeper_nodes" : { "Properties" : { "Instance" : { "Plugin": "instance-vagrant", "Properties": { "Box": "bento/ubuntu-16.04" } }, "Flavor" : { "Plugin": "flavor-zookeeper", "Properties": { "type": "member", "IPs": ["192.168.1.200", "192.168.1.201", "192.168.1.202"] } } } } } }
  56. 56. Current Status
  57. 57. Support more platforms 59 • Compute: • Bare-metal: HP OneView, MAAS, RackHD • Public cloud: AWS, GCP • MacOS X (HyperKit); Docker containers • Coming soon: Azure, IBM, Digital Ocean, Packet, libvirt • Other resource types • AWS - vpc, subnets, gateways, etc.
  58. 58. Improve usability 60 • Templates • Complex scripts and configuration in any format; no more escape quotes in JSON • Fetch templates from remote repositories • Playbooks • CLI - flags, prompts — config driven and dynamic • Share “playbooks” from remote repositories
  59. 59. Improve core system 61 • High Availability — Swarm Mode or etcd • New Plugin types — Metadata and Events • Metadata: cluster-wide sysfs and reflection • Events - publish / subscribe • Remote client access: infrakit -H host:port to remote cluster
  60. 60. Road Map
  61. 61. Use Cases 63 • Support container orchestration • bootstrapping + day N management • API for cluster autoscaling • k8s, Docker Swarm Mode • Bare-metal + GPU provisioning • IoT — LinuxKit integration / custom kernel deployment
  62. 62. Improve usability 64 • Finalize API / Schema for 1.0 • Make it easy to consume • Simplify setup - fewer daemons and binaries • Embeddable / vendor API • Sensible CLI for stable / experimental features • Make it easy to extend / contribute • metadata / instance plugins • playbooks / reusable templates • community CI / compatibility testing • Documentation
  63. 63. Improve core system 65 • Provisioning of diverse resource types • networks / proxies / load balancers • GPU • Stability / performance of core controllers • Asynchronous messaging - mqtt, natsd, amqp • Monitoring + Health check SPI
  64. 64. Support more platforms 66 • Direct libvirt / KVM / CUDA • Better bare-metal / hardware ops integration • Kernel image build pipeline — LinuxKit Build, test, and deploy clusters from infrastructure definitions to kernel images
  65. 65. Get involved https://github.com/docker/infrakit dockercommunity.slack.com: #infrakit
  66. 66. Learn More at OSCON - Mindy Preston, Amir Chaudhry’s “MirageOS 3: Smaller, lighter, and more transparent” Wednesday 4:15 pm - David Chung, Bill Farner “InfraKit: A toolkit for infrastructure orchestration” Thursday 11 am
  67. 67. THANK YOU

×