Build your own container-based system with the Moby project Docker Community Edition—an open source product that lets you build, ship, and run containers—is an assembly of modular components built from an upstream open source project called Moby. Moby provides a “Lego set” of dozens of components, the framework for assembling them into specialized container-based systems, and a place for all container enthusiasts to experiment and exchange ideas. Patrick Chanezon and Mindy Preston explain how you can leverage the Moby project to assemble your own specialized container-based system, whether for IoT, cloud, or bare-metal scenarios. Patrick and Mindy explore Moby’s framework, components, and tooling, focusing on two components: LinuxKit, a toolkit to build container-based Linux subsystems that are secure, lean, and portable, and InfraKit, a toolkit for creating and managing declarative, self-healing infrastructure. Along the way, they demo how to use Moby, LinuxKit, InfraKit, and other components to quickly assemble full-blown container-based systems for several use cases and deploy them on various infrastructures.

1. Patrick Chanezon, @chanezon
David Chung, @dchungsf
Mindy Preston, @mindypreston
Build your own container-based system
with the Moby project
May 2017

2. French
Polyglot
Platforms
Software Plumber
San Francisco
Developer Relations
@chanezon

3. Docker

4. The world needs
tools of mass innovation

5. A programmable Internet would be the ultimate
tool of mass innovation

6. A commercial product,
built on
a development platform,
built on
infrastructure,
built on
standards.
Docker is building a stack to program the Internet

7. Docker is building a stack to program the Internet
CE
EE

8. enterprise edition
Ubuntu
Fedora
Mac
Azure
CentOS
Windows 10
AWS
Debian
community edition
Ubuntu
Windows Server
Azure
CentOS
Suse
Red Hat
AWS
Oracle Linux

9. Orchestration
Container Runtime
OS
Infrastructure Management
Container Platform Layers
Application Services

10. Docker is a platform made of components
Raft Store
Node
Identity
Secrets
Routing
Mesh
Overlay
Networking
Swarm Orchestration
Engine
Application Services

11. 12,000,000,000
11,000,000,000
10,000,000,000
9,000,000,000
8,000,000,000
7,000,000,000
6,000,000,000
5,000,000,000
4,000,000,000
3,000,000,000
2,000,000,000
1,000,000,000
Notary
runC
containerd
HyperKit , VPNKit, DataKit
SwarmKit
libcontainer
libnetwork
InfraKit
2013 2014 2015 2016 2017
1M
2014
PULLS
1B
2015
PULLS
6B
2016
PULLS
12B
2017
PULLS
linuxKit

12. LinuxKit
A toolkit for building secure, portable and lean
operating systems for containers

13. Taking Docker
multi-platform
“I want Docker for X”

14. Desktop Server Cloud
I want Docker for…

15. Not every platform
provides a Linux
subsystem

16. Not every platform provides a Linux subsystem
Orchestration
Container Runtime
Linux Subsystem
Infrastructure Management
Application Services

17. The container
movement needs a
secure, lean,
portable
subsystem

18. The container movement needs
a secure, lean, portable Linux subsystem.
introducing

19. Only works with
containers
- Smaller attack
surface
- Immutable
infrastructure
- Sandboxed system
services
- Specialized patches
and configuration
Incubator for
security innovations
- Wireguard,
Landlock, KSPP
- MirageOS type
safe system
daemons
Community-first
security process
- Linux is too big
for any one
company to
secure it
- Participate in
existing Linux
security efforts
1. LinuxKit: a SECURE Linux subsystem

20. - Minimal size, minimal boot time
- All system services are containers
- Everything can be removed or
replaced
2. LinuxKit: a LEAN Linux subsystem

21. - Desktop, server, IoT, mainframe
- Intel & ARM
- Bare metal & virtualized
3. LinuxKit: a PORTABLE Linux subsystem

22. Docker and Microsoft collaborate to bring
Linux containers to Windows
+ +

23. https://github.com/linuxkit/linuxkit
Get Started with LinuxKit

24. Moby
An open framework to assemble specialized
container systems without reinventing the wheel.

25. Pioneers 2013 - 2014

26. Production Model: open-source!

27. Use case: cloud native apps on Linux server
Early Adopters 2015 - 2016

28. Production Model: OPEN COMPONENTS

29. Mainstream 2017 - 2018
Containers are spreading to every category of computing:
server, datacenter, cloud, IoT, desktop, mobile…

30. Case study:
Specializing Docker for the mainstream
Desktop Server Cloud

31. The open component model shows its limits…

32. The auto industry has solved this problem:
COMMON ASSEMBLIES.

33. Scaling the Docker production model: share components AND
ASSEMBLIES.

34. It’s time to take our ecosystem to the next level…
By collaborating on components AND COMMON ASSEMBLIES.

36. – Library of 80+ components
– Package your own
components as containers
– Reference assemblies
deployed on millions of nodes
– Create your own assemblies
or start from an existing one
A framework to assemble
specialized container
systems without
reinventing the wheel.

37. Docker uses Moby for its
open-source
– Thousands of contributors,
hundreds of patches/week
– Component development
– Specialized assembly
development
– Integration tests
– Architecture design
– Integration with other projects
– Experimentation and bleeding
edge features

38. Docker uses Moby for its
open-source...
and so can you!
– Community-run
– Open governance inspired by
the Fedora project
– Plays well with existing
projects - no donation
necessary!

39. Moby and Docker

40. What it means for you
Moby helps you
innovate without tying
you to Docker
System BuildersDocker Users
Docker will better leverage
the ecosystem to innovate
faster for you

41. Moby transforms multi-month R&D
projects into weekend projects.

42. locked-down
Linux with
remote
attestation
Weekend
project #1:
Notary

43. custom CI/CD
stack
Weekend
project #2:
Notary Registry Docker Builder
+

44. custom CI/CD
stack
+ Debian
+ Terraform
Weekend
project #3:
Notary Docker Builder
+
Registry

45. “RedisOS”
Weekend
project #4:

46. "RedisOS"
for Windows
"RedisOS"
for Mac
"RedisOS"
for bare metal
HyperKit
bare metal

47. Etcd clustering
on Google Cloud
Weekend
project #5:

48. SSHD
Kubernetes on
the Mac
Weekend
project #6:
HyperKit

49. Getting Started
- Blog https://mobyproject.org/blog
- Twitter @moby
- Github moby/moby

50. Let’s take containers mainstream!

52. InfraKit
A toolkit for building declarative, self-healing
infrastructure.

53. What is it?
53
• Launched at LinuxCon, Berlin in October, 2016.
• Toolkit for building declarative, self-managing
distributed applications
• Active management with active controllers
• scaling groups, rolling updates
• monitoring / health checks
• connecting nodes to L4 / ingress
• Declarative infrastructure

54. Architecture
CLI
API

55. container orchestration
Where does it fit?
55
kubectl run nginx --image=nginx
gcloud container node-pools list --zone us-
central1-f --cluster MyWorkers
aws autoscaling update-auto-scaling-group
--auto-scaling-group-name MyWorkers
docker create service nginx …
infrakit group describe workers
az vmss create --resource-group vmss-
test-1 --name MyWorkers
container orchestration
infrastructure orchestrationinfrastructure orchestration
list, err :=
group.Controller.Describe(“workers”)

56. App Opscontainer orchestrationApp Ops
One console across environments
56
kubectl run nginx --image=nginx docker create service nginx …
infrakit group describe workers
container orchestration
infrastructure orchestration
list, err :=
group.Controller.Describe(“workers”)
AWS RackHDAZ GCP OneVIEWMAASKVM VMW
Cloud Ops Hardware OpsCluster Ops

57. Configuration
Example config file (zk.conf): Group configuration = Instance + Flavor
{
"Properties": {
/* raw configuration */
}
}
{
"groups" : {
"my_zookeeper_nodes" : {
"Properties" : {
"Instance" : {
"Plugin": "instance-vagrant",
"Properties": {
"Box": "bento/ubuntu-16.04"
}
},
"Flavor" : {
"Plugin": "flavor-zookeeper",
"Properties": {
"type": "member",
"IPs": ["192.168.1.200", "192.168.1.201", "192.168.1.202"]
}
}
}
}
}
}

58. Current Status

59. Support more platforms
59
• Compute:
• Bare-metal: HP OneView, MAAS, RackHD
• Public cloud: AWS, GCP
• MacOS X (HyperKit); Docker containers
• Coming soon: Azure, IBM, Digital Ocean,
Packet, libvirt
• Other resource types
• AWS - vpc, subnets, gateways, etc.

60. Improve usability
60
• Templates
• Complex scripts and configuration in any format;
no more escape quotes in JSON
• Fetch templates from remote repositories
• Playbooks
• CLI - flags, prompts — config driven and
dynamic
• Share “playbooks” from remote repositories

61. Improve core system
61
• High Availability — Swarm Mode or etcd
• New Plugin types — Metadata and Events
• Metadata: cluster-wide sysfs and reflection
• Events - publish / subscribe
• Remote client access:
infrakit -H host:port to remote cluster

62. Road Map

63. Use Cases
63
• Support container orchestration
• bootstrapping + day N management
• API for cluster autoscaling
• k8s, Docker Swarm Mode
• Bare-metal + GPU provisioning
• IoT — LinuxKit integration / custom kernel
deployment

64. Improve usability
64
• Finalize API / Schema for 1.0
• Make it easy to consume
• Simplify setup - fewer daemons and binaries
• Embeddable / vendor API
• Sensible CLI for stable / experimental features
• Make it easy to extend / contribute
• metadata / instance plugins
• playbooks / reusable templates
• community CI / compatibility testing
• Documentation

65. Improve core system
65
• Provisioning of diverse resource types
• networks / proxies / load balancers
• GPU
• Stability / performance of core controllers
• Asynchronous messaging - mqtt, natsd, amqp
• Monitoring + Health check SPI

66. Support more platforms
66
• Direct libvirt / KVM / CUDA
• Better bare-metal / hardware ops integration
• Kernel image build pipeline — LinuxKit
Build, test, and deploy clusters from infrastructure
definitions to kernel images

67. Get involved
https://github.com/docker/infrakit
dockercommunity.slack.com:
#infrakit

69. Learn More at OSCON
- Mindy Preston, Amir Chaudhry’s
“MirageOS 3: Smaller, lighter, and more transparent”
Wednesday 4:15 pm
- David Chung, Bill Farner
“InfraKit: A toolkit for infrastructure orchestration”
Thursday 11 am

70. THANK YOU