Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Extended and embedding: containerd update & project use cases

143 views

Published on

A talk given at FOSDEM 2020 in the containers devroom on the current status of the CNCF containerd project as well as a dive into the ways users are extending and embedding containerd in other platforms and projects.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Extended and embedding: containerd update & project use cases

  1. 1. Extending and embedding: containerd project use cases A 2020 FOSDEM containerd project update Phil Estes Distinguished Engineer & CTO, IBM Cloud Platform CNCF containerd project maintainer
  2. 2. State of containerd
  3. 3. What is containerd ● A “Container runtime” ○ Below platforms (Docker, Kubernetes) ○ Above lower level runtimes (runc, Kata, Firecracker, gVisor) ● Resource Manager ○ Container processes ○ Image artifacts ○ Filesystem snapshots ○ Metadata and dependencies ● Tightly scoped ○ 100% maintainer approval required to increase scope ○ Built-in CRI plugin only scope increase
  4. 4. State of containerd ● 5th project to graduate within the CNCF - February 2019 ● Broad support and contribution from across the ecosystem ○ Over 200 individual contributors; represent > 100 companies ○ 13 maintainers represent 9 different companies ● All major cloud providers using containerd ● Supports Linux and Windows platforms, multiple architectures ● Added sub-projects to governance (Rust-based ttrpc; image encryption)
  5. 5. containerd 1.3 ● Windows support for shim V2 API ● Device mapper snapshotter (Amazon Firecracker team contribution) ● New plugin interface for processing layers (encryption, compression) ● (CRI) Support for per-pod container shim
  6. 6. In progress ● Remote snapshotter for sharing snapshots in a cluster ● cgroups v2 ● Windows CRI ● Mount and resource management ● Image encryption
  7. 7. Who is using containerd? ● Public Clouds ● Kubernetes Infra ● End Users ● DevOps Tools ● Custom Sandboxes
  8. 8. How is containerd used? ● Library ○ Go client API ■ oras, BuildKit, Weaveworks Ignite, IBM Cloud Functions, OpenFaaS “faasd”, Alibaba PouchContainer ○ Extensibility ■ Amazon ECR resolver, Azure Teleport, remote snapshotters [cvmfs, stargz] ○ Imports/Subprojects (cri-o use of containerd/cgroups) ● Kubernetes Runtime ○ CRI-containerd ■ IBM Kubernetes Service, GKE, Ticketmaster, Alibaba, microk8s, KinD, k3s, AWS Fargate ● Daemon ○ Docker, BuildKit
  9. 9. Embedding/Extending Use Cases
  10. 10. Architecture
  11. 11. API
  12. 12. API - CRI - CRI gRPC API exposed from containerd - Kubelet can be configured to use containerd as runtime
  13. 13. API - containerd - gRPC API, used by Go client - Low level access to components - Mirrors internal component interfaces - Snapshots, Content, Containers, Task, Events, etc
  14. 14. Core
  15. 15. Backend
  16. 16. Plugins
  17. 17. Plugins - Backend
  18. 18. Plugins - Backend - No re-compilation required - Proxy plugins for content store and snapshotters - Runtime shims are separate binaries implementing shim interface
  19. 19. Plugins - Client 1. Override services with service options 2. Customize push and pull with remote options type RemoteOpt func WithImageHandler(h images.Handler) RemoteOpt func WithImageHandlerWrapper(w func(images.Handler) images.Handler) RemoteOpt func WithResolver(resolver remotes.Resolver) RemoteOpt type ServicesOpt func WithContainerService(containerService containersapi.ContainersClient) ServicesOpt func WithContentStore(contentStore content.Store) ServicesOpt func WithDiffService(diffService diff.DiffClient) ServicesOpt func WithEventService(eventService EventService) ServicesOpt func WithImageService(imageService imagesapi.ImagesClient) ServicesOpt func WithLeasesService(leasesService leases.Manager) ServicesOpt func WithNamespaceService(namespaceService namespacesapi.NamespacesClient) ServicesOpt func WithSnapshotters(snapshotters map[string]snapshots.Snapshotter) ServicesOpt func WithTaskService(taskService tasks.TasksClient) ServicesOpt 2 1
  20. 20. Plugins - custom containerd binary ● Add a file with import to cmd/containerd/ in your fork. ● Create your own main.go of containerd package main import ( "fmt" "os" "github.com/containerd/containerd/cmd/containerd/command" // import built-in plugins from cmd/containerd/builtins.go _ "github.com/mygithub/customplugin" ) func main() { app := command.App() if err := app.Run(os.Args); err != nil { fmt.Fprintf(os.Stderr, "containerd: %sn", err) os.Exit(1) } }
  21. 21. Example Snapshotter Proxy Plugin // Snapshot service manages snapshots service Snapshots { rpc Prepare(PrepareSnapshotRequest) returns (PrepareSnapshotResponse); rpc View(ViewSnapshotRequest) returns (ViewSnapshotResponse); rpc Mounts(MountsRequest) returns (MountsResponse); rpc Commit(CommitSnapshotRequest) returns (google.protobuf.Empty); rpc Remove(RemoveSnapshotRequest) returns (google.protobuf.Empty); rpc Stat(StatSnapshotRequest) returns (StatSnapshotResponse); rpc Update(UpdateSnapshotRequest) returns (UpdateSnapshotResponse); rpc List(ListSnapshotsRequest) returns (stream ListSnapshotsResponse); rpc Usage(UsageRequest) returns (UsageResponse); } - implement Snapshotter gRPC API - backend requests are proxied to plugin
  22. 22. External snapshotter ● Configure with proxy_plugins ● Build as an external plugin [proxy_plugins] [proxy_plugins.customsnapshot] type = "snapshot" address = "/var/run/mysnapshotter.sock" package main import( "net" "log" "github.com/containerd/containerd/api/services/snapshots/v1" "github.com/containerd/containerd/contrib/snapshotservice" ) func main() { rpc := grpc.NewServer() sn := CustomSnapshotter() service := snapshotservice.FromSnapshotter(sn) snapshots.RegisterSnapshotsServer(rpc, service) // Listen and serve l, err := net.Listen("unix", "/var/run/mysnapshotter.sock") if err != nil { log.Fatalf("error: %vn", err) } if err := rpc.Serve(l); err != nil { log.Fatalf("error: %vn", err) } }
  23. 23. Runtime Plugins
  24. 24. Runtime shim v2 API ● Minimal and scoped to the execution lifecycle of a container ● Binary naming convention ○ Type io.containerd.runsc.v1 -> Binary containerd-shim-runsc-v1
  25. 25. Runtime Plugins - Task Service service Task { rpc State(StateRequest) returns (StateResponse); rpc Create(CreateTaskRequest) returns (CreateTaskResponse); rpc Start(StartRequest) returns (StartResponse); rpc Delete(DeleteRequest) returns (DeleteResponse); rpc Pids(PidsRequest) returns (PidsResponse); rpc Pause(PauseRequest) returns (google.protobuf.Empty); rpc Resume(ResumeRequest) returns (google.protobuf.Empty); rpc Checkpoint(CheckpointTaskRequest) returns (google.protobuf.Empty); rpc Kill(KillRequest) returns (google.protobuf.Empty); rpc Exec(ExecProcessRequest) returns (google.protobuf.Empty); rpc ResizePty(ResizePtyRequest) returns (google.protobuf.Empty); rpc CloseIO(CloseIORequest) returns (google.protobuf.Empty); rpc Update(UpdateTaskRequest) returns (google.protobuf.Empty); rpc Wait(WaitRequest) returns (WaitResponse); rpc Stats(StatsRequest) returns (StatsResponse); rpc Connect(ConnectRequest) returns (ConnectResponse); rpc Shutdown(ShutdownRequest) returns (google.protobuf.Empty); }
  26. 26. How is containerd used? ● Library ○ Go client API ■ oras, BuildKit, Weaveworks Ignite, IBM Cloud Functions, OpenFaaS “faasd”, Alibaba PouchContainer ○ Extensibility ■ Amazon ECR resolver, Azure Teleport, remote snapshotters [cvmfs, stargz] ○ Imports/Subprojects (cri-o use of containerd/cgroups) ● Kubernetes Runtime ○ CRI-containerd ■ IBM Kubernetes Service, GKE, Ticketmaster, Alibaba, microk8s, KinD, k3s, AWS Fargate ● Daemon ○ Docker, BuildKit
  27. 27. Thank You!

×