The document discusses the Stargz Snapshotter, a remote snapshotting tool for containerd that enables lazy pulling of container images. It highlights various features and functionalities, including integration with Kubernetes and other container runtimes, as well as references to related technologies and optimizations. Additionally, it provides links to various resources and examples illustrating its use and performance benefits.

1. Copyright(c)2020 NTT Corp. All Rights Reserved
Stargz Snapshotter:
0 pull
containerd
2 9 /

2. Copyright(c)2020 NTT Corp. All Rights Reserved
/
2 @ :

3. Copyright(c)2020 NTT Corp. All Rights Reserved
pull
pull run

4. Copyright(c)2020 NTT Corp. All Rights Reserved
Pull % %
p t4 H
e [ . u 6
0
0
l
1 e ] 0 0
. 0
0 a
.
r . 6
6 4
2 0 2

5. Copyright(c)2020 NTT Corp. All Rights Reserved
Pull (CRI)
API
kubectl apply
pull
kubelet
Pod
CRI
pull OCI
Docker containerd
runc gVisor Kata Containers
(CRI)
(OCI)

6. Copyright(c)2020 NTT Corp. All Rights Reserved
containerd
l CNCF graduated
l GKE[1] AWS Fargate[2] AKS(preview)[3] IKS[4] 6
l CRI CRI Kubernetes
l
l Kata Containers AWS Firecracker
l CRI
https://github.com/containerd/containerd
[1] https://cloud.google.com/kubernetes-engine/docs/concepts/using-containerd ; [2] https://aws.amazon.com/jp/blogs/containers/under-the-hood-fargate-data-plane/ ; [3] https://azure.microsoft.com/ja-
jp/updates/azure-kubernetes-service-aks-support-for-containerd-runtime-is-in-preview/ ; [4] https://www.ibm.com/cloud/blog/ibm-cloud-kubernetes-service-supports-containerd

7. Copyright(c)2020 NTT Corp. All Rights Reserved
Demo

8. Copyright(c)2020 NTT Corp. All Rights Reserved
containerd Stargz Snapshotter lazypull
l containerd 8
l Google CRFS 8 stargz
lazypull
l stargz eStargz 8
containerd Stargz Snapshotter
https://github.com/containerd/stargz-snapshotter
Stargz
Snapshotter
Kubelet
OCI
8
lazypull
stargz
pull
Slacker[1],CernVM-FS[2],Filegrain[3],Microsoft Teleportation[4],Google CRFS[5]
[1] https://www.usenix.org/conference/fast16/technical-sessions/presentation/harter [2] https://github.com/containerd/containerd/issues/2943 [3] https://github.com/AkihiroSuda/filegrain, [4]
https://stevelasker.blog/2019/10/29/azure-container-registry-teleportation/ [5] https://github.com/google/crfs

9. Copyright(c)2020 NTT Corp. All Rights Reserved
OCI stargz eStargz
Stargz
Snapshotter
stargz eStargz
lazypull
lazypull
pull run
(ctr-remote )
(DockerHub )
l stargz Google CRFS https://github.com/google/crfs 9 lazypull
l eStargz
l 9 OCI
• lazypull
• lazypull stargz/eStargz
squash

10. Copyright(c)2020 NTT Corp. All Rights Reserved
l
l 0 11 1 0
l https://github.com/moby/buildkit/pull/1402
CI
l Go stargz 1 CRFS 0Go
0
l https://github.com/golang/go/issues/30829
Serverless FaaS cold start
l https://github.com/knative/serving/issues/5913

11. Copyright(c)2020 NTT Corp. All Rights Reserved
l 1 1
• Dockerfile1RUN 1 1
l 1 1
l 1
l Docker[1] OCI[2] 1
[1] https://docs.docker.com/registry/spec/manifest-v2-2/ ; [2] https://github.com/opencontainers/image-spec/tree/v1.0.1

12. Copyright(c)2020 NTT Corp. All Rights Reserved
Pull API
GET /v2/<image>/blobs/GET /v2/<image>/manifests/
l API 1
l API API
l Docker[1] OCI[2] 2
l 2 optional Range Request 2 1
• 2
[1] https://docs.docker.com/registry/spec/api/ ; [2] https://github.com/opencontainers/distribution-spec/tree/v1.0.0-rc0

13. Copyright(c)2020 NTT Corp. All Rights Reserved
( )pull
rootfs
GET /v2/<image>/blobs/
GET /v2/<image>/manifests/

14. Copyright(c)2020 NTT Corp. All Rights Reserved
containerd remote snapshotter (1.4 )
l
l
l pull
Remote
Snapshotter

15. Copyright(c)2020 NTT Corp. All Rights Reserved
Stargz Snapshotter lazypull
Stargz
Snapshotter
rootfs FUSE
l
l stargz/eStargz pull
Lazy
pull
GET /v2/<image>/manifests/
GET /v2/<image>/blobs/
stargz
eStargz

16. Copyright(c)2020 NTT Corp. All Rights Reserved
tar
bin/bash
bin/ls
etc/passw
d
etc/group
usr/bin/apt
layer = tar (+ )
GET /v2/<image>/blobs/
l 1
6 1
l gzip

17. Copyright(c)2020 NTT Corp. All Rights Reserved
stargz
tar.gz stargz
bin/bash
bin/ls
etc/passwd
etc/group
usr/bin/apt
TOCEntries
HTTP Range Request
pull
bin/bash
bin/ls
etc/passwd
etc/group
usr/bin/apt
gzip
7
1
l Google CRFS https://github.com/google/crfs lazypull
l Seekable tar.gz
• Range Request 1 7
• tar.gz7 7
17
7

18. Copyright(c)2020 NTT Corp. All Rights Reserved
bin/bash
usr/bin/apt
etc/passwd
etc/group
bin/ls
GET /v2/<image-name>/blobs/
GET
Range: bytes=1048-2047

19. Copyright(c)2020 NTT Corp. All Rights Reserved
eStargz prefetch
stargz eStargz
bin/ls
usr/bin/apt
entrypoint.sh
bin/bash
bin/ls
usr/bin/apt
entrypoint.sh
bin/bash Range Request
landmark file
TOCEntriesTOCEntries
l eStargz stargz
l 1 9 NW

20. Copyright(c)2020 NTT Corp. All Rights Reserved
prefetch eStargz
bin/bash
entrypoint.sh
GET /v2/<image-name>/blobs/
GET
Range: bytes=0-1047

21. Copyright(c)2020 NTT Corp. All Rights Reserved
prefetch
l Docker 21 stargz eStargz
l eStargz
l
21
l 2 2
eStargzproc
ctr-remote

22. Copyright(c)2020 NTT Corp. All Rights Reserved
l Hello Bench [Harter, et al. 2016]2 2 2
• 2DockerHub
• ”print hello world”2
• ”up and running”
l : EC2 Oregon (m5.2xlarge, Ubuntu 20.04)
l : Docker Hub (docker.io)
l Stargz Snapshotter : b53e8fe8d37751753bc623b037729b6a6d9c1122
l pull create run 95 100 2
[Harter et al. 2016] Tyler Harter, Brandon Salmon, Rose Liu, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau. "Slacker: Fast Distribution with Lazy
Docker Containers". 14th USENIX Conference on File and Storage Technologies (FAST ’16). February 22–25, 2016, Santa Clara, CA, USA
NTT
pull create run
print hello world
up and running

23. Copyright(c)2020 NTT Corp. All Rights Reserved
NTT
0 5 10 15 20 25
estargz
stargz
legacy
python:3.7 (print “hello”)
pull create run [sec]
Prefetch

24. Copyright(c)2020 NTT Corp. All Rights Reserved
NTT
0 5 10 15 20 25 30
estargz
stargz
legacy
gcc:9.2.0 (printf(“hello”); )
pull create run [sec]

25. Copyright(c)2020 NTT Corp. All Rights Reserved
0 5 10 15 20 25
estargz
stargz
legacy
glassfish:4.1-jdk8 ( “Running GlassFish” )
pull create run
NTT
[sec]

26. Copyright(c)2020 NTT Corp. All Rights Reserved
early stage
l NW CDN lazy pull
l READ 2
l 2
https://github.com/containerd/stargz-snapshotter
l CRI 6 containerd lazypull
l Remote snapshotter > 1.4
pull
l Stargz Snapshotter stargz eStargz
lazypull
l stargz: Google CRFS https://github.com/google/crfs lazypull
l eStargz: stargz prefetch