The container land has been heavily influenced by Docker. After all, the huge ecosystem building Docker images and tools is hard to ignore. Docker, defines images as lists of layers. This helps container engines cache reused layers and avoid re-downloading the whole root filesystem when minor changes get pushed. The, fairly new, OCI image spec is adopting the same approach in standardising the image format.
However, layers can be chanky in size and hard to manage. Additionally, it would be better to increase the caching granularity to single files (say /bin/bash) instead of large layers. Physicists in the LHC computing GRID had a similar problem. They solced it with CernVM-FS. This talk explores the idea of using it for container image distribution.
George Lestaris is a software engineer working at Pivotal in the Garden team, the container runtime of Cloud Foundry (CF). Before Pivotal, he spent time with high throughput computing on the LHC computing grid and cloud computing research in CERN. He has given talks before in CHEP 2013 regarding virtual clusters and their use in high-energy physics, in PyCon UK 2015 on interactive cloud experimentation and in the CF summit Santa Clara 2016 on containers in CF.
-- https://www.youtube.com/watch?v=g7dJCMh20iE
4. Container image format
• Container images are formalized in: Docker, AppC
(ACI) and OCI Image spec
• Generally: image is the combination of:
• a set of layers
• metadata
9. • Different image formats - different distributions
mechanisms
• Docker: download layers through HTTP
connections from a registry
• Helps reusing layers of base images
• Efficient container image fetching by parallelizing
the downloads
Container image distribution
16. • Network file system
• no packages and layers —> files and directories
• FUSE
• Lazily downloads the used files
• Deduplication Downloaded files get cached using
a content addressable storage
using a network filesystem
19. • Most images are based on a Linux distribution
• redis 3.2.3
• Image size: 190 MB (Compressed 74 MB)
• Used to boot: 11 MB - 5.7 %
• node 6.5.0 5.4 %
• nginx 1.11 3.1 %
Applications use a small fragment of the image
20. • nginx 1.10 to 1.11:
• Real changes: 4.02 MB
• Layer changes: 58 MB (two of the three layers)
• 14.4 times the size of the diff
• nginx 1.9 to 1.10: 4.8 times the size of the diff
Small changes between versions
22. • Small tool to create containers
• Low-level interface - not supposed to be a
container runtime
• Used by container runtimes (Docker, Garden)
internally
runC
25. • All VMs create a redis:3.2.3 container in parallel
• Comparing runC, Docker and Docker with warm cache
• Run the server and ping (wait for the server to came
up)
Scenario
redis-server --daemonize yes
while ! redis-cli ping; do
echo 'retrying'
done
26.
27. • IPFS: InterPlanetary file system
• Deduplication Content addressed storage for
object
• History Versioned objects
• Decentralized P2P transfers
• Objects are files, directories or changes
(commits)
Other approaches
28. • CI server
• Large clusters that parallelly fetch images
• Network contention
• Maintaining a private registry
• Serverless (?)
Use cases