Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Linux Plumbers Conference 2014, Düsseldorf 
Docker and the Linux kernel 
Cristian S., Docker Inc. 
cristian@docker.com
What is Docker ?
The Matrix From Hell
Another Matrix From Hell
Solution: 
the intermodal shipping container
Solved!
Solution to the deployment problem: 
the Linux container
Solved!
High level overview 
• Uses namespaces & cgroups 
• Runs on mainline kernels 
• Lower overhead than VMs 
• Can run a full ...
Docker's code 
• Licensed under the Apache 2 license 
• There's no paid/premium/commercial 
version 
• Docker and its code...
Use cases 
• Application development, 
testing, packaging & deployment 
• PaaS/SaaS/cloud infrastructure 
• Application & ...
Under the hood 
• Exec drivers provide the execution environment 
(virtualization/container tech) 
– native (libcontainer ...
Kernel requirements 
• Kernel 3.8 is the absolute minimum 
(except RHEL's 2.6.32) 
• Stable & supported kernels >= 3.10 ar...
AUFS graph driver stability & 
performance 
• Operations with containers are faster than devicemapper & 
(sometimes) btrfs...
Devicemapper graph driver stability 
& performance 
• Uses loopback mounted block devices by default 
• Allows EXT4 or XFS...
btrfs graph driver stability & 
performance 
• BTRFS bugs are a problem for Docker 
• Using the RAID like features of BTRF...
Performance & stability 
• Generally speaking, the kernel has been stable 
– Fixes have been made to file systems, namespa...
Kernel recommendations 
• Avoid running kernels no longer supported 
by your Linux distribution (kernel 3.8 lts-raring 
fr...
What can Docker do for the kernel? 
• Can be used to test the kernel's stability and 
evaluate its performance 
• Changes ...
What can a kernel developer do 
with Docker? 
• Find bottlenecks in the kernel's code 
• Run a KVM VM in a container 
• St...
How can someone contribute to Docker? 
What about kernel developers? 
• You don't necessarily have to write code 
• Provid...
Thank you! Questions? 
http://docker.com 
@docker
Upcoming SlideShare
Loading in …5
×

Docker and the Linux Kernel

13,927 views

Published on

Docker presentation at Linux Plumbers conference 2014

Published in: Technology
  • Be the first to comment

Docker and the Linux Kernel

  1. 1. Linux Plumbers Conference 2014, Düsseldorf Docker and the Linux kernel Cristian S., Docker Inc. cristian@docker.com
  2. 2. What is Docker ?
  3. 3. The Matrix From Hell
  4. 4. Another Matrix From Hell
  5. 5. Solution: the intermodal shipping container
  6. 6. Solved!
  7. 7. Solution to the deployment problem: the Linux container
  8. 8. Solved!
  9. 9. High level overview • Uses namespaces & cgroups • Runs on mainline kernels • Lower overhead than VMs • Can run a full system with an init or a single service/process • Snapshot based approach to build one image on top of another
  10. 10. Docker's code • Licensed under the Apache 2 license • There's no paid/premium/commercial version • Docker and its code are free and will be free • Lives at: https://github.com/docker/docker/
  11. 11. Use cases • Application development, testing, packaging & deployment • PaaS/SaaS/cloud infrastructure • Application & service isolation • Stress testing & benchmarking (including the Linux kernel)
  12. 12. Under the hood • Exec drivers provide the execution environment (virtualization/container tech) – native (libcontainer based) and LXC – are platform dependent – native is the default • Graph drivers are the storage providers – aufs, devicemapper, btrfs and vfs – vfs shouldn't be used (only used by the tests by default) – PRs open on GitHub for ZFS and OverlayFS • Existing graph & exec drivers only support Linux
  13. 13. Kernel requirements • Kernel 3.8 is the absolute minimum (except RHEL's 2.6.32) • Stable & supported kernels >= 3.10 are recommended • BTRFS has special requirements • Kernel configuration can be checked using https://raw.githubusercontent.com/docker/ docker/master/contrib/check-config.sh
  14. 14. AUFS graph driver stability & performance • Operations with containers are faster than devicemapper & (sometimes) btrfs • Known problems – Stale NFS file handle, cap_set_file, invalid argument on mount – Causes troubles on btrfs, remote file systems and many file systems which aren't ext3/ext4 – Direct IO problems & poor performance for IO intensive workloads • Limitations – Requires aufs-tools for auplink to dereference hard links – No support for hard links across layers – Limit of 127 layers – Can't be used on Fedora/RHEL/all systems which don't apply AUFS patches and don't ship aufs-tools • AUFS is developed outside of the mainline kernel tree • Update the kernel using distro updates
  15. 15. Devicemapper graph driver stability & performance • Uses loopback mounted block devices by default • Allows EXT4 or XFS to be used • Known problems – EBUSY errors (fix to be tested) – Potential file system corruption bug with ext4 ● Might be caused by loopback mounted block devices – Older kernels: space not freed when freed on the file system, various kernel errors and problems • Limitations – Devicemapper requires configuration of the storage to avoid the use of loopback mounted block devices • Update the kernel using distro updates
  16. 16. btrfs graph driver stability & performance • BTRFS bugs are a problem for Docker • Using the RAID like features of BTRFS is likely to cause data loss • Known problems – Data can be corrupted if exotic mount options are used – Kernel 3.8 & older than the latest stable kernels can cause data loss & corruption – The file system becomes slower the more data is written & stored on it – Performance degrades quickly & fragmentation is a problem – Balancing the file system to fix fragmentation could trigger some bugs • BTRFS is used automatically if Docker's root folder is on BTRFS • Update the kernel using distro updates • Using the latest minor version of supported kernels is recommended
  17. 17. Performance & stability • Generally speaking, the kernel has been stable – Fixes have been made to file systems, namespaces, cgroups, netfilter, aufs, btrfs, devicemapper and other kernel components – Recent kernels and the newest minor versions of LTS kernels have fixed a lot of problems, including devicemapper and btrfs problems – Some PID 1 issues are still being discussed – Changes around the cgroups and the namespaces have been made; this has also helped improve stability for containers • Performance is actively being studied – Docker's code has been improved to use less memory & be faster – Performance needs to be studied on the kernel side to achieve better scalability by a) making fewer syscalls in Docker where possible, b) making those syscalls faster in the kernel
  18. 18. Kernel recommendations • Avoid running kernels no longer supported by your Linux distribution (kernel 3.8 lts-raring from Ubuntu 12.04.x) • Open Docker issues on github; post errors (full kernel panics, btrfs check output, Docker daemon logs, `docker info`, `docker version` and `uname -a` output) when encountering errors • File bug reports with your distribution
  19. 19. What can Docker do for the kernel? • Can be used to test the kernel's stability and evaluate its performance • Changes made to the kernel can be tested easily against Docker • Can be used to test the running kernel to avoid breaking user space compatibility • Makes it simple to mix workloads for stress testing and performance testing • Exposes some hard to trigger kernel problems
  20. 20. What can a kernel developer do with Docker? • Find bottlenecks in the kernel's code • Run a KVM VM in a container • Stress testing for file systems, network, namespaces and the kernel in general • Large scale testing with containers • Hardware testing in containers • Network testing with advanced topologies
  21. 21. How can someone contribute to Docker? What about kernel developers? • You don't necessarily have to write code • Providing feedback, doing code review or telling the right kernel developer to do that are also useful helpful • Suggestions on how to debug specific kernel bugs and bugs in general are welcome • Contributing to Docker is also helping the kernel because Docker relies on the kernel
  22. 22. Thank you! Questions? http://docker.com @docker

×