Docker internals session by Sangam Biradar at the Cloud Native, Bangalore September 2019 meetup

1. Docker Internals
Sept. 7, 2019
Venue :- BRIDGEi2i Analytics Solutions Pvt. Ltd.
Docker Bangalore Joint Meetup with
Cloud Native Group of Bangalore

2. 2
$Whoami
Sangam Biradar
Github-sangam14
Twitter-@BiradarSangam
$curl engineitops.com
engineitops.icu
- My Personal Blog Website
- Over 100+ blog post on docker | Golang and kubernetes and cloud
- Unique 50000+ hits | Dockerlabs Active Contributor
- Educated around 1000+ student
- Research Papers Presented/Published in the Conference Proceedings(IEEE)
1.“Build Minimal Docker Container Using Golang’’- presented and published in the proceedings of
the International Conference ICICCS2018 at Vaigai College Of Engineering, Madurai 14 June 18
- 2."Algo_Seer: System for Extracting and Searching Algorithms in Scholarly Big Data"-
presented and published in the proceedings of the International Conference ICICV 2019 Springer
Lecture Notes on Data Engineering and Communications Technologies. Francis Xavier
Engineering College,Tirunelveli.14 feb 19 Series Editor: Xhafa, Fatos ISSN: 2367-4512

3. Outline
3
● What is container – overview
● The building Blocks [ namespaces , Control
groups , copy-on-write storage , union filesystem ]
● Container Runtime
● Demo

4. Container Overview
In short…..
1

5. Container
● A definition says “Containers are an abstraction at the app
layer that packages code and dependencies together”. It
practically means just the Application and its dependent
binaries and libraries are packaged into a container with no
extra baggage of an operating system.
5

6. “
How its work then?
6

7. 7
Pre –virtualization - To hypervisor
base virtualization world
To container base virtualization
world

8. 8
Daemon running on the host OS

9. The building
Blocks
[ namespaces , Control groups , copy-on-write storage , union
filesystem ]
9

10. Cgroups
(control Group)
● first idea that should come to mind is of resource management
● Resource metering and limiting memory
CPU
block I/O
● network*
Device node (/dev/*) access control Crowd control
10

11. ● Each subsystem has a hierarchy (tree)
separate hierarchies for CPU, memory, block I/O...
● Hierarchies are independent
the trees for e.g. memory and CPU can be different
● Each process is in a node in each hierarchy
think of each hierarchy as a different dimension or axis
● Each hierarchy starts with 1 node (the root)
Initially, all processes start at the root node*
● Each node = group of processes
sharing the same resources
11

12. ● Keeps track of pages used by each group:
file (read/write/mmap from block devices)
anonymous (stack, heap, anonymous mmap)
active (recently accessed)
inactive (candidate for eviction)
● Each page is “charged” to a group
● Pages can be shared across multiple groups
● e.g. multiple processes reading from the same files
when pages are shared, only one group “pays” for a page
12
Memory Cgroups

13. Memory limits
● Each group can have its own limits
● limits are optional
two kinds of limits: soft and hard limits
● Soft limits are not enforced
● they influence reclaim under memory pressure
● Hard limits will trigger a per-group OOM killer
● Limits can be set for different kinds of memory
● physical memory
● kernel memory
● total memory
13

14. BlKIO Cgroup
● Keeps track of I/Os for each group
per block device
read vs write
sync vs async
● Set (limits) for each group
per block device , read vs write , ops vs bytes
● Set relative weights for each group
● Note: most writes go through the page cache
● so classic writes will appear to be unthrottled at first
14

15. ● Automatically set traffic class or priority,
for traffic generated by processes in the group
● Only works for egress traffic
● Net_cls will assign traffic to a class
● class then has to be matched with tc/iptables, otherwise
traffic just flows normally
● Net_prio will assign traffic to a priority
● priorities are used by queuing disciplines
15
Net_cls Net_prio Cgroup

16. ● Pid – (Process isolation)
● Net – (Network isolation)
● mnt – ( Filesystem mount points isolation)
● UTS – (Nodename and Domainname isolation)
● IPC – (Inter Process Communication resource isolation)
● User - User name space
Namespaces
16

17. Namespace sharing
17

18. Container Runtime
18

19. Create own container
using golang
19
IEEE PAPER : https://ieeexplore.ieee.org/author/37086701647

20. 20
https://www.technoarete.org/common_abstract/pdf/IJER
CSE/v5/i3/Ext_97135.pdf
International Journal of Engineering Research in Computer Science and Engineering (IJERCSE)
Vol 5, Issue 3, March 2018
Build Minimal Docker Container Using Golang
[1] Biradar Sangam.M

21. 21

22. http://dockerlabs.collabnix.com/pres
entation/docker-internals.html#/
22
Demo

23. 23
Thanks!
Any questions?
You can find me at twitter @BiradarSangam or visit
engineItops.com

24. 24
http://engineitops.com/cloud-
native-and-containerisation-joint-
meetup-with-docker-bangalore-
docker-internals-sangam-biradar/
Refer blog post:-