Доклад: Все что вы хотели знать о CRIU, но стеснялись спросить... Андрей Вагин пришёл в Parallels в 2006 году, занимается разработкой ядра Linux, контейнерной системы OpenVZ и сравнительно молодого проекта CRIU. В 2008 году закончил МФТИ.

1. Andrey Vagin <avagin@openvz.org><
CRIU - Checkpoint/Restore in User-space

2. 2
Agenda
● CRIU and use-cases
● History
● Current state
● Under the hood
● Kernel impact
● How to integrate with/into CRIU
● P.haul
● Questions

3. 3
History
●
Berkeley Lab Checkpoint/Restart (BLCR) (2003)
– Load a kernel module and link with a library
●
DMTCP: Distributed MultiThreaded CheckPointing (2004-2006)
– Preload a library
●
OpenVZ (2005)
– OpenVZ kernel
●
Linux Checkpoint/Restart by Oren Laadan (2008)
– A non-mainline kernel
●
CRIU (2011)
OpenVZ
2005
BLCR
2003
Linux C/R
2008
CRIU
2011
DMTCP
2007

4. 4
What is C/R and how can it be used?
C/R is the ability to save states of processes
and to restore them later.
Usage scenarios:
– Failure recovery
– Live migration
– RKU (seamless kernel update)
– Rollback to the previous state
– Speed up of slow-boot services
– HPC issues

5. 5
Who is CRIU user?

6. 6
How does this work?
Kernel objects Process tree
crtools
Image files
Name-spaces
Files
Sockets
Pipes
001101
101010
110001
011010
000011
010101
001101
101010
110001
011010
000011
010101
001101
101010
110001
011010
000011
010101
001101
101010
110001
011010
000011
010101
001101
101010
110001
011010
000011
010101
001101
101010
110001
011010
000011
010101

7. 8
Dump
● Parasite code
– Receive file descriptors
– Dump memory content
– Prctl(), sigaction, pending signals, timers, etc.
● Ptrace
– freeze processes
– Inject a parasite code
● Netlink
– Get information about sockets, netns
● Procfs
/proc/PID/maps, /proc/PID/map_files/,
/proc/PID/status, /proc/PID/mountinfo

8. 10
Restore
● Collect shared objects
● Restore name-spaces
● Create a process tree
– Restore SID, PGID
– Restore objects, which should be inherited
● Files, sockets, pipes, ...
● Restore per-task properties.
● Restore memory
● Sim! Sala bim!
● Awesome
Namespaces
Processes

9. 11
sigreturn()

10. 12
New features in a kernel
● Parasite code injection (by Tejun Heo)
– Read task states, that are currently retrieved by a task only about itself
● The kcmp() system call
– Helps checking which kernel objects are shared between processes
● Proc map_files directory
– Find out what exact file is mapped
– Mappings sharing info
● A bunch of prctl extensions
– Set various private stuff on task/mm objects (c/r-only feature)
● Last-pid sysctl
– Restore task with desired PID value

11. 13
New features in a kernel
● Sockets information dumping via netlink (sock_diag)
– Extendable sockets state retrieving engine
● TCP repair mode
– Read intimate state of a TCP connection
and reconstructs it from scratch on a freshly created socket
● Virtual net devices indexes
– Allows to restore network devices in a namespace
● Socket peeking offset
– Allows peeking sockets queues (reading without removing data from queue)
● Task memory tracking
– incremental snapshots, online migration

12. 14
How to integrate with CRIU
● Action scripts
– block/unblock network
– setup namespaces
– post-dump and post-restore
● RPC, shared library
● Plugins

13. 15
RPC and libcriu.so
● Easy to use from other languages
– The protocol is based on protobuf messages
● Allow to use CRIU for unprivileged processes
– CRIU still requires root privileges to run
– UNIX domain sockets support passing credentials
● Self-dump
– A process can request to dump itself

14. 16
Plugins
● Unknown file types
● External dependencies
– Unix sockets (dbus, journald, rsyslog, etc)
– Unknown character and block devices.
– External bind-mounts
– External net devices
– External something else

15. 17
Community

16. 18
In a Nutshell, CRIU...
.... has had 4,375 commits made by 36 contributors
representing 58,688 lines of code
... is mostly written in C
with a very low number of source code comments
... has a young, but established codebase
maintained by a large development team
with stable Y-O-Y commits
... estimated cost $ 787,432
https://www.ohloh.net/p/criu#

17. 19
Where is CRIU now?

18. 20
P.haul (process hauler) - Live migration using CRIU
Live migration using CRIU
● Iterative
● Optimal
● Customizable
#./p.haul ovz 100 10.30.25.213
Migration succeeded
total time is ~2.86 sec
frozen time is ~1.99 sec
( ['0.27', '0.18', '1.55'] )
restore time is ~0.86 sec
img sync time is ~0.32 sec

19. 21
P.haul
Pre-dump and sync FS
Freeze, dump, sync FS
Restore
Kill
Resume
Post-dump
Resume
Post-resume

20. Thank you
http://criu.org