CRIU (Checkpoint/Restore In Userspace) is a Linux application designed for checkpointing and restoring the state of running applications. It supports various usage scenarios including live migration, reboot-less kernel upgrades, and debugging, with significant contributions and support from multiple organizations. The latest version, v1.3rc1, works with unmodified Linux kernels and has been integrated into major Linux distributions.

1. CRIU:
Time and Space Travel Service
for Linux Applications
Kir Kolyshkin
Texas Linux Fest, 14 Jun 2014

2. 2
Agenda
What is CRIU?
Project history and state
Usage scenarios

Live migration

Reboot-less kernel upgrade

Slow services startup

Advanced debugging and testing

and more...

3. 3
What is CRIU?
Checkpoint Restore In Userspace
Checkpoint
or
Dump
Restore
or
Restart
Full
info
about
state

4. 4
CRIU pre-history
●
OpenVZ project
●
Containers live migration feature
●
Containers → Upstream Linux
●
1500+ kernel patches from us
●
Kernel-level checkpoint-restore merge failed
●
User-level checkpoint-restore ...

5. 5
Why in userspace?
Kernel
User-space
Dump:
- ptrace
- /proc
- netlink
- syscalls
Restore:
- syscalls
Process
kmod
C/R API

6. 6
Some history
Project started almost 3 years ago
– an RFC on kernel memory API extension
– small command line tool
– minimal dump of process' internals
First release
– v0.1 -- 23 Jul 2012 (x86 and basic stuff)
Since then
– Kernel part completed a year ago (150+ kernel patches:
new APIs for reading and setting process' state)

7. 7
Current project state
The latest release
– v1.3rc1
– supports x86_64 & ARM & AARM64
– support features that typical apps use
– works on unmodified linux-3.11+
– Included into Debian, Fedora, Ubuntu, Arch, SUSE, Gentoo, CoreOS...
Explicitly checked
– Apache, nginx, Oracle*, mysql, mongodb
– ssh/sshd, openvpn, cron, sendmail
– Java, gcc, make
– VNC + { gimp, mplayer, blender, supertux }
– Screen + { bash, top, tcpdump, tar/bz2 }
* some kernel tweaks required

8. 8
Some vitals
- 55K lines of code
- 150+ kernel patches
- contribs from Google, Huawei, Samsung, Canonical

9. 9
Usage scenarios
●
Live migration
●
incl. Docker, LXC, OpenVZ containers
●
Kernel upgrade w/o reboot
●
Slow services startup
●
Periodic snapshots (HPC)
●
Advanced debugging and testing

10. 10
Live migration
Host A Host B

11. 11
Live migration
Host A Host B
Shared FS
Pre-migrate memory
with memory tracker
http://criu.org/P.Haul

12. 12
Load balancing on cluster
Host A
Host C
Host B

13. 13
Power saving on cluster
Host A
Host C
Host B

14. 14
Node maintenance
Host A Host B

15. 15
Kernel upgrade w/o reboot
Host
Kernel A
Kexec
Kernel B

16. 16
Slow services startup
time# service foo start
Service readiness
Spawn process
Load config
Top-up caches
Initialize resource pools
Ready
T
100%

17. 17
Slow services startup
time
Tt < T
Ready
Spawn process
100%
Service readiness
# service foo restore

18. 18
Periodic snapshots
time
Memory tracker helps
to keep images smaller

19. 19
HPC
time
Power
failure
0% 20% 40% 60% 60%

20. 20
Advanced debugging
Production Host
Application
in trouble
Developer Host
Debugger

21. 21
Advanced testing
...
New test
or
new hardware
?

22. 22
More (funny) use cases
Forgot to launch your program in screen
– Live-migrate it there
Playing a game without the save button
– Snapshot it
[Put your own use case here]
http://criu.org/Usage_scenarios

23. 23
Recap
●
Started as containers live-migration tool
●
General tool to dump/restore apps state
●
v1.2 + Linux-3.11+ can do the trick
●
A lot of interesting technologies
●
Memory tracker
●
Migration of TCP connections
●
Injecting your code into a running application
●
Detecting kernel objects sharing
●
etc.

24. 24
Resources
http://criu.org – main site, documentation
http://git.criu.org – git repo with tool sources
http://plus.google.com/+CRIU page
criu@openvz.org mailing list
Kir Kolyshkin <kir@openvz.org> that's me
Thank you!