One might find it ironic that some of the world's fastest supercomputers -- vast clusters capable of trillions of floating point operations per second -- can take upwards of a half an hour to reboot in between jobs. While we often talk about the density advantages of containers, it's the opposite approach that we use in the High Performance Computing world! Here, we use exactly 1 system container per node, giving it unlimited access to all of the host's CPU, Memory, Disk, IO, and Network. And yet we can still leverage the management characteristics of containers -- security, snapshots, live migration, and instant deployment to recycle each node in between jobs. In this talk, we'll examine a reference architecture and some best practices around containers in HPC environments.

1. Streamlining HPC Workloads
with Containers
@DustinKirkland

2. what does
high-performance computing
look like?

3. Wikipedia
says...

4. Or perhaps in China...

5. Google image search shows...

6. The
university
student
learns...

7. HackerNews suggests...

8. Your DevOps engineer launches...
x1.32xlarge

9. But
then
there
is
your
real,
actual
data
center...

10. what do all of these
have in common?

11. a lot,
actually

12. they’re all running
Linux

13. directly on the
bare metal itself

14. performance is
maximized

15. overhead is
minimized

16. big problems are
distributed across a cluster

17. everyone prefers a
clean environment

18. virtual machines
always involve overhead

19. VM MonitorVMXON VMXOFF
Guest
VM EntryVM Exit

20. oh, and let’s reboot
a datacenter

22. BIOS is checking memory for problems…
Scanning 1,199,511,627,776 bytes…
This may take several minutes…
Running test 1 of 8: 1.0% complete
Overall test status: 0.1% complete
Time elapsed: 17m23s
Status:
No problems have been found yet.

24. so let’s have a look at
containers

25. first,
process containers

26. awesome for HPC
functions

30. LXD

31. second,
machine containers

33. ➢ Ultra fast “vm-lite” guests (bare metal speed)
➢ Any distribution of Linux - e.g. Ubuntu, CentOS
➢ Starts in less than 1 second
➢ 15x density of KVM or ESX for idle workloads
host A
nova-lxd lxc cli
lxdkernel
other restful apps
lxc
machine
LXD REST API
host B
lxc
machine
lxdkernel
host C host D host ...
lxc
machine
lxc
machine
lxc
machine
lxdkernel lxdkernel lxdkernel

34. ➢ Ultra fast “vm-lite” guests (bare metal speed)
➢ Any distribution of Linux - e.g. Ubuntu, CentOS
➢ Starts in less than 1 second
➢ 15x density of KVM or ESX for idle workloads
host A
nova-lxd lxc cli
lxdkernel
other restful apps
LXD REST API
host B
lxdkernel
host C host D host ...
lxdkernel lxdkernel lxdkernel
lxc
machine
lxc
machine
lxc
machine
lxc
machine
lxc
machine

35. CPU Cores
CPU Cycles
Memory
Disk Space
Disk IO
Network IO
One LXD container,
with 100% of the system:
“alloy” mode

36. exclusive access
to system resources

37. but secured from the
underlying hardware and OS

38. cgroups, user namespaces,
apparmor, seccomp

39. instant
startup

40. looks like a machine,
Linux on Linux

41. zero
latency

42. zero
overhead

43. identical
performance

44. snapshot
restore

45. live
migration

46. demo

47. ubuntu.com/lxd
github.com/lxc
linuxcontainers.org