More Related Content Similar to XPDDS19 Keynote: Unikraft Weather Report (20) More from The Linux Foundation (19) XPDDS19 Keynote: Unikraft Weather Report1. Simon Kuenzer <simon.kuenzer@neclab.eu>
Senior Researcher, NEC Laboratories Europe GmbH
Xen Summit 2019, Chicago
Weather Report
This work has received funding from the European Union’s Horizon 2020 research and
innovation program under grant agreements no. 675806 (“5G CITY”) and no. 825377
(“UNICORE”). This work reflects only the author’s views and the European
Commission is not responsible for any use that may be made of the information it
contains.
4. 4 © NEC Corporation 2019
Unikernels as VMs
Unikernels
Kernel
Hardware
Kernel
App B
Libs B
Traditional VMs
Hypervisor
App A
Libs A
Hardware
Hypervisor
5. 5 © NEC Corporation 2019
Unikernels as VMs
App A
Libs A
App B
Libs B
Unikernels
Kernel
Hardware
Kernel
App B
Libs B
Traditional VMs
Hypervisor
App A
Libs A
Hardware
Hypervisor
6. 6 © NEC Corporation 2019
Unikernels as VMs
▌Unikernels are purpose-built
Thin kernel layer, only what application needs
Single monolithic binary that contains OS and application
App A
Libs A
App B
Libs B
Unikernels
Kernel
Hardware
Kernel
App B
Libs B
Traditional VMs
Hypervisor
App A
Libs A
Hardware
Hypervisor
7. 7 © NEC Corporation 2019
Unikernels as VMs
▌Unikernels are purpose-built
Thin kernel layer, only what application needs
Single monolithic binary that contains OS and application
▌No isolation within Unikernel, done with hypervisor
One application Flat and single address space
App A
Libs A
App B
Libs B
Unikernels
Kernel
Hardware
Kernel
App B
Libs B
Traditional VMs
Hypervisor
App A
Libs A
Hardware
Hypervisor
8. 8 © NEC Corporation 2019
Unikernels as VMs
▌Unikernels are purpose-built
Thin kernel layer, only what application needs
Single monolithic binary that contains OS and application
▌No isolation within Unikernel, done with hypervisor
One application Flat and single address space
▌Further advantages from specialization
Performance and efficiency; reduced attack vector; small memory footprint
App A
Libs A
App B
Libs B
Unikernels
Kernel
Hardware
Kernel
App B
Libs B
Traditional VMs
Hypervisor
App A
Libs A
Hardware
Hypervisor
9. 9 © NEC Corporation 2019
Unikernel Framework
Motivation
▌Support wide range of use cases
▌Simplify building and optimizing
▌Simplify porting of existing applications
▌Common and shared code base for Unikernel creators
▌Support different hypervisors and CPU architectures
10. 10 © NEC Corporation 2019
Unikernel Framework
Motivation
▌Support wide range of use cases
▌Simplify building and optimizing
▌Simplify porting of existing applications
▌Common and shared code base for Unikernel creators
▌Support different hypervisors and CPU architectures
▌Concept: “Everything is a library”
Decomposed OS functionality
▌Two components:
1. Library Pool
2. Build Tool
11. 11 © NEC Corporation 2019
architecture
libs
main
libs
platform
libs
libarm32arch.olibx86_64arch.o
libxenplat.o
drivers
libconsole.o
libixgbe.o
libvirtio.o
network stack
liblwip.o
libtcpip.o
libhttp.o
memory allocators
libbuddy.o
libheap.o
libmempool.o
filesystems
libvfs.o
libfat.o
libext3.o
runtimes
libocaml.o
libpython.o
liberlang.o
schedulers
libcoop.o
libpreempt.o
librt.o
and more…
standard libs
musl.o
libnewlibc.o
libopenssl.o
1) Library Pool
libkvmplat.o liblinuxuplat.o
libmipsarch.o
libxenplat.o
12. 12 © NEC Corporation 2019
architecture
libs
main
libs
platform
libs
libarm32arch.olibx86_64arch.o
libxenplat.o
drivers
libconsole.o
libixgbe.o
libvirtio.o
network stack
liblwip.o
libtcpip.o
libhttp.o
memory allocators
libbuddy.o
libheap.o
libmempool.o
filesystems
libvfs.o
libfat.o
libext3.o
runtimes
libocaml.o
libpython.o
liberlang.o
schedulers
libcoop.o
libpreempt.o
librt.o
MyApplication
and more…
standard libs
musl.o
libnewlibc.o
libopenssl.o
1) Library Pool
libkvmplat.o liblinuxuplat.o
libmipsarch.o
libxenplat.o
Select/create
application
1 MyLib_A
13. 13 © NEC Corporation 2019
architecture
libs
main
libs
platform
libs
libarm32arch.olibx86_64arch.o
libxenplat.o
drivers
libconsole.o
libixgbe.o
libvirtio.o
network stack
liblwip.o
libtcpip.o
libhttp.o
memory allocators
libbuddy.o
libheap.o
libmempool.o
filesystems
libvfs.o
libfat.o
libext3.o
runtimes
libocaml.o
libpython.o
liberlang.o
schedulers
libcoop.o
libpreempt.o
librt.o
MyApplication
and more…
standard libs
musl.o
libnewlibc.o
libopenssl.o
1) Library Pool
libkvmplat.o liblinuxuplat.o
libmipsarch.o
libxenplat.o
Select/create
application
1
Selectand
configurelibraries
2
MyLib_A
14. 14 © NEC Corporation 2019
Unikernels
architecture
libs
main
libs
platform
libs
libarm32arch.olibx86_64arch.o
libxenplat.o
drivers
libconsole.o
libixgbe.o
libvirtio.o
network stack
liblwip.o
libtcpip.o
libhttp.o
memory allocators
libbuddy.o
libheap.o
libmempool.o
filesystems
libvfs.o
libfat.o
libext3.o
runtimes
libocaml.o
libpython.o
liberlang.o
schedulers
libcoop.o
libpreempt.o
librt.o
MyApplication
and more…
standard libs
musl.o
libnewlibc.o
libopenssl.o
1) Library Pool
libkvmplat.o liblinuxuplat.o
libmipsarch.o
libxenplat.o
Select/create
application
1
Selectand
configurelibraries
2
Build3Run4
myapp_xen_x86-64 myapp_kvm_x86-64 myapp_bare_arm64 etc.
MyLib_A
15. 15 © NEC Corporation 2019
2) Build Tool
16. 16 © NEC Corporation 2019
2) Build Tool
▌KConfig based and Makefile “Magic”
17. 17 © NEC Corporation 2019
2) Build Tool
▌KConfig based and Makefile “Magic”
▌Type “make menuconfig”
Choose options in the menu that you want for your application
Choose your target platform(s), e.g., Xen, KVM, bare-metal, Linux
18. 18 © NEC Corporation 2019
2) Build Tool
▌KConfig based and Makefile “Magic”
▌Type “make menuconfig”
Choose options in the menu that you want for your application
Choose your target platform(s), e.g., Xen, KVM, bare-metal, Linux
▌Save your config and type “make”
20. 20 © NEC Corporation 2019
Timeline
▌Early 2017: NEC-Internal project launch; 0.1
Build system
Initial port from Mini-OS and Solo5/KVM
21. 21 © NEC Corporation 2019
Timeline
▌Early 2017: NEC-Internal project launch; 0.1
Build system
Initial port from Mini-OS and Solo5/KVM
▌Dec/2017: Public Launch; RELEASE-0.2 Titan
As Xen Incubator project
Arm32 Xen, x86 Xen, x86 KVM, x86 Linux
Binary buddy allocator (heap)
Cooperative scheduling
22. 22 © NEC Corporation 2019
Timeline
▌Early 2017: NEC-Internal project launch; 0.1
Build system
Initial port from Mini-OS and Solo5/KVM
▌Dec/2017: Public Launch; RELEASE-0.2 Titan
As Xen Incubator project
Arm32 Xen, x86 Xen, x86 KVM, x86 Linux
Binary buddy allocator (heap)
Cooperative scheduling
▌Feb/2019: RELEASE-0.3 Iapetus
Arm64 support for KVM
Networking (uknetdev, lwip, virtio-net)
Initial VFS with in-RAM filesystem
newlib
23. 23 © NEC Corporation 2019
Timeline
▌Early 2017: NEC-Internal project launch; 0.1
Build system
Initial port from Mini-OS and Solo5/KVM
▌Dec/2017: Public Launch; RELEASE-0.2 Titan
As Xen Incubator project
Arm32 Xen, x86 Xen, x86 KVM, x86 Linux
Binary buddy allocator (heap)
Cooperative scheduling
▌Feb/2019: RELEASE-0.3 Iapetus
Arm64 support for KVM
Networking (uknetdev, lwip, virtio-net)
Initial VFS with in-RAM filesystem
newlib
▌Upcoming release: RELEASE-0.4 Rhea
Language support: C++, Python, etc.
Block & console devices
9pfs filesystem (Xen, KVM)
Libraries: intel-intrinsics, libunwind, libuuid, pthread-embedded,
compiler-rt, eigen, fp16, fxdiv, pthreadpool
24. 24 © NEC Corporation 2019
Changed lines of code by affiliation
▌3 contributors
▌Project Launch
86020
14903
27720
11736
32149
86020
31369
60401
2017 2018 2019
NEC Politehnica University of Bucharest Arm Other
▌~15 contributors
▌*Many patches
still under review
▌~20 contributors
▌Start of student
projects
26. 26 © NEC Corporation 2019
Documentation
▌http://unikraft.org
▌Generated by Sphinx
▌Part of base repository: /doc/
27. 27 © NEC Corporation 2019
Patch Reviews
▌Patchwork: http://patchwork.unikraft.org
▌Coordination of reviewers
▌Review status and reviewer/maintainer in charge visible
28. 28 © NEC Corporation 2019
Automated Testing (coming soon)
▌Today: Two-branch model: master & staging
Staging: Latest accepted patches
Master: Manual follow-up: Releases and tested code
29. 29 © NEC Corporation 2019
Automated Testing (coming soon)
▌Today: Two-branch model: master & staging
Staging: Latest accepted patches
Master: Manual follow-up: Releases and tested code
▌Jenkins for automate testing
Common combinations of configurations and libraries
Execution test on common hypervisors and architectures
(Xen, KVM, Linux, x86, Arm)
30. 30 © NEC Corporation 2019
Automated Testing (coming soon)
▌Today: Two-branch model: master & staging
Staging: Latest accepted patches
Master: Manual follow-up: Releases and tested code
▌Jenkins for automate testing
Common combinations of configurations and libraries
Execution test on common hypervisors and architectures
(Xen, KVM, Linux, x86, Arm)
▌Goal
Integrate test results into review processes
32. 32 © NEC Corporation 2019
Binary Compatible Unikernels
▌Even with complete library pool,
manual porting is non-trivial
Existing build system need to be ported or
instrumented (e.g., cross-compilation)
Pre-compiled binaries cannot be executed
(e.g., proprietary executables)
33. 33 © NEC Corporation 2019
Binary Compatible Unikernels
▌Even with complete library pool,
manual porting is non-trivial
Existing build system need to be ported or
instrumented (e.g., cross-compilation)
Pre-compiled binaries cannot be executed
(e.g., proprietary executables)
▌ELF binary compatibility, Linux ABI
Same executable for Linux should run on
Unikraft without recompilation
ELF loader
System call emulation
[1] Pierre et. al, A Binary-Compatible Unikernel, VEE’19
[2] Kiviti et. al, OSv—Optimizing the Operating System for Virtual Machines, USENIX ATC ’14
Image: https://en.wikipedia.org/wiki/Executable_and_Linkable_Format
34. 34 © NEC Corporation 2019
Language Runtimes
35. 35 © NEC Corporation 2019
Language Runtimes
▌Support applications written in higher-level
language as Unikernel
Primarily Serverless and IoT use cases
36. 36 © NEC Corporation 2019
Language Runtimes
▌Support applications written in higher-level
language as Unikernel
Primarily Serverless and IoT use cases
▌Upcoming:
C++
Rust
Go
Ruby
Javascript (v8)
Python
37. 37 © NEC Corporation 2019
Language Runtimes
▌Support applications written in higher-level
language as Unikernel
Primarily Serverless and IoT use cases
▌Upcoming:
C++
Rust
Go
Ruby
Javascript (v8)
Python
Unikernel Image sizes
(uncompressed, Xen)
Micropython 828 KB
Python 2 3,9 MB
Go runtime 552 KB
38. 38 © NEC Corporation 2019
NFV
▌Virtualized Network Functions
Package vNF directly as VM with Unikraft
Remove maintenance effort of hosting OS
Minimal OS overhead
Minimal OS noise
High networking performance & throughput
39. 39 © NEC Corporation 2019
NFV
▌Virtualized Network Functions
Package vNF directly as VM with Unikraft
Remove maintenance effort of hosting OS
Minimal OS overhead
Minimal OS noise
High networking performance & throughput
▌Click
Programmable vNF
40. 40 © NEC Corporation 2019
NFV
▌Virtualized Network Functions
Package vNF directly as VM with Unikraft
Remove maintenance effort of hosting OS
Minimal OS overhead
Minimal OS noise
High networking performance & throughput
▌Click
Programmable vNF
▌Intel DPDK
Dataplane development Kit
SDK for building high-performance VNFs
Directly build Unikernel instead of kernel-bypassing application
41. 41 © NEC Corporation 2019
Application Porting
▌Initial set of ported application
Typical for cloud deployments
Learn about missing components in Unikraft
42. 42 © NEC Corporation 2019
Application Porting
▌Initial set of ported application
Typical for cloud deployments
Learn about missing components in Unikraft
▌Webservers
Nginx, lighttpd
43. 43 © NEC Corporation 2019
Application Porting
▌Initial set of ported application
Typical for cloud deployments
Learn about missing components in Unikraft
▌Webservers
Nginx, lighttpd
▌Databases
Memcached, Redis, sqlite
44. 44 © NEC Corporation 2019
Application Porting
▌Initial set of ported application
Typical for cloud deployments
Learn about missing components in Unikraft
▌Webservers
Nginx, lighttpd
▌Databases
Memcached, Redis, sqlite
▌Machine Learning
Pytorch
45. 45 © NEC Corporation 2019
Application Porting
▌Initial set of ported application
Typical for cloud deployments
Learn about missing components in Unikraft
▌Webservers
Nginx, lighttpd
▌Databases
Memcached, Redis, sqlite
▌Machine Learning
Pytorch
▌Goal: Only minimal effort to integrate to build system
Makefile.uk
Config.uk
46. 46 © NEC Corporation 2019
Application Porting
▌Initial set of ported application
Typical for cloud deployments
Learn about missing components in Unikraft
▌Webservers
Nginx, lighttpd
▌Databases
Memcached, Redis, sqlite
▌Machine Learning
Pytorch
▌Goal: Only minimal effort to integrate to build system
Makefile.uk
Config.uk
SQLite Unikernel
Image size (uncompressed, Xen) 844 KB
47. 47 © NEC Corporation 2019
Xenstore
Xen Dom0-Disaggregation
▌Disaggregate functionality from Dom0
Xen
Linux
Dom0
xl
toolstack
48. 48 © NEC Corporation 2019
Xenstore
Xen Dom0-Disaggregation
▌Disaggregate functionality from Dom0
XenStore stub domain (e.g., cxenstore)
Xen
Linux
Dom0
xl
toolstack
49. 49 © NEC Corporation 2019
Xenstore
Xen Dom0-Disaggregation
▌Disaggregate functionality from Dom0
XenStore stub domain (e.g., cxenstore)
Xen
Linux
Qemu
HVM DomUDom0
xl
toolstack
50. 50 © NEC Corporation 2019
Xenstore
Xen Dom0-Disaggregation
▌Disaggregate functionality from Dom0
XenStore stub domain (e.g., cxenstore)
QEMU stub domain
Xen
Linux
Qemu
HVM DomUDom0
xl
toolstack
51. 51 © NEC Corporation 2019
Hardening
▌Already small attack vector due to specialization
52. 52 © NEC Corporation 2019
Hardening
▌Already small attack vector due to specialization
▌Common attack prevention features need to be
implemented1, for instance:
ASLR (via boot loader or toolstack)
Stack canaries
Page protection bits
Heap integrity checks
[1] NCC Group, Assessing Unikernel Security,
https://www.nccgroup.trust/us/our-research/assessing-unikernel-security/
53. 53 © NEC Corporation 2019
Hardening
▌Already small attack vector due to specialization
▌Common attack prevention features need to be
implemented1, for instance:
ASLR (via boot loader or toolstack)
Stack canaries
Page protection bits
Heap integrity checks
▌Potential to enable enhanced preventions with little
performance costs due to direct access to privileged
functionality
e.g., malloc() maps pages, free() unmaps pages
[1] NCC Group, Assessing Unikernel Security,
https://www.nccgroup.trust/us/our-research/assessing-unikernel-security/
56. 56 © NEC Corporation 2019
Talks
▌Check-out related talks:
Wednesday, 11:55am
Secure Unikraft Applications with Solo5
Haibo Xu, Arm
Wednesday, 1:50pm
When Unikraft Meets Arm64
Jia He, Arm
Wednesday, 3:45pm
A Journey to Mirage OS as Xen PVH
Marek Marczykowski-Górecki, Invisible Things Lab
57. 57 © NEC Corporation 2019
Talks
▌Check-out related talks:
Wednesday, 11:55am
Secure Unikraft Applications with Solo5
Haibo Xu, Arm
Wednesday, 1:50pm
When Unikraft Meets Arm64
Jia He, Arm
Wednesday, 3:45pm
A Journey to Mirage OS as Xen PVH
Marek Marczykowski-Górecki, Invisible Things Lab
▌Design Sessions
Tuesday, 2:40pm
Building Stub Domains with Unikraft
Wednesday, 4:35pm
A Journey through Unikraft's Build System