1
A PLATFORM FOR AUTOMATING
UNIKERNELS COMPILATION AND
DEPLOYMENT
UNIK
2
VIRTUALIZATION STACK
Redundancy in the stack
– e.g. Isolation
Application Config
Application
Language Runtime
Shared Libraries
Docker Runtime
OS User Processes
OS Kernel
Virtual HW Drivers
Hypervisor
Hardware Drivers
Hardware
The aim is to run single
Application with a single user
on a single server
3
KERNEL COMPLEXITY - PROTECTION
Application safe from user
Application safe from
application User safe from user
4
INEFFICIENCY
• Needless permission check, it is hard and an updated
model from time sharing computer from the 50s, 60s
• Microservices architecture duplicate what Linux
did for us
• Kernel include a lot of unnecessary drivers that
not being used: floppy, USB …
• Update and patches using yum bring a lot of
unnecessary components
5
SECURITY
• Very large attack surface
• A lot of exploits target linux.
It is harder to attack
hypervisor - not expose to
the internet
• Microservices architecture
Sharing – Kernel, Memory,
filesystem, hardware
The only thing make it safe is kernel extension
like: cgroup
6
LINUX KERNEL LANGUAGES
7
SOURCE LINES OF CODE
Small Applications: 10Ks
Medium to large applications: 100Ks
Really huge applications: 1Ms
8
2.4
5.2
11
12.6
13.5
15.9
22
0
5
10
15
20
25
Linux kernel
2.4.2
Linux kernel
2.6.0
Linux kernel
2.6.29
Linux kernel
2.6.32
Linux kernel
2.6.35
Linux kernel 3.6 Linux kernel pre-
4.2
2001 2003 2009 2009 2010 2012 2015
Linux Kernel SLOC
9
59
104
215
283
324
419
0
50
100
150
200
250
300
350
400
450
Debian 2.2 Debian 3.0 Debian 3.1 Debian 4.0 Debian 5.0 Debian 7.0
2000 2002 2005 2007 2009 2012
Debian SLOC
10
HOW DID WE GET HERE ? EVOLUTION !
Unix was supported us the entire way!
11
DECADES OF BACKWARDS COMPATIBILITY
What can linux run on ?
What can run on linux ?
Anything !
Anything !
12
TRADE OFF
VS
Compatibility Efficiency
13
Make it works.
Make it right.
Make it fast.
14
Or is it ?
But how ?
What can we do to make it better ?
Rewrite ?
does it make sense to write one from scratch ?
Probably not – it is too much work !
15
True, linux is monolithic, and I agree that microkernels are
nicer... As has been noted (not only by me), the linux kernel
is a minuscule part of a complete system: Full sources for
linux currently runs to about 200kB compressed. And all of
that source is portable, except for this tiny kernel that you
can (provably: I did it) re-write totally from scratch in less
than a year without having /any/ prior knowledge.
– Linus Torvalds, 1992
16
{uni-} {kernel}
a bridge between
applications and
the actual data
processing done
at the hardware
level.
One; having
or consisting
of one.
17
Application
Kernel
TRADITIONAL APPROACH
libc
libz
iconv
openGL
gtk
libgmp libtlc
Libstd++ libgcc
18
Application
Kernel
UNIKERNEL APPROACH
libc
libz
iconv
openGL
gtk
libgmp libtlc
Libstd++ libgcc
19
• Based on library OS
• Contains only what is needed
• Single process (micro services architecture)
• Single address space
• No virtual memory isolation
• No context switching
• No usermode/kernelmode
• Real immutable infrastructure
UNIKERNELS
20
App Binary
App Config
App Deps
Virt, HW Drivers
Langue runtime
ApplicationRuntime
Packaging Tool Unikernel!
UNIKERNEL CREATION
21
UNIKERNEL STACK
• Unikernels deploy directly
against the hypervisor
• Unikernels have their own
network stack
• Unikernels have their own
virtualize memory presented
as hardware
• Unikernel are completely self
contained & ideally
immutable
Hypervisor
1
0
.
1
0
.
1
.
1
1
0
.
1
0
.
1
.
2
1
0
.
1
0
.
1
.
3
1
0
.
1
0
.
1
.
4
1
0
.
1
0
.
1
.
5
1
0
.
1
0
.
1
.
6
1
0
.
1
0
.
1
.
7
22
HOW CAN UNIKERNELS HELP ADDRESS OUR
PROBLEMS? Application Config
Application
Language Runtime
Shared Libraries
Docker Runtime
OS User Processes
OS Kernel
Virtual HW Drivers
Hypervisor
Hardware Drivers
Hardware
Minimal layers of isolation
and abstraction
Includes only what is really
needed
Less code, fewer bugs, easy
to reason about
23
DOCKER STACK VS. UNIKERNEL STACK
Application Binary
+ Library OS
Hypervisor
Hardware Drivers
Hardware
Application Config
Application
Language Runtime
Shared Libraries
Docker Runtime
OS User Processes
OS Kernel
Virtual HW Drivers
Hypervisor
Hardware Drivers
Hardware
24
DOCKER STACK VS. UNIKERNEL STACK
Application Binary
+ Library OS
Hypervisor
Hardware Drivers
Hardware
Application Config
Application
Language Runtime
Shared Libraries
Docker Runtime
OS User Processes
OS Kernel
Hardware Drivers
Hardware
Hardware isolation
25
UNIKERNEL ADVANTAGES
• No other users, no multi-user support
• No permission checks – you can utilize 100% of your hardware
• Isolation at the virtual hardware – only ! share only hardware
• Minimal virtual machine ~1 gb in size, minimal unikernel is
tiny, kb in size
• Very short boot time
• A tiny custom surface of attack, less likely to be effected by
a public exploit
• Real immutable infrastructure – perfect fit to micro services architecture
26
Backward compatibility Forward compatibility
POSIX compliance Language specifics
27
BENCHMARK
30
COOL DEMOS
31
is an open-source
tool written in Go for compiling
applications into unikernels and
deploying those unikernels across a
variety of cloud providers,
embedded devices (IoT), as well as
a developer laptop or workstation.
32
unik build --path example-app/ --base unikernel-type --language language
--provider provider-name --name image-name
unik run --instanceName instance-name –imageName image-name
UNIK WORKFLOW
33
UNIK IS NOT OPINIONATED !
Unikernel types Cloud providers
Processor architectures
34
35
UNIK INTEGRATION WITH DOCKER
Docker API can be used to create unikernel via UniK
36
UNIK INTEGRATION WITH KUBERNETES
Unikernels support was added to Kubernetes by the UniK team by adding UniK as a
container runtime to K8s - in the same way that Docker and rkt are container
runtimes, UniK is now also available as a "container" runtime for k8s.
37
UNIK KUBERNETES ARCHITECTURE
unikernels
Now one can
deploy a unikernel
apps alongside
regular kubernetes
containerized apps.
39
41
42
INTERNET OF THINGS
UniK will Push
Unikernel
To Raspberry Pi
Unikernel will
communicate with
the Panini toaster
Toaster will make
Panini
We will eat Panini
bread
43
WORRIED ABOUT IOT DDOS?
THINK UNIKERNELS
Could such attacks be prevented
if IoT devices were running bare
metal unikernels?
Read our blog !
45
.NET ON UNIKERNEL
46
SERVERLESS
47
OpenSource
48
UNIK COMMUNITY
49
EMC-Advanced-Dev
http://www.github.com/emc-advanced-dev
Idit Levine
CTO of Cloud Platform Team &
Office of the CTO, EMC
@idit_levine
Scott Weiss
Software Engineer, EMC
@ilackarms
Yuval Kohavi
Software Engineer, EMC
@uvgroovy
50
@Idit_Levine
51

Unik Slides

  • 1.
    1 A PLATFORM FORAUTOMATING UNIKERNELS COMPILATION AND DEPLOYMENT UNIK
  • 2.
    2 VIRTUALIZATION STACK Redundancy inthe stack – e.g. Isolation Application Config Application Language Runtime Shared Libraries Docker Runtime OS User Processes OS Kernel Virtual HW Drivers Hypervisor Hardware Drivers Hardware The aim is to run single Application with a single user on a single server
  • 3.
    3 KERNEL COMPLEXITY -PROTECTION Application safe from user Application safe from application User safe from user
  • 4.
    4 INEFFICIENCY • Needless permissioncheck, it is hard and an updated model from time sharing computer from the 50s, 60s • Microservices architecture duplicate what Linux did for us • Kernel include a lot of unnecessary drivers that not being used: floppy, USB … • Update and patches using yum bring a lot of unnecessary components
  • 5.
    5 SECURITY • Very largeattack surface • A lot of exploits target linux. It is harder to attack hypervisor - not expose to the internet • Microservices architecture Sharing – Kernel, Memory, filesystem, hardware The only thing make it safe is kernel extension like: cgroup
  • 6.
  • 7.
    7 SOURCE LINES OFCODE Small Applications: 10Ks Medium to large applications: 100Ks Really huge applications: 1Ms
  • 8.
    8 2.4 5.2 11 12.6 13.5 15.9 22 0 5 10 15 20 25 Linux kernel 2.4.2 Linux kernel 2.6.0 Linuxkernel 2.6.29 Linux kernel 2.6.32 Linux kernel 2.6.35 Linux kernel 3.6 Linux kernel pre- 4.2 2001 2003 2009 2009 2010 2012 2015 Linux Kernel SLOC
  • 9.
    9 59 104 215 283 324 419 0 50 100 150 200 250 300 350 400 450 Debian 2.2 Debian3.0 Debian 3.1 Debian 4.0 Debian 5.0 Debian 7.0 2000 2002 2005 2007 2009 2012 Debian SLOC
  • 10.
    10 HOW DID WEGET HERE ? EVOLUTION ! Unix was supported us the entire way!
  • 11.
    11 DECADES OF BACKWARDSCOMPATIBILITY What can linux run on ? What can run on linux ? Anything ! Anything !
  • 12.
  • 13.
    13 Make it works. Makeit right. Make it fast.
  • 14.
    14 Or is it? But how ? What can we do to make it better ? Rewrite ? does it make sense to write one from scratch ? Probably not – it is too much work !
  • 15.
    15 True, linux ismonolithic, and I agree that microkernels are nicer... As has been noted (not only by me), the linux kernel is a minuscule part of a complete system: Full sources for linux currently runs to about 200kB compressed. And all of that source is portable, except for this tiny kernel that you can (provably: I did it) re-write totally from scratch in less than a year without having /any/ prior knowledge. – Linus Torvalds, 1992
  • 16.
    16 {uni-} {kernel} a bridgebetween applications and the actual data processing done at the hardware level. One; having or consisting of one.
  • 17.
  • 18.
  • 19.
    19 • Based onlibrary OS • Contains only what is needed • Single process (micro services architecture) • Single address space • No virtual memory isolation • No context switching • No usermode/kernelmode • Real immutable infrastructure UNIKERNELS
  • 20.
    20 App Binary App Config AppDeps Virt, HW Drivers Langue runtime ApplicationRuntime Packaging Tool Unikernel! UNIKERNEL CREATION
  • 21.
    21 UNIKERNEL STACK • Unikernelsdeploy directly against the hypervisor • Unikernels have their own network stack • Unikernels have their own virtualize memory presented as hardware • Unikernel are completely self contained & ideally immutable Hypervisor 1 0 . 1 0 . 1 . 1 1 0 . 1 0 . 1 . 2 1 0 . 1 0 . 1 . 3 1 0 . 1 0 . 1 . 4 1 0 . 1 0 . 1 . 5 1 0 . 1 0 . 1 . 6 1 0 . 1 0 . 1 . 7
  • 22.
    22 HOW CAN UNIKERNELSHELP ADDRESS OUR PROBLEMS? Application Config Application Language Runtime Shared Libraries Docker Runtime OS User Processes OS Kernel Virtual HW Drivers Hypervisor Hardware Drivers Hardware Minimal layers of isolation and abstraction Includes only what is really needed Less code, fewer bugs, easy to reason about
  • 23.
    23 DOCKER STACK VS.UNIKERNEL STACK Application Binary + Library OS Hypervisor Hardware Drivers Hardware Application Config Application Language Runtime Shared Libraries Docker Runtime OS User Processes OS Kernel Virtual HW Drivers Hypervisor Hardware Drivers Hardware
  • 24.
    24 DOCKER STACK VS.UNIKERNEL STACK Application Binary + Library OS Hypervisor Hardware Drivers Hardware Application Config Application Language Runtime Shared Libraries Docker Runtime OS User Processes OS Kernel Hardware Drivers Hardware Hardware isolation
  • 25.
    25 UNIKERNEL ADVANTAGES • Noother users, no multi-user support • No permission checks – you can utilize 100% of your hardware • Isolation at the virtual hardware – only ! share only hardware • Minimal virtual machine ~1 gb in size, minimal unikernel is tiny, kb in size • Very short boot time • A tiny custom surface of attack, less likely to be effected by a public exploit • Real immutable infrastructure – perfect fit to micro services architecture
  • 26.
    26 Backward compatibility Forwardcompatibility POSIX compliance Language specifics
  • 27.
  • 28.
  • 29.
    31 is an open-source toolwritten in Go for compiling applications into unikernels and deploying those unikernels across a variety of cloud providers, embedded devices (IoT), as well as a developer laptop or workstation.
  • 30.
    32 unik build --pathexample-app/ --base unikernel-type --language language --provider provider-name --name image-name unik run --instanceName instance-name –imageName image-name UNIK WORKFLOW
  • 31.
    33 UNIK IS NOTOPINIONATED ! Unikernel types Cloud providers Processor architectures
  • 32.
  • 33.
    35 UNIK INTEGRATION WITHDOCKER Docker API can be used to create unikernel via UniK
  • 34.
    36 UNIK INTEGRATION WITHKUBERNETES Unikernels support was added to Kubernetes by the UniK team by adding UniK as a container runtime to K8s - in the same way that Docker and rkt are container runtimes, UniK is now also available as a "container" runtime for k8s.
  • 35.
    37 UNIK KUBERNETES ARCHITECTURE unikernels Nowone can deploy a unikernel apps alongside regular kubernetes containerized apps.
  • 36.
  • 37.
  • 38.
    42 INTERNET OF THINGS UniKwill Push Unikernel To Raspberry Pi Unikernel will communicate with the Panini toaster Toaster will make Panini We will eat Panini bread
  • 39.
    43 WORRIED ABOUT IOTDDOS? THINK UNIKERNELS Could such attacks be prevented if IoT devices were running bare metal unikernels? Read our blog !
  • 40.
  • 41.
  • 42.
  • 43.
  • 44.
    49 EMC-Advanced-Dev http://www.github.com/emc-advanced-dev Idit Levine CTO ofCloud Platform Team & Office of the CTO, EMC @idit_levine Scott Weiss Software Engineer, EMC @ilackarms Yuval Kohavi Software Engineer, EMC @uvgroovy
  • 45.
  • 46.