Deployment and orchestration of Unikernels in the NFV Infrastructure
Stefano Salsano
Project coordinator – Superfluidity project
Univ. of Rome Tor Vergata, Italy / CNIT, Italy
Software Networks and 5G: from network programmability to SDN/NFV combination
for effective network slicing - Workshop @ EuCNC 2017, Oulu, Finland, June 11th 2017
A super-fluid, cloud-native, converged edge system
Outline
• Superfluid NFV : towards sub 10 ms service instantiation
• Unikernels and their orchestration using VIMs
(Virtual Infrastructure Managers)
• “Quick & dirty” implementation of Unikernel orchestration over
OpenStack, OpenVIM and Nomad – Performance evaluation
• Extending ETSI NFV models to support Unikernels orchestation.
XEN / OpenVIM “clean” implementation. See our live demo!!
2
Superfluid Network Function VIrtualization
Superfluidity Goals
• Instantiate network functions and services on-the-fly
• Run them anywhere in the network (core, aggregation, edge), across
heterogeneous infrastructure environments (computing and networking),
taking advantage of specific hardware features, such as high performance
accelerators, when available
Superfluidity Approach
• Decomposition of network components and services into elementary and
reusable primitives (“Reusable Functional Blocks – RFBs”)
• Platform-independent abstractions, permitting reuse of network functions
across heterogeneous hardware platforms
3
The Superfluidity vision
4
Current NFV
technology
Granularity
Time scale
Superfluid
NFV
technology
Days, Hours Minutes Seconds Milliseconds
Big VMs
Small
components
Micro
operations • From VNF
Virtual Network Functions
to RFB
Reusable Functional Blocks
• Heterogeneous RFB execution
environments
- Hypervisors
- Modular routers
- Packet processors
…
Towards sub 10 ms service instantiation
Why a superfluid NFV
• Quick provisioning of services: Just-In-Time (JIT) proxies, firewalls, on-the-fly
monitoring
• Quick migration of services
• Optimized use of resources thanks to dynamic sharing
5
Unikernels: a tool for superfluid virtualization
Containers
e.g. Docker
• Lightweight (not enough?)
• Poor isolation
6
Hypervisors (traditional VMs)
e.g. XEN, KVM, wmware…
• Strong isolation
• Heavyweight
Unikernels
Specialized VMs (e.g. MiniOS, ClickOS…)
• Strong isolation
• Very Lightweight
• Very good security properties
They break the “myth” of VMs being heavy weight…
Outline
• Superfluid NFV : towards sub 10 ms service instantiation
• Unikernels and their orchestration using VIMs
(Virtual Infrastructure Managers)
• “Quick & dirty” implementation of Unikernel orchestration over
OpenStack, OpenVIM and Nomad – Performance evaluation
• Extending ETSI NFV models to support Unikernels orchestation.
XEN / OpenVIM “clean” implementation. See our live demo!!
7
What is a Unikernel?
• Specialized VM: single
application + minimalistic OS
• Single address space,
co-operative scheduler so low
overheads
• Unikernel virtualization
platforms extend existing
hypervisors (e.g. XEN)
driver1
driver2
app1
(e.g., Linux, FreeBSD)
KERNELSPACEUSERSPACE
app2
appNdriverN
Vdriver1
vdriver2
app
SINGLEADDRESS
SPACE
8
General purpose OS Unikernel
a minimalistic OS
(e.g., MiniOS, Osv)
Unikernels (ClickOS) memory footprint and boot time
VM configuration: MiniOS, 1 VCPU, 8MB RAM, 1 VIF
• 4 ms
• 87.77 ms
9
Boot time, state of the art results
Recent results from Superfluidity,
by redesigning the XEN toolstack
Memory footprint
• Hello world guest VM : 296 KB
• Ponger (ping responder) guest VM : ~700KB
Unikernels (ClickOS) memory footprint and boot time
VM configuration: MiniOS, 1 VCPU, 8MB RAM, 1 VIF
10
Boot time, state of the art results
Memory footprint
• Hello world guest VM : 296 KB
• Ponger (ping responder) guest VM : ~700KB
Recent results from Superfluidity,
by redesigning the XEN toolstack
• 4 ms
• 87.77 ms
VM instantiation and boot time
typical performance (no Unikernels)
11
Orchestrator
request
VIM
operations
Virtualization
Platform
Guest OS (VM)
Boot time
1-2 s
5-10 s
~1 s
VM instantiation and boot time
typical performance (no Unikernels)
12
Orchestrator
request
VIM
operations
Virtualization
Platform
Guest OS (VM)
Boot time
1-2 s
~1 ms
~1 ms
XEN Hypervisor
Enhancements
Unikernels Unikernels and Hypervisor can provide
low instantiation times for “Micro-VNF”
VM instantiation and boot time
typical performance (no Unikernels)
13
Orchestrator
request
VIM
operations
Virtualization
Platform
Guest OS (VM)
Boot time
1-2 s
~1 ms
~1 ms
XEN Hypervisor
Enhancements
Unikernels
Can we improve VIM
performances?
Unikernels and Hypervisor can provide
low instantiation times for “Micro-VNF”
Outline
• Superfluid NFV : towards sub 10 ms service instantiation
• Unikernels and their orchestration using VIMs
(Virtual Infrastructure Managers)
• “Quick & dirty” implementation of Unikernel orchestration over
OpenStack, OpenVIM and Nomad – Performance evaluation
• Extending ETSI NFV models to support Unikernels orchestation.
XEN / OpenVIM “clean” implementation. See our live demo!!
14
Performance analysis and Tuning of
Virtual Infrastructure Managers (VIMs) for Unikernel VNFs
• We considered 3 VIMs (OpenStack, Nomad, OpenVIM)
15
- General model of the VNF instantiation process, mapping of the
operations of the 3 VIMs in the general model
- (Quick & dirty) modifications to VIMs to instantiate Micro-VNFs
based on ClickOS Unikernel
- Performance Evaluation
Virtual Infrastructure Managers (VIMs)
We considered three VIMs :
• OpenStack Nova
– OpenStack is composed by subprojects
– Nova: orchestration and management of computing resources ---> VIM
– 1 Nova node (scheduling) + several compute nodes (which interact with the hypervisor)
– Not tied to a specific virtualization technology
• Nomad by HashiCorp
– Minimalistic cluster manager and job scheduler
– Nomad server (scheduling) + Nomad clients (interact with the hypervisor)
– Not tied to a specific virtualization technology
• OpenVIM
– NFV specific VIM, originally developed by the OpenMANO open source project, now
maintained in the context of ETSI OSM 16
Reference Model of the VNF instantiation process
17
Mapping of the reference model to the considered VIMs
18
Results – ClickOS instantiation times
(OpenStack, Nomad, OpenVIM)
19
OpenStack Nova
Nomad
seconds
seconds
OpenVIM
seconds
There is no comparison implied…
• NB: the purpose of the work is NOT to compare OpenStack vs. Nomad.
The goal is to understand how both behave and find ways to reduce
instantiation times.
• A direct comparison makes few sense. OpenStack is a much more
complete framework in terms of offered functionality and different
types of supported hypervisors. Moreover, the comparison is unfair
also because for the Nomad case we have developed a driver only
targeted to support the Xen/Click OS case.
20
Outline
• Superfluid NFV : towards sub 10 ms service instantiation
• Unikernels and their orchestration using VIMs
(Virtual Infrastructure Managers)
• “Quick & dirty” implementation of Unikernel orchestration over
OpenStack, OpenVIM and Nomad – Performance evaluation
• Extending ETSI NFV models to support Unikernels orchestation.
XEN / OpenVIM “clean” implementation. See our live demo!!
21
Extending the ETSI NFV models to support Unikernels
22
• In the NFV models, a Virtual Network Function (VNF) is decomposed in
Virtual Deployment Units (VDU)
• We extended the VDU information elements in the model to support
Unikernel VDUs (based on the ClickOS Unikernel)
• “Regular” VDUs based on traditional VMs and Unikernel VDUs can
coexist in the same service descriptor
Working prototype (come and see our demo!)
23
Orchestrator
RDCL 3D
VIM
OpenVIM
XEN
We enhanced XEN to
support both regular VMs
(HVM) and Click Unikernels
NSD
NSD
NSD
ETSI release 2
descriptors
NSD
NSD
VNFD
Our orchestrator
prototype
(RDCL 3D) uses
the enhanced VDU
descriptors and
interacts with
OpenVIM
OpenVIM has been
enhanced to support
XEN and Unikernels
Working prototype (come and see our demo!)
24
This is a regular
VM (XEN HVM)
These are 3
Unikernel VMs
(ClickOS)
Conclusions
• We have considered the optimization of Unikernel virtualization and the needed
enhancements to Virtual Infrastructure Managers to support Unikernels.
• In the Superfluidity vision, Unikernels are interesting as they support the
decomposition of network services in “smaller” components that can be deployed
on the fly (NB: Unikernels are complementary to other approaches!)
• We have extended NFV Infrastructure in order to support Unikernel virtualization in
addition to traditional VMs. In particular we have enhanced XEN and OpenVIM (and
developed quick prototypes for OpenStack and Nomad).
• There is still room for improvement of VIM performance…
– e.g. OpenStack (~ 1 s), Nomad (~ 300 ms), OpenVIM (~ 2-300 ms)
25
Conclusions
• Unikernel virtualization can provide VM instantiation and boot time in the order of ms,
VM memory footprints in the order of few MB.
• It is possible to add Unikernel support in VIMs… VIMs are currently designed for
generality, the challenge is to specialize them in a flexible way, keeping the
compatibility with the mainstream versions and achieving the desired performance.
• Unikernels could prove useful in specific contexts. The final goal is to design systems
that exploit the most efficient solutions (VMs, container, Unikernels) depending on
several factors.
26
Thank you. Questions?
Contacts
Stefano Salsano
University of Rome Tor Vergata / CNIT
stefano.salsano@uniroma2.it
These tools are available on github (Apache 2.0 license)
https://github.com/superfluidity/RDCL3D
https://github.com/netgroup/vim-tuning-and-eval-tools
http://superfluidity.eu/
The work presented here only covers a subset of the work performed in the project
27
References
• SUPERFLUIDITY project Home Page http://superfluidity.eu/
• G. Bianchi, et al. “Superfluidity: a flexible functional architecture for 5G networks”, Transactions on
Emerging Telecommunications Technologies 27, no. 9, Sep 2016
• P. L. Ventre, C. Pisa, S. Salsano, G. Siracusano, F. Schmidt, P. Lungaroni,
N. Blefari-Melazzi, “Performance Evaluation and Tuning of Virtual Infrastructure Managers for
(Micro) Virtual Network Functions”,
IEEE NFV-SDN Conference, Palo Alto, USA, 7-9 November 2016
http://netgroup.uniroma2.it/Stefano_Salsano/papers/salsano-ieee-nfv-sdn-2016-vim-performance-for-unikernels.pdf
• S. Salsano, F. Lombardo, C. Pisa, P. Greto, N. Blefari-Melazzi,
“RDCL 3D, a Model Agnostic Web Framework for the Design and Composition of NFV Services”,
submitted paper, https://arxiv.org/abs/1702.08242
28
References – Speed up of Virtualization Platforms / Guests
• J. Martins, M. Ahmed, C. Raiciu, V. Olteanu, M. Honda, R. Bifulco, F. Huici, “ClickOS and the art
of network function virtualization”, NSDI 2014, 11th USENIX Conference on Networked
Systems Design and Implementation, 2014.
• F. Manco, J. Martins, K. Yasukata, J. Mendes, S. Kuenzer, F. Huici,
“The Case for the Superfluid Cloud”, 7th USENIX Workshop on Hot Topics in Cloud Computing
(HotCloud 15), 2015
• Recent unpublished results are included in this presentation:
S. Salsano, F. Huici, “Superfluid NFV: VMs and Virtual Infrastructure Managers speed-up for
instantaneous service instantiation”, invited talk at EWSDN 2016 workshop, 10 October 2016,
The Hague, Netherlands
http://www.slideshare.net/stefanosalsano/superfluid-nfv-vms-and-virtual-infrastructure-managers-speedup-for-instantaneous-service-instantiation
29
The SUPERFLUIDITY project has received funding from the European Union’s Horizon
2020 research and innovation programme under grant agreement No.671566
(Research and Innovation Action).
The information given is the author’s view and does not necessarily represent the view
of the European Commission (EC). No liability is accepted for any use that may be
made of the information contained.
30

Deploying of Unikernels in the NFV Infrastructure

  • 1.
    Deployment and orchestrationof Unikernels in the NFV Infrastructure Stefano Salsano Project coordinator – Superfluidity project Univ. of Rome Tor Vergata, Italy / CNIT, Italy Software Networks and 5G: from network programmability to SDN/NFV combination for effective network slicing - Workshop @ EuCNC 2017, Oulu, Finland, June 11th 2017 A super-fluid, cloud-native, converged edge system
  • 2.
    Outline • Superfluid NFV: towards sub 10 ms service instantiation • Unikernels and their orchestration using VIMs (Virtual Infrastructure Managers) • “Quick & dirty” implementation of Unikernel orchestration over OpenStack, OpenVIM and Nomad – Performance evaluation • Extending ETSI NFV models to support Unikernels orchestation. XEN / OpenVIM “clean” implementation. See our live demo!! 2
  • 3.
    Superfluid Network FunctionVIrtualization Superfluidity Goals • Instantiate network functions and services on-the-fly • Run them anywhere in the network (core, aggregation, edge), across heterogeneous infrastructure environments (computing and networking), taking advantage of specific hardware features, such as high performance accelerators, when available Superfluidity Approach • Decomposition of network components and services into elementary and reusable primitives (“Reusable Functional Blocks – RFBs”) • Platform-independent abstractions, permitting reuse of network functions across heterogeneous hardware platforms 3
  • 4.
    The Superfluidity vision 4 CurrentNFV technology Granularity Time scale Superfluid NFV technology Days, Hours Minutes Seconds Milliseconds Big VMs Small components Micro operations • From VNF Virtual Network Functions to RFB Reusable Functional Blocks • Heterogeneous RFB execution environments - Hypervisors - Modular routers - Packet processors …
  • 5.
    Towards sub 10ms service instantiation Why a superfluid NFV • Quick provisioning of services: Just-In-Time (JIT) proxies, firewalls, on-the-fly monitoring • Quick migration of services • Optimized use of resources thanks to dynamic sharing 5
  • 6.
    Unikernels: a toolfor superfluid virtualization Containers e.g. Docker • Lightweight (not enough?) • Poor isolation 6 Hypervisors (traditional VMs) e.g. XEN, KVM, wmware… • Strong isolation • Heavyweight Unikernels Specialized VMs (e.g. MiniOS, ClickOS…) • Strong isolation • Very Lightweight • Very good security properties They break the “myth” of VMs being heavy weight…
  • 7.
    Outline • Superfluid NFV: towards sub 10 ms service instantiation • Unikernels and their orchestration using VIMs (Virtual Infrastructure Managers) • “Quick & dirty” implementation of Unikernel orchestration over OpenStack, OpenVIM and Nomad – Performance evaluation • Extending ETSI NFV models to support Unikernels orchestation. XEN / OpenVIM “clean” implementation. See our live demo!! 7
  • 8.
    What is aUnikernel? • Specialized VM: single application + minimalistic OS • Single address space, co-operative scheduler so low overheads • Unikernel virtualization platforms extend existing hypervisors (e.g. XEN) driver1 driver2 app1 (e.g., Linux, FreeBSD) KERNELSPACEUSERSPACE app2 appNdriverN Vdriver1 vdriver2 app SINGLEADDRESS SPACE 8 General purpose OS Unikernel a minimalistic OS (e.g., MiniOS, Osv)
  • 9.
    Unikernels (ClickOS) memoryfootprint and boot time VM configuration: MiniOS, 1 VCPU, 8MB RAM, 1 VIF • 4 ms • 87.77 ms 9 Boot time, state of the art results Recent results from Superfluidity, by redesigning the XEN toolstack Memory footprint • Hello world guest VM : 296 KB • Ponger (ping responder) guest VM : ~700KB
  • 10.
    Unikernels (ClickOS) memoryfootprint and boot time VM configuration: MiniOS, 1 VCPU, 8MB RAM, 1 VIF 10 Boot time, state of the art results Memory footprint • Hello world guest VM : 296 KB • Ponger (ping responder) guest VM : ~700KB Recent results from Superfluidity, by redesigning the XEN toolstack • 4 ms • 87.77 ms
  • 11.
    VM instantiation andboot time typical performance (no Unikernels) 11 Orchestrator request VIM operations Virtualization Platform Guest OS (VM) Boot time 1-2 s 5-10 s ~1 s
  • 12.
    VM instantiation andboot time typical performance (no Unikernels) 12 Orchestrator request VIM operations Virtualization Platform Guest OS (VM) Boot time 1-2 s ~1 ms ~1 ms XEN Hypervisor Enhancements Unikernels Unikernels and Hypervisor can provide low instantiation times for “Micro-VNF”
  • 13.
    VM instantiation andboot time typical performance (no Unikernels) 13 Orchestrator request VIM operations Virtualization Platform Guest OS (VM) Boot time 1-2 s ~1 ms ~1 ms XEN Hypervisor Enhancements Unikernels Can we improve VIM performances? Unikernels and Hypervisor can provide low instantiation times for “Micro-VNF”
  • 14.
    Outline • Superfluid NFV: towards sub 10 ms service instantiation • Unikernels and their orchestration using VIMs (Virtual Infrastructure Managers) • “Quick & dirty” implementation of Unikernel orchestration over OpenStack, OpenVIM and Nomad – Performance evaluation • Extending ETSI NFV models to support Unikernels orchestation. XEN / OpenVIM “clean” implementation. See our live demo!! 14
  • 15.
    Performance analysis andTuning of Virtual Infrastructure Managers (VIMs) for Unikernel VNFs • We considered 3 VIMs (OpenStack, Nomad, OpenVIM) 15 - General model of the VNF instantiation process, mapping of the operations of the 3 VIMs in the general model - (Quick & dirty) modifications to VIMs to instantiate Micro-VNFs based on ClickOS Unikernel - Performance Evaluation
  • 16.
    Virtual Infrastructure Managers(VIMs) We considered three VIMs : • OpenStack Nova – OpenStack is composed by subprojects – Nova: orchestration and management of computing resources ---> VIM – 1 Nova node (scheduling) + several compute nodes (which interact with the hypervisor) – Not tied to a specific virtualization technology • Nomad by HashiCorp – Minimalistic cluster manager and job scheduler – Nomad server (scheduling) + Nomad clients (interact with the hypervisor) – Not tied to a specific virtualization technology • OpenVIM – NFV specific VIM, originally developed by the OpenMANO open source project, now maintained in the context of ETSI OSM 16
  • 17.
    Reference Model ofthe VNF instantiation process 17
  • 18.
    Mapping of thereference model to the considered VIMs 18
  • 19.
    Results – ClickOSinstantiation times (OpenStack, Nomad, OpenVIM) 19 OpenStack Nova Nomad seconds seconds OpenVIM seconds
  • 20.
    There is nocomparison implied… • NB: the purpose of the work is NOT to compare OpenStack vs. Nomad. The goal is to understand how both behave and find ways to reduce instantiation times. • A direct comparison makes few sense. OpenStack is a much more complete framework in terms of offered functionality and different types of supported hypervisors. Moreover, the comparison is unfair also because for the Nomad case we have developed a driver only targeted to support the Xen/Click OS case. 20
  • 21.
    Outline • Superfluid NFV: towards sub 10 ms service instantiation • Unikernels and their orchestration using VIMs (Virtual Infrastructure Managers) • “Quick & dirty” implementation of Unikernel orchestration over OpenStack, OpenVIM and Nomad – Performance evaluation • Extending ETSI NFV models to support Unikernels orchestation. XEN / OpenVIM “clean” implementation. See our live demo!! 21
  • 22.
    Extending the ETSINFV models to support Unikernels 22 • In the NFV models, a Virtual Network Function (VNF) is decomposed in Virtual Deployment Units (VDU) • We extended the VDU information elements in the model to support Unikernel VDUs (based on the ClickOS Unikernel) • “Regular” VDUs based on traditional VMs and Unikernel VDUs can coexist in the same service descriptor
  • 23.
    Working prototype (comeand see our demo!) 23 Orchestrator RDCL 3D VIM OpenVIM XEN We enhanced XEN to support both regular VMs (HVM) and Click Unikernels NSD NSD NSD ETSI release 2 descriptors NSD NSD VNFD Our orchestrator prototype (RDCL 3D) uses the enhanced VDU descriptors and interacts with OpenVIM OpenVIM has been enhanced to support XEN and Unikernels
  • 24.
    Working prototype (comeand see our demo!) 24 This is a regular VM (XEN HVM) These are 3 Unikernel VMs (ClickOS)
  • 25.
    Conclusions • We haveconsidered the optimization of Unikernel virtualization and the needed enhancements to Virtual Infrastructure Managers to support Unikernels. • In the Superfluidity vision, Unikernels are interesting as they support the decomposition of network services in “smaller” components that can be deployed on the fly (NB: Unikernels are complementary to other approaches!) • We have extended NFV Infrastructure in order to support Unikernel virtualization in addition to traditional VMs. In particular we have enhanced XEN and OpenVIM (and developed quick prototypes for OpenStack and Nomad). • There is still room for improvement of VIM performance… – e.g. OpenStack (~ 1 s), Nomad (~ 300 ms), OpenVIM (~ 2-300 ms) 25
  • 26.
    Conclusions • Unikernel virtualizationcan provide VM instantiation and boot time in the order of ms, VM memory footprints in the order of few MB. • It is possible to add Unikernel support in VIMs… VIMs are currently designed for generality, the challenge is to specialize them in a flexible way, keeping the compatibility with the mainstream versions and achieving the desired performance. • Unikernels could prove useful in specific contexts. The final goal is to design systems that exploit the most efficient solutions (VMs, container, Unikernels) depending on several factors. 26
  • 27.
    Thank you. Questions? Contacts StefanoSalsano University of Rome Tor Vergata / CNIT stefano.salsano@uniroma2.it These tools are available on github (Apache 2.0 license) https://github.com/superfluidity/RDCL3D https://github.com/netgroup/vim-tuning-and-eval-tools http://superfluidity.eu/ The work presented here only covers a subset of the work performed in the project 27
  • 28.
    References • SUPERFLUIDITY projectHome Page http://superfluidity.eu/ • G. Bianchi, et al. “Superfluidity: a flexible functional architecture for 5G networks”, Transactions on Emerging Telecommunications Technologies 27, no. 9, Sep 2016 • P. L. Ventre, C. Pisa, S. Salsano, G. Siracusano, F. Schmidt, P. Lungaroni, N. Blefari-Melazzi, “Performance Evaluation and Tuning of Virtual Infrastructure Managers for (Micro) Virtual Network Functions”, IEEE NFV-SDN Conference, Palo Alto, USA, 7-9 November 2016 http://netgroup.uniroma2.it/Stefano_Salsano/papers/salsano-ieee-nfv-sdn-2016-vim-performance-for-unikernels.pdf • S. Salsano, F. Lombardo, C. Pisa, P. Greto, N. Blefari-Melazzi, “RDCL 3D, a Model Agnostic Web Framework for the Design and Composition of NFV Services”, submitted paper, https://arxiv.org/abs/1702.08242 28
  • 29.
    References – Speedup of Virtualization Platforms / Guests • J. Martins, M. Ahmed, C. Raiciu, V. Olteanu, M. Honda, R. Bifulco, F. Huici, “ClickOS and the art of network function virtualization”, NSDI 2014, 11th USENIX Conference on Networked Systems Design and Implementation, 2014. • F. Manco, J. Martins, K. Yasukata, J. Mendes, S. Kuenzer, F. Huici, “The Case for the Superfluid Cloud”, 7th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 15), 2015 • Recent unpublished results are included in this presentation: S. Salsano, F. Huici, “Superfluid NFV: VMs and Virtual Infrastructure Managers speed-up for instantaneous service instantiation”, invited talk at EWSDN 2016 workshop, 10 October 2016, The Hague, Netherlands http://www.slideshare.net/stefanosalsano/superfluid-nfv-vms-and-virtual-infrastructure-managers-speedup-for-instantaneous-service-instantiation 29
  • 30.
    The SUPERFLUIDITY projecthas received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No.671566 (Research and Innovation Action). The information given is the author’s view and does not necessarily represent the view of the European Commission (EC). No liability is accepted for any use that may be made of the information contained. 30