Open Source NFV:
Upstream Headwaters to Full Deployment
Chris Wright
Chief Technologist
Red Hat, Inc.
Software is eating the
world Marc Andreessen
Open Source
Software is eating the
world
World without Open Source Software
No Amazon
No Google
No eBay
No PayPal
No Yahoo!
No Salesforce
No Linkedin
No Twitter
No Facebook
No Alibaba
IT Trend – Infrastructure
CPU
RAM
DISK
B
U
S
NET
SERVE
R
SERVERS
STORAGE
N
E
T
W
O
R
K
DATACENTE
R
IT Trend – Development
IT Trend – Business
Commoditization Innovation
Modern IT
Everybody wants…
Massive Scale
Agile Business
...to operate like large scale web companies
Gum sales
declining
Transportation
FinanceRetail
IT Infrastructure
Media
Software Disrupts Traditional Business
DISRUPT YOURSELF OR BE DISRUPTED
Open Source NFV
Network Functions Virtualization (NFV)
Network Functions are trapped in function specific HW
Expensive to procure, integrate, deploy and operate
Virtualize Network Functions (VNFs)
Distribute VNFs on COTS-based IaaS – a Cloud
Bring modern data center architecture to Telco network
Steer traffic with SDN
Traffic must traverse chain of functions in well-defined order
Why NFV?
Reduce time-to-market for new services
Improve business agility
Compete with over-the-top web-based services
Reduce CAPEX and OPEX costs
Virtual Network Function as a Service
Access Router
WAN Optimization
Firewall
CG-NAT
ADCIPS
NG-Firewall
DPI
Monitoring QoS
(ETSI #2)
Network Functions
Example: VNFaaS
Requires service chaining support in OpenDaylight, OpenStack
Neutron
Enable Nova instances as nodes in a Neutron service chain
SFC support in OpenDaylight
Performance
DPDK-accelerated Open vSwitch
Reliability
HA instances in Nova
Virtualized Mobile Base Station
-Multiple RAN technologies from
multiple vendors to be consolidated
on a single BS to improve utilization
-BS virtualization can share
resources among multiple logical
RAN nodes from different systems.
-C-RAN for efficient resource
utilization among different physical
BSs.
-Baseband Unit (BBU) pool with high
performance servers and real-time
processing for signaling capacity
(ETSI #4)
Example: C-RAN/vRAN
Determinism
Real-time patches to KVM
Performance
DPDK accelerated Open vSwitch
Reliability at scale
HA instances in OpenStack Nova
HA OpenStack
Fault management and reporting
IPv6 support – kernel through Nova, Neutron
NFV is not just OpenStack
NFV is a broad set of Communities
ETSI NFV driving industry trends, documenting
functional requirements
OPNFV integrating open source components,
communicating needs upstream
OpenStack abstraction/integration of virtual
compute, network, storage
libvirt, qemu/KVM, Ceph,
OpenDaylight, Open vSwitch
Underlying infrastructure
management
Upstream first
Successful changes should be designed, written and proposed
with upstream
Maintaining significant out-of-tree patches across multiple
projects is unsustainable
Upstream First
Requires discipline (small changes, discussed first)
Relationships (your priority might not be their priority)
Many communities – not just OpenStack!
Open Source NFV Summary
NFV requires a collection of multiple projects
Use-cases require features which touch many components
Maintaining forks of multiple projects is unsustainable
Multiple service providers and NEPs have similar requirements
Engaging with upstream projects early is required for success
Computing
Hardware
Storage
Hardware
Network
Hardware
Hardware resources
Virtualisation Layer
Virtualised
Infrastructure
Manager(s)
VNF
Manager(s)
VNF 2
OSS/BSS
NFVI
VNF 3VNF 1
Virtual
Computing
Virtual
Storage
Virtual
Network
EMS 2 EMS 3EMS 1
Service, VNF and Infrastructure
Description
Or-Vi
Or-Vnfm
Vi-Vnfm
Os-Ma
Se-Ma
Ve-Vnfm
Nf-Vi
Vn-Nf
Vl-Ha
Orchestrator
OpenStack
Linux
KVM OVS + DPDK
OPNFV
OpenDaylight
Open vSwitch (OVS)
Multi-layer virtual switch
Configuration managed w/ OVSDB
Flow tables controlled by OpenFlow
Provides connection between VMs on same host
Provides uplink to physical network via host NIC
Data fast path in-kernel
Challenges
kernel networking stack can be bottleneck
64 byte packet processing rates suffer
Microflows vs. megaflows
DPDK
Library for userspace packet processing
Directly manages NIC with userspace poll mode driver (PMD)
Drivers for both physical and virtual NICs
Polls driver NIC for packets, NIC DMAs directly to application
buffers
Platform specific optimizations
Hugepages, NUMA and cacheline aware
Batched packet processing
CPU instructions (SSE4, AVX, etc)
OVS + DPDK
Intel reports improved packet processing rates
NFV Relevant Open Source Projects
Project LOC Change rate Cost
Linux 17,285,731 6000/mo $500M
QEMU 902,252 500/mo $25M
libvirt 566,977 250/mo $15M
OpenDaylight 2,326,260 2000/mo $67M
OpenStack 2,526,686 5000/mo $72M
Open vSwitch 224,121 200/mo $5M
DPDK 1,466,904 200/mo $42M
http://www.openhub.net
Putting it all together
OpenStack Compute
KVM
OVS+DPDK
OpenStack Compute
KVM
OVS+DPDK
OpenStack Compute
KVM
OVS+DPDK
OpenDaylight
Controller
OpenStack
Networking
OpenFlow/OVSDB
ML2_ODL
OPNFV Bootstrap/Getting Started
with Foreman/Quickstack
Quick Overview/Review
Foreman
Jenkins
Khaleesi
Genesis
QuickStack
Build Scheduler
Ansible Test Framework
Host Provisioning
Foreman Initiated Puppet
Host Provisioning
Configures OpenStack with
Puppet
Current Deployment
Single POD in Intel lab with donated hardware
Baremetal hosts
Foreman server and hosts running CentOS 7
Admin (management), Public and Private networks
Minimum 2 servers per instance (more with HA)
OpenDaylight integration with OpenStack
Topology Overview
Foreman Server
OpenStack
Compute
OpenStack
Network/Control
OpenDaylightAdmin Network
Private Network
Public Network
Foreman Overview
ForemanForeman Proxy Puppet Master
DNS
DHCP
TFTP
BMC
Puppet CA
Host Groups
Host DB
IP Management
KickStart Templates
Global Params
Genesis
QuickStack
OpenStack
OpenDaylight
Provisioning Work Flow
Put Host into Build Mode with BMC
Reboot
Host PXE Boots using KickStart
Template
Foreman provides host associated OS
install + Puppet
Host Reboots into OS and starts
Puppet
Puppet applies Genesis -> ODL and
Quickstack
Foreman/QuickStack – OPNFV Arno Release
OpenStack HA + OpenDaylight achieved
Automatic deployment of OPNFV target system to virtual or
baremetal setups
Bootable Foreman/QuickStack ISO to assist in deployment
Functest Integration in a OPNFV CI environment
OPNFV – OpenStack HA + Opendaylight
Only installer to achieve O/S integration with ODL for Arno
All O/S services required for OPNFV run in HA -
https://wiki.opnfv.org/get_started/get_started_system_state
Includes automatically provisioned external network for tenant
access
Uses Ceph clustering backend for Cinder
Foreman/QuickStack Automatic Deployment
Uses a simple bash “deploy.sh” script to deploy an entire
OpenStack HA + ODL setup (3 control nodes, x compute nodes)
Minimal parameters needed – only need IPMI, mac address info
of baremetal
Network information and O/S parameters are dynamically
determined at runtime
Deploys to either baremetal or virtual machines
Automatic Deployment Process (baremetal)
deploy.sh -base_config my_settings.yml
Vagrantfile + OPNFV settings created for JumpHost
Vagrant up installs Khaleesi + provisions JumpHost
Khaleesi executes and installs Foreman pre-configured
with OPNFV nodes
Khaleesi issues API call to Foreman to rebuild and
provision nodes using QuickStack puppet modules
Foreman/QuickStack ISO
To be installed on the JumpHost as it's base OS
Based on CentOS 7
Contains artifacts to be able to deploy after installation
CI + Functest Integration
Foreman/QuickStack integrated into OPNFV CI environment
OPNFV CI environment is controlled by Jenkins on Linux
Foundation hardware
Functest team triggers a set of tests to execute after a deploy:
Vping VNF test
ODL Integration test suite
Tempest Smoke Tests
Rally benchmark tests
Thank you

OPNFV: Upstream Headwaters to Full Deployment

  • 1.
    Open Source NFV: UpstreamHeadwaters to Full Deployment Chris Wright Chief Technologist Red Hat, Inc.
  • 2.
    Software is eatingthe world Marc Andreessen
  • 3.
    Open Source Software iseating the world
  • 4.
    World without OpenSource Software No Amazon No Google No eBay No PayPal No Yahoo! No Salesforce No Linkedin No Twitter No Facebook No Alibaba
  • 5.
    IT Trend –Infrastructure CPU RAM DISK B U S NET SERVE R SERVERS STORAGE N E T W O R K DATACENTE R
  • 6.
    IT Trend –Development
  • 7.
    IT Trend –Business Commoditization Innovation
  • 8.
    Modern IT Everybody wants… MassiveScale Agile Business ...to operate like large scale web companies
  • 9.
  • 10.
  • 11.
    DISRUPT YOURSELF ORBE DISRUPTED
  • 12.
  • 13.
    Network Functions Virtualization(NFV) Network Functions are trapped in function specific HW Expensive to procure, integrate, deploy and operate Virtualize Network Functions (VNFs) Distribute VNFs on COTS-based IaaS – a Cloud Bring modern data center architecture to Telco network Steer traffic with SDN Traffic must traverse chain of functions in well-defined order
  • 14.
    Why NFV? Reduce time-to-marketfor new services Improve business agility Compete with over-the-top web-based services Reduce CAPEX and OPEX costs
  • 15.
    Virtual Network Functionas a Service Access Router WAN Optimization Firewall CG-NAT ADCIPS NG-Firewall DPI Monitoring QoS (ETSI #2) Network Functions
  • 16.
    Example: VNFaaS Requires servicechaining support in OpenDaylight, OpenStack Neutron Enable Nova instances as nodes in a Neutron service chain SFC support in OpenDaylight Performance DPDK-accelerated Open vSwitch Reliability HA instances in Nova
  • 17.
    Virtualized Mobile BaseStation -Multiple RAN technologies from multiple vendors to be consolidated on a single BS to improve utilization -BS virtualization can share resources among multiple logical RAN nodes from different systems. -C-RAN for efficient resource utilization among different physical BSs. -Baseband Unit (BBU) pool with high performance servers and real-time processing for signaling capacity (ETSI #4)
  • 18.
    Example: C-RAN/vRAN Determinism Real-time patchesto KVM Performance DPDK accelerated Open vSwitch Reliability at scale HA instances in OpenStack Nova HA OpenStack Fault management and reporting IPv6 support – kernel through Nova, Neutron
  • 19.
    NFV is notjust OpenStack
  • 20.
    NFV is abroad set of Communities ETSI NFV driving industry trends, documenting functional requirements OPNFV integrating open source components, communicating needs upstream OpenStack abstraction/integration of virtual compute, network, storage libvirt, qemu/KVM, Ceph, OpenDaylight, Open vSwitch Underlying infrastructure management
  • 21.
    Upstream first Successful changesshould be designed, written and proposed with upstream Maintaining significant out-of-tree patches across multiple projects is unsustainable
  • 22.
    Upstream First Requires discipline(small changes, discussed first) Relationships (your priority might not be their priority) Many communities – not just OpenStack!
  • 23.
    Open Source NFVSummary NFV requires a collection of multiple projects Use-cases require features which touch many components Maintaining forks of multiple projects is unsustainable Multiple service providers and NEPs have similar requirements Engaging with upstream projects early is required for success
  • 24.
    Computing Hardware Storage Hardware Network Hardware Hardware resources Virtualisation Layer Virtualised Infrastructure Manager(s) VNF Manager(s) VNF2 OSS/BSS NFVI VNF 3VNF 1 Virtual Computing Virtual Storage Virtual Network EMS 2 EMS 3EMS 1 Service, VNF and Infrastructure Description Or-Vi Or-Vnfm Vi-Vnfm Os-Ma Se-Ma Ve-Vnfm Nf-Vi Vn-Nf Vl-Ha Orchestrator OpenStack Linux KVM OVS + DPDK OPNFV OpenDaylight
  • 26.
    Open vSwitch (OVS) Multi-layervirtual switch Configuration managed w/ OVSDB Flow tables controlled by OpenFlow Provides connection between VMs on same host Provides uplink to physical network via host NIC Data fast path in-kernel Challenges kernel networking stack can be bottleneck 64 byte packet processing rates suffer Microflows vs. megaflows
  • 27.
    DPDK Library for userspacepacket processing Directly manages NIC with userspace poll mode driver (PMD) Drivers for both physical and virtual NICs Polls driver NIC for packets, NIC DMAs directly to application buffers Platform specific optimizations Hugepages, NUMA and cacheline aware Batched packet processing CPU instructions (SSE4, AVX, etc)
  • 28.
    OVS + DPDK Intelreports improved packet processing rates
  • 29.
    NFV Relevant OpenSource Projects Project LOC Change rate Cost Linux 17,285,731 6000/mo $500M QEMU 902,252 500/mo $25M libvirt 566,977 250/mo $15M OpenDaylight 2,326,260 2000/mo $67M OpenStack 2,526,686 5000/mo $72M Open vSwitch 224,121 200/mo $5M DPDK 1,466,904 200/mo $42M http://www.openhub.net
  • 30.
    Putting it alltogether OpenStack Compute KVM OVS+DPDK OpenStack Compute KVM OVS+DPDK OpenStack Compute KVM OVS+DPDK OpenDaylight Controller OpenStack Networking OpenFlow/OVSDB ML2_ODL
  • 31.
  • 32.
    Quick Overview/Review Foreman Jenkins Khaleesi Genesis QuickStack Build Scheduler AnsibleTest Framework Host Provisioning Foreman Initiated Puppet Host Provisioning Configures OpenStack with Puppet
  • 33.
    Current Deployment Single PODin Intel lab with donated hardware Baremetal hosts Foreman server and hosts running CentOS 7 Admin (management), Public and Private networks Minimum 2 servers per instance (more with HA) OpenDaylight integration with OpenStack
  • 34.
  • 35.
    Foreman Overview ForemanForeman ProxyPuppet Master DNS DHCP TFTP BMC Puppet CA Host Groups Host DB IP Management KickStart Templates Global Params Genesis QuickStack OpenStack OpenDaylight
  • 36.
    Provisioning Work Flow PutHost into Build Mode with BMC Reboot Host PXE Boots using KickStart Template Foreman provides host associated OS install + Puppet Host Reboots into OS and starts Puppet Puppet applies Genesis -> ODL and Quickstack
  • 37.
    Foreman/QuickStack – OPNFVArno Release OpenStack HA + OpenDaylight achieved Automatic deployment of OPNFV target system to virtual or baremetal setups Bootable Foreman/QuickStack ISO to assist in deployment Functest Integration in a OPNFV CI environment
  • 38.
    OPNFV – OpenStackHA + Opendaylight Only installer to achieve O/S integration with ODL for Arno All O/S services required for OPNFV run in HA - https://wiki.opnfv.org/get_started/get_started_system_state Includes automatically provisioned external network for tenant access Uses Ceph clustering backend for Cinder
  • 40.
    Foreman/QuickStack Automatic Deployment Usesa simple bash “deploy.sh” script to deploy an entire OpenStack HA + ODL setup (3 control nodes, x compute nodes) Minimal parameters needed – only need IPMI, mac address info of baremetal Network information and O/S parameters are dynamically determined at runtime Deploys to either baremetal or virtual machines
  • 41.
    Automatic Deployment Process(baremetal) deploy.sh -base_config my_settings.yml Vagrantfile + OPNFV settings created for JumpHost Vagrant up installs Khaleesi + provisions JumpHost Khaleesi executes and installs Foreman pre-configured with OPNFV nodes Khaleesi issues API call to Foreman to rebuild and provision nodes using QuickStack puppet modules
  • 42.
    Foreman/QuickStack ISO To beinstalled on the JumpHost as it's base OS Based on CentOS 7 Contains artifacts to be able to deploy after installation
  • 43.
    CI + FunctestIntegration Foreman/QuickStack integrated into OPNFV CI environment OPNFV CI environment is controlled by Jenkins on Linux Foundation hardware Functest team triggers a set of tests to execute after a deploy: Vping VNF test ODL Integration test suite Tempest Smoke Tests Rally benchmark tests
  • 44.

Editor's Notes