3. 3
DatacenterSolutionsGroup
TheCloudNativePath
Sources: BT on NFV in 2012; HPE http://community.hpe.com/t5/Telecom-IQ/HP-s-4-Stages-of-NFV/ba-p/6797122
Note: The timelines provided in the diagram have shifted, however, the transformation phases defined by HP in
2012 are still valid
decoupled
virtualized
cloudified
decomposed
2014 2015 2016 2017
functionsoutcomes
Standardize,
Open platforms
Better utilization
Elasticity and speed
Composable services
7. 7
DatacenterSolutionsGroup
AddresskeyChallengesincontainersBareMetal
* Open Source: Available on Intel github https://github.com/Intel-Corp | NFD at https://github.com/kubernetes-incubator/node-feature-discovery
Node Feature Discovery
CPU Manager for
Kubernetes
Native Huge page support
for Kubernetes
VHOST USER
SR-IOV
Open Source: CNI plug-in - V2.0 June ‘17
Upstream K8s: TBD
Open Source: CNI plug-in - V1.0 Sep ‘17
Open Source: CNI plug-in - V2.0 April ‘17
Open Source: Nov. ‘16
Upstream K8: Incubation Graduation TBD
Open Source: V1.2 April ‘17
Upstream K8: Phase 1 - V1.8 Sept ‘17
Upstream K8: V1.8 Sept ‘17
SOFTWAREAVAILABILITY*
Upstream collected: V5.7.2 June ‘17 ;
5.8.0
Challengesbeingaddressed Solution
Multiple network interfaces for VNFs
High performance Data Plane (E-W)
High performance Data Plane (N-S)
Ability to request/allocate platform capabilities
CPU Core-Pinning and isolation for K8s pods
Dynamic Huge Page allocation
Platform telemetry information
KubernetesNetworking
DataPlane Acceleration
EnhancePlatformAwareness(EPA)
Telemetry
11. 11
DatacenterSolutionsGroup
ContainerBaremetalExperienceKits
Reference Architecture
Reference Architecture
Installation Scripts
Reference Architecture
User Guide
Platform Telemetry
Technical Application Note
Feature Brief
Demo
Enhance Platform Awareness
Feature Brief
White Paper
Technical Application Note
Benchmark Test Report
Demo
Kubernetes Networking
Feature Brief
Technical Application Note
Demo
All collateral can be found at: https://networkbuilders.intel.com/network-technologies/container-experience-kits
12. 12
DatacenterSolutionsGroup
EngagewithIntel
Open Source
POC
Experience Kits
Best Practice Guidelines
Software community Engagewithintel
Container
Capabilities
CONTAINER
NETWORKING
Intel is addressing key challenges to using containers for NFV use cases
Most of these have been open sourced already
Explore more information available on Intel’s Network Builders site
https://networkbuilders.intel.com/network-technologies/container-experience-kits
VNF
16. 16
DatacenterSolutionsGroup
AddresskeyChallengesincontainersBareMetal
* Open Source: Available on Intel github https://github.com/Intel-Corp | NFD at https://github.com/kubernetes-incubator/node-feature-discovery
Node Feature Discovery
CPU Manager for
Kubernetes
Native Huge page support
for Kubernetes
VHOST USER
SR-IOV
Open Source: CNI plug-in - V2.0 June ‘17
Upstream K8s: TBD
Open Source: CNI plug-in - V1.0 Sep ‘17
Open Source: CNI plug-in - V2.0 April ‘17
Open Source: Nov. ‘16
Upstream K8: Incubation Graduation TBD
Open Source: V1.2 April ‘17
Upstream K8: Phase 1 - V1.8 Sept ‘17
Upstream K8: V1.8 Sept ‘17
SOFTWAREAVAILABILITY*
Upstream collected: V5.7.2 June ‘17 ;
5.8.0
Challengesbeingaddressed Solution
Multiple network interfaces for VNFs
High performance Data Plane (E-W)
High performance Data Plane (N-S)
Ability to request/allocate platform capabilities
CPU Core-Pinning and isolation for K8s pods
Dynamic Huge Page allocation
Platform telemetry information
KubernetesNetworking
DataPlane Acceleration
EnhancePlatformAwareness(EPA)
Telemetry
17. 17
DatacenterSolutionsGroup
MultipleNetworkInterfacesforVNFs
Kubernetes support only one Network interface – “eth0”
In NFV use cases, it is required to provide multiple network
interfaces to the virtualized operating environment of the VNF
eth0
Pod
eth1
eth2
eth0
interface
Pod
Container
ContainerContainer
Container
ContainerContainer
problem
Usecases
Functional separation of control and data network planes link
aggregation/bonding for redundancy of the network
Support for implementation of different network SLAs
Network segregation and Security
reference
Multus CNI – https://github.com/Intel-Corp/multus-cni
Native Kubernetes - Mailing list with details on discussions:
https://groups.google.com/forum/#!forum/kubernetes-sig-network
NetworkControlFlowwithMultus PodNetworkInterfaceswithMultus
KUBELET
SR-
IOV
Logging
Firewall
net0 net1
eth0
FlannelLinuxBridge
Kubernetes Pod
SR-IOV
net1
SR-IOV
net0eth0
LINUX BRIDGE VF0 VF1
18. 18
DatacenterSolutionsGroup
VhostUserCNIPlugin
No Container Networking with software acceleration
for NFV, particularly for East – West Traffic
problem
solution
Virtio_user/ vhost_user performance better than VETH pairs
Supports VPP as well as DPDK OVS
Vhost_user CNI plugin enables K8s to leverage data plane
acceleration
reference
https://github.com/intel/vhost-user-net-plugin (V1.0 Sep ’17)
NIC
eth0
OVS- DPDK/VPP
vhostuser
Kubernetes Pod
Container
VNF Application
DPDK
virtio_user
19. 19
DatacenterSolutionsGroup
DPDK–SR-IOVCNIPlugin
Lack of support for physical platform resource isolation
No guaranteed network IO performance
No support for Data Plane Networking
problem
solution
Allows SR-IOV support in Kubernetes via a CNI plugin
Supports two modes of operation:
1. SR-IOV: SR-IOV VFs are allocated to pod network namespace
2. DPDK: SR-IOV VFs are bounded to DPDK drivers in user space
reference
github.com/kubernetes-incubator/node-feature-discovery
Kernel
SR-IOV Enabled Network Interface
Kubernetes Pod
Container
VNF Application
VFVF VF
DPDK
uio_pci_generic/igb_uio/vfio-pci
20. 20
DatacenterSolutionsGroup
NodeFeatureDiscovery
No way to identify hardware capabilities or configuration
Inability for workload to request certain hardware feature
problem
solution
Node Feature Discovery (NFD) brings Enhanced Platform
Awareness (EPA) in K8s
NFD detects resources on each node in a Kubernetes cluster
and advertises those features
NFD allows matching of workload to platform capabilities
reference
github.com/Intel-Corp/sriov-cni
SR-IOV Network Features Single Root I/O Virtualization
AVX CPUID Features: Intel® Advances Vector Extensions 512 (Intel® AVX-512)
Turbo
Boost
Intel® Turbo Boost Technology processor accelerator
NodeFeatureDiscoveryLabelDetails
NODE 1
NFD
DISCOVERY
POD
NODE 2
NFD
DISCOVERY
POD
SR-IOV
AVX
Turbo
Boost
Application A
Application B
POD label:
Application B
Application A
POD label:
SR-IOV
AVX
MASTER
ETCD
NODE 1
NODE 2
NODEFEATUReDISCOVERYINK8s
21. 21
DatacenterSolutionsGroup
CPUManagerforKubernetes–CPUPinningandIsolation
Kubernetes has no mechanism to support core pinning and
isolation
Results in high priority workloads not achieving SLAs
problem
solution
CPU-Manager-For-Kubernetes introduces core pinning and
isolation to K8s without requiring changes to the code base
CMK guarantees high priority workloads are pinned to
exclusive cores
Gives a performance boost to high priority applications
Negates the noisy neighbor* scenario
reference
https://github.com/Intel-Corp/CPU-Manager-for-Kubernetes
WITHOUTCMK:CPUPinningandIsolation
Core0
CPU0 CPU1
Target
Workload
Core1
CPU2 CPU3
Noisy
Neighbor
Workload
Core0
CPU0 CPU1
Target
Workload
Core1
CPU2 CPU3
Noisy
Neighbor
Workload
WITHCMK:CPUPinningandIsolation
Noisy
Neighbor
Workload
* Noisy Neighbor Workload: An application that causes other virtual applications that share
the infrastructure to suffer from uneven performance
22. 22
DatacenterSolutionsGroup
HugepageNativeSupportinKubernetes
No resource management of Huge Pages in kubernetes
Responsibility of the cluster operator to handle it manually
problem
solution
Huge Pages introduced as first class resource in kubernetes
Support for Huge Pages via hugetlbfs enabled through a memory backed volume plugin
Inherent accounting of Huge Pages
Automatic relinquishing of Huge Pages in case of unexpected process termination
reference
Alpha support for pre-allocated hugepages
Hugetlbfs support based on empty dir volume plugin
23. 23
DatacenterSolutionsGroup
PlatformTelemetrySystemsupportinKubernetes
Compute Network Storage
Intel Run Sure Technologies
Resource
Telemetry
Interfaces
Open Collection
BasePlatform
Intel Infrastructure Management Technologies
See: Platform Service Assurance site (not including containers specific data):
https://networkbuilders.intel.com/network-technologies/serviceassurance
Container
Container and
Platform Telemetry
Platform
Telemetry
Container
Telemetry
Container
Telemetry
24. 24
DatacenterSolutionsGroup
Kuryr-Kubernetes
• Using Kuryr, we can also enable
Neutron Advanced Networking
features like Security groups, NAT etc.
• We can use any Neutron Plugin with
this like OVS, Linux Bridge, Midonet,
VPP
Openstack Neutron
API
Kuryr plugin
kubelet
Container Network Interface(CNI)
Kubernetes Network plugin
Kuryr–Kubernetes
Networkingcontrolflowdiagram
Pod Network ns
container container
br-tap
OVS Bridge Port Midonet Port
Linux Bridge Port
Any Neutron
Plug-in Port Type
VETH pair
tap
Bridge
Kuryr-Kubernetes
Networkingdataflowdiagram
Example:ContainerNetworkingusingKuryr
25. 25
DatacenterSolutionsGroup
Socket App
Nova
DPDK Pod Non DPDK
Pod
BSD Sockets API
Layer 4
Layer 3
Layer 2
Network
Appliance
Virt
I/O
Virt
I/O
Neutron
Virtual
Machine
MasterVMForContainersENABLINGDPDKinNestedContainers
One Virtual Machine to many Containers
Target: 1k Containers per VM
Container Data Plane performance using existing & well
supported abstractions
objectives
Usecase
Elasticity and scalability of containerized VNF application in VM
benefits
VT-x ring de-privileging to move the VM and Container into user
space, making it accessible to the user space vSwitch with just a
single copy
Standard Virtio interface that supports both interrupt and poll
modes, VNF and Cloud based applications
Standard Vhost shared memory interface between DPDK vSwitch
and VNF
solutions
Enabling DPDK in containers using VIRTIO
Using Kuryr–Kubernetes, orchestrate the Dataplane networking for
DPDK workload using the infrastructure vSwitch
26. 26
DatacenterSolutionsGroup
EngagewithIntel
Open Source
POC
Experience Kits
Best Practice Guidelines
Software community Engagewithintel
Container
Capabilities
CONTAINER
NETWORKING
Intel is addressing key challenges to using containers for NFV use cases
Many of these have been open sourced already
Explore more information available on Intel’s Network Builders site
https://networkbuilders.intel.com/network-technologies/container-experience-kits
VNF