Intel NFVi Enabling Kit Demo/Lab

Presented by
M Jay
Muthurajan.Jayakumar@intel.com
March 20, 2018

Copyright © 2017, Intel Corporation. All rights reserved.
*Other names and brands may be claimed as the property of others.
2
Optimization Notice: Intel’s compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations
that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other
optimizations. Intel does not guarantee the availability, functionality, or eﬀectiveness of any optimization on microprocessors not
manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors.
Certain optimizations not specifc to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable
product User and Reference Guides for more information regarding the specifc instruction sets covered by this notice. Cost reduction
scenarios described are intended as examples of how a given Intel- based product, in the specifed circumstances and confgurations,
may aﬀect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction. Intel
technologies’ features and benefts depend on system confguration and may require enabled hardware, software or service activation.
Performance varies depending on system confguration. No computer system can be absolutely secure. Check with your system
manufacturer or retailer or learn more at https://networkbuilders.intel.com/network-technologies/intelselectfasttrackkit.
No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.
Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a
particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in
trade.
This document contains information on products, services and/or processes in development. All information provided here is subject
to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps.
The products and services described may contain defects or errors known as errata which may cause deviations from published
specifications. Current characterized errata are available on request. Copies of documents which have an order number and are
referenced in this document may be obtained by calling 1-800-548-4725 or by visiting www.intel.com/design/literature.htm.
© 2017 Intel Corporation. Intel, the Intel logo, and Xeon are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other
countries.
Legal Information

3
Send your inputs to
M Jay
Muthurajan.Jayakumar@intel.com

Acknowledgements
All Contributing & Facilitating Intel® Select Solution Fast Track Kit
4
John Morgan & PAE
Team
Ai Bee, Lim Chiu-Pi, Shih
M Jay Nancy Yadav Hu, Xuekun
Zhang, Jerry Larry Wang Joel Schuetze
Auernheimer, Joel A Langley, Todd C Dirk Blevins
Sy Jong Choi Eric Hu Andrew Duignan
Ren Shu
Sujata Tibrewala Purvi Thakkar Ruparel, Chandresh
Anju Saraswat And Each and Everyone
Contributing &
Facilitating
Intel® Select Solution
Fast Track Kit

5
To cross the “NFVi-ready” ODM chasm

6
• Intel own a big portion of NFVi
software
• We need to ensures that the
combination of Software
Ingredients for ODM’s Intel
Platforms operate predictably.
• Major concern from ODM
1. Getting SW running
2. Deliver matching performance with
Intel.
ODM’s Biggest Chasm: Software
Open Source software stack
based on ETSI* NFV reference Architecture
Intel® QuickAssist Technology Drivers
Intel® Ethernet Drivers: 10 and 40 GbE
Linux* Fedora* OS
KVM Hypervisor
DPDK*
Open vSwitch*
OpenStack* Cloud OS
OpenDaylight* Controller*
OPNFV* project
Volume Server
Processor AccelerationNIC Silicon Storage

7
ETSI NFVI Architecture - Intel Select Fast Track Kit Mapping
Image Caption 10pt gray text

8
Agenda
BKC Installera) Plus
b) Base
Tests H/W & S/W
Checklist

9
INTEL NFVi EK – Intel® NFVi Enabling Kit
9
An easy-to-use, automatic, self-contained toolkit to
accelerate ODM* benchmarking NFVi-ready server
designs on Whitley platform. Help ODM to catch up the
business opportunity in 5G wave.
INTEL NFVi EK is a toolkit
Help ODM to ramp capability fast on NFVi
Ensure integration conformance align to NFVi BKC
Based on golden benchmark to characterize baseline performance
Leverage open methodology to generate KPI** w/ VNF workload
Automatic, self-contained with required scripts and SW components
Easy diagnosis and optimization
Benchmark NSB/ OPNFVNFVi BKC
Intel NFVi Enabling Kit
(INTEL NFVi EK)
(Automatic test suite: Conformance test, test cases,
performance indicators, report, analyze, etc.)
* ODM: Traditional ODMs with limited or no Comm expertise
** KPI: Key Performance Indicator from NSB/OPNFV which defines open and well-known
benchmark environment, methodology and VNF workloads
Conformance Baseline Perf NFVi/Network
/VNF KPIs

10
INTEL NFVi EK - Performance Baseline
• As a standard benchmark for DPDK, QAT
and Openvswitch
• With a verified traffic generator.
• KPI:
• Single core performance, verified platform
health level.
• Easily setup with NIC simple configuration
• Alignment with NPG benchmarking team
release performance report.
• Deliver a html version of RFC2544
throughput report.
INTEL NFVi EK
Ubuntu
P1 P2
IXIA
P3 P4
P3P1 P2 P4
DPDK
Traffic Gen L3fwd

11
Do You Want To Get DPDK Cookbook?

12
https://software.intel.com/en-us/article/get-the-dpdk-cookbook

13
Agenda
Needs of NFV Infrastructure
Quantitative Approach
Intel Select Solution for NFVi
What to Give and What to Get
Call For Action

14
Agenda
Needs of NFV Infrastructure
Quantitative Approach
Intel Select Solution for NFVi
What to Give and What to Get
Call For Action
1) Pkt Processing
3) Speed / Connections per sec
2) Crypto / Compression

15
Needs - What is important for your NFV Infrastructure?
A) Functional Correctness – Working well - with all features combinations
B) Extracting Full Performance from your Server – with above combinations
C) Tools to Probe and report status about both A) and B)
D) Tools to control and correct A) and B)
E) All Of The Above

16
E) All Of The Above

17
1) Functional Correctness
Change is the only constant. Lot of Moving Parts / Variables
Which CPU Version to go with which BIOS version? (Refer Document & implement)
Which Driver version? (Refer release notes and implement)
How many Getting Started Guides and Dear Customer Letters can you read?
How to feel comfortably confident about functional correctness ?
Base Platform --> h/w version firmware version OS Version
Drivers --> NIC PF Version DPDK Version QAT Version
Virtualization --> QEMU Version Libvirt Version NIC VF Version
NIC h/w, f/w --> NC Firmware NIC Transceivers
NIC Cable and
Length
Open Source
Agility Frequent Patches in all software releases &
in all firmware
releases

18
With those questions on Functionality, let us look at Performance

19
How do you know your NIC card is plugged in most optimal slot?
Not all slots
are made
equal !
2) Is Performance Optimal?
Green Servers
How do you know the system is optimized for performance or optimized for power?
Not all slots are made equal

20
How do you know memory is sufficient?
How do you know memory and I/O are optimally placed for NUMA?
I/O Plugged in CPU1’s Slot
How much memory do you
see in CPU1 node?
ZERO !
CPU 0 CPU 1
CPU 0 has only One
Channel memory
populated.
How do you know during run time …
Your Cores, you think you dedicated for tight polling…
Are not taken away by Linux Scheduler …

21
What if?
What if?
If there is a way you can compare with a reference platform
That has all the matching components – from patches, functionality point of view
That, also, quantitatively tells what conformance requirements are
Base Platform --> h/w version firmware version OS Version
Drivers --> NIC PF Version DPDK Version QAT Version
Virtualization --> QEMU Version Libvirt Version NIC VF Version
NIC h/w, f/w --> NC Firmware NIC Transceivers
NIC Cable and
Length
Open Source
Agility Frequent Patches in all software releases &
in all firmware
releases

22
Welcome to
Intel® Select Fast Track Kit for NFV Infrastructure

23
https://builders.intel.com/docs/networkbuilders/intel-select-fast-track-kit-for-nfvi.pdf

24
Blueprint & Corresponding Optimized NFV Server

25
Developers Benefit

26

27

28
Building Blocks – Intel Select Fast Track Kit

29
Configuration – Intel Select Fast Track Kit

30
OK. I am interested. Where can I get more info?

31
E) All Of The Above

32
Conformity Checklist - What if I want to build 2 systems –
1) One For highest Performance &
2) Another For Price / Performance sweet spot
How to measure each one - as how it stands in the scale?

33
Two Targets For Conformity – 1) Base, 2) Plus
BASE
Platform
PLUS
Platform

34
OK.
Conformity – What does it measure?
raw numbers?
or at
application level?

35
In 3 Dimensions - 1) Pkt Processing Throughput,
2) Security /Compression processing
3) Connections per second
1) Pkt Processing
(DPDK RFC 2544)
3) Speed / Connections per sec
OpenSSL
OpenSSL/NGiNX

36
1) Pkt Processing
(DPDK RFC 2544)
3) Connetions per sec
2) Crypto / compression
36

37
1) Pkt Processing
(DPDK RFC 2544)
3) Connections per sec
Next:
2) Security & Compression
OpenSSL / NGINX
OpenSSL

Where to Get Intel_NFVI_EK from?
git@github.intel.com:lwang47/Intel_NFVI_EK.git
https://github.intel.com/lwang47/Intel_NFVI_EK

40
For example Reference: Open source Doc at
http://fast.dpdk.org/doc/perf/DPDK_17_08_Intel_NIC_performance_report.pdf

Pre-Work That Was Done
41

Pre-Work that was done
apt-get update
apt-get install build-essential
apt-get install git
uname –r
Note down what it returns and use in the following ‘uname –r’
apt-get install pciutils libudev-dev libboost-dev linux-headers-`uname -r`
zlib1g-dev libssl-dev
apt-get install git

Preparing Server
cd /root
mkdir work
cd /root/work
mv /downloads/Intel_NFVI_EK.zip /root/work/.
unzip as shown next

Install Intel_NFVI_EK-dev
Intel_NFVI_EK.zip = => unzip to get Intel_NFVI_EK-dev.zip
[password intel.123]
* Intel_NFVI_EK-dev.zip = => unzip to get the contents

Directory Structure after extracting

Getting the Menu
sudo su –
cd /root/work
mv Intel_NFVI_EK-dev Intel_NFVI_EK
1st :To get the menu loaded
# python /root/work/Intel_NFVI_EK/src/tools/menu.py
2nd: To get the conformance report:-
# python /root/work/Intel_NFVI_EK/src/exec/server_info.py
1st : Menu Getting Loaded
Ping
10.27.156.74

Menu option 1 - Guide

Option 2 – Download BKC s/w packages
PROXY
ERRORS?

50
These Are End Results.
How to get there?
Is there checklist to get us there?
S/W
Checklist ?
H/W
Checklist ?

51
S/W
Checklist ?

52
BASE
Platform
PLUS
Platform
H/W
Checklist ?

53
What If My Checklist does not match?

54
E) All Of The Above

Base Versus Plus
./ubuntu.sh base Vs ./ubuntu plus
55

Checking Hardware
56
112
cores
Node 0
Siblings
Node 1
Siblings
Passed

Memory
57
NIC

NIC Firmware
58
NIC Slot

Chipset
59
QAT HW
NVMe
BIOS Version
BIOS Config

Drivers & OS - Version
60
QAT Driver
DPDK Version
NIC Drivers
QAT Driver

Virtualization
61
KVM/QEMU
Libvirt

62
How Can I find My Checklist does match?

Congratulations ! Your System Passed !!
63

Suppose If s/w did Not Confirm, What is the Next
Step?
Install All Those Needed correct software
64
E) All Of The Above

Lot Of Moving Parts
65

Manual Install Versus Auto Install
66
BKC
Installer

Contents of BKC Installer – 1) Ubuntu
67

Contents of BKC Installer – 1) Ubuntu
68

Demo
69
Attached the Instruction to run the system as well:-
simplify steps.
cd /root/work
./env.sh
Skylake SP 8160
Working Steps: 2,3,4,5,9
10.5.250.16
username: root
password: root
Skylake SP 6140
Working Steps: 2,3,4,5
10.5.250.23
username: root
password: root
Broadwell E5-2695 v4
Working Steps: 2,3,4,5,8
10.5.251.37
username: root
password: root245
Skylake SP 6130 for running QAT Sample Application only, 2 is
not required as all pre-download are done.
Working Steps: 2,9
10.5.248.116
username: root
password: root
Note: test result is at
/root/work/Intel_NFVI_EK/output/qat_Throughput.html

Contents of BKC Installer
70
RHEL

What To Expect: Installer – Directory & Files
71

Streamlined BKC Install
72

Pre Seed
ubuntu16-uefi.seed
73
Must Change

Trex* (Open Source Linux* Foundation s/w)
74

75

Trex – Configuring dpdk_setup
76
Python ./dpdk_setup_ports.py -i

77

Grafana – Windows – Page 1of 5
78
Step 4) Download nssm
Why? This will run Grafana as windows service.
How? https://nssm.cc/download
Ref: 1) https://www.youtube.com/watch?v=QJxk0V5i6qM
2) http://docs.grafana.org/installation/windows/
Step 1) Open a browser and type in
https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana-4.6.1.windows-x64.zip
Step 2) Unzip grafana-4.6.1.windows-x64.zip
Change 3000 to 8080
Step 5) Create a directory bin in c:/
Copy the nssm.exe to c:/bin
Step 3) Go To Config directory. Copy sample.ini to custom.ini. Edit custom.ini file. Remove the Semicolon before the word
http_port. Change 3000 to 8080 for http_port. Save custom.ini file

rfc2544
79

80
QAT Sample Code: ./cpa_sample_code
Intel® NFVI Select Compliance Design
Requirements
1) Pkt Processing
(DPDK RFC 2544)
3) Connections per sec

81

82

Step 6:
Open Command Prompt & Issue the two commands
cd bin
nssm install Grafana
83
Step 7: Follow the steps in screenshot
7a) Application
Point the path to grafana-server.exe

Step 7b) Application->Details -> Delayed Start
84
Step 7c) Logon as local system a/c

Step 8: C:binServices.msc
85
User name = admin
Password = admin
Step 9: Right Click & Start Grafana
Step 10: Open a Browser
127.0.0.1:8080/login

86
https://hodgkins.io/windows-metric-dashboards-with-influxdb-and-grafana
Tips & BKMs
Create a directory bin in c:/
Copy the nssm.exe to c:/bin
Extract both downloaded nssm and grafana in place in c:/downloads
Grafana – Windows – Page 5 of 5

Ports & Addresses
87
Usage Port # File Directory Notes
server “127.0.0.1” “25826” 25826 – is the port where
collectd will “transmit” on
the network i/f [andfluxDB
will listen for CollectD
metrics
Specified in CollectD config
file collectd.conf
/etc/collectd/collectd.conf And InfluxDB will listen for
CollectD metrics through
this same port – and it will
be mentioned in InfluxDB
conf file
[[collectd]]
bind-address =
“:25826”
“:25826” InfluxDB binds to
collectd with this <ip-
addr>:25826
/etc/influxdb/influxdb.c
onf
Collectd config uses this
port to inject into the
network to server
bind-address = “:8083” “:8083” Admin service for
influxdb
onf
Note, for admin
purposes, bind-address
becomes “<IP-
Address>:8083”
bind-address = “:8086” “:8086” Admin service for
influxdb
onf
Note, for data exchange
purposes, bind-address
becomes “<IP-
Address>:8086”
<ip-addr>:8086 :8086 In the GUI Grafana data source Influxdb config has
http: defined with
this port

collectd
Collectd in dynamic environment
https://www.youtube.com/watch?v=PWVAAQ2N2_k
88
Communication between InfluxDB and Collectd
https://groups.google.com/forum/#!topic/influxdb/4oDzeSGc-qY

89

Call To Action
Get the 2 Documents
List the Gives & Gets
Work on Your Gives & Gets
Contact M Jay Muthurajan.Jayakumar@intel.com
90

Optimization Notice
91
Optimization Notice
Intel’s compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that
are not unique to Intel microprocessors. These optimizations include SSE2®, SSE3, and SSSE3 instruction sets and
other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on
microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended
for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for
Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information
regarding the specific instruction sets covered by this notice.
Notice revision #20110804

Back up
92

/root/trex/v2.31/automation
93

94
/root/trex/v2.31/automation/trex_control_plane

95
/root/trex/v2.31/automation/trex_control_plane/unit_tests

Starting the Script
96

Check the BKC Version
97

Post Install Info
98

Contents – Looking Deeper
99

txt.zip
100

WW41
101

Dmesg with BKC_ww41
102

dmesg continues
103

104

Dmesg ..
105

ww41_BKC - Verify for Base Qualification
106

107

108

Virtualization
109

Still You Can Improve !
110

111

Collectd
Sudo apt-get update
112
https://github.com/collectd/collectd

sudo service collectd start
service collectd status
113
In case modifying conf file,
Service collectd restart

ls –al /etc/collectd
114
cat /etc/collectd/collection.conf
cat /etc/collectd/collect.conf

Apache2
Https://www.youtube.com/watch?v=JBghNRlF-I8
https://www.youtube.com/watch?v=i1vB7JnPvuE
Sudo apt-get update
Sudo apt-get install apache2
apachectl start
But there is no GUI to see the Browser interface of apache. How to do that?
115

Install full GUI
https://www.youtube.com/watch?v=6e0PuU2EHp8
Sudo tasksel
Select Ubuntu Gnome Desktop
Sudo reboot
PLEASE NOTE:
ssh still works with the updated latest password, whereas, new GUI only remembers
old original password – which in the BKC was password
How come it remembers old password? Or is this password from GUI install?
Open firefox browser and type in the IP address e.g., 192.168.0.7
116

Time zone setting - Good to have in UTC
117
Note Down Release and Code Name – 16.04 xenial
Install curl first
Sudo apt-get update
Sudo apt-get install curl
Installing InfluxDB
You should get response
Sudo apt-get update
It gives this error

Sudo apt-get install influxdb -y
118
Check the file /etc/influxdb/influxdb.conf – for admin section enabled and bind-address = “:8083”
Check the file /etc/influxdb/influxdb.conf – for http section enabled and bind-address = “:8086”

Verify both admin and http web interface
119
192.168.0.7:8083 192.168.0.7:8086
Time to Add Data Sources?
Let us enhance collectd with snmp-mibs for query
sudo apt-get install snmp-mibs-downloader
Let us go influxdb config file to enable collectd
Sudo vi etc/influxdb/influxdb.conf
See how is collectd section defined
It is disabled. Now let us enable it
Let it take its effect. sudo systemctl restart influxdb

service infludb status # You will see now influxdb talking to collectd on UDP
120

404 error on API interface – This is what you will get
https://github.com/influxdata/influxdb/issues/436
121

InfluxDB
122
https://docs.influxdata.com/influxdb/v1.3/concepts/key_concepts/

Reference
123
https://github.com/cisco-system-traffic-generator/trex-stateless-gui/releases/tag/v4.0
RFC2544
https://gerrit.fd.io/r/gitweb?p=csit.git;a=tree;f=resources;hb=HEAD
How to Install Grafana on Ubuntu 16.04 and Pull In home Assistant Data from InfluxDB
https://www.youtube.com/watch?v=oexrAKLQ_LI
InfluxDB – Ubuntu wget
wget https://dl.influxdata.com/influxdb/releases/influxdb_1.3.7_amd64.deb
sudo dpkg -i influxdb_1.3.7_amd64.deb
InfluxDB – Windows wget
unzip influxdb-1.3.7_windows_amd64.zip

Collectd
Collectd First Steps https://collectd.org/wiki/index.php/First_steps
Send CollectD Metric to InfluxDB https://anomaly.io/collectd-metrics-to-
influxdb/
Collectd https://collectd.org/
Collectd plug-ins https://collectd.org/wiki/index.php/Table_of_Plugins
https://collectd.org/wiki/index.php/First_steps
124

125
https://collectd.org/download.shtml

Hands-on TRex
sudo su -
cd /home/ubuntu/root/trex/v2.31
python dpdk_nic_bind.py - - status
python dpdk_nic_bind.py –u 0000:03:00.0
python dpdk_nic_bind py –u 0000:03:00.1
python dpdk_nic_bind.py –b ixgbe 03:00.0 03:0.1
126
sudo python dpdk_setup_ports.py -i
cp cfg/simple_cfg.yaml /etc/trex_cfg.yaml
sudo python dpdk_setup_ports.py -i

Collectd Plug-ins
Grafana – influxdb – collectd - demo
127

Apps trying to take all resources
128

Limit the scope of resource usage
129
Run again - ./t-rex-64 –c 4 -i
Still has to ask for less memory

Still reduced to limit_memory to 256
130
Looked at Huge page

131

Intel NFVi Enabling Kit Demo/Lab

More Related Content

What's hot

Similar to Intel NFVi Enabling Kit Demo/Lab

More from Michelle Holley

Recently uploaded

Intel NFVi Enabling Kit Demo/Lab