• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
How to Speak Intel DPDK KNI for Web Services.
 

How to Speak Intel DPDK KNI for Web Services.

on

  • 2,973 views

How to Speak Intel DPDK KNI for Web Services.

How to Speak Intel DPDK KNI for Web Services.
31 Oct, 2013
SAKURA Internet Research Center
Senior Researcher / Naoto MATSUMOTO

Statistics

Views

Total Views
2,973
Views on SlideShare
2,951
Embed Views
22

Actions

Likes
2
Downloads
0
Comments
9

1 Embed 22

https://twitter.com 22

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

19 of 9 previous next Post a comment

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • @naotomatsumoto
    Thanks for the suggestion. I indeed use SRIOV, direct pass-through a VF (virtual function) to the VM under KVM hypervisor. I will continue trying and let you know if I can make KNI in VM working.
    Are you sure you want to
    Your message goes here
    Processing…
  • Now I think that the idea of DPDK is to encourage people to use it's user-level DPDK library for better performance. The KNI module is for compatibility support in order for legacy socket application to work on top of DPDK. As a result,it makes sense that the performance of KNI is not as good as standard interrupt mode driver without using DPDK.
    Are you sure you want to
    Your message goes here
    Processing…
  • Please try it using SR-IOV on VMM and Attache to directly Intel NIC for VM and using KNI if you must need it scenario.
    Are you sure you want to
    Your message goes here
    Processing…
  • Nop, It's only Beta-metal server scenario.
    Are you sure you want to
    Your message goes here
    Processing…
  • Hi Naoto MATSUMOTO,

    I'm trying to use the same steps in your slides, but running it inside a VM. I followed the Intel DPDK programming guide, section 9.1 and 9.2. Eventually in VM, I enable the kni module and I could see the vEth0_0.

    However, my vEth0_0 in VM could not receive any packet. Have you every tried this scenario? Thank you!
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    How to Speak Intel DPDK KNI for Web Services. How to Speak Intel DPDK KNI for Web Services. Presentation Transcript

    • 31 Oct, 2013 SAKURA Internet Research Center Senior Researcher / Naoto MATSUMOTO
    • Our LAB’s Goal: To research and deploy about near future technology* for global IT industory. *The timescale is between in 3 to 5 years. Source: SAKURA Internet Research Center 10/2013, Project THORN.
    • Intel® DPDK KNI Overview 1) 2) 3) 4) Intel® DPDK source codes for linux were released at End of 2012. 6WIND Announces dpdk.org Open-Source Project at April 2013. Added KNI(Kernel Network Interface) function at Mid 2013. dpdk-1.5.0r1 was released at Oct 2013. Web Services HTTPD (without App modification) User Kernel Linux Sockets vEth0_0 Intel DPDK KNI Intel DPDK (PMD drivers) Physical NIC(s) FYI: Disruptive IP Networking with Intel DPDK on Linux @SlideShare http://www.slideshare.net/naotomatsumoto/disruptive-ip-networking-with-intel-dpdk-on-linux Source: SAKURA Internet Research Center 10/2013, Project THORN.
    • Buildup Intel® DPDK KNI Env 1) Install Intel® DPDK # uname –a Linux 2.6.32-358.6.1.el6.x86_64 #1 SMP Tue Apr 23 16:15:13 CDT 2013 x86_64 x86_64 x86_64 GNU # cd /opt; tar xzvf ./dpdk-1.5.0r1.tar.gz; cd dpdk-1.5.0r1 # make config T=x86_64-default-linuxapp-gcc # make install T=x86_64-default-linuxapp-gcc # make # rmmod ixgbe # modprobe uio; insmod build/kmod/igb_uio.ko; insmod build/kmod/rte_kni.ko kthread_mode=multiple # mkdir -p /hugetlb # mount -t hugetlbfs nodev /hugetlb # echo 4096 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages 2) Intel ® DPDK PMD Testing # build/app/testpmd -c7 -n3 -- -i --nb-cores=2 --nb-ports=2 3) Configure Intel® DPDK KNI module # # # # export RTE_SDK=/opt/dpdk-1.5.0r1 cd /opt/dpdk-1.5.0r1/examples/kni make ./build/kni -c 0xf0 -n 4 -- -p 0x3 --config="(0,4,6,8),(1,5,7,9)" & 4) Intel® DPDK KNI Operation # ifconfig vEth0 192.168.0.1 # ethtool -d vEth0 # tcpdump -i vEth0 Source: SAKURA Internet Research Center 10/2013, Project THORN.
    • Intel® DPDK KNI boot sequence # ./build/kni -c 0xf0 -n 4 -- -p 0x3 --config="(0,4,6,8),(1,5,7,9)“ EAL: EAL: EAL: : EAL: EAL: EAL: : EAL: EAL: : APP: KNI: APP: KNI: Detected lcore 0 as core 0 on socket 0 Detected lcore 1 as core 1 on socket 0 Detected lcore 2 as core 2 on socket 0 Setting up memory... Ask a virtual area of 0x2097152 bytes Virtual area found at 0x7f1d8a000000 (size = 0x200000) Web Services HTTPD TSC frequency is ~3299939 KHz Master core 4 is ready (tid=8b3bc800) Initialising port 0 ... pci: 0c:00:00 8086:10fb Initialising port 1 ... pci: 0c:00:01 8086:10fb Figure 2. Kernel NIC Application Packet Flow, 8.0 Kernel NIC Interface Sample Application Intel® Data Plane Development Kit (Intel® DPDK) Sample Applications User Guide September 2013 Checking link status ..........................................................................................done Port 0 Link Up - speed 10000 Mbps - full-duplex Port 1 Link Down APP: Lcore 5 is reading from port 1 APP: Lcore 6 is writing to port 0 APP: Lcore 7 is writing to port 1 APP: Lcore 4 is reading from port 0 Source: SAKURA Internet Research Center 10/2013, Project THORN.
    • Intel® DPDK KNI Operation # ip link show 21: vEth0_0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 22: vEth1_0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff # ip addr add 192.168.0.1/24 dev vEth0_0 # ethtool vEth0_0 Settings for vEth0_0: Supported ports: [ FIBRE ] Supported link modes: 10000baseT/Full Supported pause frame use: No Supports auto-negotiation: No Advertised link modes: 10000baseT/Full Advertised pause frame use: No Advertised auto-negotiation: No Speed: 10000Mb/s Duplex: Full Port: FIBRE PHYAD: 0 Transceiver: external Auto-negotiation: off Supports Wake-on: d Wake-on: d : Link detected: yes Source: SAKURA Internet Research Center 10/2013, Project THORN.
    • Intel® DPDK KNI Monitoring # vnstat -l -i vEth0_0 Monitoring vEth0_0... rx: (press CTRL-C to stop) 800.90 Mbit/s 1601806 p/s tx: 0 kbit/s 0 p/s # tcpdump -i vEth0_0 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vEth0_0, link-type EN10MB (Ethernet), capture size 65535 bytes 00:55:46.407878 IP 192.168.0.9.bre > 192.168.0.1.spytechphone: UDP, length 22 00:55:46.407884 IP 192.168.0.9.bre > 192.168.0.1.spytechphone: UDP, length 22 00:55:46.407886 IP 192.168.0.9.bre > 192.168.0.1.spytechphone: UDP, length 22 00:55:46.407887 IP 192.168.0.9.bre > 192.168.0.1.spytechphone: UDP, length 22 00:55:46.407889 IP 192.168.0.9.bre > 192.168.0.1.spytechphone: UDP, length 22 00:55:46.407890 IP 192.168.0.9.bre > 192.168.0.1.spytechphone: UDP, length 22 00:55:46.407891 IP 192.168.0.9.bre > 192.168.0.1.spytechphone: UDP, length 22 00:55:46.407892 IP 192.168.0.9.bre > 192.168.0.1.spytechphone: UDP, length 22 # dropwatch Initalizing null lookup method dropwatch> *Does not Effect… Source: SAKURA Internet Research Center 10/2013, Project THORN.
    • Intel® DPDK KNI perf check # perf record ./build/kni -c 0xf0 -n 4 -- -p 0x3 --config="(0,4,6,8),(1,5,7,9)“ # perf report Samples: 662K of event 'cycles', Event count (approx.): 660643513353 41.29% kni kni [.] kni_allocate_mbufs 30.80% kni kni [.] main_loop 9.24% kni kni [.] ixgbe_recv_pkts 8.93% kni kni [.] rte_delay_us 5.22% kni kni [.] rte_kni_tx_burst : # perf stat ./build/kni -c 0xf0 -n 4 -- -p 0x3 --config="(0,4,6,8),(1,5,7,9)" ^C./build/kni: Interrupt Performance counter stats for './build/kni -c 0xf0 -n 4 -- -p 0x3 --config=(0,4,6,8),(1,5,7,9)': 123704.616399 274 13 9,996 493,863,586,483 309,423,280,316 164,745,543,981 585,543,402,812 task-clock context-switches CPU-migrations page-faults cycles stalled-cycles-frontend stalled-cycles-backend instructions 99,577,432,345 branches 116,202,928 branch-misses 43.487826969 seconds time elapsed # 2.845 CPUs utilized # 0.002 K/sec # 0.000 K/sec # 0.081 K/sec # 3.992 GHz # 62.65% frontend cycles idle # 33.36% backend cycles idle # 1.19 insns per cycle # 0.53 stalled cycles per insn # 804.961 M/sec # 0.12% of all branches [83.33%] [83.33%] [66.67%] [83.33%] [83.33%] [83.33%] Source: SAKURA Internet Research Center 10/2013, Project THORN.
    • Intel® DPDK KNI CPU check # ./build/kni -c 0xf0 -n 4 -- -p 0x3 --config="(0,4,6,8),(1,5,7,9)“ & # pkt-gen –i ix0 -f tx -l 64 -D 00:0C:BD:00:E8:15 (on Traffic Generator side) # top top - 18:31:52 up 20 min, 4 users, load average: 4.17, 3.31, 1.98 Tasks: 337 total, 3 running, 334 sleeping, 0 stopped, 0 zombie Cpu0 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu1 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu2 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu3 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu4 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu5 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu6 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu7 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu8 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu9 : 0.0%us, 97.0%sy, 0.0%ni, 3.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu10 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu11 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 32831124k total, 9119152k used, 23711972k free, 18796k buffers Swap: 7565304k total, 0k used, 7565304k free, 370952k cached Source: SAKURA Internet Research Center 10/2013, Project THORN.
    • Thanks for your interest. SAKURA Internet Research Center. Source: SAKURA Internet Research Center 10/2013, Project THORN.