Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Disruptive IP Networking with Chelsio 40GbE H/W Offload NIC


Published on

Disruptive IP Networking with Chelsio 40GbE H/W Offload NIC
09 Dec, 2013
SAKURA Internet Research Center
Senior Researcher / Naoto MATSUMOTO

Published in: Technology, Business
  • Be the first to comment

Disruptive IP Networking with Chelsio 40GbE H/W Offload NIC

  1. 1. 09 Dec, 2013 SAKURA Internet Research Center Senior Researcher / Naoto MATSUMOTO
  2. 2. Enabling ToDo 1) Download from support web 2) Install Drivers # # # # # tar xzvf ./ChelsioUwire- cd ChelsioUwire- make UNAME_R=2.6.32-358.el6 make UNAME_R=2.6.32-358.el6 install reboot © Copyright 2013 Chelsio Communications
  3. 3. Linux Kernel module check # modinfo t4_tom filename: /lib/modules/2.6.32-358.6.1.el6.x86_64/updates/kernel/drivers/net/offload/t4_tom/t4_tom.ko version: license: GPL author: Chelsio Communications description: TCP offload module for Chelsio T4/T5-based network cards srcversion: 677C1016F4208F8F816F0FF depends: toecore,ipv6,cxgb4 vermagic: 2.6.32-358.6.1.el6.x86_64 SMP mod_unload modversions parm: activated:whether to enable TOE at init time or not (int) parm: unsupported_allow_unload:allow UNSUPPORTED unloading of module (int) parm: cop_managed_offloading:all connection offloading decision managed by COP (int) parm: offload_vlan:Only Offload connections on the indicated VLAN (int) parm: tp_multi_chan:allow active opens to use TP c-channel mask (int) parm: send_page_order:order of page allocation for sendmsg (int)
  4. 4. ethtool check # ethtool eth1 Settings for eth1: Supported ports: [ ] Supported link modes: 40000baseSR4/Full Supported pause frame use: No Supports auto-negotiation: No Advertised link modes: Not reported Advertised pause frame use: No Advertised auto-negotiation: No Speed: 40000Mb/s Duplex: Full Port: Other PHYAD: 255 Transceiver: external Auto-negotiation: off Supports Wake-on: bg Wake-on: d Current message level: 0x000000ff (255) drv probe link timer ifdown ifup rx_err tx_err Link detected: yes
  5. 5. NIC Dirver Tuning w/ script # /opt/ChelsioUwire- Discovering Chelsio T4/T5 devices ... Configuring Chelsio T4/T5 devices ... Tuning eth1 IRQ table length 12 Writing 1 in /proc/irq/66/smp_affinity Writing 2 in /proc/irq/67/smp_affinity Writing 4 in /proc/irq/68/smp_affinity Writing 8 in /proc/irq/69/smp_affinity Writing 10 in /proc/irq/70/smp_affinity Writing 20 in /proc/irq/71/smp_affinity Writing 40 in /proc/irq/72/smp_affinity Writing 80 in /proc/irq/73/smp_affinity Writing 100 in /proc/irq/74/smp_affinity Writing 200 in /proc/irq/75/smp_affinity Writing 400 in /proc/irq/76/smp_affinity Writing 800 in /proc/irq/77/smp_affinity eth1 now up and tuned :
  6. 6. Build Packet Generator /w NETMAP 1) Creat Bootable USB memory for Packet Generator/Receiver Download Bin image from Burn to USB Memory using Win32 Disk Imager for multiple Traffic Generators. 2) Check vnstat on Reciver side (40GbE H/W NIC) # vnstat -i eth1 -l Monitoring eth1... (press CTRL-C to stop) rx: 17.68 Gbit/s 35352503 p/s tx: 2.54 Mbit/s 5087 p/s Source: SAKURA Internet Research Center. 11/2012: Project THORN
  7. 7. 10/40GbE-NIC Performance check 10/40GbE-NIC 64Byte Short Packet Processing Bechmark (RX) 1.39Gbit/s Need more Traffic Generators 1.41Gbit/s 2.81Mpps 1.39Gbit/s 2.62Mpps Corei7-3930K CPU @ 3.20GHz / 32GB DDR3-DIMM / PCI Express 3.0 using linux standard pkt-gen with NETMAP. Mpps (Million Packet/sec) SOURCE: SAKURA Internet Research Center. 12/2013 Project THORN.
  8. 8. 10/40/100GbE Networking GOAL
  9. 9. Thanks for your interest. SAKURA Internet Research Center.