Successfully reported this slideshow.

Disruptive IP Networking with Chelsio 40GbE H/W Offload NIC

1,664 views

Published on

Disruptive IP Networking with Chelsio 40GbE H/W Offload NIC
09 Dec, 2013
SAKURA Internet Research Center
Senior Researcher / Naoto MATSUMOTO

Published in: Technology, Business
  • Be the first to comment

Disruptive IP Networking with Chelsio 40GbE H/W Offload NIC

  1. 1. 09 Dec, 2013 SAKURA Internet Research Center Senior Researcher / Naoto MATSUMOTO
  2. 2. Enabling ToDo 1) Download from support web http://service.chelsio.com/ 2) Install Drivers # # # # # tar xzvf ./ChelsioUwire-2.8.0.0.tar.gz cd ChelsioUwire-2.8.0.0 make UNAME_R=2.6.32-358.el6 make UNAME_R=2.6.32-358.el6 install reboot © Copyright 2013 Chelsio Communications
  3. 3. Linux Kernel module check # modinfo t4_tom filename: /lib/modules/2.6.32-358.6.1.el6.x86_64/updates/kernel/drivers/net/offload/t4_tom/t4_tom.ko version: 2.8.0.0 license: GPL author: Chelsio Communications description: TCP offload module for Chelsio T4/T5-based network cards srcversion: 677C1016F4208F8F816F0FF depends: toecore,ipv6,cxgb4 vermagic: 2.6.32-358.6.1.el6.x86_64 SMP mod_unload modversions parm: activated:whether to enable TOE at init time or not (int) parm: unsupported_allow_unload:allow UNSUPPORTED unloading of module (int) parm: cop_managed_offloading:all connection offloading decision managed by COP (int) parm: offload_vlan:Only Offload connections on the indicated VLAN (int) parm: tp_multi_chan:allow active opens to use TP c-channel mask (int) parm: send_page_order:order of page allocation for sendmsg (int)
  4. 4. ethtool check # ethtool eth1 Settings for eth1: Supported ports: [ ] Supported link modes: 40000baseSR4/Full Supported pause frame use: No Supports auto-negotiation: No Advertised link modes: Not reported Advertised pause frame use: No Advertised auto-negotiation: No Speed: 40000Mb/s Duplex: Full Port: Other PHYAD: 255 Transceiver: external Auto-negotiation: off Supports Wake-on: bg Wake-on: d Current message level: 0x000000ff (255) drv probe link timer ifdown ifup rx_err tx_err Link detected: yes
  5. 5. NIC Dirver Tuning w/ script # /opt/ChelsioUwire-2.8.0.0/tools/t4_perftune.sh Discovering Chelsio T4/T5 devices ... Configuring Chelsio T4/T5 devices ... Tuning eth1 IRQ table length 12 Writing 1 in /proc/irq/66/smp_affinity Writing 2 in /proc/irq/67/smp_affinity Writing 4 in /proc/irq/68/smp_affinity Writing 8 in /proc/irq/69/smp_affinity Writing 10 in /proc/irq/70/smp_affinity Writing 20 in /proc/irq/71/smp_affinity Writing 40 in /proc/irq/72/smp_affinity Writing 80 in /proc/irq/73/smp_affinity Writing 100 in /proc/irq/74/smp_affinity Writing 200 in /proc/irq/75/smp_affinity Writing 400 in /proc/irq/76/smp_affinity Writing 800 in /proc/irq/77/smp_affinity eth1 now up and tuned :
  6. 6. Build Packet Generator /w NETMAP 1) Creat Bootable USB memory for Packet Generator/Receiver Download Bin image from http://info.iet.unipi.it/~luigi/netmap/ Burn to USB Memory using Win32 Disk Imager for multiple Traffic Generators. 2) Check vnstat on Reciver side (40GbE H/W NIC) # vnstat -i eth1 -l Monitoring eth1... (press CTRL-C to stop) rx: 17.68 Gbit/s 35352503 p/s tx: 2.54 Mbit/s 5087 p/s Source: SAKURA Internet Research Center. 11/2012: Project THORN
  7. 7. 10/40GbE-NIC Performance check 10/40GbE-NIC 64Byte Short Packet Processing Bechmark (RX) 1.39Gbit/s Need more Traffic Generators 1.41Gbit/s 2.81Mpps 1.39Gbit/s 2.62Mpps Corei7-3930K CPU @ 3.20GHz / 32GB DDR3-DIMM / PCI Express 3.0 using linux standard pkt-gen with NETMAP. Mpps (Million Packet/sec) SOURCE: SAKURA Internet Research Center. 12/2013 Project THORN.
  8. 8. 10/40/100GbE Networking GOAL
  9. 9. Thanks for your interest. SAKURA Internet Research Center.

×