Enabling Converged Networks with iSCSI over Data Center Bridging


Published on

Joe Gervais,

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Enabling Converged Networks with iSCSI over Data Center Bridging

  1. 1. Enabling Converged Networks with iSCSI over Data Center Bridging<br />Joe Gervais<br />Senior Director, Product Marketing, Emulex<br />
  2. 2. Agenda<br />iSCSI over Data Center Bridging (DCB) Defined<br />Enabling Convergence<br />Companion Technologies For 10G iSCSI over DCB<br />Sample Topology<br />2<br />
  3. 3. iSCSI over DCB Defined<br />3<br />
  4. 4. iSCSI over DCB<br />iSCSI over Data Center Bridging (DCB) protocols a.k.a. Enterprise iSCSI<br />Data Center Bridging Capabilities eXchange Protocol (DCBX)<br />Supports centralized configuration of DCB and related protocols<br />Initiated by endpoints (hosts/storage) announce themselves with iSCSI Type/Length Value (TLV)<br />Enhanced Transmission Selection<br />Provides priority groups with bandwidth controls<br />Priority Flow Control (PFC)<br />Enable lossless Ethernet operation to provide deterministic performance<br />What’s needed<br />iSCSI HBA supporting iSCSI over DCB<br />Could be a NIC – some limitations<br />10G Ethernet Switch supporting DCB<br />10G Storage Array supporting iSCSI over DCB<br />4<br />
  5. 5. Data Center Bridging Exchange Protocol (DCBX)<br />Automatic “handshake” to allow device to exchange Enhanced Ethernet capabilities<br /><ul><li> Simplifies configuration
  6. 6. Identify misconfigurations
  7. 7. Allows switch to override server parameters</li></ul> Based on Link Layer Discovery Protocol (LLDP)<br />
  8. 8. Lossless Ethernet<br />Receive Queues<br />Transmit Queues<br />Ethernet Link<br />Priority<br />Priority<br />Zero<br />One<br />Two<br />Three<br />Four<br />Five<br />Six<br />Seven<br />Zero<br />One<br />Two<br />Three<br />Four<br />Five<br />Six<br />Seven<br />PAUSE<br />STOP<br />Essential to have lossless, consistent, reliable storage when sharing the same network regardless of LAN traffic conditions<br />DCB designed to address this concern using PFC for different traffic types<br />Uses VLAN priorities - each can be paused<br />Ensures zero data loss when receive queues are full<br />Implementation specifics may limit efficiency of PFC<br />Hardware iSCSI vs. Software iSCSI<br />
  9. 9. Lossless Ethernet - Hardware iSCSI<br />App<br />App<br />App<br />App<br />App<br />App<br />Server Hardware<br />OS<br />OS<br />OS<br />OS<br />OS<br />OS<br />Hypervisor<br />OS<br />LAN app traffic<br />Hardware iSCSI<br />iSCSI Buffer Pool<br />NIC Buffer Pool<br />iSCSI Target<br />Adapter Hardware<br />LAN<br />Initiates iSCSI transactions, instructs OS to pre-allocate buffers for transaction<br />PG2<br />PG1<br />CNA Port<br />
  10. 10. Lossless Ethernet - Software iSCSI<br />App<br />App<br />App<br />App<br />App<br />App<br />Server Hardware<br />OS<br />OS<br />OS<br />OS<br />OS<br />OS<br />iSCSI traffic suffers from unnecessary pause frames especially with concurrent TCP/UDP traffic… therefore not eliminating jitter<br />Hypervisor<br />OS<br />LAN app traffic<br />Software iSCSI<br />NIC Buffer Pool<br />Shared between LAN and iSCSI traffic<br />iSCSI Target<br />Adapter Hardware<br />PG1<br />LAN<br />PG2<br />No iSCSI session awareness, hence cannot pre-allocate buffers in shared OS buffer pool. Buffer over-run results in upstream Pause Frames<br />Priority<br />Pause<br />Frame<br />CNA Port<br />QOS is compromised<br />
  11. 11. Enabling Convergence<br />9<br />
  12. 12. Barriers to Convergence<br />iSCSI on Gigabit Ethernet best practices architected separate Ethernet network for iSCSI traffic<br />Servers - iSCSI on separate NICs<br />Switches – iSCSI on separate switches<br />Rational<br />Congestion causes packet loss, creating non-deterministic performance<br />TCP/IP error recovery expensive from a predictable latency perspective<br />In virtualized environments, best practices employ multiple NICs for traffic isolation (iSCSI, vMotion, production, management, etc.)<br />Enables effective bandwidth caps between services, ensures available bandwidth at server<br />10<br />
  13. 13. Enabling Converged iSCSI<br />Large Enterprise<br />Provide Deterministic Performance<br />Requires Lossless Ethernet (DCB)<br />Support Convergence<br />DCB support<br />Multichannel Support<br />Enterprise class management<br />Small / Medium Enterprise<br />Support 1 -> 10 migration<br />Multichannel support<br />DCB support<br />Support 1G backwards compatibility<br />10GBASE-T support<br />Easy-to-use tools for configuration/management<br />DCBX based network configuration<br />10GBaseT<br />Lossless Ethernet<br />Multichannel<br />User Experience<br />
  14. 14. User Experience<br />Hardware iSCSI enhances other aspects of the user experience<br />
  15. 15. User Experience - Scalability and Reliability<br />Hardware iSCSI<br />Software iSCSI<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />OS<br />OS<br />OS<br />OS<br />OS<br />OS<br />OS<br />OS<br />VM1<br />VM2<br />VM3<br />VM26<br />VM1<br />VM2<br />VM3<br />VM17<br />…<br />…<br />53% more<br />Ethernet<br />Switch<br />Ethernet<br />Switch<br />iSCSI storage<br />iSCSI storage<br />Hardware iSCSI demonstrated support for more VM’s per server, given performance objective of 5k IOPs per VM<br />Reduced need for physical servers translate to power and cost savings<br />Attachments to iSCSI targets maintained in adapter hardware – immunity to host disruptions that could lead to session loss in software initiators<br />
  16. 16. OneCommand Manager provides a single, uniform management interface<br />Procedures for attaching to iSCSI targets are the same across supported OS<br />Alternatively, can use VMware vSphere Client to manage OneConnect iSCSI adapter<br />SCSI Management API (IMA) plugin provided for VMware<br />Hardware iSCSI device viewed in vSphere Client under Storage Adaptersin the Configuration tab<br />Less complicated than software iSCSI, which requires configuring a Vmkernel connection to ESX, then binding to the physical NIC<br />User Experience - Manageability<br />
  17. 17. Boot from SAN <br />OneConnect iSCSI adapter can attach to an iSCSI target at boot time<br />iSCSISelect Utility is used to specify and configure a boot device on a particular LUN<br />Invoked at boot-time by pressing CTRL-S<br />Configures iSCSI initiator to attach to desired target<br />LUN can be configured as boot device<br />
  18. 18. Current versions of vSphere don’t directly support DCB protocols<br />No APIs for a NIC to support DCBX and PFC.<br />Since HBAs/CNAs offload iSCSI, DCB transparent to OS<br />DCBX and PFC all handled by device<br />Host to Device interaction all at SCSI layer<br />iSCSI over DCB with VMware<br />Hardware iSCSI<br />App<br />App<br />App<br />App<br />OS<br />OS<br />OS<br />OS<br />VM1<br />VM2<br />VM3<br />VM26<br />…<br />Ethernet<br />Switch<br />iSCSI storage<br />
  19. 19. Companion Technologies For 10G iSCSI over DCB<br />17<br />
  20. 20. Technology<br />Universal Multi-Channel (UMC) presents multiple (typically 4) PCI functions per port to the OS <br />Partitions the port into independent channels of user-defined bandwidth<br />Switch-agnostic<br />Multiple channels can be segmented based on MAC address and/or customer-provided VLAN ID (C-VLAN)<br />Benefits<br />Retains networking best practices in 1G to 10G migration<br />Physical 1G port becomes logical 10G port<br />Transparent to operating systems<br />OS still sees dedicated NIC devices<br />Complementary to DCB traffic shaping with ETS<br />Bandwidth controls at both logical NIC and priority group level<br />Multi-Channel<br />Unique PCI function per logical port<br />Physical Port<br />Channels<br />Network/IP<br />Storage/iSCSI<br />MultifunctionAdapters<br />Network/IP<br />Network/IP<br />
  21. 21. Low Cost Media Options for iSCSI over DCB<br />10GBASE-T<br />Backwards compatible with Gigabit Ethernet<br />Uses Cat6/Cat6A unshielded twisted pair cabling with RJ-45 connectors<br />Supports 100m distances<br />Top of Rack, End of Row (EOR)<br />SFP+ Direct Attach Copper<br />iSCSI adapter with SFP cage<br />Twinaxial Cable with SFP modules on ends <br />Supports 5-7m distances <br />Top of Rack<br />Optical versions available supporting 100m or more<br />
  22. 22. Sample iSCSI over DCB Topology<br />20<br />
  23. 23. Enterprise iSCSI Solution Example<br />iSCSI Configuration today<br />Sample Solution Overview<br />PowerEdge servers (R710) in VMware cluster<br />EqualLogic array (PS6010XVS)<br />PowerConnect switch (B-8000e)<br />Emulex Enterprise iSCSI HBAs<br />vCenter iSCSI IMA Plug-in<br />IMA<br />Plugin<br />
  24. 24. Enterprise iSCSI Solution Example<br />DCBX iSCSI TLV<br />DCBX iSCSI TLV<br />DCBX Enable PFC<br />DCBX Enable 9000B MTU<br />DCBX Enable ETS PG2 BW 1G<br />DCBX Enable PFC<br />DCBX Enable 9000B MTU<br />DCBX Enable ETS PG2 BW 1G<br />
  25. 25. Emulex OneConnect<br />10 Gigabit Ethernet products for Dell PowerEdge Servers<br />NIC, iSCSI and FCoE models<br />iSCSI over DCB support with OCe11000 family of products<br />Built in support for DCB with Microsoft Windows, VMware vSphere, Red Hat and Novell SUSE Linux operating systems<br />SFP+ and 10GBASE-T models<br />
  26. 26. iSCSI over DCBBenefits Summary<br />Enables convergence of LAN and SAN traffic<br />DCB enables mixed traffic<br />Multichannel enables multiple logical NICs in one 10G link<br />Deterministic performance in converged, mixed traffic environment<br />Lossless Ethernet using Priority Flow Control enables support<br />Ease of use<br />Centralized configuration via switch with DCBX support<br />Hosts, storage announce capabilities with iSCSI TLV<br />Switch enables PFC, Jumboframes, other parameters via DCBX<br />Easy server integration<br />iSCSI HBAs such as Emulex OneConnect provide transparent support<br />24<br />
  27. 27. See http://www.emulex-dell.com/<br />To register for the Emulex quarterly Dell newsletter <br />For information on Emulex OneConnect products supporting iSCSI over DCB<br />For information about other Emulex solutions for Dell<br />For iSCSI over DCB white paper<br />Resources & Call to Action<br />
  28. 28. Thank you<br />26<br />