Enabling Converged Networks with iSCSI over Data Center Bridging

Enabling Converged Networks with iSCSI over Data Center Bridging
Joe Gervais
Senior Director, Product Marketing, Emulex

Agenda
iSCSI over Data Center Bridging (DCB) Defined
Enabling Convergence
Companion Technologies For 10G iSCSI over DCB
Sample Topology
2

iSCSI over DCB Defined
3

iSCSI over DCB
iSCSI over Data Center Bridging (DCB) protocols a.k.a. Enterprise iSCSI
Data Center Bridging Capabilities eXchange Protocol (DCBX)
Supports centralized configuration of DCB and related protocols
Initiated by endpoints (hosts/storage) announce themselves with iSCSI Type/Length Value (TLV)
Enhanced Transmission Selection
Provides priority groups with bandwidth controls
Priority Flow Control (PFC)
Enable lossless Ethernet operation to provide deterministic performance
What’s needed
iSCSI HBA supporting iSCSI over DCB
Could be a NIC – some limitations
10G Ethernet Switch supporting DCB
10G Storage Array supporting iSCSI over DCB
4

Data Center Bridging Exchange Protocol (DCBX)
Automatic “handshake” to allow device to exchange Enhanced Ethernet capabilities
Simplifies configuration
Identify misconfigurations
Allows switch to override server parameters
Based on Link Layer Discovery Protocol (LLDP)

Lossless Ethernet
Receive Queues
Transmit Queues
Ethernet Link
Priority
Priority
Zero
One
Two
Three
Four
Five
Six
Seven
Zero
One
Two
Three
Four
Five
Six
Seven
PAUSE
STOP
Essential to have lossless, consistent, reliable storage when sharing the same network regardless of LAN traffic conditions
DCB designed to address this concern using PFC for different traffic types
Uses VLAN priorities - each can be paused
Ensures zero data loss when receive queues are full
Implementation specifics may limit efficiency of PFC
Hardware iSCSI vs. Software iSCSI

Lossless Ethernet - Hardware iSCSI
App
App
App
App
App
App
Server Hardware
OS
OS
OS
OS
OS
OS
Hypervisor
OS
LAN app traffic
Hardware iSCSI
iSCSI Buffer Pool
NIC Buffer Pool
iSCSI Target
Adapter Hardware
LAN
Initiates iSCSI transactions, instructs OS to pre-allocate buffers for transaction
PG2
PG1
CNA Port

Lossless Ethernet - Software iSCSI
App
App
App
App
App
App
Server Hardware
OS
OS
OS
OS
OS
OS
iSCSI traffic suffers from unnecessary pause frames especially with concurrent TCP/UDP traffic… therefore not eliminating jitter
Hypervisor
OS
LAN app traffic
Software iSCSI
NIC Buffer Pool
Shared between LAN and iSCSI traffic
iSCSI Target
Adapter Hardware
PG1
LAN
PG2
No iSCSI session awareness, hence cannot pre-allocate buffers in shared OS buffer pool. Buffer over-run results in upstream Pause Frames
Priority
Pause
Frame
CNA Port
QOS is compromised

Enabling Convergence
9

Barriers to Convergence
iSCSI on Gigabit Ethernet best practices architected separate Ethernet network for iSCSI traffic
Servers - iSCSI on separate NICs
Switches – iSCSI on separate switches
Rational
Congestion causes packet loss, creating non-deterministic performance
TCP/IP error recovery expensive from a predictable latency perspective
In virtualized environments, best practices employ multiple NICs for traffic isolation (iSCSI, vMotion, production, management, etc.)
Enables effective bandwidth caps between services, ensures available bandwidth at server
10

Enabling Converged iSCSI
Large Enterprise
Provide Deterministic Performance
Requires Lossless Ethernet (DCB)
Support Convergence
DCB support
Multichannel Support
Enterprise class management
Small / Medium Enterprise
Support 1 -> 10 migration
Multichannel support
DCB support
Support 1G backwards compatibility
10GBASE-T support
Easy-to-use tools for configuration/management
DCBX based network configuration
10GBaseT
Lossless Ethernet
Multichannel
User Experience

User Experience
Hardware iSCSI enhances other aspects of the user experience

User Experience - Scalability and Reliability
Hardware iSCSI
Software iSCSI
App
App
App
App
App
App
App
App
OS
OS
OS
OS
OS
OS
OS
OS
VM1
VM2
VM3
VM26
VM1
VM2
VM3
VM17
…
…
53% more
Ethernet
Switch
Ethernet
Switch
iSCSI storage
iSCSI storage
Hardware iSCSI demonstrated support for more VM’s per server, given performance objective of 5k IOPs per VM
Reduced need for physical servers translate to power and cost savings
Attachments to iSCSI targets maintained in adapter hardware – immunity to host disruptions that could lead to session loss in software initiators

OneCommand Manager provides a single, uniform management interface
Procedures for attaching to iSCSI targets are the same across supported OS
Alternatively, can use VMware vSphere Client to manage OneConnect iSCSI adapter
SCSI Management API (IMA) plugin provided for VMware
Hardware iSCSI device viewed in vSphere Client under Storage Adaptersin the Configuration tab
Less complicated than software iSCSI, which requires configuring a Vmkernel connection to ESX, then binding to the physical NIC
User Experience - Manageability

Boot from SAN
OneConnect iSCSI adapter can attach to an iSCSI target at boot time
iSCSISelect Utility is used to specify and configure a boot device on a particular LUN
Invoked at boot-time by pressing CTRL-S
Configures iSCSI initiator to attach to desired target
LUN can be configured as boot device

Current versions of vSphere don’t directly support DCB protocols
No APIs for a NIC to support DCBX and PFC.
Since HBAs/CNAs offload iSCSI, DCB transparent to OS
DCBX and PFC all handled by device
Host to Device interaction all at SCSI layer
iSCSI over DCB with VMware
Hardware iSCSI
App
App
App
App
OS
OS
OS
OS
VM1
VM2
VM3
VM26
…
Ethernet
Switch
iSCSI storage

Companion Technologies For 10G iSCSI over DCB
17

Technology
Universal Multi-Channel (UMC) presents multiple (typically 4) PCI functions per port to the OS
Partitions the port into independent channels of user-defined bandwidth
Switch-agnostic
Multiple channels can be segmented based on MAC address and/or customer-provided VLAN ID (C-VLAN)
Benefits
Retains networking best practices in 1G to 10G migration
Physical 1G port becomes logical 10G port
Transparent to operating systems
OS still sees dedicated NIC devices
Complementary to DCB traffic shaping with ETS
Bandwidth controls at both logical NIC and priority group level
Multi-Channel
Unique PCI function per logical port
Physical Port
Channels
Network/IP
Storage/iSCSI
MultifunctionAdapters
Network/IP
Network/IP

Low Cost Media Options for iSCSI over DCB
10GBASE-T
Backwards compatible with Gigabit Ethernet
Uses Cat6/Cat6A unshielded twisted pair cabling with RJ-45 connectors
Supports 100m distances
Top of Rack, End of Row (EOR)
SFP+ Direct Attach Copper
iSCSI adapter with SFP cage
Twinaxial Cable with SFP modules on ends
Supports 5-7m distances
Top of Rack
Optical versions available supporting 100m or more

Sample iSCSI over DCB Topology
20

Enterprise iSCSI Solution Example
iSCSI Configuration today
Sample Solution Overview
PowerEdge servers (R710) in VMware cluster
EqualLogic array (PS6010XVS)
PowerConnect switch (B-8000e)
Emulex Enterprise iSCSI HBAs
vCenter iSCSI IMA Plug-in
IMA
Plugin

Enterprise iSCSI Solution Example
DCBX iSCSI TLV
DCBX iSCSI TLV
DCBX Enable PFC
DCBX Enable 9000B MTU
DCBX Enable ETS PG2 BW 1G
DCBX Enable PFC
DCBX Enable 9000B MTU
DCBX Enable ETS PG2 BW 1G

Emulex OneConnect
10 Gigabit Ethernet products for Dell PowerEdge Servers
NIC, iSCSI and FCoE models
iSCSI over DCB support with OCe11000 family of products
Built in support for DCB with Microsoft Windows, VMware vSphere, Red Hat and Novell SUSE Linux operating systems
SFP+ and 10GBASE-T models

iSCSI over DCBBenefits Summary
Enables convergence of LAN and SAN traffic
DCB enables mixed traffic
Multichannel enables multiple logical NICs in one 10G link
Deterministic performance in converged, mixed traffic environment
Lossless Ethernet using Priority Flow Control enables support
Ease of use
Centralized configuration via switch with DCBX support
Hosts, storage announce capabilities with iSCSI TLV
Switch enables PFC, Jumboframes, other parameters via DCBX
Easy server integration
iSCSI HBAs such as Emulex OneConnect provide transparent support
24

See http://www.emulex-dell.com/
To register for the Emulex quarterly Dell newsletter
For information on Emulex OneConnect products supporting iSCSI over DCB
For information about other Emulex solutions for Dell
For iSCSI over DCB white paper
Resources & Call to Action

Thank you
26

Enabling Converged Networks with iSCSI over Data Center Bridging

by Emulex Corporation on Jun 13, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

Statistics

Enabling Converged Networks with iSCSI over Data Center Bridging — Presentation Transcript