The document compares FCoE (Fibre Channel over Ethernet) and iSCSI (Internet Small Computer Systems Interface) in the context of data center convergence and performance. It discusses the benefits of each protocol in terms of throughput, latency, simplified connectivity, and virtualization capabilities, as well as considerations for network architecture. The author emphasizes the increasing importance of Ethernet infrastructure and the potential advantages of adopting either protocol based on existing system investments and operational needs.

1. FCoE vs. iSCSIMaking the ChoiceStephen Foskettstephen@fosketts.net@SFoskettFoskettServices.comBlog.Fosketts.netGestaltIT.com

2. This is Not a Rah-Rah Session

3. This is Not a Cage Match

4. First, let’s talk about convergence and Ethernet

5. Converging on ConvergenceData centers rely more on standard ingredientsWhat will connect these systems together?IP and Ethernet are logical choices

6. Drivers of Convergence

7. What's in it for you?

8. The Storage Network Roadmap

9. Performance: Throughput vs. LatencyHigh-ThroughputLow Latency

10. Serious Performance10 GbE is faster than most storage interconnectsiSCSI and FCoE both can perform at wire-rate

11. Latency is Critical TooLatency is even more critical in shared storage

12. Benefits Beyond Speed10 GbE takes performance off the table (for now…)But performance is only half the story:Simplified connectivityNew network architectureVirtual machine mobility

13. Server Connectivity1 GbE Network1 GbE Cluster4G FC Storage10 GbE(Plus 6 Gbps extra capacity)

14. FlexibilityNo more rats-nest of cablesServers become interchangeable unitsSwappableBrought on line quicklyFew cable connections Less concern about availability of I/O slots, cards and portsCPU, memory, chipset are deciding factor, not HBA or network adapter

15. Changing Data CenterPlacement and cabling of SAN switches and adapters dictates where to install serversConsiderations for placing SAN-attached servers:Cable types and lengthsSwitch locationLogical SAN layoutApplies to both FC and GbEiSCSISANsUnified 10 GbE network allows the same data and storage networking in any rack position

16. Virtualization: Performance and FlexibilityPerformance and flexibility benefits are amplified with virtual servers10 GbE acceleration of storage performance, especially latency – “the I/O blender”Can allow performance-sensitive applications to use virtual servers

17. Virtual Machine MobilityMoving virtual machines is the next big challengePhysical servers are difficult to move around and between data centersPent-up desire to move virtual machines from host to host and even to different physical locations

18. Enhanced 10 Gb EthernetEthernet and SCSI were not made for each other

19. SCSI expects a lossless transport with guaranteed delivery

20. Ethernet expects higher-level protocols to take care of issues

21. “Data Center Bridging” is a project to create lossless Ethernet

22. AKA Data Center Ethernet (DCE), Converged Enhanced Ethernet (CEE)

23. iSCSI and NFS are happy with or without DCB

24. DCB is a work in progress

25. FCoE requires PFC (Qbb or PAUSE), DCBX (Qaz)

26. QCN (Qau) is still not readyPriority Flow Control (PFC)802.1QbbCongestion Management (QCN)802.1QauBandwidth Management (ETS)802.1QazPAUSE802.3xData Center Bridging Exchange Protocol (DCBX)Traffic Classes 802.1p/Q

27. Flow ControlPAUSE (802.3x)Reactive not proactive (like FC credit approach)Allows a receiver to block incoming traffic in a point-to-point Ethernet linkPriority Flow Control 802.1Qbb)Uses an 8-bit mask in PAUSE to specify 802.1p prioritiesBlocks a class of traffic, not an entire linkRatified and shippingSwitch ASwitch BGraphic courtesy of EMCResult of PFC:Handles transient spikesMakes Ethernet losslessRequired for FCoE

28. Bandwidth ManagementEnhanced Transmission Selection (ETS) 802.1QazLatest in a series of attempts at Quality of Service (QoS)Allows “overflow” to better-utilize bandwidthData Center Bridging Exchange (DCBX) protocolAllows devices to determine mutual capabilitiesRequired for ETS, useful for othersRatified and shipping10 GE Link Realized Traffic UtilizationOffered TrafficHPC Traffic3G/s3G/s2G/s3G/s3G/s2G/sStorage Traffic3G/s3G/s3G/s3G/s3G/s3G/sLAN Traffic4G/s5G/s3G/s3G/s4G/s6G/sGraphic courtesy of EMC

29. Congestion NotificationNeed a more proactive approach to persistent congestionQCN 802.1QauNotifies edge ports of congestion, allowing traffic to flow more smoothlyImproves end-to-end network latency (important for storage)Should also improve overall throughputNot quite readyGraphic courtesy of Broadcom

30. Now we can talk aboutiSCSI vs. FCoE

31. Why Choose a Protocol?

32. SAN History: SCSIEarly storage protocols were system-dependent and short distanceMicrocomputers used internal ST-506 disksMainframes used external bus-and-tag storageSCSI allowed systems to use external disksBlock protocol, one-to-many communicationExternal enclosures, RAIDReplaced ST-506 and ESDI in UNIX systemsSAS dominates in servers; PCs use IDE (SATA)Copyright 2006, GlassHouse Technologies

33. The Many Faces of SCSI“SCSI”SASiSCSI“FC”FCoE

34. Comparing Protocols

35. Why Go iSCSI?iSCSI targets are robust and matureJust about every storage vendor offers iSCSI arraysSoftware targets abound, too (Nexenta, Microsoft, StarWind)Client-side iSCSI is strong as wellWide variety of iSCSI adapters/HBAsSoftware initiators for UNIX, Windows, VMware, MacSmooth transition from 1- to 10-gigabit EthernetPlug it in and it works, no extensions requirediSCSI over DCB is rapidly appearing

36. iSCSI Support Matrix

37. iSCSI Reality Check

38. The Three-Fold Path of Fibre Channel

39. FCoE Spotters’ GuideFibre Channel over Ethernet (FCoE)FC-BB-5Bandwidth Management (ETS)802.1QazPriority Flow Control (PFC)802.1QbbCongestion Management (QCN)802.1QauFibre ChannelEthernetFibre Channel over Ethernet (FCoE)

40. Why Go FCoE?Large FC install base/investmentStorage arrays and switchesManagement tools and skillsAllows for incremental adoptionFCoE as an edge protocol promises to reduce connectivity costsEnd-to-end FCoE would be implemented laterI/O consolidation and virtualization capabilitiesMany DCB technologies map to the needs of server virtualization architecturesAlso leverages Ethernet infrastructure and skills

41. Who’s Pushing FCoE and Why?Cisco wants to move to an all-Ethernet futureBrocade sees it as a way to knock off Cisco in the Ethernet marketQlogic, Emulex, and Broadcom see it as a differentiator to push siliconIntel wants to drive CPU upgradesNetApp thinks their unified storage will win as native FCoE targetsEMC and HDS want to extend their dominance of high-end FC storageHP, IBM, and Oracle don’t care about FC anyway

42. FCoE Reality Check

43. Comparing Protocol EfficiencySource: UjjwalRajbhandari, Dell Storage Product Marketinghttp://www.delltechcenter.com/page/Comparing+Performance+Between+iSCSI,+FCoE,+and+FC

44. Comparing Protocol ThroughputSource: UjjwalRajbhandari, Dell Storage Product Marketinghttp://www.delltechcenter.com/page/Comparing+Performance+Between+iSCSI,+FCoE,+and+FC

45. Comparing Protocol CPU EfficiencySource: UjjwalRajbhandari, Dell Storage Product Marketinghttp://www.delltechcenter.com/page/Comparing+Performance+Between+iSCSI,+FCoE,+and+FC

46. Conclusion

47. Counterpoint: Why Ethernet?Why converge on Ethernet at all?Lots of work just to make Ethernet perform unnatural acts!Why not InfiniBand?Converged I/O already worksExcellent performance and scalabilityWide hardware availability and supportKinda pricey; another new networkWhy not something else entirely?Token Ring would have been great!FCoTRGet on the Ring

48. Thank You!Stephen Foskettstephen@fosketts.nettwitter.com/sfoskett+1(508)451-9532FoskettServices.comblog.fosketts.netGestaltIT.comTechFieldDay.com40