Fibre Channel over Ethernet (FCoE), iSCSI and the Converged Data Center

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    Notes on slide 1

    Questions for the audience:How many have FC today?How many have iSCSI today?How many have 10GbE today?

    WHY need it?WHAT it is?HOW it will happen!

    Let’s look at the rack server environment today. In today’s environment… Servers connect to the LAN with multiple network interface cards (NICs)Servers require HBAs to connect to an existing Fibre Channel SANMany of today’s data centers are running at 1 Gigabit Ethernet speedMultiple server adapters require more power and cooling costs

    Buy the “best available” grade of cable to help future proof. OM4 under discussion (buy OM3 today). Cat6a available today.For 2008 FCoE deployments, Twinax helps meet time-to-market needs with a low power/cost solution until 10GBase-T has more time to mature.Very few customers have deployed 10Gb Ethernet. Some may question adding a new cabling infrastructure, but should be limited to early “Phase 1” testing and deployments.

    FC has a large and well managed install base (EMC has shipped over 4M Connectrix switch ports); $10B install base according to BrocadeDrive for I/O consolidationiSCSI strength was to use existing network infrastructure and skill set to offer lower-cost connectivity for consolidation, but management of large iSCSI environments is cumbersomeNot rip and replace or new management paradigm, but incremental transition

    Replication first (iFCP EOL)IB = special transport layer and new management; used in HPCFCoE is layer 2 only, so can use FCIP under a shared FC management with FC+FCIP+FCoE; or use Ethernet native replication from Symm/CLARiiON

    Same host drivers, FC management of storage.With motion – bubble analogy of FC in Jumbo Frame Ethernet and “popped” in CNS.Discuss CNA vs. NIC (NIC on 2nd click) for deployment options. CNA has hardware offload, NIC does not.

    FCoE is getting traction because it maintains the channel-like performance of FC and low protocol overhead. Note that FCoE is Data Center only (Layer 2 – no routing).Encapsulation (not bridging) to attach to FC

    Discussion after arrows gone – CNA (not NIC) for 1st generation since need FC traffic to go to hypervisor as FC, not through Vswitch (not lossless) or in guest.

    Also useful for iSCSI

    Also useful for iSCSI

    TitleMonth Year

    TitleMonth Year

    FCoE is NOT the death of iSCSI (just like iSCSI was not the death of FC) – plenty of room for both; both add to the overall SAN market (meet the needs of the growing virtualization market – want more networked devices).This is also the place to say reassuring things about EMC as \"arms supplier\" in the protocol wars – whatever protocol fits your needs (customer), we (EMC) are the experts.

    33

    34

    35

    37

    3 Favorites

    Fibre Channel over Ethernet (FCoE), iSCSI and the Converged Data Center - Presentation Transcript

    1. Fibre Channel over Ethernet (FCoE), iSCSI and the Converged Data Center Stuart Miniman, Technologist, Office of the CTO EMC Corporation © Copyright 2009 EMC Corporation. All rights reserved. 1
    2. Objectives After this session you will be able to:  Describe FCoE and iSCSI and how they fit into existing storage and networking infrastructures.  Compare and contrast the structure and functionality of the FCoE and iSCSI protocol stacks.  Understand how FCoE and iSCSI solutions provide storage networking options for Ethernet, including 10 Gb Ethernet. © Copyright 2009 EMC Corporation. All rights reserved. 2
    3. Agenda  Storage Networking, FCoE and iSCSI  iSCSI and FCoE Protocols  Ethernet Enhancements  Security in the SAN  Deployments of FCoE and iSCSI  Conclusion and Summary © Copyright 2009 EMC Corporation. All rights reserved. 3
    4. Rack Server Environment Today Ethernet Fibre Channel iSCSI SAN  Servers connect to LAN, NAS and iSCSI SAN with NICs 1 Gigabit Ethernet  Servers connect to FC SAN with HBAs  Many environments today are still 1 Gigabit Ethernet 1 Gigabit Ethernet  Multiple server adapters, multiple cables, power and cooling costs 1 Gigabit Fibre Channel Ethernet LAN – Storage is a separate network Ethernet HBAs (including iSCSI) NICs Fibre Channel SAN Note: NAS will continue to be part of the solution. Everywhere that you see Ethernet or 10Gb Ethernet in this Storage presentation, NAS can be considered Rack-mounted servers part of the unified storage solution © Copyright 2009 EMC Corporation. All rights reserved. 4
    5. 10Gb Ethernet allows for Converged Data Center  Maturation of 10 Gigabit Ethernet – 10 Gigabit Ethernet allows replacement of n x 1Gb with a much smaller number (start with 2) of 10Gb Adapters – Many storage applications require > 1Gb bandwidth Single Wire for SAN 10 GbE Network and Storage LAN  10 Gigabit Ethernet simplifies server, network and storage infrastructure – Reduces the number of cables and server adapters – Lowers capital expenditures and administrative costs – Reduces server power and cooling costs – Blade servers and server virtualization drive consolidated bandwidth 10 Gigabit Ethernet is the answer! iSCSI and FCoE both leverage this inflection point © Copyright 2009 EMC Corporation. All rights reserved. 5
    6. 10 Gigabit Ethernet Cabling Type / Cable Distance + – Connector Optical OM2 OM2 82m; Provides Optical (multimode) / (orange) OM3 extended historically LC or OM3 300m distance for 1% of (aqua) backbone or overall core Ethernet ports Copper / Twinax 5m Passive = Very Distance SFP+DA low power limited to (direct attach) rack Copper Cat6 or Cat6 55m; Keep existing High power (10GBase-T) / Cat6a Cat 6a cabling layout and cost RJ-45 100m (> 1 B ports) today and patch panel infrastructure *10GBase-CX is another copper option, not expected in most 10 Gb Ethernet © Copyright 2009 EMC Corporation. All rights reserved. 6
    7. Why iSCSI? Initiator Delivery of iSCSI Target Protocol Data Unit (PDU) SCSI for SCSI functionality SCSI (initiator, target, data iSCSI read / write, etc.) iSCSI Reliable data transport and delivery (TCP TCP Provides IP routing TCP Windows, ACKs, (Layer 3) capability so IP packets can find their IP ordering, etc.) way through the network IPsec IPsec Provides physical Link Link network capability (Layer 2 Ethernet, IP Network Cat 5, MAC, etc.) © Copyright 2009 EMC Corporation. All rights reserved. 7
    8. Why a New Option for FC Customers?  FC has a large and well managed install base – Want a solution that is attractive for customers with FC expertise / investment – Previous convergence options did not allow for incremental adoption  Requirement for a Data Center solution that can provide I/O consolidation – 10 Gigabit Ethernet makes this option available  Leveraging Ethernet infrastructure and skill set has always been attractive FCoE allows an Ethernet-based SAN to be introduced into the FC-based Data Center without breaking existing administrative tools and workflows © Copyright 2009 EMC Corporation. All rights reserved. 8
    9. Protocol Comparisons FC over Ethernet (no TCP/IP) App App Applications Applications SCSI SCSI SCSI SCSI iSCSI FC FC FC FC FC SRP Encapsulation Encapsulation iFCP FCIP TCP Layer Layer TCP TCP IP IP IP FCoE FCoE Base Base FC Infiniband Ethernet Ethernet Transport Transport Block storage FC replication New transport with TCP/IP over IP FC management and drivers Low latency, high bandwidth © Copyright 2009 EMC Corporation. All rights reserved. 9
    10. FCoE Extends FC on a Single Network Server sees storage traffic as FC Network FC Ethernet Driver Driver Network FCoE SW Stack Standard Converged 10G NIC Network Adapter FC storage 2 Lossless Ethernet Links options FC network Converged Network Switch Ethernet FC SAN sees host as FC © Copyright 2009 EMC Corporation. All rights reserved. 10
    11. Time To Widespread Adoption 1980 1990 2000 2010 10 Gigabit Ethernet 02 09? Standard Widespread 73 83 93 iSCSI Defined Standard Widespread 00 02 08 Defined Widespread Standard 85 94 03 Defined Standard Widespread 07 09 ?? Defined Standard? © Copyright 2009 EMC Corporation. All rights reserved. 11
    12. Standards for Next Generation Data Center Two emerging parallel industry standards seek to drive I/O consolidation in large data centers over time:  Fibre Channel over Ethernet  Converged Enhanced Ethernet (FCoE) protocol (CEE) – Developed by International – Developed by Ethernet IEEE Committee for Information Data Center Bridging Task Group Technology Standards (INCITS) – Converged Enhanced Ethernet T11 Fibre Channel Interfaces creates an Ethernet environment Technical Committee that drops frames as rarely as – Fibre Channel over Ethernet Fibre Channel allows native Fibre Channel to – Technology commonly referred to travel unaltered over Ethernet as Lossless Ethernet – FC-BB-5 standard is making – IEEE standards targeting progress towards ratification in ratification in 2009/2010 2009 – Requirement for FCoE; Enhancement for iSCSI Companies working on the standard committees Key participants: Brocade, Cisco, EMC, Emulex, HP, IBM, Intel, QLogic, Sun, others © Copyright 2009 EMC Corporation. All rights reserved. 12
    13. Agenda  Storage Networking, FCoE and iSCSI  iSCSI and FCoE Protocols  Ethernet Enhancements  Security in the SAN  Deployments of FCoE and iSCSI  Conclusion and Summary © Copyright 2009 EMC Corporation. All rights reserved. 13
    14. iSCSI and FCoE Framing  iSCSI is SCSI functionality transported using TCP/IP for delivery and routing in a standard Ethernet/IP environment Ethernet iSCSI Data CRC iSCSI Frame IP TCP Header TCP/IP and iSCSI require CPU processing  FCoE is FC frames encapsulated in Layer 2 Ethernet frames designed to utilize a Lossless Ethernet environment – Large maximum size of FC requires Ethernet Jumbo Frames – No TCP, so Lossless environment required – No IP routing Ethernet Header Header Header FCoE CRC EOF FCS FC Payload FC FCoE Frame © Copyright 2009 EMC Corporation. All rights reserved. FC Frame 14
    15. iSCSI Concepts  iSCSI session: One initiator and one target – Multiple TCP connections allowed in a session  Exploit network parallelism  Error recovery possible across connections  Most communication is based on SCSI – For example, Ready to Transmit (R2T) for target flow control  Important iSCSI additions to SCSI – Immediate and unsolicited data to avoid round trip – Login phase for connection setup  Text-based parameter negotiation – Explicit logout for clean teardown © Copyright 2009 EMC Corporation. All rights reserved. 15
    16. SCSI-to-iSCSI Mapping SCSI Command and Data iSCSI PDU iSCSI PDU iSCSI PDU iSCSI PDU Header Data Header Data Header Data Header Data IP IP IP IP IP IP IP IP IP Packet Packet Packet Packet Packet Packet Packet Packet Packet iSCSI PDU alignment with packets varies © Copyright 2009 EMC Corporation. All rights reserved. 16
    17. FCoE Frame Formats  Ethernet frames give a 1:1 encapsulation of FC frames – No segmenting FC frames across FCoE Frame Format multiple Ethernet frames Bit 0 Bit 31 – FCoE flow control is Ethernet based Destination MAC Address  BB Credit/R_RDY replaced by Pause/PFC mechanism Source MAC Address  FC frames are large, require Jumbo IEEE 802.1Q Tag frames ET = FCoE Ver Reserved – Max FC payload size is 2112 bytes Reserved – Max FCoE frame size is 2180 bytes Reserved  Also created an FCoE Initialization Reserved SOF Protocol (FIP) for: Encapsulated FC Frame – Discovery (Including FC-CRC) – Login EOF Reserved – To determine if the MAC address is FCS server provided (SPMA) or fabric provided (FPMA) © Copyright 2009 EMC Corporation. All rights reserved. 17
    18. Storage Drivers and Virtualization vNIC vSCSI vNIC vSCSI Vswitch VMkernel Hypervisor storage stack NIC C FC NIC C FC N HBA N HBA A A LAN traffic LAN traffic iSCSI traffic FCoE follows FC iSCSI traffic FC traffic path *iSCSI initiator can also be in the VM © Copyright 2009 EMC Corporation. All rights reserved. 18
    19. Storage Drivers and Virtualization SW SW FCoE FCoE vNIC vSCSI vNIC vSCSI vSwitch is not Lossless Vswitch VMkernel Hypervisor storage stack No FCoE access here currently NIC FC NIC FC HBA HBA FCoE software in the guest would send traffic through the vSwitch to the vNIC © Copyright 2009 EMC Corporation. All rights reserved. 19
    20. Multipathing Mechanisms for iSCSI and FCoE  Ethernet trunking and NIC Teaming – Link layer (2), below TCP, transparent to iSCSI and FCoE  Multiple TCP connections – In a single iSCSI session (layer 5) – Same or different hardware (Ethernet) ports – Difficult when TCP and iSCSI are offloaded  Multiple iSCSI/FCoE sessions or paths – Multipathing software (e.g., PowerPath) above iSCSI or FCoE – Same or different hardware (e.g., Ethernet) ports  iSCSI also supports HTTP-style redirects – Target has been temporarily or permanently moved © Copyright 2009 EMC Corporation. All rights reserved. 20
    21. Agenda  Storage Networking, FCoE and iSCSI  iSCSI and FCoE Protocols  Ethernet Enhancements  Security in the SAN  Deployments of FCoE and iSCSI  Conclusion and Summary © Copyright 2009 EMC Corporation. All rights reserved. 21
    22. Lossless Ethernet  Limit the environment only to the Data Center – FCoE is Layer 2 only  IEEE 802.1 Data Center Bridging (DCB) is the standards task group  Converged Enhanced Ethernet (CEE) is an industry consensus term which covers three link level features – Priority Flow Control (PFC, IEEE 802.1Qbb) – Enhanced Transmission Selection (ETS, IEEE 802.1Qaz) – Data Center Bridging Exchange Notification (DCBX, currently part of IEEE 802.1Qaz, leverages 802.1AB (LLDP))  Data Center Ethernet is a Cisco term for CEE plus additional functionality including Congestion Notification (IEEE 802.1Qau) Enhanced Ethernet provides the Lossless Infrastructure which will enable FCoE and augment iSCSI storage traffic . © Copyright 2009 EMC Corporation. All rights reserved. 22
    23. PAUSE and Priority Flow Control  PAUSE transforms Ethernet into a lossless fabric  Classical 802.3x PAUSE is rarely implemented since it stops all traffic  Priority Flow Control (PFC), formerly known as Per Priority PAUSE (PPP) or Class Based Flow Control – PFC will be limited to Data Center  A new PAUSE function that can halt traffic according to priority tag while allowing traffic at other priority levels to continue – Creates lossless virtual lanes  Per priority link level flow control – Only affect traffic that needs it – Ability to enable it per priority – Not simply 8 x 802.3x PAUSE Switch A Switch B © Copyright 2009 EMC Corporation. All rights reserved. 23
    24. Enhanced Transmission Selection and Data Center Bridging Exchange Protocol (DCBX) Enhanced Transmission Selection (ETS) provides a common management framework for bandwidth management  Allows configuration Offered Traffic 10 GE Link Realized Traffic Utilization of HPC & storage traffic to have 3G/s HPC Traffic 2G/s appropriately higher 3G/s 3G/s 2G/s 3G/s priority  When a given load in 3G/s Storage Traffic 3G/s a class does not fully 3G/s 3G/s 3G/s 3G/s utilize its allocated bandwidth, ETS allows other traffic classes to use the available 3G/s 4G/s 6G/s 3G/s LAN Traffic 5G/s bandwidth 4G/s  Maintain low latency treatment of certain t1 t2 t3 t1 t2 t3 traffic classes Data Center Bridging Exchange Protocol (DCBX) is responsible for configuration of link parameters for DCB functions  Determines which devices support Enhanced Ethernet functions © Copyright 2009 EMC Corporation. All rights reserved. 24
    25. Beyond Link Level Switch  Congestion notification – IEEE 802.1Qau targeted for April 2010 Transmit Queue ratification Switch  Allows a switch to notify attached ports to slow down transmission due to heavy traffic, in order to reduce the chances of packet drops or Th Receive Buffer ro network deadlocks ttl e  Moves the management of congestion back to the edge, which helps alleviate network-wide bottlenecks e ttl ro Th  Layer 2 multipathing – IETF TRILL - TRansparent Interconnection of Lots of Links  Used with the Spanning Tree Protocol to provide more efficient bridging and bandwidth aggregation  Focuses on a bridging capability that will increase bandwidth by allowing and aggregating multiple network paths  Standards are stable; products are coming soon © Copyright 2009 EMC Corporation. All rights reserved. 25
    26. 40 and 100 Gigabit Ethernet  IEEE P802.3ba Task Force states that bandwidth requirements for computing and networking applications are growing at different rates, which necessitates two distinct data rates, 40 Gb/s and 100 Gb/s  IEEE target for standard completion of 40 GbE & 100 GbE is 2010  40 GbE products shipping today supporting existing fiber plant and plan is for 100 GbE to also support 10m copper, 100m MMF (use OM4 for extended reach) and SMF  Cost of 40 GbE or 100 GbE is currently 5 – 10 x 10 GbE – Adoption will become more economically attractive at 2.5x which will take a couple of years © Copyright 2009 EMC Corporation. All rights reserved. 26
    27. Agenda  Storage Networking, FCoE and iSCSI  iSCSI and FCoE Protocols  Ethernet Enhancements  Security in the SAN  Deployments of FCoE and iSCSI  Conclusion and Summary © Copyright 2009 EMC Corporation. All rights reserved. 27
    28. Storage Security Services in the SAN  Storage Area Networking began as a homogeneous interconnection of a limited amount of nodes, Fibre Channel specific  Security in the FC SAN relied on physical security to protect data  The introduction of TCP/IP networks required more proven security functionality – Leverage existing security best practices for the network – Add functionality to FC to protect what is now an open network  Security functionality in the SAN – Authentication – Authorization – Access Control – Confidentiality  Adding FCoE as a possible SAN interconnect leverages existing mechanisms while requiring new functionality to protect at the Ethernet Layer © Copyright 2009 EMC Corporation. All rights reserved. 28
    29. Security Functionality  Authentication – DH-CHAP for end to end and end node to switch authentication in FC – FCAP, etc.. for FC switch to switch authentication – CHAP for end to end authentication in iSCSI  Authorization – Storage ports perform device masking, allocating storage access based on authenticated initiators  Access Control – Zoning in Fibre Channel limits the scope of discovery based on end node identity – VSAN in Fibre Channel provides segregation of traffic based on end node identity – Discovery Domains in iSCSI can limit the scope of discovery based on end node identity – VLANs in the IP domain allow for either port based or packet based isolation – ACLs utilized in FCoE to allow or deny traffic to a Ethernet port  Confidentiality – IPSEC for protection of data in transit for iSCSI and interswitch traffic for FCIP © Copyright 2009 EMC Corporation. All rights reserved. 29
    30. Agenda  Storage Networking, FCoE and iSCSI  iSCSI and FCoE Protocols  Ethernet Enhancements  Security in the SAN  Deployments of FCoE and iSCSI  Conclusion and Summary © Copyright 2009 EMC Corporation. All rights reserved. 30
    31. FCoE and iSCSI FCoE iSCSI Ethernet FC expertise / install base No FC expertise needed FC management Layer 2 Ethernet Leverage Ethernet/IP expertise Supports distance Use FCIP for distance connectivity (L3 IP routing) 10 Gigabit Ethernet Strong virtualization affinity Lossless Ethernet Standards in process Standards since 2003 © Copyright 2009 EMC Corporation. All rights reserved. 31
    32. iSCSI Deployment  iSCSI grew to > 10% of SAN market revenue in 2008 *  Many deployments are small environments, which replace DAS – Strong affinity in SMB/commercial markets  Seeing strong growth of Unified Storage – Supports iSCSI, FC, and NAS  iSCSI with 10 Gigabit Ethernet becoming Ethernet available iSCSI SAN * According to IDC, 3/09 © Copyright 2009 EMC Corporation. All rights reserved. 32
    33. FCoE Server Phase (Today)  FCoE with direct attach of server to Converged Network Switch at top of rack or end of row  Tightly controlled solution Ethernet LAN  Server 10 GE adapters may be CNA or NIC  Storage is still a separate network Converged Network Switch FC Attach Fibre Channel SAN 1 Gb NICs FC HBAs 10 GbE CNAs Storage Ethernet Rack Mounted FC Servers © Copyright 2009 EMC Corporation. All rights reserved. 33
    34. FCoE Network Phase (2009 / 2010)  Converged Network Switches move out of the rack from a tightly controlled environment into a unified network  Maintains existing LAN and SAN management Ethernet LAN Overlapping domains may compel cultural adjustments Ethernet Network (IP, FCoE) and CNS Converged Network Switch Fibre Channel SAN FC Attach 10 GbE CNAs Storage Ethernet FC Rack Mounted Servers © Copyright 2009 EMC Corporation. All rights reserved. 34
    35. FCoE Storage Phase (2010+)  Single Ethernet network for IP and storage traffic Ethernet LAN  End-to-End Ethernet with native FCoE (future)  End-to-end solution still maintains upper layer FC at host and storage FCoE Converged Network Storage Switch Fibre Channel & FCoE FC & FCoE SAN attach 10GbE CNAs Ethernet Storage Rack Mounted FC Servers © Copyright 2009 EMC Corporation. All rights reserved. 35
    36. Agenda  Storage Networking, FCoE and iSCSI  iSCSI and FCoE Protocols  Ethernet Enhancements  Security in the SAN  Deployments of FCoE and iSCSI  Conclusion and Summary © Copyright 2009 EMC Corporation. All rights reserved. 36
    37. Convergence at 10 Gigabit Ethernet  Two paths to a Converged Network – iSCSI purely Ethernet – FCoE allows for mix of FC and Ethernet (or all Ethernet LAN Ethernet)  FC that you have today or buy tomorrow will plug into this in the future  Choose based on scalability, management, and skill set Converged Network Switch iSCSI/FCoE Storage 10 GbE CNAs FC SAN Ethernet Rack Mounted FC Servers © Copyright 2009 EMC Corporation. All rights reserved. 37
    38. Summary  A converged data center environment can be built using 10Gb Ethernet – Ethernet Enhancements are required for FCoE and will assist iSCSI  Choosing between FCoE and iSCSI will be based on customer existing infrastructure and skill set  10 Gigabit Ethernet solutions will take time to mature – Active industry participation is creating standards that allow solutions that can integrate into existing data centers – FCoE and iSCSI will follow the Ethernet roadmap to 40 and 100 Gigabit in the future The Converged Data Center allows Storage and Networking to leverage operational and capital efficiencies © Copyright 2009 EMC Corporation. All rights reserved. 38
    39. Related Session and References  Considerations for Deploying Fibre Channel Over Ethernet – Tuesday / Thursday  FCoE in the EMC Topology Guide http://elabnavigator.emc.com  Industry site with consolidated information http://www.fcoe.com/  EMC FCoE Videos – search “FCoE” on YouTube  EMC FCoE whitepaper http://www.emc.com/collateral/hardware/white-papers/h5916-intro-to-fcoe-wp.pdf  T11 FCoE activity http://www.t11.org/fcoe  IEEE 802.1 Data Center Bridging task group page http://www.ieee802.org/1/pages/dcbridges.html  Optimizing iSCSI in VMware environment http://virtualgeek.typepad.com/virtual_geek/2009/01/a-multivendor-post-to-help-our-m © Copyright 2009 EMC Corporation. All rights reserved. 39
    40. Stuart Miniman, Technologist, Office of the CTO EMC Corporation

    + Stuart MinimanStuart Miniman, 8 months ago

    custom

    3241 views, 3 favs, 3 embeds more stats

    This presentation is from the EMC World 2009 confer more

    More info about this presentation

    © All Rights Reserved

    Go to text version

    • Total Views 3241
      • 3010 on SlideShare
      • 231 from embeds
    • Comments 0
    • Favorites 3
    • Downloads 210
    Most viewed embeds
    • 229 views on http://nohype.tumblr.com
    • 1 views on http://www.tumblr.com
    • 1 views on http://static.slidesharecdn.com

    more

    All embeds
    • 229 views on http://nohype.tumblr.com
    • 1 views on http://www.tumblr.com
    • 1 views on http://static.slidesharecdn.com

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories

    Tags

    What's New

    © 2009 SlideShare Inc. All rights reserved.