Unified Fabric Infrastructure
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
862
On Slideshare
862
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
28
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Unified Fabric InfrastructureGilles ChekrounDistinguished Engineer – Data Center Virtualization Team - EMEAR10-OCT-2012© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 1
  • 2. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public 2 2
  • 3. • Virtualization Requires Network Based Storage Traditional Virtualized App App App App App App App App App App App App App App App Many Apps, OS OS OS OS 1 Application… OS OS OS OS OS OS OS OS OS or “VMs”… OS OS Transition c Hypervisor ...1 Server ...1 Server, or “Host”20,000,00017,500,000 Tipping Point15,000,00012,500,00010,000,000 7,500,000 5,000,000 2,500,000 0 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 Virtualized Non-Virtualized Source: IDC, Nov 2010 3
  • 4. • Fewer physical objects in the Data Center, better asset utilization with the same workloads Conservation of space, power, cooling • Fewer technologies, specialized expertise, greater economies of scale 10GbE Enables Consolidation of Networks, Reduction in Specialized Networks LAN SAN A SAN B Cluster Specialized High Specialized Fibre Channel Performance Storage Network Workload Network© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public 4 4
  • 5. It’s Ethernet!! Ethernet Model Has Proven Benefits Ethernet Economic Model FC Economic Model  Embedded on  Always a Stand-Up Card Motherboard  Specialized Drivers  Integrated into O/S  Few Suppliers  Many Suppliers  Mainstream Technology  Specialized Technology  Widely Understood  Special Expertise  Interoperability by Design  Interoperability by Test© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public 5 5
  • 6. FCoE Benefits ‒ Both block I/O & Ethernet traffic Mapping of FC Frames  Fewer Cables over Ethernet co-exist on same cable Enables FC to Run on  Fewer adapters needed a Lossless Ethernet  Overall less power Network  Interoperates with existing ‒ Management of SAN remain SAN’s Ethernet consistent  No Gateway Fibre Channel Traffic 6
  • 7. • Enables Shared LAN and SAN Links for Access Layer TCO Benefits Consolidate I/O Backup • Eliminate adapter and Production Active Production Standby switch ports Clustering • Eliminate cables Management • Avoid air dams SAN A SAN B • Reduce cable maintenance Server • Reduce power Standardize I/OSAN Edge • Faster server deployment • Enhance server consolidationLAN Access • Enhance storage consolidation 7
  • 8. LAN SAN-A SAN-B LAN SAN-A SAN-B 4 4 2 2 Nearly Half the Cables 16 Servers Enet FC Total 16 Servers Enet FC TotalAdapters 16 16 32 Adapters 16 0 16Switches 2 2 4 Switches 2 0 2Cables 36 36 72 Cables 36 4 40Mgmt. Pts. 2 2 4 Mgmt. Pts. 2 0 2 8
  • 9. FC FC SAN ExtensionFICON VSAN FC SAN Security ZoningiSCSI FICON QoS FCIPFCoE SAN Fabric  Preserves FC investments  Simplifies SAN-attach of servers 9
  • 10. • Parallel LAN/SAN InfrastructureLAN SAN A SAN B • Inefficient use of Network Infrastructure • 5+ connections per server— higher adapter and cabling costs Adds downstream port costs; cap-ex and op-ex Each connection adds additional points of failure in the fabric • Longer lead time for server provisioning • Multiple fault domains— complex diagnostics • Management complexity Ethernet FC 1
  • 11. • Reduction of server adapters LAN SAN A SAN B • Simplification of access layer and cabling • Gateway free implementation— fits in installed base of existing LAN and SAN Nexus 5000 • L2 Multipathing Access— Distribution • Lower total cost of ownership • Fewer cables • Investment protection (LANs and SANs)Standard Server Cisco UCS • Consistent operational model Enhanced Ethernet and FCoE Ethernet FC 1
  • 12. Cisco Nexus 5020 HP Server Cisco UCS Cisco Nexus 2232 June 2008 Jan 2009 June 2009 May 2010 Emulex CNA Dell Server June Cisco Nexus 4000 Mellanox FCoE EMC Storage Q4 Summer 2008 2009 November 2009 Gateway 2010 2010 2008 2009 2010 2011 Timeline QLogic CNA Cisco Nexus 5010 IBM Storage HP Storage Broadcom CNIC Summer 2008 Jan 2009 June 2009 April 2010 Late 2010 Cisco FCoE Brocade 8000 FCoE Standard Net App Storage Intel CNA Multi-hop FCoE on Market Leader Switch Finalized Oct 2009 June 2010 Nexus and MDS June 2008 August 2008 June 2009 March 2011Virtualization: VMware, Oracle Operating Systems: Microsoft Windows Server, Red Hat Enterprise Linux,Virtual Box, and Microsoft Hyper-V Oracle Enterprise Linux and Novell SUSE Linux Enterprise All Support FCoE 1
  • 13. • A Virtual MDS Dedicated Storage VDC – Converged Interfaces LAN SAN • Model for host/target interfaces, not ISLs • Separate VDC running ONLY storage Nexus 7000 related protocols LAN Storage VDC VDC AGGREGATION • Ingress Ethernet traffic is split based on frame ether type • FCoE traffic is processed in the context of the Storage VDC Nexus 7000 LAN Storage VDC VDC ACCESS LAN Storage VDC VDC VDCs Offer Fault Isolation for Converged I/O Higher AvailabilityFCoE © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public 1
  • 14. SAN LAN LAN/SANScalability Nexus 7000 MDS 9500 Nexus 5000 Nexus 4000 MDS 9200 Nexus 1010 Nexus B22 Nexus 3000 Nexus 2000 MDS 9100 Nexus 1000V Cisco NX-OS: One OS from the Hypervisor to the Data Center Core Convergence VM-Aware 10 GbE Fabric Cloud Mobility Networking switching Extensibility 1
  • 15. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
  • 16. Operating System / Applications SCSI Layer FCP iSCSI FCP FCP FCP SRP FCIP iFCP TCP TCP TCP IP IP IP FCoE FC Ethernet IB 1, 2, 4, 8, 10, 16 Gbps 1, 10, 40, 100 . . . Gbps 10, 20 Gbps FCoE Is Non Routable, Localized DC Transport Solution with Lower Protocol Overhead than FCIP or iSCSI© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public 16 16
  • 17. Fibre Channel Switch Input Fabric Output Port X PortFabric NPVSwitch E_Port E_Port F_Port NP_Port SwitchFabric TE_Port TE_PortSwitch End F_Port N_Port Node End F_Port N_Port Node 1
  • 18. • Port Trunking and ChannelingPort Channels Higher aggregate bandwidth Hardware-based load balancing Only supported on switch to switch connections (E_Port to E_Port and NP_Port to F_Port)Trunking Trunking E_Port (TE_Port) Carries tagged frames from multiple VSANs Enhanced ISL (EISL) link Trunking E_Port (TE_Port) Standardization of Enhanced Capabilities is less common across vendors in the Fibre Channel market than You May Be Used to in the LAN market 1
  • 19. • VLANs or 802.1q for FCA Virtual SAN (VSAN) Provides a Methodto Allocate Ports within a Physical Fabricand Create Virtual Fabrics Analogous to VLANs in Ethernet Physical SAN Islands Are Virtualized onto a Virtual fabrics created from larger cost- Common SAN Infrastructure effective redundant physical fabric Reduces wasted ports of a SAN island approach Fabric events are isolated per VSAN which gives further isolation for High Availability FC Features can be configured on a per VSAN basis. 1
  • 20. • Fibre Channel Flow Control Fibre Channel Switch Source regulated flow control B2B Credits used to ensure that FC transport is lossless 16 # of credits negotiated between ports when link is R_RDY brought up # Credits decremented with each packet placed on the wire Packet  Independent of packet size  If # credits = 0, packet transmission is stopped # of credits incremented with each “transfer ready” 16 received 15 B2B Credits need to be taken into consideration as distance and/or bandwidth increases 16 Host 2
  • 21. • Fibre Channel Forwarding – Just Like OSPF• FSPF routes traffic based on destination domain ID• For FSPF a domain ID identifies a VSAN in a single switch This limits the max number of switches that can be supported in the Fabric to 239.• FSPF performs hop-by-hop routing• FSPF uses total cost as the metric to determine most efficient path• FSPF supports equal cost load balancing across links• Link cost can be manually adjusted 2
  • 22. • In Beginning… Start with a host and a target that need to communicate FABRIC A FABRIC B  Typical Host has 2 HBA’s (one per fabric) each with a WWN  Target has multiple ports to connect to multiple fabrics Connect them to a FC Switch  Port Type Negotiation  Speed Negotiation FC Switch is part of the SAN “fabric” Most commonly, dual fabrics are deployed for redundancy Host 2
  • 23. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public 23 2
  • 24. High-Speed Ethernet Server Adoption 2
  • 25. • Multi core CPU architectures driving increased network bandwidth demands• Virtual Machines driving increased I/O connections and I/O bandwidth per server• Low latency 10GE affordability (even optics…)• Increased adoption of NAS (NFS/CIFS) and iSCSI• Consolidation of networks Unified Fabrics with Ethernet• Ubiquity of large scale Ethernet networks Extensive range of management, diagnostic and troubleshooting tools Massive base of manufacturers, suppliers and integrators  Competition, price, services and innovation 2
  • 26. • FCoE is fully defined in FC-BB-5 standard• FCoE works alongside additional technologies to make I/O Consolidation a reality T11 IEEE 802.1 FCoE DCB FC on FC on Other other network Network PFC ETS DCBX media Media Lossless Priority Configuration Ethernet Grouping Verification FC-BB-5 802.1Qbb 802.1Qaz 802.1Qaz 2
  • 27. • IEEE 802.1Qbb • VLAN Tag enables 8 priorities for Ethernet traffic • PFC enables Flow Control on a Per-Priority basis using PAUSE frames (802.1p). • Therefore, we have the ability to have lossless and lossy priorities at the same time on the same wire •Allows FCoE to operate over a lossless priority independent of other priorities FCoE Ethernet Wire 2
  • 28. • IEEE 802.1Qaz • Allows you to create priority groups • Can guarantee bandwidth • Can assign bandwidth percentages to groups • Not all priorities need to be used or in groups 80% 20% 80% 20% FCoE Ethernet Wire 2
  • 29. D C B X• IEEE 802.1Qaz • Allows network devices to “Hello?” advertise their identities and “Hello?” “Hello.” capabilities over the network “Look’s Like We All Speak the • Enables hosts to pick up proper “Hello.” Same Language.” configuration from the network “Hello?” • Enables switches to verify proper configuration “Hello.” • Provides support for: • PFC • ETS • Applications (e.g., FCoE) Ethernet Wire 2
  • 30. • Low power consumption SAN A LAN SAN B• Low cable cost SFP+ CX-1 Copper (SFF 8431)• Low transceiver latency Application Server Application Server• Low error rate (1017) Application Server Application Server• Thinner cable with higher bend radius 16x10 GE Cables 16x10 GE Cables Application Server Application Server Supports 10GE passive direct attached up to 10 meters Application Server Application Server• Easier to manage cabling solution reduces Application Server deployment time Application Server Application Server• All copper cables are contained within the rack Application Server Application Server Application Server Application Server Application Server 3
  • 31. First Generation Second Generation 10GE Today Mid-2008 Mid-2009 FCoE in Software Open-FCoE.org And Other NIC Vendors• Multiple components• Full height/length • Single chip • Half height/length • Less than half the • Brings the economy of scale power of Ethernet to SAN • Same support as • Field upgrade-able NICs HBAs 3
  • 32. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public 32 3
  • 33. • It’s Fibre Channel• From a Fibre Channel standpoint it’s FC connectivity over a new type of cable called… Ethernet• From an Ethernet standpoints it’s Yet another ULP (Upper Layer Protocol) to be transported FC-4 ULP Mapping FC-4 ULP Mapping FC-3 Generic Services FC-3 Generic Services FC-2 Framing & Flow Control FC-2 Framing & Flow Control FC-1 Encoding FCoE Logical End Point FC-0 Physical Interface Ethernet Media Access Control Ethernet Physical Layer 3
  • 34. Ethernet 12 Bytes (MAC Addresses) + Header 4 Bytes (802.1Q Tag) FCoE Header 16 Bytes FCTotal: 2180 Bytes Header 24 Bytes FC Payload Up to 2112 Bytes 4 Bytes FCoE Standard (FC-BB-5) Requires Jumbo Support; 2.5KB = “Baby Jumbo” 1 Byte (EOF) + 3 Bytes (Padding) CRC EOF FCS 4 Bytes 3
  • 35. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public 35 3
  • 36. Target Same host to target communication Ethernet Fabric FC Fabric  Host has 2 CNA’s (one per fabric)  Target has multiple ports to connect to fabric Connect to a DCB capable switch  Port Type Negotiation (FC port type will be handled by FIP)  Speed Negotiation  DCBX Negotiation DCB Capable Switch Access switch is a Fibre Channel Forwarder (FCF) Acting as an FCF Dual fabrics are still deployed for redundancy Unified Wire CNA Server 3
  • 37. Ethernet FC • Ethernet LAN and Fibre Channel SAN• Physical and Logical separation of LAN and SAN traffic FC• Additional Physical and Logical separation of SAN fabrics Fabric ‘A’ Fabric ‘B’ L3 L2 Nexus 7000 MDS 9000 Nexus 5000 NIC HBA Isolation Convergence 3
  • 38. Ethernet FC • Ethernet LAN and Ethernet SAN• Same topologies as existing networks, but using Nexus Unified Fabric Ethernet switches for SANs FCoE• Physical and Logical separation of LAN Fabric ‘A’ Fabric ‘B’ and SAN traffic L3 L2• Additional Physical and Logical separation of SAN fabrics Nexus 7000 Nexus 7000 Nexus 5000 Nexus 5000• Ethernet SAN Fabric carries FC/FCoE & IP based storage (iSCSI, NAS, …)• Common components: Ethernet Capacity, common parts sparing and Cost NIC CNA or CNA Isolation Convergence 3
  • 39. Ethernet FC • Sharing Access Layer for LAN and SAN Converged FCoE Link Dedicated FCoE Link• Shared Physical, Separate Logical LAN and SAN traffic at Access Layer FCoE FC• Physical and Logical separation of LAN and SAN traffic at Aggregation Layer Fabric ‘A’ Fabric ‘B’ L3• Additional Physical and Logical separation MDS 9000 of SAN fabrics L2• Storage VDC for additional management and operation separation Nexus 7000• Higher I/O, HA, fast re-convergence for Nexus 5000 host LAN traffic CNA Isolation Convergence 3
  • 40. Maintaining Dual SAN Fabrics with Overlay Ethernet FC Converged FCoE Link • LAN and SAN traffic share physical switches Fabric ‘A’ LAN/SAN Dedicated • LAN and SAN traffic use dedicated links between FCoE Link switches Fabric ‘B’ • All Access and Aggregation switches are FCoE FCF switches L3 • Dedicated links between switches are VE_Ports L2 • Storage VDC for additional operation separation FCF at high function agg/core VE • Improved HA, load sharing and scale for LAN vs. traditional STP topologies FCF FCF • SAN can utilize higher performance, higher Nexus 7000 Nexus 5000 … density, lower cost Ethernet switches for the … aggregation/core FCoE FC CNA Isolation Convergence© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public 40 40
  • 41. Maintaining Dual SAN Fabrics with Overlay Ethernet FC • LAN and SAN traffic share physical Converged Fabric ‘A’ FCoE Link switches Dedicated • LAN and SAN traffic use dedicated links Fabric ‘B’ FCoE Link between switches • All Access and Aggregation switches are L3 FCoE FCF switches L2 • Dedicated links between switches are FCF VE_Ports VE • Storage VDC for additional operation separation at high function agg/core FCF FCF Nexus 7000 … • Improved HA, load sharing and scale for Nexus 5000 LAN vs. traditional STP topologies … • SAN can utilize higher performance, higher density, lower cost Ethernet switches for CNA FCoE FC the aggregation/core Isolation Convergence© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public 41 41
  • 42. Converged Network with Dedicated Links • Maintaining Dual SAN Fabrics with FabricPath Ethernet FC Converged FCoE Link  FabricPath enabled for LAN traffic Fabric ‘A’ Dedicated  Dual Switch core for SAN A & SAN B Fabric ‘B’ FCoE Link  All Access and Aggregation switches are FCoE FCF switches L3  Dedicated links between switches are L2 FCF FCF VE_Ports  Storage VDC for additional operation VE separation at high function agg/core  Improved HA and scale over vPC (ISIS, 7000 Nexus FCF FCF RPF, … and N+1 redundancy) Nexus 5000  SAN can utilize higher performance, higher density, lower cost Ethernet switches CNA FCoE FCIsolation Convergence 4
  • 43. Ethernet FC Converged • SAN Separation at the Access Switch FCoE Link Dedicated FCoE Link• LAN and SAN traffic share physical switches and links Fabric ‘A’ Fabric ‘B’• FabricPath enabled• All Access switches are FCoE FCF switches L3• VE_Ports to each neighbor Access switch L2• Single process and database (FabricPath) for forwarding• Improved (N + 1) redundancy for LAN & SAN 10,20 10 30 20,30 FCF FCF• Sharing links increases fabric flexibility and scalability Nexus 7000 Nexus 5000• Distinct SAN ‘A’ & ‘B’ for zoning isolation and multipathing redundancy 10,20 20,30 10 30 FCoE CNA1 CNA2 Array1 Array2 *Not currently supported Isolation Convergence 4
  • 44. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public 44 44
  • 45. Please fill evaluation formsThank you.