Cisco Catalyst 6500 Switch Architecture

10,353 views

Published on

This session presents an in-depth study of the architecture of the Cisco Catalyst 6500 Series Switch and how the core components work together to enable high-performance packet switching. Topics include a discussion of the latest supervisors, forwarding engines, and switching modules, as well as key hardware components. This session is for enterprise and service provider attendees who have worked with Cisco Catalyst 6500 Series Switches and wish to further their understanding of the switch forwarding architecture. An understanding of multilayer switching technologies and the basic operation of the Cisco Catalyst 6500 Series is assumed. Basic knowledge of routing protocols as well as traditional campus design is recommended.

Published in: Technology, Business

Cisco Catalyst 6500 Switch Architecture

  1. 1. Catalyst 6500 Architecture BRKARC-3465
  2. 2. Cisco Live & Networkers Virtual Special Offer – Save $100 Cisco Live has a well deserved reputation as one the industry’s best educational values. With hundreds of sessions spanning four educational programs — Networkers, Developer Networker, Service Provider, IT Management, you can build a custom curriculum that can make you a more valuable asset to your workplace and advance your career goals. Cisco Live and Networkers Virtual immerses you in all facets of Cisco Live, from participating in live keynotes and Super Sessions events to accessing session content to networking with your peers. Visit www.ciscolivevirtual.com and register for Cisco Live and Networkers Virtual. To get $100 USD off the Premier pass, which provides access to hundreds of technical sessions, enter “slideshareFY11”. BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2
  3. 3. Session Goal To provide you with a thorough understanding of the Catalyst® 6500 switching architecture, packet flow, forwarding engine functions, and key feature operation. BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 33
  4. 4. Agenda Chassis and Power Supplies Supervisor Engine and Switch Fabric Architecture Module Architecture Layer 2 Forwarding IP Unicast Forwarding IPv4 Multicast Forwarding Netflow, ACL and QoS Processing Packet Walks BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 4
  5. 5. Catalyst 6500 Chassis Architecture Modular chassis in a variety of form factors 3, 4, 6, 9, and 13-slot versions Enhanced (“E”) chassis offer higher system power capacity and better signal integrity 3, 4, 6, and 9-slot versions Classic switching bus traces/connectors Crossbar fabric traces/connectors Redundant power supplies Fan tray for system cooling 6509-NEB-A and 6509-V-E chassis offers redundant fan trays and air filtration Redundant voltage termination (VTT)/clock modules Redundant MAC address EEPROMs Note: All non-E Series chassis are End-of-Sale except for the WS-C6509-NEB-A. BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5
  6. 6. Catalyst 6503/6503E and 6504E Power Supply Power Supply Clock Clock EEPROM SLOT 1 Dual Channels Fan Tray SLOT 2 Dual Channels SLOT 3 Dual Channels SLOT 4 Dual Channels Switch Fabric Shared Bus 6504-E 6503-E 5 RU 4 RU Note: CEF720 modules not supported in Catalyst 6503 (non-E) chassis BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 6
  7. 7. Catalyst 6506/6509 and 6506E/6509E 6509-E 6506-E Clock Clock EEPROM SLOT 1 Dual Channels 12 RU SLOT 2 Dual Channels 15 RU SLOT 3 Dual Channels SLOT 4 Dual Channels SLOT 5 Dual Channels 6509-V-E SLOT 6 Dual Channels Fan Tray SLOT 7 Dual Channels 21 RU SLOT 8 Dual Channels SLOT 9 Dual Channels Switch Fabric Shared Bus Power Supply Power Supply BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 7
  8. 8. Catalyst 6513 Clock Clock EEPROM SLOT 1 Single Channel SLOT 2 Single Channel SLOT 3 Single Channel SLOT 4 Single Channel SLOT 5 Single Channel 19 RU SLOT 6 Single Channel Fan Tray SLOT 7 Single Channel SLOT 8 Single Channel SLOT 9 Dual Channels SLOT 10 Dual Channels SLOT 11 Dual Channels SLOT 12 Dual Channels Note: Modules requiring dual fabric SLOT 13 Dual Channels Channels (6816,all 67xx except 6724) and WiSM will not work in Switch Fabric Shared Bus slots 1-8. Power Supply Power Supply BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8
  9. 9. Catalyst 6513-E Clock Clock EEPROM SLOT 1 Dual Channels SLOT 2 Dual Channels Fan Tray (removed from rear) SLOT 3 Dual Channels SLOT 4 Dual Channels SLOT 5 Dual Channels 19 RU SLOT 6 Dual Channels SLOT 7 Dual Channels SLOT 8 Dual Channels SLOT 9 Dual Channels SLOT 10 Dual Channels SLOT 11 Dual Channels SLOT 12 Dual Channels Note: Modules requiring dual fabric SLOT 13 Dual Channels Fan Tray Channels (6816,all 67xx except 6724) and WiSM will not work in Switch Fabric Shared Bus slots 1-8. Power Supply Power Supply BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9
  10. 10. Power Supplies Power Supplies for six, nine and thirteen slot chassis are located at front bottom of chassis Power Supplies for three and four slot chassis are located in the rear AC Power Supplies DC Power Supplies 950W ** 950W 1000W ** 1300W ** 1300W ** 2500W 1400W 2700W 2500W ** 4000W 2700W 6000W 3000W 4000W ** Now EOS 6000W 8700W BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10
  11. 11. Power Supply Redundancy The Catalyst 6500 Can Utilize Two Power Supplies to Work in Either Combined or Redundant Mode Redundant Mode Combined Mode Catalyst 6500 Catalyst 6500 50% 50% 83% 83% Power Supply 1 Power Supply 2 Power Supply 1 Power Supply 2 • Each power supply operates at ~50% capacity • Each power supply provides up to 83% of its capacity • Neither supply operates at >60% or <40% capacity • The total system power available is 167% of the capacity of a single supply • If one fails, the second supply can power the system on its own • If one fails, the second supply may not be able to power the system on its own - this could result in devices or • This is the default and recommended configuration for linecards being shut down the power supplies • This is not the recommended mode for production BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 11
  12. 12. Power Management All Supervisors and Linecards have a power value programmed in EEPROM - this is used to identify how much power to reserve from the power budget. Use the Power Calculator on cisco.com to determine the power supply and heat dissipation requirements - http://www.cisco.com/go/powercalculator If insufficient power is available, the system powers down Power Devices, then switching modules, then services modules. Powered Devices and modules are powered off from highest numbered to lowest numbered (port or slot). BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12
  13. 13. Agenda Chassis and Power Supplies Supervisor Engine and Switch Fabric Architecture Module Architecture Layer 2 Forwarding IP Unicast Forwarding IPv4 Multicast Forwarding Netflow, ACL and QoS Processing Packet Walks BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 13
  14. 14. Supervisor 720 720Gbps crossbar fabric 512/512MB (3A/B) or 1/1GB (3BXL) DRAM PFC3A/B/BXL forwarding engine daughter card Internal RP and SP bootflash (64MB each) Integrated RP/SP CPUs on MSFC3 daughter card (600MHz Optional 512MB and 1GB CF MIPS) bootflash upgrade for SP - WS-CF-UPG-1GB= Supported from Cisco IOS 12.2(14)SX and Catalyst OS - WS-CF-UPG= (512MB)* 8.1(1) Dual external compact flash slots 2 x GbE uplink ports— 2 x SFP <or> 1 x SFP and 1 x 10/100/1000 * Internal CF Adapter comes standard on all Supervisor 720s ordered with 12.2(18)SXE5 or newer BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 14
  15. 15. Supervisor 720/PFC3 Architecture ADJ FIB contains IPv4/IPv6 Fabric interface/replication engine to QoS TCAM prefixes and MPLS contains QoS contains interface with fabric and bus, and for rewrite info entries multicast/SPAN replication ACL entries Hardware for ACL 1Gbps inband SFP / NetFlow table TCAM counters channel SFP GETX for stats and features MSFC 3 ACE ACE Counter Counter Flash QoS QoS Adj Adj FIB FIB ACL ACL Flash RP 1 Gbps NetFlow TCAM TCAM TCAM ACL TCAM RP NetFlow TCAM TCAM TCAM TCAM TCAM contains DRAM DRAM Complex Complex Port ASIC Port ASIC security and feature ACL 1 Gbps entries Flash Flash SP SP L3/4 Engine L3/4 Engine DRAM Complex Complex DRAM MET MET Fabric Interface Fabric Interface L2 Engine Switch Fabric Switch Fabric 20 Gbps and and 16 x 20G Traces 16 x 20G Traces Replication Engine Replication Engine L2 Table (64K) PFC3 L2 Table (64K) DBUS 16 Gbps L2 Table EOBC RBUS Bus contains Traces # 1 to 16 MAC entries BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 15
  16. 16. Supervisor 720-10G 720-Gbps crossbar fabric Internal RP (64MB) and SP (1GB) boot devices PFC3C/CXL forwarding engine daughter card Single external compact flash slot Integrated RP/SP CPUs on MSFC3 Uplink ports: daughter card 2 x 10GE (X2) 2 x SFP Supported from Cisco IOS 1 x 10/100/1000 12.2(33)SXH All uplink ports are available in a NOT supported with Catalyst OS redundant Supervisor configuration 1GB SP / 1GB RP DRAM 2 USB ports (1 console, 1 device - currently disabled) BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 16
  17. 17. Supervisor 720-10G Architecture L2, L3 and L4 Additional fabric channels to operations 10 Gigabit Ethernet uplinks support 10GE interfaces combined in a single ASIC Traces # 1 to 18 10GE SFP GETX 10GE PFC3C/XL ACE ACE Counter Counter Switch Fabric Switch Fabric (#19) Quad PHY MUX PHY Quad PHY Quad PHY Quad QoS QoS FIB FIB ACL ACL NetFlow TCAM TCAM TCAM NetFlow TCAM TCAM TCAM 20 x 20G Traces 20 x 20G Traces (#20) Port ASIC Port ASIC Port ASIC Port ASIC L2 // L3 // L4 Forwarding, L2 L3 L4 Forwarding, 16 Gbps 16 Gbps Security, QoS ACL Security, QoS ACL MSFC 3 Engine Engine Flash 20 Gbps Metropolis Fabric Fabric Flash DRAM RP RP Fabric ASIC ASIC ASIC L2 Table (96k) L2 Table (96k) DRAM Complex Complex MET MET Flash Flash SP SP 20 Gbps Hyperion Fabric ASIC Fabric ASIC Fabric ASIC Bus Interface Bus Interface 1 Gbps DRAM DRAM Complex Complex 1 Gbps Multicast/SPAN Multicast/SPAN Multicast/SPAN Larger L2 Table DBUS 16 Gbps EOBC RBUS Bus BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 17
  18. 18. Supervisor 32 Classic Supervisor—no fabric, uses 16-Gbps bus only PFC3B forwarding engine daughter card SP CPU (400MHz Sibyte) 2 10GE Xenpak + 1 10/100/1000 RJ-45 uplink ports MSFC2a routing engine 512MB/512MB DRAM Internal CF bootdisk (256MB) and MSFC2A bootflash (64MB) External CF slot Uplink options: 8 SFP + 1 10/100/1000 8 1GE SFP + 2 10GE + 1 10/100/1000 1 10/100/1000 RJ-45 uplink ports Supported from Cisco IOS 12.2(18)SXF BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 18
  19. 19. Supervisor 32-8G/PFC3 Architecture PFC3B exactly the same as on Supervisor 720-3B GbE Uplinks Flash ACE QoS QoS FIB FIB ADJ Flash ACE TCAM TCAM ADJ SP CPU Counter Counter TCAM TCAM DRAM DRAM 1 Gbps Port ASIC Port ASIC Flash Flash L3/4 1 Gbps ACL L3/4 NetFlow RP CPU ACL NetFlow DRAM DRAM TCAM TCAM Engine Engine MSFC2a Daughter Card MET MET L2 Engine PFC3B Daughter Replication Replication L2 Table L2 Table Card Engine Engine Bus attached only; no 16 Gbps DBUS fabric support Bus RBUS EOBC BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 19
  20. 20. Supervisor 32-10GE/PFC3 Architecture Dual port ASICs to support two 10GE interfaces 10GE 10/100/1000 Uplinks Uplink Flash Flash ACE QoS QoS FIB FIB ADJ SP CPU SP CPU 1 Gbps ACE TCAM TCAM ADJ TCAM TCAM DRAM DRAM Port Port Port Port Counter Counter Flash Flash ASIC ASIC ASIC ASIC 1 Gbps RP CPU RP CPU ACL ACL L3/4 L3/4 DRAM DRAM NetFlow NetFlow FPGA TCAM TCAM Engine Engine MSFC2a Daughter Card FPGA MUX MUX L2 Engine PFC3B Replication Replication Daughter L2 Table Engine Engine L2 Table Card MET MET 16 Gbps DBUS Bus RBUS EOBC BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 20
  21. 21. Supervisor Chassis Requirements Supervisor 720 (all variants) and Supervisor 32 (all variants) require: Catalyst 6500 or 6500-E chassis High speed fan tray (FAN2/E-FAN) E-Fan cannot be used in non-E Chassis Fan2 cannot be used in E-Series 2500W power supply (AC or DC) or greater 3000W supply for new AC deployments since 2500W AC is End-of-Sale Specific chassis slots: Slot 1 or 2 in 3/4 slot Slot 5 or 6 in 6/9 slot Slot 7 or 8 in 13 slot BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 21
  22. 22. Supervisor 720 Switch Fabric 720-Gbps crossbar switch fabric provides Multiple conflict-free paths between switching modules Fabric channels run at up to 20-Gbps full duplex 20-Gbps in / 20-Gbps out per channel Works with all fabric-capable modules Fabric channels auto-sync speed on per-slot basis (8-Gbps or 20-Gbps) Two fabric channels per slot in 6503-E 6504-E 6506-E 6509-E 6509-V-E In 6513 / 6513-E: One fabric channel slots 1–8 Two fabric channels slots 9–13 “Dual-fabric channel” modules not supported in slots 1–8 of 6513 / 6513-E BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 22
  23. 23. Monitoring Fabric and Bus Utilization Show Platform Hardware Capacity Fabric 6509E#show platform hardware capacity fabric 6509E#show platform hardware capacity fabric Switch Fabric Resources Switch Fabric Resources Bus utilization: current: 25%, peak was 75% at 19:28:31 UTC Mon Feb 222009 Bus utilization: current: 25%, peak was 75% at 19:28:31 UTC Mon Feb 2009 Fabric utilization: Ingress Fabric utilization: Ingress Egress Egress Module Chanl Speed rate peak Module Chanl Speed rate peak rate peak rate peak 11 00 20G 10% 50% @13:49 06Jan09 20% 50% @13:49 06Jan09 20G 10% 50% @13:49 06Jan09 20% 50% @13:49 06Jan09 11 11 20G 20% 50% @13:49 06Jan09 10% 50% @13:49 06Jan09 20G 20% 50% @13:49 06Jan09 10% 50% @13:49 06Jan09 22 00 20G 0% 1% @20:30 13Jan09 0% 1% @20:46 06Jan09 20G 0% 1% @20:30 13Jan09 0% 1% @20:46 06Jan09 22 11 20G 0% 1% @20:47 16Jan09 0% 1% @16:52 06Jan09 20G 0% 1% @20:47 16Jan09 0% 1% @16:52 06Jan09 33 00 20G 20% 40% @13:49 06Jan09 0% 0% @13:49 06Jan09 20G 20% 40% @13:49 06Jan09 0% 0% @13:49 06Jan09 66 00 20G 0% 1% @17:44 06Jan09 0% 1% @00:36 08Jan09 20G 0% 1% @17:44 06Jan09 0% 1% @00:36 08Jan09 88 00 8G 0% 3% @16:33 09Feb09 50% 100% @13:49 06Jan09 8G 0% 3% @16:33 09Feb09 50% 100% @13:49 06Jan09 BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 23
  24. 24. Policy Feature Card 3 Mandatory daughter card for supervisor engine enabling key hardware features: Layer 2 switching IPv4 and IPv6 unicast forwarding IPv4 and IPv6 multicast forwarding Security ACLs QoS/policing NetFlow accounting MPLS*/VRF-lite Bidir PIM, GRE/v6 tunnels, CoPP PFC3C and PFC3CXL also support VSS * MPLS on all but PFC3A VSS is covered in RST-3468 Cisco Catalyst Virtual Switch (VSS) BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 24
  25. 25. Reference: PFC3 Comparison Feature PFC3A PFC3B / BXL PFC3C / CXL FIB TCAM 256K 256K / 1M 256K / 1M Adjacency Table 1M 1M 1M 128K (115K) / 128K (115K) / NetFlow Table 128K (64K) 256K (230K) 256K (230K) MAC Table 64K (32K) 64K (32K) 96K (80K) IPv6 Hardware Hardware Hardware Bidir PIM Hardware Hardware Hardware Native MPLS No (VRF Lite Only) Yes Yes EoMPLS No Yes Yes NAT, Tunnels Hardware Hardware Hardware uRPF Check Yes (Multipath) Yes (Multipath) Yes (Multipath) VSS No No Yes =BRKARC-3465 from earlier versionits affiliates. All rights reserved. Change © 2010 Cisco and/or Cisco Confidential 25
  26. 26. Reference: PFC3 Comparison (Cont.) Feature PFC3A PFC3B / BXL PFC3C / CXL ACL TCAM 32K/4K Dual-Bank 32K/4K Dual-Bank 32K/4K Dual-Bank PACLs Yes Yes Yes ACL Counters No Yes Yes QoS TCAM 32K/4K 32K/4K 32K/4K ACL Labels 512 4K 4K ACL LOUs 64 64 64 CPU Rate Limiters 8 (L3), 2 (L2) 8 (L3), 2 (L2) 8 (L3), 2 (L2) CoPP Yes Yes Yes User-Based Policing Yes Yes Yes Egress Policing Yes Yes Yes HSRP/VRRP Groups Protocol limit Protocol limit Protocol limit Unique MAC/Interface Yes Yes Yes =BRKARC-3465 from earlier versionits affiliates. All rights reserved. Change © 2010 Cisco and/or Cisco Confidential 26
  27. 27. High-Level Forwarding Engine Logic Frame Frame Input Layer 2 Input Layer 2 received received lookup lookup FIB TCAM Layer 2 Table FIB lookup FIB lookup ACL TCAM Router Router Yes Input QoS Input QoS Input ACL Input ACL MAC? MAC? lookup lookup lookup lookup No QoS TCAM NetFlow Table NetFlow lookup NetFlow lookup Input QoS Input QoS lookup lookup QoS TCAM Input ACL Output QoS Output QoS Output ACL Output ACL Output Layer 2 Output Layer 2 Input ACL lookup lookup lookup lookup lookup lookup lookup lookup ACL TCAM QoS TCAM ACL TCAM Layer 2 Table Bridged Bridged Output QoS Output QoS Output ACL Output ACL Transmit Transmit NetFlow NetFlow lookup lookup lookup lookup frame frame NetFlow Table QoS TCAM ACL TCAM BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 27
  28. 28. PFC TCAM Technology Value 1 TCAM—Ternary Content Addressable Memory Value 2 Value 3 Leveraged heavily in Mask 1 Catalyst 6500 Value 4 FIB, ACL, QoS, NetFlow all utilize Value 5 TCAM memory Value 6 All entries accessed in parallel— Value 7 fixed performance independent Value 8 of number of entries Value 1 Memory consists of groups of Value 2 values and associated masks Value 3 Mask 2 8:1 ratio of values to masks Value 4 Masks are used to “wildcard” Value 5 some portion of values Value 6 Value 7 Value 8 Masks Values BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 28
  29. 29. Generic TCAM Lookup Logic 1 3 Compare Generate Packet Fields Lookup 0110xxxx 01101010 011010xx 110110xx 1 Key Lookup Key 000111xx 2 2 11111100 101101xx 3 1. Relevant fields read 100111xx 4 from contents of packet 000000xx 5 010010xx 6 2. Lookup key created 1=“Compare” 111111xx 7 0=“Mask” 3. As lookup key 001100xx 8 compared to value entries, associated 0111xxxx 1 mask applied 1011xxxx 2 11110000 1101xxxx 3 4. Longest match returns HIT! 0110xxxx 4 Result result 1110xxxx 45 0011xxxx 6 Result format varies depending on lookup type 0000xxxx 7 1000xxxx 8 BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Masks Cisco Confidential Values 29
  30. 30. Agenda Chassis and Power Supplies Supervisor Engine and Switch Fabric Architecture Module Architecture Layer 2 Forwarding IP Unicast Forwarding IPv4 Multicast Forwarding Netflow, ACL and QoS Processing Packet Walks BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 30
  31. 31. Classic Module DBUS Example: WS-X6416-GBIC RBUS Classic Module Port Port Port Port Port Port Port Port ASIC ASIC ASIC ASIC ASIC ASIC ASIC ASIC 4xGE 4xGE 4xGE 4xGE DBUS Port ASICs for physical RBUS connectivity, buffering, and queuing Classic Module Port Port ASIC ASIC 48x10/100 Example: WS-X6148A-RJ-45 BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 31
  32. 32. Reference: Classic Module Family Copper 10/100 and 10/100/1000: Fiber 10, 100, and GE: WS-X6148A-GE-TX WS-X6024-10FL-MT WS-X6148A-GE-45AF WS-X6148-FE-SFP WS-X6148-GE-TX WS-X6224-100FX-MT WS-X6148V-GE-TX WS-X6324-100FX-SM WS-X6148-GE-45AF WS-X6324-100FX-MM WS-X6148-RJ-21 WS-X6408-GBIC WS-X6148-RJ21V WS-X6148-RJ-45V WS-X6408A-GBIC WS-X6148A-RJ-45 WS-X6416-GBIC WS-X6148A-45AF WS-X6416-GE-MT WS-X6148X2-RJ-45 WS-X6148X2-45AF WAN: WS-X6148-RJ-45 WS-X6182-2PA WS-X6148-21AF WS-X6148-45AF Service Modules: WS-X6196-RJ-21 WS-X6380-NAM WS-X6196-21AF WS-X6381-IDS3 WS-X6248A-TEL WS-X6248-TEL WS-X6066-SLB-APC9 WS-X6248-RJ-45 WS-X6066-SLB-S-K9 WS-X6316-GE-TX WS-X6624-FXS WS-X6348-RJ-45 WS-X6608-E1 WS-X6348-RJ21V WS-X6608-T1 WS-X6348-RJ-45V BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 32
  33. 33. CEF256 Module Example: WS-X6516-GBIC DBUS 8Gbps Fabric RBUS Channel CEF256 Module Fabric Fabric Interface Interface Fabric interface LCDBUS to interface with LCRBUS fabric and bus Replication Replication Engine Engine Port Port Port Port Port Port Port Port ASIC ASIC ASIC ASIC ASIC ASIC ASIC ASIC 4xGE 4xGE 4xGE 4xGE Replication engine for local SPAN/multicast Local linecard bus for replication ASIC interconnection BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 33
  34. 34. CEF256 Module with DFC Example: WS-X6516-GBIC with WS-F6K-DFC Layer 2/4 Engine for L2 and ACL/QoS lookups 8Gbps Fabric Layer 3 Channel Engine for CEF256 FIB/Adj and Layer 2/4 Layer 2/4 L3 L3 Module NetFlow Fabric Fabric lookups Engine Engine Engine Engine with DFC Interface Interface DFC LCDBUS LCRBUS Replication Replication Engine Engine Port Port Port Port Port Port Port Port ASIC ASIC ASIC ASIC ASIC ASIC ASIC ASIC 4xGE 4xGE 4xGE 4xGE BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 34
  35. 35. Reference: CEF256 Module Family 10GE: WAN: WS-X6502-10GE* WS-X6582-2PA Copper 10/100 and 10/100/1000: Service Modules: WS-X6548-RJ-21* WS-SVC-NAM-1 WS-X6548-RJ-45* WS-SVC-NAM-2 WS-X6548-GE-TX WS-SVC-IDSM2-BUN-K9 WS-X6548V-GE-TX WS-SVC-FWM-1-K9 WS-X6548-GE-45AF WS-SVC-SSL-1 WS-X6516-GE-TX WS-SVC-CSG-1 Fiber 100 and GE: WS-SVC-CMM WS-X6524-100FX-MM* WS-SVC-AON-1-K9 WS-X6516-GBIC* WS-SVC-ADM-1-K9, WS-X6516A-GBIC* WS-X6816-GBIC (dCEF256)** WS-SVC-AGM-1-K9 DFCs: WS-SVC-CSG-1 WS-F6K-DFC WS-SVC-IPSEC-1 WS-F6K-DFC3A WS-SVC-MWAM-1 WS-F6K-DFC3B WS-SVC-PSD-1 WS-F6K-DFC3BXL WS-SVC-WLAN-1-K9 * Supports optional DFC/DFC3, ** Requires DFC/DFC3 BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 35
  36. 36. CEF720 Module Example: WS-X6748-SFP 20Gbps Fabric DBUS 20Gbps Fabric Channel RBUS Channel Complex A Complex B CEF720 Module Fabric Fabric Bus Bus Bus Bus Fabric Fabric Interface & Interface & Interface Interface Interface Interface Interface & Interface & Replication Replication Replication Replication Engine CFC Engine Engine Engine Port Port Port Port Port Port Port Port ASIC ASIC ASIC ASIC ASIC ASIC ASIC ASIC 12xGE 12xGE 12xGE 12xGE Combined fabric interface and replication engine Transparent bus interface Bus interface for control information only!! BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 36
  37. 37. CEF720 Module with DFC3 Example: WS-X6748-SFP with WS-F6700-DFC3B 20Gbps Fabric 20Gbps Fabric Channel Channel Complex A Complex B CEF720 Module Fabric Fabric Layer 2 Layer 2 Layer 2 Layer 2 Fabric Fabric with DFC3 Interface & Interface & Engine Engine Engine Engine Interface & Interface & Replication Replication Replication Replication Engine Engine L3/4 Engine Engine L3/4 Engine Engine Port Port Port Port DFC3 Port Port Port Port ASIC ASIC ASIC ASIC ASIC ASIC ASIC ASIC 12xGE 12xGE 12xGE 12xGE Layer3/4 Engine for FIB/Adj, ACL, QoS and NetFlow Layer 2 lookups Engine for L2 lookups BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 37
  38. 38. Catalyst 6500 Linecards dCEF720 Linecards – WS-X6708-10G-3C/CXL The dCEF720 linecards are the fourth generation of high performance linecards to be introduced into the Catalyst 6500 linecard family… Supervisor Supervisor Switch Fabric Switch Fabric dCEF720 dCEF720 Linecards have no Linecards have no EOBC 20Gb 20Gb connection to the connection to the Data or Results Data or Results FABRIC FABRIC BACKPLANE INTF BACKPLANE INTF FABRIC FABRIC buses buses INTF INTF INTF INTF They connect into They connect into FPGA FPGA FPGA FPGA FIRE FIRE FIRE FIRE the Switch fabric the Switch fabric ASIC ASIC ASIC ASIC via dual 20Gb via dual 20Gb DFC3 DFC3 Fabric channels Fabric channels FPGA MUX FPGA MUX FPGA MUX FPGA MUX They come with a They come with a FPGA MUX FPGA MUX FPGA MUX FPGA MUX DFC3/4 pre- DFC3/4 pre- installed with no installed with no PORT PORT ASIC ASIC PORT PORT ASIC ASIC PORT PORT ASIC ASIC PORT PORT ASIC ASIC PORT PORT ASIC ASIC PORT PORT ASIC ASIC PORT PORT ASIC ASIC PORT PORT ASIC ASIC option for a CFC option for a CFC 7 7 5 5 4 4 1 1 8 8 6 6 3 3 2 2 BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 38
  39. 39. Catalyst 6500 Linecards dCEF720 Linecards – WS-X6716-10x-3C/CXL The dCEF720 linecards are the fourth generation of high performance linecards to be introduced into the Catalyst 6500 linecard family… Supervisor Supervisor Switch Fabric Switch Fabric dCEF720 dCEF720 Linecards have no Linecards have no EOBC 20Gb 20Gb connection to the connection to the Data or Results Data or Results FABRIC FABRIC BACKPLANE INTF BACKPLANE INTF FABRIC FABRIC buses buses INTF INTF INTF INTF FPGA FPGA FPGA FPGA They connect into They connect into FIRE FIRE FIRE FIRE the Switch fabric the Switch fabric ASIC ASIC ASIC ASIC via dual 20Gb via dual 20Gb DFC3 DFC3 Fabric channels Fabric channels PORT PORT PORT PORT PORT PORT PORT PORT ASIC ASIC ASIC ASIC ASIC ASIC ASIC ASIC They come with a They come with a MUX MUX MUX MUX MUX MUX MUX MUX DFC3/4 pre- DFC3/4 pre- installed with no installed with no MUX MUX MUX MUX MUX MUX MUX MUX MUX MUX MUX MUX MUX MUX MUX MUX option for a CFC option for a CFC 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 10 11 12 9 10 11 12 13 14 15 16 13 14 15 16 BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 39
  40. 40. Reference: CEF720 Module Family 10GE: Service Modules: WS-X6704-10GE* WS-SVC-WiSM-1-K9 WS-X6708-10G** DFCs: WS-X6716-10G** WS-X6700-CFC (Centralized WS-X6716-10T** Forwarding Card) Copper 10/100/1000: WS-F6700-DFC3A WS-X6748-GE-TX* WS-F6700-DFC3B WS-F6700-DFC3BXL Fiber GE: WS-F6700-DFC3C WS-X6724-SFP* WS-F6700-DFC3CXL WS-X6748-SFP* *Supports optional DFC3 ** Requires DFC3 BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 40
  41. 41. Distributed Forwarding One or more modules have a local forwarding engine (DFC—Distributed Forwarding Card) Central engine and distributed engines perform different lookups independently and simultaneously Implementation is fully distributed All hardware from PFC is present on the DFC Full Layer 2, Layer 3, ACL/QoS information downloaded from Supervisor Ingress DFC performs all lookups locally Deterministic, highly scalable—Not flow-based NOT just for local switching— destination interface irrelevant DFCs always require Cisco IOS software on both the SP and the RP BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 41
  42. 42. DFC3x and PFC3x Interoperability The use of a DFC3 requires it to operate with the equivalent PFC3 version - a mix of PFC3 and DFC3 versions will result in the system operating at the lowest common denominator PFC3A PFC3B PFC3BXL PFC3C PFC3CXL DFC3A Operate as Operate as Operate as Operate as PFC3A PFC3A PFC3A PFC3A DFC3B Operate as Operate as Operate as Operate as DFC3A PFC3B PFC3B PFC3B DFC3BXL Operate as Operate as Operate as Operate as DFC3A DFC3B PFC3B and PFC3BXL DFC3B DFC3C Operate as Operate as Operate as Operate as DFC3A DFC3B PFC3B and PFC3C DFC3B DFC3CXL Operate as Operate as Operate as Operate as DFC3A DFC3B DFC3BXL DFC3C BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 42
  43. 43. Mixing Different Linecard Architectures When utilizing Centralized Forwarding, the backplane will operate in one of three modes—these modes are determined by the combination of linecards installed in the chassis, from which module the traffic is sourced from and to which module the traffic is destined. Mode Description Illustration • Between non fabric modules and between a non fabric and a fabric enabled linecard • Throughput – 15 Mpps (@ 64 byte frames) Data H Data H FLOW THROUGH • Bandwidth – 16 Gbps of bandwidth shared throughou • Data Bus frame size is variable; min of 4 cycles (64B Data) on the DBus for every frame +1 wait cycle Bus • Between fabric linecards when a non fabric linecard is in the chassis. Data H D H • Throughput – 15 Mpps (@ 64 byte frames); independent of frame size for CEF256 and CEF720 TRUNCATED • Bandwidth – 16 G shared for classic; 8 G per CEF256; Bus 20 G/channel CEF720. • Data Bus frame size is variable; min of 4 cycles (64 Bytes Data) on the Data Bus for every frame. • When only ALL fabric enabled linecards in a chassis • Throughput – 30 Mpps (@ any frame size) D H D H D H D H • Bandwidth – 8 G CEF256; 20 G/channel CEF720 COMPACT • Data Bus frame size is constant (compact header); 2 cycles (32 B Data) on the DBus for every frame + no wait cycle Bus BRKARC-3465 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 43

×