Nguyen Van Linh, profile picture
Uploaded byNguyen Van Linh
PDF, PPTX1,051 views

Brkarc 3601

The document discusses the Nexus 7000 and 7700 series switches, including their architecture, modules, and deployment models. It provides an overview of the chassis, supervisor engines, I/O modules, and NX-OS software. Examples of common use cases and design considerations are presented. Key capabilities like VPC, OTV, and VDCs are also covered, along with real-world design examples. The goal is to provide an understanding of the flexible and scalable architecture to enable various data center designs.

Embed presentation

Download as PDF, PPTX
Nexus 7000/7700 Architecture and Deployment Models Satish Kondalam Technical Marketing Engineer
Session Abstract This session will discuss the foundations of the Nexus 7000 and 7700 series switches, including chassis, I/O modules, and NX-OS software. Examples will show common use-cases for different module types and considerations for module interoperability. The focus will then shift to key platform capabilities and features – including VPC,OTV, VDCs, and others – along with real-world designs and deployment models.
Session Goals • To provide an understanding of the Nexus 7000 / Nexus 7700 switching architecture, which provides the foundation for flexible, scalable Data Centre designs • To examine key Nexus 7000 / Nexus 7700 design building blocks and illustrate common design alternatives leveraging those features and functionalities • To see how the Nexus 7000 / Nexus 7700 platform plays in emerging technologies and architectures
• Introduction to Nexus 7000 / Nexus 7700 • Nexus 7000 / Nexus 7700 Architecture • Chassis, Supervisor Engines and NX-OS software, I/O modules (M2/F2E/F3) • I/O Module Interoperability • Fabric Architecture • Hardware Forwarding • Generic Designs with Nexus 7000 / Nexus 7700 • STP/VPC, L4-7 services integration, VDCs, VRF/MPLS VPNs, OTV Agenda
Introduction to Nexus 7000 / Nexus 7700 Platform Data-centre class Ethernet switches designed to deliver high performance, high availability, system scale, and investment protection Designed for wide range of Data Centre deployments, focused on feature-rich 10G/40G/100G density and performance I/O Modules Supervisor Engines Fabrics Chassis
Nexus 7000 General purpose DC switching w/10/40/100G Nexus 7700 Targeted at Dense 40G/100G deployments CommonFoundation • Same release vehicles, versioning, feature-sets • Common configuration model • Common operational model • Common fabric ASICs (Fab2) and architecture • Same central arbitration model • Same VOQ/QOS model • Identical forwarding ASICs (F2E, F3) • Consistent hardware feature sets • Consistent hardware scale Nexus 7000 / Nexus 7700 – Common Foundation
• Introduction to Nexus 7000 / Nexus 7700 • Nexus 7000 / Nexus 7700 Architecture • Chassis, Supervisor Engines and NX-OS software, I/O modules (M2/F2E/F3) • I/O Module Interoperability • Fabric Architecture • Hardware Forwarding • Generic Designs with Nexus 7000 / Nexus 7700 • STP/VPC, L4-7 services integration, VDCs, VRF/MPLS VPNs, OTV Agenda
Nexus 7000 Chassis Family Front Rear 21RU N7K-C7010 25RU Front RearN7K-C7018 Front RearN7K-C7009 14RU Nexus 7010 Nexus 7018 Nexus 7009 Front N7K-C7004 7RU Rear Nexus 7004 Front Back Side Side Side Side Side Back
Nexus 7700 Chassis Family Front Rear 26RU N77-C7718 Nexus 7718 Front Rear 14RU N77-C7710 Nexus 7710 Front Rear 9RU N77-C7706 Nexus 7706 Front Back Front Back Front Back
Nexus 7702 Chassis NX-OS 7.2 and later Front RearN77-C7702 Front Back 3RU One Supervisor Engine Two Power Supplies One F3 Series I/O Module One Fan Tray (3 Fans) No Fabric Modules!
Supervisor Engine 2 / 2E • Provides all control plane and management functions • Connects to fabric via 1G inband interface • Interfaces with I/O modules via 1G switched EOBC • Onboard central arbiter ASIC Controls access to fabric bandwidth via dedicated arbitration path to I/O modules Console Port Management Ethernet N7K-SUP2/N7K-SUP2E USB Host Ports ID and Status LEDs Supervisor Engine 2 (Nexus 7000) Supervisor Engine 2E (Nexus 7000 / Nexus 7700) Base performance High performance One quad-core 2.1GHz CPU with 12GB DRAM Two quad-core 2.1GHz CPU with 32GB DRAM USB Log Flash USB Expansion Flash N77-SUP2E ID and Status LEDs Console Port Management Ethernet USB Expansion Flash
Supervisor Engine 2 / 2E Architecture Fabric ASIC VOQs I/O Controller Main CPU Main CPU NVRAM DRAM Bootflash (eUSB) USB expansion USB logflash USB device port Console Mgmt0 Central ArbiterSwitched EOBC 12GB/32GB2.1GHz Quad-Core 32MB 2.1GHz Quad-Core Sup2E Only To Fabric Modules To Module VOQs Dedicated Arbitration Path Dedicated Arbitration Path To Module CPUs Switched 1GE EOBC 1GE Inband 2GB
Nexus 7000 / 7700 I/O Module Families M1 1G and 10G M2 10G / 40G / 100G F1 10G F2 10G F2E 10G F3 10G / 40G / 100G F2E 10G F3 10G / 40G / 100G F3 closes the F/M feature gap!
Nexus 7000 M2 I/O Modules • 10G / 40G / 100G M2 I/O modules • Share common hardware architecture – multi-chipset • Two integrated forwarding engines (120Mpps) • Layer 2/Layer 3 forwarding with L3/L4 services (ACL/QOS) and advanced features (MPLS/OTV/GRE etc.) • Large forwarding tables (900K FIB/ 128K ACL) N7K-M224XP-23L / N7K-M206FQ-23L / N7K-M202CF-22L N7K-M224XP-23L N7K-M206FQ-23L N7K-M202CF-22L Module Port Density Optics Bandwidth M2 10G 24 x 10G (plus Nexus 2000 FEX support) SFP+ 240G M2 40G 6 x 40G (or up to 24 x 10G via breakout) QSFP+ 240G M2 100G 2 x 100G CFP 200G
Fabric ASIC Nexus 7000 M2 I/O Module Architecture N7K-M224XP-23L / N7K-M206FQ-23L / N7K-M202CF-22L To Fabric Modules Front Panel Ports VOQsVOQsForwarding Engine Replication Engine Replication Engine LinkSec + 12 X 10G MAC -or- 3 X 40G MAC -or- 1 X 100G MAC VOQs VOQs Forwarding Engine Replication Engine Replication Engine LinkSec + 12 X 10G MAC -or- 3 X 40G MAC -or- 1 X 100G MAC LC CPU EOBC Arbitration Aggregator To Central Arbiters
Nexus 7000 / Nexus 7700 F2E I/O Modules • 48-port 1G/10G with SFP/SFP+ transceivers • 480G full-duplex fabric connectivity • System-on-chip (SOC) forwarding engine design 12 independent SOC ASICs • Layer 2/Layer 3 forwarding with L3/L4 services (ACL/QOS) • Interoperability with M1/M2, in Layer 2 mode on Nexus 7000 Proxy routing for inter-VLAN/L3 traffic N7K-F248XP-25E / N7K-F248XT-25E / N77-F248XP-23E 7000: Supported in NX-OS release 6.1(2) and later 7700: Supported in NX-OS release 6.2(2) and later Nexus 7000 N7K-F248XP-25E Nexus 7000 N7K-F248XT-25E Nexus 7700 N77-F248XP-23E
4 X 10G SOC 1 Nexus 7000 F2E Module Architecture N7K-F248XP-25E / N7K-F248XT-25E To Fabric Modules 1 Front Panel Ports (SFP/SFP+) 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 4 X 10G SOC 2 4 X 10G SOC 3 4 X 10G SOC 4 4 X 10G SOC 5 4 X 10G SOC 6 4 X 10G SOC 7 4 X 10G SOC 8 4 X 10G SOC 9 4 X 10G SOC 10 4 X 10G SOC 11 4 X 10G SOC 12 Fabric ASIC LC CPU EOBC Arbitration Aggregator To Central Arbiters to ARB to LC CPU LC Inband
4 X 10G SOC 1 Nexus 7700 F2E Module Architecture N77-F248XP-23E 1 Front Panel Ports (SFP/SFP+) 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 4 X 10G SOC 2 4 X 10G SOC 3 4 X 10G SOC 4 4 X 10G SOC 5 4 X 10G SOC 6 4 X 10G SOC 7 4 X 10G SOC 8 4 X 10G SOC 9 4 X 10G SOC 10 4 X 10G SOC 11 4 X 10G SOC 12 To Fabric Modules Fabric ASIC Fabric ASIC LC CPU EOBC Arbitration Aggregator To Central Arbiters to ARB to LC CPU LC Inband
Nexus 7000 F3 I/O Modules • 10G / 40G / 100G F3 I/O modules • Share common hardware architecture • SOC-based forwarding engine design 6 independent SOC ASICs per module • Layer 2/Layer 3 forwarding with L3/L4 services (ACL/QOS) and advanced features (MPLS/OTV/GRE/VXLAN etc.) • Require Supervisor Engine 2 / 2E N7K-F348XP-25 / N7K-F312FQ-25 / N7K-F306CK-25 Module Port Density Optics Bandwidth F3 10G 48 x 1/10G (plus Nexus 2000 FEX support) SFP+ 480G F3 40G 12 x 40G (or up to 48 x 10G via breakout) QSFP+ 480G F3 100G 6 x 100G CPAK 550G N7K-F348XP-25 N7K-F312FQ-25 N7K-F306CK-25
Nexus 7700 F3 I/O Modules • 10G / 40G / 100G F3 I/O modules • Share common hardware architecture • SOC-based forwarding engine design 6 independent SOC ASICs per 10G module 12 independent SOC ASICs per 40G/100G module • Layer 2/Layer 3 forwarding with L3/L4 services (ACL/QOS) and advanced features (MPLS/OTV/GRE/VXLAN etc.) N7K-F348XP-25 / N7K-F312FQ-25 / N7K-F306CK-25 Module Port Density Optics Bandwidth F3 10G 48 x 1/10G (plus Nexus 2000 FEX support) SFP+ 480G F3 40G 24 x 40G (or up to 76 x 10G + 5 x 40G via breakout) QSFP+ 960G F3 100G 12 x 100G CPAK 1.2T N77-F348XP-23 N77-F324FQ-25 N77-F312CK-26
8 X 10G SOC 1 Nexus 7000 F3 48-Port 1G/10G Module Architecture N7K-F348XP-25 8 X 10G SOC 6 8 X 10G SOC 5 8 X 10G SOC 4 8 X 10G SOC 3 8 X 10G SOC 2 1 Front Panel Ports (SFP/SFP+) 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 To Fabric Modules Fabric ASIC EOBC LC Inband to ARB x 6… FSA CPU 1G switch To Central Arbiters …x 6 to FSA CPU Arbitration Aggregator
FSA CPU Fabric Services Accelerator (FSA) for F3 • High-performance module CPU with on-board acceleration engines • 6Gbps inband connectivity from SOCs to FSA • Multi-Mpps packet processing • 2 X 2GB dedicated DRAM • Performance/scale boost for distributed fabric services, including sampled Netflow and BFD (roadmap) • Other potential applications include distributed ARP/ping processing, data plane packet analysis (wireshark), network probing, etc. 6 x 1Gbps Module Inband I/O 2GB DRAM Dual-Core LC CPU Acceleration Engines 2GB DRAM EOBC
Nexus 7000 F3 12-Port 40G Module Architecture N7K-F312FQ-25 To Fabric Modules 2 X 40G SOC 6 2 X 40G SOC 5 2 X 40G SOC 4 2 X 40G SOC 3 2 X 40G SOC 2 2 X 40G SOC 1 Fabric ASIC Front Panel Ports (QSFP+) EOBC LC Inband to ARB x 6… FSA CPU 1G switch To Central Arbiters …x 6 to FSA CPU Arbitration Aggregator 1 2 3 4 5 6 7 8 9 10 11 12
Nexus 7000 F3 6-Port 100G Module Architecture N7K-F306CK-25 To Fabric Modules 1 X 100G SOC 6 1 X 100G SOC 5 1 X 100G SOC 4 1 X 100G SOC 3 1 X 100G SOC 2 1 X 100G SOC 1 Fabric ASIC 2 3 4 5 61 Front Panel Ports (CPAK) EOBC LC Inband to ARB x 6… FSA CPU 1G switch To Central Arbiters …x 6 to FSA CPU Arbitration Aggregator
8 X 10G SOC 1 Nexus 7700 F3 48-Port 1G/10G Module Architecture N77-F348XP-23 To Fabric Modules 8 X 10G SOC 6 8 X 10G SOC 5 8 X 10G SOC 4 8 X 10G SOC 3 8 X 10G SOC 2 1 Front Panel Ports (SFP/SFP+) 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 Fabric ASIC Fabric ASIC EOBC LC Inband to ARB x 6… FSA CPU 1G switch To Central Arbiters …x 6 to FSA CPU Arbitration Aggregator
2 X 40G SOC 1 Nexus 7700 F3 24-Port 40G Module Architecture N77-F324FQ-25 Front Panel Ports (QSFP+) To Fabric Modules 2 X 40G SOC 12 2 X 40G SOC 11 2 X 40G SOC 10 2 X 40G SOC 9 2 X 40G SOC 8 2 X 40G SOC 7 2 X 40G SOC 6 2 X 40G SOC 5 2 X 40G SOC 4 2 X 40G SOC 3 2 X 40G SOC 2 Fabric ASIC Fabric ASIC EOBC LC Inband to ARB x 12… To Central Arbiters …x 12 to FSA CPU FSA CPU 1G switch Arbitration Aggregator 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
1 X 100G SOC 1 Nexus 7700 F3 12-Port 100G Module Architecture N77-F312CK-26 Front Panel Ports (CPAK) 2 3 4 5 6 7 8 9 10 11 121 To Fabric Modules 1 X 100G SOC 12 1 X 100G SOC 11 1 X 100G SOC 10 1 X 100G SOC 9 1 X 100G SOC 8 1 X 100G SOC 7 1 X 100G SOC 6 1 X 100G SOC 5 1 X 100G SOC 4 1 X 100G SOC 3 1 X 100G SOC 2 Fabric ASIC Fabric ASIC EOBC LC Inband to ARB x 12… To Central Arbiters …x 12 to FSA CPU FSA CPU 1G switch Arbitration Aggregator
F3 Module 40G and 100G Flows • Virtual Queuing Index (VQI) sustains 10G, 40G, or 100G traffic flow based on destination interface type • No single-flow limit – full 40G/100G flow support Ingress Modules 10G 10G 40G 40G 100G Spines Spines Spines Spines Fabrics Egress Interfaces Destination VQIs 1 VQI 1 VQI 1 VQI 1 VQI 1 VQI
• Introduction to Nexus 7000 / Nexus 7700 • Nexus 7000 / Nexus 7700 Architecture • Chassis, Supervisor Engines and NX-OS software, I/O modules (M2/F2E/F3) • I/O Module Interoperability • Fabric Architecture • Hardware Forwarding • Generic Designs with Nexus 7000 / Nexus 7700 • STP/VPC, L4-7 services integration, VDCs, VRF/MPLS VPNs, OTV Agenda
I/O Module Interoperability • General module interoperability rule is: “+/-1 generation” in same Virtual Device Context (VDC) • Layer 3 forwarding behaviour is key difference between interop models: • “Proxy Forwarding” • “Ingress Forwarding” with Lowest Common Denominator
L2 Host A 10.1.10.100 vlan 10 Host B 10.1.20.100 vlan 20 L2 Proxy Forwarding Model • F2E modules run in pure Layer 2 mode – all L3 functions disabled • M2 modules host SVIs and other L3 functions • From F2E perspective, Router MAC reachable via M2 modules • All packets destined to Router MAC forwarded through fabric toward one M2 module, selected via port-channel hash • M2 modules(s) perform all L3 forwarding and policy, pass packets back over fabric to output port • Key consideration: M-series L3 routing capacity versus F-series front-panel port count – How much Layer 3 routing is required? M2 + F2E VDC F2Emodules M2modules interface vlan 10 ip address 10.1.10.1/24 ! interface vlan 20 ip address 10.1.20.1/24 Router MAC reached via M2 modules M2 modules host SVIs MAC Table rtr-mac → M2 modules Port-channel hash selects M2 module
Ingress Forwarding with Lowest Common Denominator Model • F3 module interoperability always “Ingress Forwarding” – NO proxy forwarding Ingress module receiving packet makes all forwarding decisions for that packet • Supported feature set and scale based on Lowest Common Denominator Feature available if all modules support the feature Table sizes based on lowest capacity F3 + M2 VDC -or- F3 + F2E VDC Module Types in VDC Layer 2 Layer 3 VPC MPLS OTV Fabric Path VXLAN Table Sizes F3 ✓ ✓ ✓ ✓ ✓ ✓ ✓ F3 size F3 + M2 ✓ ✓ ✓ ✓ ✓ ✗ ✗ F3 size F3 + F2E ✓ ✓ ✓ ✗ ✗ ✓ ✗ F2E size M2 + F2E + F3 Not supported Not all features supported by software today…
Module Interoperability Use Cases • M2 + F2E VDC • Provide higher-density 1G/10G while supporting M2 features and L3 functions • Full internet routes, MPLS VPNs • FabricPath with increased MAC address scale (proxy L2 learning) • F2E + F3 VDC • Introduction of 40G/100G into existing 10G environments • Migration to larger table sizes • Transition to additional features/functionality (OTV, MPLS, VXLAN, etc.) • M2 + F3 VDC • Introduce higher 1G/10G/40G/100G port-density while maintaining feature-set • Avoid proxy-forwarding model for module interoperability • Migrate to 40G/100G interfaces with full-rate flow capability M2 F2E F2E F3 M2 F3
• Introduction to Nexus 7000 / Nexus 7700 • Nexus 7000 / Nexus 7700 Architecture • Chassis, Supervisor Engines and NX-OS software, I/O modules (M2/F2E/F3) • I/O Module Interoperability • Fabric Architecture • Hardware Forwarding • Generic Designs with Nexus 7000 / Nexus 7700 • STP/VPC, L4-7 services integration, VDCs, VRF/MPLS VPNs, OTV Agenda
Crossbar Switch Fabric Modules • Provide interconnection of I/O modules • Nexus 7000 and Nexus 7700 fabrics based on Fabric 2 ASIC • Each installed fabric increases available per-payload slot bandwidth • Different I/O modules leverage different amount of available fabric bandwidth • Access to fabric bandwidth controlled using QOS-aware central arbitration with VOQ N7K-C7018-FAB-2 N7K-C7010-FAB-2 N7K-C7009-FAB-2 Fabric Module Supported Chassis Per-fabric module bandwidth Max fabric modules Total bandwidth per slot Nexus 7000 Fabric 2 7009 / 7010 / 7018 110Gbps per slot 5 550Gbps per slot Nexus 7700 Fabric 2 7706 / 7710 / 7718 220Gbps per slot 6 1.32Tbps per slot N77-C7718-FAB-2 N77-C7710-FAB-2 N77-C7706-FAB-2
Nexus 7000 / Nexus 7700 implement 3-stage crossbar switch fabric • Stages 1 and 3 on I/O modules • Stage 2 on fabric modules Ingress ModuleEgress Module Ingress Module 220G (4 x 55G) Egress Module Multistage Crossbar 1st stage 2nd stage 3rd stage Fabric ASIC Fabric ASIC Fabric ASICFabric ASIC 1.32T 1st stage 3rd stage 550G 110G (2 x 55G) 1 Fabric ASIC 2 3 4 5Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC Fabric Modules Nexus 7000 Nexus 7700 Fabric ASIC Fabric ASIC Fabric Modules Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC 1 Fabric ASIC 2 3 4 5 Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC 6 Fabric ASIC
110Gbps220Gbps330Gbps440Gbps550Gbps Local Fabric (480G) Local Fabric (240G) I/O Module Capacity – Nexus 7000 One fabric: • Any port can pass traffic to any other port in VDC Three fabrics: • 240G M2 module has maximum bandwidth Five fabrics: • 480G F2E/F3 module has maximum bandwidth Fabric 2 Modules 1Fabric ASIC 2Fabric ASIC 3Fabric ASIC 4Fabric ASIC 5Fabric ASIC per slot bandwidth
What About Nexus 7004? • Nexus 7004 has no fabric modules • Each I/O module has local fabric with 10 available fabric channels • I/O modules connect “back-to-back” via 8 fabric channels • Two fabric channels “borrowed” to connect supervisor engines Sup Slot 2Sup Slot 1 M2/F2E/F3 Module 4 M2/F2E/F3 Module 3 Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC 2 * 55G fabric channels 8 * 55G local fabric channels interconnect I/O modules (440G)
220Gbps440Gbps660Gbps880Gbps1100Gbps1320Gbps Local Fabric #1 (480G) Local Fabric #1 (960G) Local Fabric #1 (1.2T) Fabric #2 Fabric #2 Fabric #2 I/O Module Capacity – Nexus 7700 One fabric: • Any port can pass traffic to any other port in VDC Three fabrics: • 480G F2E/F3 10G module has maximum bandwidth Five fabrics: • 960G F3 40G module has maximum bandwidth Six fabrics: • 1.2T F3 100G module has maximum bandwidth per slot bandwidth Fabric 2 Modules 1 Fabric ASICs 2 Fabric ASICs 3 Fabric ASICs 4 Fabric ASICs 5 Fabric ASICs 6 Fabric ASICs
• Nexus 7702 has no fabric modules • Single I/O module – all traffic locally switched • Two fabric channels connect to supervisor engine F3 Module Supervisor What About Nexus 7702? 1* 55G fabric channel Fabric ASIC Fabric ASIC Fabric ASIC
• Introduction to Nexus 7000 / Nexus 7700 • Nexus 7000 / Nexus 7700 Architecture • Chassis, Supervisor Engines and NX-OS software, I/O modules (M2/F2E/F3) • I/O Module Interoperability • Fabric Architecture • Hardware Forwarding • Generic Designs with Nexus 7000 / Nexus 7700 • STP/VPC, L4-7 services integration, VDCs, VRF/MPLS VPNs, OTV Agenda
Hardware Forwarding Lookups • Layer 2 and Layer 3 packet flow virtually identical in hardware • Forwarding engine / decision engine pipeline provides consistent L2 and L3 lookup performance • Pipelined architecture also performs ingress and egress ACL, QOS, and Netflow lookups, affecting final forwarding result
M2 Forwarding Engine Hardware • Two hardware forwarding engines integrated on every M2 I/O module • Layer 2 switching (with hardware MAC learning) • Layer 3 IPv4/IPv6 unicast and multicast • MPLS/VPLS/EoMPLS • OTV / GRE • RACL/VACL/PACL • QOS remarking and policing policies • Ingress and egress Netflow (full and sampled) Hardware Table M-Series Modules without Scale License M-Series Modules with Scale License MAC Address Table 128K 128K FIB TCAM 128K IPv4 / 64K IPv6 900K IPv4 / 350K IPv6 Classification TCAM (ACL/QOS) 64K 128K Netflow Table 1M 1M
FE Daughter Card L2 Engine L3 Engine Classification (ACL/QOS) Netflow Layer 3 FIB FIB TCAM CL TCAM Policing Netflow Table From I/O Module Replication Engines To I/O Module Replication Engines Ingress Parser MAC Table L2 Lookup (pre-L3) L2 Lookup (post-L3) Final Results M-Series Forwarding Engine Architecture FE Daughter Card Ingress lookup pipeline Egress lookup pipeline • Ingress MAC table lookups • Port-channel hash result • Ingress ACL/QOS classification • Ingress Netflow collection • FIB TCAM and adjacency table lookups for Layer 3 forwarding • ECMP hashing • Ingress policing HDR • Egress ACL/QOS classification • Egress Netflow collection • Egress policing • Egress MAC lookups • Receive packet header for lookup from Replication Engine • Return final result (destination + priority) to Replication Engine
F2E Forwarding Engine Hardware • 4 x 10G SOC with decision engine • Layer 2 switching (with hardware MAC learning) • Layer 3 IPv4/ IPv6 unicast and multicast • FabricPath forwarding • RACL/VACL/PACL • QOS remarking and policing policies • Ingress sampled Netflow Hardware Table F2E Capacity MAC Address Table 16K FIB TCAM 32K IPv4/16K IPv6 Classification TCAM (ACL/QOS) 16K Per F2E Module
F3 Forwarding Engine Hardware • 8 x 10G, 2 x 40G, or 1 x 100G SOC with decision engine • Layer 2 switching (with hardware MAC learning) • Layer 3 IPv4/ IPv6 unicast and multicast • FabricPath forwarding • RACL/VACL/PACL • QOS remarking and policing policies • Ingress sampled Netflow • MPLS/VPLS/EoMPLS • OTV / GRE tunnels • LISP • VXLAN Hardware Table F3 Capacity MAC Address Table 64K FIB TCAM 64K IPv4/32K IPv6 Classification TCAM (ACL/QOS) 16K
MAC Table FIB TCAM CL TCAM Layer 3 FIB Classification (ACL/QOS/SNF) Policing From Ingress Port Logic To Ingress Buffer L2 Lookup (pre-L3) L2 Lookup (post-L3) Final Results Ingress Parser F2E/F3 Decision Engine Decision Engine F2E/F3 SOC HDR • Ingress MAC table lookups • Port-channel hash result • Ingress ACL/QOS/SNF classification • FIB TCAM and adjacency table lookups for Layer 3 forwarding • ECMP hashing • Ingress policing • Egress ACL/QOS classification • Egress policing • Egress MAC lookups • Receive packet from Port Logic block • Send payload to Ingress Buffer • Send header to Decision Engine • Return final result (destination + priority) to Ingress Buffer Ingress lookup pipeline Egress lookup pipeline PKT
• Introduction to Nexus 7000 / Nexus 7700 • Nexus 7000 / Nexus 7700 Architecture • Chassis, Supervisor Engines and NX-OS software, I/O modules (M2/F2E/F3) • I/O Module Interoperability • Fabric Architecture • Hardware Forwarding • Generic Designs with Nexus 7000 / Nexus 7700 • STP/VPC, L4-7 services integration, VDCs, VRF/MPLS VPNs, OTV Agenda
Nexus 7000 / Nexus 7700 Design Building Blocks Foundational: • Spanning Tree (RSTP+/MST) • Virtual Port Channel (VPC) • Virtual Routing and Forwarding (VRF) and MPLS VPNs Innovative: • Remote Integrated Service Engine (RISE) • Virtual Device Context (VDC) • Overlay Transport Virtualisation (OTV)
STP → Virtual Port Channel (VPC) • Eliminates STP blocked ports, leveraging all available uplink bandwidth and minimising reliance on STP • Provides active-active HSRP • Works seamlessly with current network designs/topologies • Works with any module type (M2/F2E/F3) • Most customers have taken this step No blocking ports STP blocks one uplink Topology without VPC L2 L3 Topology with VPC L2 L3
…L2 L3 L3 Collapsed Core/Aggregation • Nexus 7000 / Nexus 7700 as Data Centre collapsed core/aggregation • Consolidate multiple aggregation building blocks into single switch pair • Reduce number of managed devices • Simplify East-West communication path • M-series or F-series I/O modules, depending on: • Port density, feature-set, and scale requirements • Desired level of oversubscription L2 L3 VPC Domain L3
Traditional 3-Tier Hierarchical Design … core1 core2 agg1 agg2 aggX aggY • Extremely wide customer-deployment footprint • Nexus 7000 / Nexus 7700 in both Data Centre aggregation and core • Provides high-density, high-performance 10G / 40G / 100G • Same module-type considerations as collapsed core – density, features, scale • Scales well, but scoping of failure domains imposes some restrictions • VLAN extension / workload mobility options limited L2 L3 L3
L4-7 Services Integration – VPC Connected • VPC designs well-suited for L4-7 services integration – pair of aggregation devices makes service appliance connections simple • Multiple service types possible – transparent services, appliance as gateway, active-standby or active-active models • VPC-connected appliances preferred: • Ensures that all traffic – data plane, fault-tolerance, and management – sent direct via VPC port-channels • Minimises VPC peer link utilisation in steady state • Use orphan ports with “vpc orphan-port suspend” when services appliance does not support port- channels or Layer 3 peering to VPC peer required L2 L3 VPC Primary VPC Secondary L3 Active Service Standby Service
L4-7 Services Integration – RISE • Logical integration of external services appliance with Nexus 7000 / Nexus 7700 Citrix NetScaler and Cisco Prime NAM appliance supported today • Enables tight services integration between services appliance and Nexus 7000 / Nexus 7700 switches, including: Discovery and bootstrap Automated Policy Based Routing (APBR) Route Health Injection (RHI) (future) Remote Integrated Service Engine (RISE) Physical Topology Logical Topology with RISE
RISE Auto-PBR • User configures new service in NetScaler • NetScaler sends server list and next-hop interface to Nexus 7000/7700 switch over RISE control channel • Switch automatically generates PBR route-maps and applies PBR rules in data- plane hardware to redirect target traffic – no manual configuration on switch • Client traffic destined to VIP redirected to NetScaler for processing, destination rewritten to Real server IP NetScaler MPX APBR APBR APBR rules Configure new service Client → VIP Client → Real
RISE Auto-PBR • User configures new service in NetScaler • NetScaler sends server list and next-hop interface to Nexus 7000/7700 switch over RISE control channel • Switch automatically generates PBR route-maps and applies PBR rules in data- plane hardware to redirect target traffic – no manual configuration on switch • Client traffic destined to VIP redirected to NetScaler for processing, destination rewritten to Real server IP • Return traffic redirected to rewrite Real IP to VIP NetScaler MPX APBR APBR VIP → Client Real → Client APBR rules
VDC Details Virtual Device Contexts • Create multiple logical devices out of one physical device • Provide data-plane, control-plane, and management-plane separation • Fault isolation and reduced fate sharing Infrastructure Kernel VDC 1 VDC 1 Layer 2 Protocols VLAN STP VPC CDP LACP CTS Network Stack (L2 / IPv4 / IPv6) Layer 3 Protocols OSPF BGP VRRP SNMP PIM RIB VDC 2 VDC n VDC n VDC 2 Layer 2 Protocols VLAN STP VPC CDP LACP CTS Network Stack (L2 / IPv4 / IPv6) Layer 3 Protocols OSPF BGP VRRP SNMP PIM RIB Note: VDCs do not provide a hypervisor capability, or ability to run different OS versions in each VDC
VDC Interface Allocation • Physical interfaces assigned on per VDC basis, from default/admin VDC • All subsequent interface configuration performed within the assigned VDC • A single interface cannot be shared across multiple VDCs • VDC type (“limit-resource module-type”) determines types of interfaces allowed in VDC • VDC type driven by operational goals and/or hardware restrictions, e.g.: • Mix M2 and F2E in same VDC to increase MAC scale in FabricPath • Restrict VDC to F3 only to avoid lowest common denominator • Cannot mix M1 and F3 in same VDC
VDC Interface Allocation – M2 • Allocate any interface to any VDC • But, be aware of shared hardware resources – backend ASICs may be shared by several VDCs • Best practice: allocate entire module to one VDC to minimise shared hardware resources VDC 1 VDC 2 VDC 3 VDC 4 M2-10G M2-40G
VDC Interface Allocation – F2E / F3 Modules • Allocation on port-group boundaries – aligns ASIC resources to VDCs • Port-group size varies depending on module type F3-10G 8-port port-group VDC 1 VDC 2 VDC 3 VDC 4 F2E 4-port port-group F3-40G 2-port port-group F3-100G 1-port port-group
Communicating Between VDCs • Must use front-panel ports to communicate between VDCs • No backplane inter-VDC communication • No restrictions on L2/L3 configuration, module types, or physical media type – just like interconnecting two physical switches • Copper Twinax cables (CX-1) or 40G bidi optics provide low-cost interconnect options VDC 1 VDC 2
… core1 core2 agg1 agg2 aggX aggY L2 L3 Admin Zone 1 Admin Zone 2 Admin Zone 3 L2 L3 core1 core2 Admin Zone 1 (VDC 1) Admin Zone 3 (VDC 3) Admin Zone 2 (VDC 2) Collapsed Core Design with VDCs • Maintain administrative segmentation while consolidating network infrastructure • Maintain fault isolation between zones (independent L2, routing processes per zone) • Firewalling between zones facilitated by VDC port membership model
VRF / MPLS VPNs • Provides network virtualisation – One physical network supporting multiple virtual networks • While maintaining security/segmentation and access to shared services • VRF-lite segmentation for simple/limited virtualisation environments • MPLS L3VPN for larger-scale, more flexible deployments
Agg CampusInternet MPLS Layer 3 VPN – Secure Multi-Tenant Data Centre Requirement: • Secure segmentation for hosted / enterprise data centre Solution: • MPLS Layer 3 VPNs for segmentation • MPLS PE boundary in Pod aggregation layer with VRF membership on SVIs • Direct PE-PE or PE-P-PE interconnections in core • Layer 2 with VLANs below MPLS boundary MPLS Access Pod 1 Pod 2 PE PP PE Core Agg L2 L2 L3
Any Transport! – L2, L3, MPLS OTV for Multi-Site VLAN Extension • Overlay Transport Virtualisation (OTV) provides multi-site Layer 2 Data Centre Interconnect (DCI) • Dynamic “MAC in IP” encapsulation with forwarding based on MAC “routing” table • No pseudo-wire or tunnel state maintained Site 2 Site 3 OTV – Virtual Layer 2 Interconnect Site 1 VLAN x VLAN x VLAN x Layer 2 adjacent
MAC2 MAC3MAC1 OTV at a Glance • MAC addresses advertised in routing protocol (control plane learning) between Data Centre sites • Ethernet traffic between sites encapsulated in IP: “MAC in IP” IP A IP B Ethernet Frame MAC1→MAC2 IP packet IP A→IP B Ethernet Frame MAC1→MAC2 Encap Decap OTV edge OTV edge OTV Ethernet Frame MAC1→MAC2 MAC IF MAC1 IP A MAC2 po1 MAC3 po1 MAC IF MAC1 po1 MAC2 IP B MAC3 IP B Site 2Site 1
• Current limitation – SVI (for VLAN termination at L3) and OTV overlay interface (for VLAN extension over OTV) cannot exist in same VDC • Typical designs move OTV to separate VDC, or separate switch (e.g. Nexus 7702) OTV VDC Requirement Nexus 7000/7700 SVI L2 VLAN Nexus 7000/7700 OTV L2 VLAN SVI L2 VLAN OTV L2 VLAN Nexus 7000/7700 VDC w/SVI VDC w/OTV OTVSVI L2 VLAN Nexus 7000/7700  
Key Takeaways • Nexus 7000 / Nexus 7700 switching architecture provides foundation for flexible and scalable Enterprise network designs • Nexus 7000 / Nexus 7700 design building blocks interwork and complement each other to solve customer challenges • Nexus 7000 / Nexus 7700 platform continues to evolve to support next-generation/emerging technologies and architectures
Q & A
Complete Your Online Session Evaluation Learn online with Cisco Live! Visit us online after the conference for full access to session videos and presentations. www.CiscoLiveAPAC.com Give us your feedback and receive a Cisco 2016 T-Shirt by completing the Overall Event Survey and 5 Session Evaluations. – Directly from your mobile device on the Cisco Live Mobile App – By visiting the Cisco Live Mobile Site http://showcase.genie-connect.com/ciscolivemelbourne2016/ – Visit any Cisco Live Internet Station located throughout the venue T-Shirts can be collected Friday 11 March at Registration
Thank you
Brkarc 3601

More Related Content

PDF
BRKARC-3470 - Nexus 7000 and 7700 Architecture.pdf
byssuser1d1e521
 
PDF
Brkarc 3470 - cisco nexus 7000-7700 switch architecture (2016 las vegas) - 2 ...
bykds850
 
PDF
N7K Hardware Architecture with proper explanation
byKalkiNarayana
 
PDF
N7K Hardware Architecture and it's components
byKalkiNarayana
 
PPTX
Training Nexus 7 data & operation 2024.pptx
byrudyhermawan37
 
PPT
Cisco nexus 7009 overview
byHamza Al-Qudah
 
PDF
Ebadullah- CV
bydaywalker064
 
PPT
Systemic differences between biological and technical systems
byEgils Stalidzans
 
BRKARC-3470 - Nexus 7000 and 7700 Architecture.pdf
byssuser1d1e521
 
Brkarc 3470 - cisco nexus 7000-7700 switch architecture (2016 las vegas) - 2 ...
bykds850
 
N7K Hardware Architecture with proper explanation
byKalkiNarayana
 
N7K Hardware Architecture and it's components
byKalkiNarayana
 
Training Nexus 7 data & operation 2024.pptx
byrudyhermawan37
 
Cisco nexus 7009 overview
byHamza Al-Qudah
 
Ebadullah- CV
bydaywalker064
 
Systemic differences between biological and technical systems
byEgils Stalidzans
 

Similar to Brkarc 3601

PDF
Brkarc 3454 - in-depth and personal with the cisco nexus 2000 fabric extender...
bykds850
 
PDF
Next Generation Nexus 9000 Architecture
byCisco Canada
 
PDF
Brkdct 3101
byNguyen Van Linh
 
PDF
Brkarc 3454
byNguyen Van Linh
 
PDF
Cisco Connect Vancouver 2017 - Gain insight and programmability with Cisco DC...
byCisco Canada
 
PPTX
Cisco nx os
byUtpal Sinha
 
PDF
Cisco CCNA Data Center Networking Fundamentals
byE.S.G. JR. Consulting, Inc.
 
PDF
Cisco DC Networking: Gain Insight and Programmability with
byCisco Canada
 
PPTX
Nexus 7000 Series Innovations: M3 Module, DCI, Scale
byTony Antony
 
PPTX
Introduction to nexux from zero to Hero
byDhruv Sharma
 
PPTX
01-05-N9K Hardware.pptx
byFeizhang41
 
PPTX
Cisco nexus series
byAnwesh Dixit
 
PDF
Data Centre Portfolio Update
byCisco Canada
 
PDF
DellEMC Networking Product Portfolio Guide
byLac Viet Computing Corporation
 
PDF
cisco-n2k-c2348tq-e-datasheet.pdf
byHi-Network.com
 
PPTX
Higher Speed, Higher Density, More Flexible SAN Switching
byTony Antony
 
DOCX
Cisco nexus 7000 and nexus 7700
byIT Tech
 
DOCX
Cisco nexus 7000, nexus 5000 and 2000 fa qs
byIT Tech
 
PDF
Gain Insight and Programmability with Cisco DC Networking
byCisco Canada
 
PDF
Gain Insight and Programmability with Cisco DC Networking
byCisco Canada
 
Brkarc 3454 - in-depth and personal with the cisco nexus 2000 fabric extender...
bykds850
 
Next Generation Nexus 9000 Architecture
byCisco Canada
 
Brkdct 3101
byNguyen Van Linh
 
Brkarc 3454
byNguyen Van Linh
 
Cisco Connect Vancouver 2017 - Gain insight and programmability with Cisco DC...
byCisco Canada
 
Cisco nx os
byUtpal Sinha
 
Cisco CCNA Data Center Networking Fundamentals
byE.S.G. JR. Consulting, Inc.
 
Cisco DC Networking: Gain Insight and Programmability with
byCisco Canada
 
Nexus 7000 Series Innovations: M3 Module, DCI, Scale
byTony Antony
 
Introduction to nexux from zero to Hero
byDhruv Sharma
 
01-05-N9K Hardware.pptx
byFeizhang41
 
Cisco nexus series
byAnwesh Dixit
 
Data Centre Portfolio Update
byCisco Canada
 
DellEMC Networking Product Portfolio Guide
byLac Viet Computing Corporation
 
cisco-n2k-c2348tq-e-datasheet.pdf
byHi-Network.com
 
Higher Speed, Higher Density, More Flexible SAN Switching
byTony Antony
 
Cisco nexus 7000 and nexus 7700
byIT Tech
 
Cisco nexus 7000, nexus 5000 and 2000 fa qs
byIT Tech
 
Gain Insight and Programmability with Cisco DC Networking
byCisco Canada
 
Gain Insight and Programmability with Cisco DC Networking
byCisco Canada
 

Recently uploaded

PDF
Small Business Automation: A Comprehensive Cost and ROI Guide
byAmelia Swank
 
PDF
Artificial Intelligence(AI) full Book.pdf
bydeyanimesh299
 
PDF
What is Voice User Interface (VUI) Definition & Examples.pdf
byDesign Studio UI UX
 
PPTX
Pizza Chain Market Data Scraping for Better Insights Report.pptx
byWeb Data Crawler
 
PDF
How Application Performance Monitoring Tools Are Used in Performance Testing
byhenrycavill30521
 
PDF
Top Benefits of Using KVM VPS Hosting for Growing Businesses
byEthernet Servers
 
PDF
Industrial RFID Landscape for 2025 - Readers, Tags, Antennas, Printers, etc
byRich Rogers
 
PPTX
Search Engine Responses to Conspiratorial Search Practices AANZCA 2025 Presen...
bykkasianenko
 
PPT
HDTV and DTV Standards: The United States Opts for a Digital HDTV Standard
byJeffrey Hart
 
PPTX
Cyber Security Overview-breif note .pptx
byxbitindiacom
 
PPTX
UNIT III TYPOLOGY OF CRIME AND CRIMINAL BEHAVIOUR (1).pptx
bybloodyhumanoid
 
PPTX
Cesium formate brine - A brief history - prepared by John Downs in May 2011
byJohn Downs
 
PDF
AI Driven & AI Native Development AI native development
byZainKarim7
 
PPTX
Spacecraft Guidance Quick Research Guide by Arthur Morgan
byArthur Morgan
 
PPTX
Large Language Model. LLM Audit Artificial Intelligence
byMANASCHOUDHARY22
 
PDF
Regenerative Agriculture Finance : Environmental impact
byrose mason
 
PPTX
Digital transformation success powered by EPM and NexInfo.pptx
byjayasuryanexinfo
 
PDF
A Brief Introduction About Christopher Elwell Woburn
byChristopher Elwell Woburn
 
PDF
Digital Marketing Trends in 2026: AI, Data, Content & Future Growth
byConacent Consulting
 
PPTX
AI Bot Traffic Surge: Retail Fraud Threat for Age-Restricted Websites
byFTx Identity
 
Small Business Automation: A Comprehensive Cost and ROI Guide
byAmelia Swank
 
Artificial Intelligence(AI) full Book.pdf
bydeyanimesh299
 
What is Voice User Interface (VUI) Definition & Examples.pdf
byDesign Studio UI UX
 
Pizza Chain Market Data Scraping for Better Insights Report.pptx
byWeb Data Crawler
 
How Application Performance Monitoring Tools Are Used in Performance Testing
byhenrycavill30521
 
Top Benefits of Using KVM VPS Hosting for Growing Businesses
byEthernet Servers
 
Industrial RFID Landscape for 2025 - Readers, Tags, Antennas, Printers, etc
byRich Rogers
 
Search Engine Responses to Conspiratorial Search Practices AANZCA 2025 Presen...
bykkasianenko
 
HDTV and DTV Standards: The United States Opts for a Digital HDTV Standard
byJeffrey Hart
 
Cyber Security Overview-breif note .pptx
byxbitindiacom
 
UNIT III TYPOLOGY OF CRIME AND CRIMINAL BEHAVIOUR (1).pptx
bybloodyhumanoid
 
Cesium formate brine - A brief history - prepared by John Downs in May 2011
byJohn Downs
 
AI Driven & AI Native Development AI native development
byZainKarim7
 
Spacecraft Guidance Quick Research Guide by Arthur Morgan
byArthur Morgan
 
Large Language Model. LLM Audit Artificial Intelligence
byMANASCHOUDHARY22
 
Regenerative Agriculture Finance : Environmental impact
byrose mason
 
Digital transformation success powered by EPM and NexInfo.pptx
byjayasuryanexinfo
 
A Brief Introduction About Christopher Elwell Woburn
byChristopher Elwell Woburn
 
Digital Marketing Trends in 2026: AI, Data, Content & Future Growth
byConacent Consulting
 
AI Bot Traffic Surge: Retail Fraud Threat for Age-Restricted Websites
byFTx Identity
 

Brkarc 3601

  • 2.
    Nexus 7000/7700 Architecture and DeploymentModels Satish Kondalam Technical Marketing Engineer
  • 3.
    Session Abstract This sessionwill discuss the foundations of the Nexus 7000 and 7700 series switches, including chassis, I/O modules, and NX-OS software. Examples will show common use-cases for different module types and considerations for module interoperability. The focus will then shift to key platform capabilities and features – including VPC,OTV, VDCs, and others – along with real-world designs and deployment models.
  • 4.
    Session Goals • Toprovide an understanding of the Nexus 7000 / Nexus 7700 switching architecture, which provides the foundation for flexible, scalable Data Centre designs • To examine key Nexus 7000 / Nexus 7700 design building blocks and illustrate common design alternatives leveraging those features and functionalities • To see how the Nexus 7000 / Nexus 7700 platform plays in emerging technologies and architectures
  • 5.
    • Introduction toNexus 7000 / Nexus 7700 • Nexus 7000 / Nexus 7700 Architecture • Chassis, Supervisor Engines and NX-OS software, I/O modules (M2/F2E/F3) • I/O Module Interoperability • Fabric Architecture • Hardware Forwarding • Generic Designs with Nexus 7000 / Nexus 7700 • STP/VPC, L4-7 services integration, VDCs, VRF/MPLS VPNs, OTV Agenda
  • 6.
    Introduction to Nexus7000 / Nexus 7700 Platform Data-centre class Ethernet switches designed to deliver high performance, high availability, system scale, and investment protection Designed for wide range of Data Centre deployments, focused on feature-rich 10G/40G/100G density and performance I/O Modules Supervisor Engines Fabrics Chassis
  • 7.
    Nexus 7000 General purposeDC switching w/10/40/100G Nexus 7700 Targeted at Dense 40G/100G deployments CommonFoundation • Same release vehicles, versioning, feature-sets • Common configuration model • Common operational model • Common fabric ASICs (Fab2) and architecture • Same central arbitration model • Same VOQ/QOS model • Identical forwarding ASICs (F2E, F3) • Consistent hardware feature sets • Consistent hardware scale Nexus 7000 / Nexus 7700 – Common Foundation
  • 8.
    • Introduction toNexus 7000 / Nexus 7700 • Nexus 7000 / Nexus 7700 Architecture • Chassis, Supervisor Engines and NX-OS software, I/O modules (M2/F2E/F3) • I/O Module Interoperability • Fabric Architecture • Hardware Forwarding • Generic Designs with Nexus 7000 / Nexus 7700 • STP/VPC, L4-7 services integration, VDCs, VRF/MPLS VPNs, OTV Agenda
  • 9.
    Nexus 7000 ChassisFamily Front Rear 21RU N7K-C7010 25RU Front RearN7K-C7018 Front RearN7K-C7009 14RU Nexus 7010 Nexus 7018 Nexus 7009 Front N7K-C7004 7RU Rear Nexus 7004 Front Back Side Side Side Side Side Back
  • 10.
    Nexus 7700 ChassisFamily Front Rear 26RU N77-C7718 Nexus 7718 Front Rear 14RU N77-C7710 Nexus 7710 Front Rear 9RU N77-C7706 Nexus 7706 Front Back Front Back Front Back
  • 11.
    Nexus 7702 Chassis NX-OS7.2 and later Front RearN77-C7702 Front Back 3RU One Supervisor Engine Two Power Supplies One F3 Series I/O Module One Fan Tray (3 Fans) No Fabric Modules!
  • 12.
    Supervisor Engine 2/ 2E • Provides all control plane and management functions • Connects to fabric via 1G inband interface • Interfaces with I/O modules via 1G switched EOBC • Onboard central arbiter ASIC Controls access to fabric bandwidth via dedicated arbitration path to I/O modules Console Port Management Ethernet N7K-SUP2/N7K-SUP2E USB Host Ports ID and Status LEDs Supervisor Engine 2 (Nexus 7000) Supervisor Engine 2E (Nexus 7000 / Nexus 7700) Base performance High performance One quad-core 2.1GHz CPU with 12GB DRAM Two quad-core 2.1GHz CPU with 32GB DRAM USB Log Flash USB Expansion Flash N77-SUP2E ID and Status LEDs Console Port Management Ethernet USB Expansion Flash
  • 13.
    Supervisor Engine 2/ 2E Architecture Fabric ASIC VOQs I/O Controller Main CPU Main CPU NVRAM DRAM Bootflash (eUSB) USB expansion USB logflash USB device port Console Mgmt0 Central ArbiterSwitched EOBC 12GB/32GB2.1GHz Quad-Core 32MB 2.1GHz Quad-Core Sup2E Only To Fabric Modules To Module VOQs Dedicated Arbitration Path Dedicated Arbitration Path To Module CPUs Switched 1GE EOBC 1GE Inband 2GB
  • 14.
    Nexus 7000 /7700 I/O Module Families M1 1G and 10G M2 10G / 40G / 100G F1 10G F2 10G F2E 10G F3 10G / 40G / 100G F2E 10G F3 10G / 40G / 100G F3 closes the F/M feature gap!
  • 15.
    Nexus 7000 M2I/O Modules • 10G / 40G / 100G M2 I/O modules • Share common hardware architecture – multi-chipset • Two integrated forwarding engines (120Mpps) • Layer 2/Layer 3 forwarding with L3/L4 services (ACL/QOS) and advanced features (MPLS/OTV/GRE etc.) • Large forwarding tables (900K FIB/ 128K ACL) N7K-M224XP-23L / N7K-M206FQ-23L / N7K-M202CF-22L N7K-M224XP-23L N7K-M206FQ-23L N7K-M202CF-22L Module Port Density Optics Bandwidth M2 10G 24 x 10G (plus Nexus 2000 FEX support) SFP+ 240G M2 40G 6 x 40G (or up to 24 x 10G via breakout) QSFP+ 240G M2 100G 2 x 100G CFP 200G
  • 16.
    Fabric ASIC Nexus 7000M2 I/O Module Architecture N7K-M224XP-23L / N7K-M206FQ-23L / N7K-M202CF-22L To Fabric Modules Front Panel Ports VOQsVOQsForwarding Engine Replication Engine Replication Engine LinkSec + 12 X 10G MAC -or- 3 X 40G MAC -or- 1 X 100G MAC VOQs VOQs Forwarding Engine Replication Engine Replication Engine LinkSec + 12 X 10G MAC -or- 3 X 40G MAC -or- 1 X 100G MAC LC CPU EOBC Arbitration Aggregator To Central Arbiters
  • 17.
    Nexus 7000 /Nexus 7700 F2E I/O Modules • 48-port 1G/10G with SFP/SFP+ transceivers • 480G full-duplex fabric connectivity • System-on-chip (SOC) forwarding engine design 12 independent SOC ASICs • Layer 2/Layer 3 forwarding with L3/L4 services (ACL/QOS) • Interoperability with M1/M2, in Layer 2 mode on Nexus 7000 Proxy routing for inter-VLAN/L3 traffic N7K-F248XP-25E / N7K-F248XT-25E / N77-F248XP-23E 7000: Supported in NX-OS release 6.1(2) and later 7700: Supported in NX-OS release 6.2(2) and later Nexus 7000 N7K-F248XP-25E Nexus 7000 N7K-F248XT-25E Nexus 7700 N77-F248XP-23E
  • 18.
    4 X 10G SOC1 Nexus 7000 F2E Module Architecture N7K-F248XP-25E / N7K-F248XT-25E To Fabric Modules 1 Front Panel Ports (SFP/SFP+) 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 4 X 10G SOC 2 4 X 10G SOC 3 4 X 10G SOC 4 4 X 10G SOC 5 4 X 10G SOC 6 4 X 10G SOC 7 4 X 10G SOC 8 4 X 10G SOC 9 4 X 10G SOC 10 4 X 10G SOC 11 4 X 10G SOC 12 Fabric ASIC LC CPU EOBC Arbitration Aggregator To Central Arbiters to ARB to LC CPU LC Inband
  • 19.
    4 X 10G SOC1 Nexus 7700 F2E Module Architecture N77-F248XP-23E 1 Front Panel Ports (SFP/SFP+) 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 4 X 10G SOC 2 4 X 10G SOC 3 4 X 10G SOC 4 4 X 10G SOC 5 4 X 10G SOC 6 4 X 10G SOC 7 4 X 10G SOC 8 4 X 10G SOC 9 4 X 10G SOC 10 4 X 10G SOC 11 4 X 10G SOC 12 To Fabric Modules Fabric ASIC Fabric ASIC LC CPU EOBC Arbitration Aggregator To Central Arbiters to ARB to LC CPU LC Inband
  • 20.
    Nexus 7000 F3I/O Modules • 10G / 40G / 100G F3 I/O modules • Share common hardware architecture • SOC-based forwarding engine design 6 independent SOC ASICs per module • Layer 2/Layer 3 forwarding with L3/L4 services (ACL/QOS) and advanced features (MPLS/OTV/GRE/VXLAN etc.) • Require Supervisor Engine 2 / 2E N7K-F348XP-25 / N7K-F312FQ-25 / N7K-F306CK-25 Module Port Density Optics Bandwidth F3 10G 48 x 1/10G (plus Nexus 2000 FEX support) SFP+ 480G F3 40G 12 x 40G (or up to 48 x 10G via breakout) QSFP+ 480G F3 100G 6 x 100G CPAK 550G N7K-F348XP-25 N7K-F312FQ-25 N7K-F306CK-25
  • 21.
    Nexus 7700 F3I/O Modules • 10G / 40G / 100G F3 I/O modules • Share common hardware architecture • SOC-based forwarding engine design 6 independent SOC ASICs per 10G module 12 independent SOC ASICs per 40G/100G module • Layer 2/Layer 3 forwarding with L3/L4 services (ACL/QOS) and advanced features (MPLS/OTV/GRE/VXLAN etc.) N7K-F348XP-25 / N7K-F312FQ-25 / N7K-F306CK-25 Module Port Density Optics Bandwidth F3 10G 48 x 1/10G (plus Nexus 2000 FEX support) SFP+ 480G F3 40G 24 x 40G (or up to 76 x 10G + 5 x 40G via breakout) QSFP+ 960G F3 100G 12 x 100G CPAK 1.2T N77-F348XP-23 N77-F324FQ-25 N77-F312CK-26
  • 22.
    8 X 10G SOC1 Nexus 7000 F3 48-Port 1G/10G Module Architecture N7K-F348XP-25 8 X 10G SOC 6 8 X 10G SOC 5 8 X 10G SOC 4 8 X 10G SOC 3 8 X 10G SOC 2 1 Front Panel Ports (SFP/SFP+) 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 To Fabric Modules Fabric ASIC EOBC LC Inband to ARB x 6… FSA CPU 1G switch To Central Arbiters …x 6 to FSA CPU Arbitration Aggregator
  • 23.
    FSA CPU Fabric ServicesAccelerator (FSA) for F3 • High-performance module CPU with on-board acceleration engines • 6Gbps inband connectivity from SOCs to FSA • Multi-Mpps packet processing • 2 X 2GB dedicated DRAM • Performance/scale boost for distributed fabric services, including sampled Netflow and BFD (roadmap) • Other potential applications include distributed ARP/ping processing, data plane packet analysis (wireshark), network probing, etc. 6 x 1Gbps Module Inband I/O 2GB DRAM Dual-Core LC CPU Acceleration Engines 2GB DRAM EOBC
  • 24.
    Nexus 7000 F312-Port 40G Module Architecture N7K-F312FQ-25 To Fabric Modules 2 X 40G SOC 6 2 X 40G SOC 5 2 X 40G SOC 4 2 X 40G SOC 3 2 X 40G SOC 2 2 X 40G SOC 1 Fabric ASIC Front Panel Ports (QSFP+) EOBC LC Inband to ARB x 6… FSA CPU 1G switch To Central Arbiters …x 6 to FSA CPU Arbitration Aggregator 1 2 3 4 5 6 7 8 9 10 11 12
  • 25.
    Nexus 7000 F36-Port 100G Module Architecture N7K-F306CK-25 To Fabric Modules 1 X 100G SOC 6 1 X 100G SOC 5 1 X 100G SOC 4 1 X 100G SOC 3 1 X 100G SOC 2 1 X 100G SOC 1 Fabric ASIC 2 3 4 5 61 Front Panel Ports (CPAK) EOBC LC Inband to ARB x 6… FSA CPU 1G switch To Central Arbiters …x 6 to FSA CPU Arbitration Aggregator
  • 26.
    8 X 10G SOC1 Nexus 7700 F3 48-Port 1G/10G Module Architecture N77-F348XP-23 To Fabric Modules 8 X 10G SOC 6 8 X 10G SOC 5 8 X 10G SOC 4 8 X 10G SOC 3 8 X 10G SOC 2 1 Front Panel Ports (SFP/SFP+) 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 Fabric ASIC Fabric ASIC EOBC LC Inband to ARB x 6… FSA CPU 1G switch To Central Arbiters …x 6 to FSA CPU Arbitration Aggregator
  • 27.
    2 X 40G SOC1 Nexus 7700 F3 24-Port 40G Module Architecture N77-F324FQ-25 Front Panel Ports (QSFP+) To Fabric Modules 2 X 40G SOC 12 2 X 40G SOC 11 2 X 40G SOC 10 2 X 40G SOC 9 2 X 40G SOC 8 2 X 40G SOC 7 2 X 40G SOC 6 2 X 40G SOC 5 2 X 40G SOC 4 2 X 40G SOC 3 2 X 40G SOC 2 Fabric ASIC Fabric ASIC EOBC LC Inband to ARB x 12… To Central Arbiters …x 12 to FSA CPU FSA CPU 1G switch Arbitration Aggregator 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
  • 28.
    1 X 100G SOC1 Nexus 7700 F3 12-Port 100G Module Architecture N77-F312CK-26 Front Panel Ports (CPAK) 2 3 4 5 6 7 8 9 10 11 121 To Fabric Modules 1 X 100G SOC 12 1 X 100G SOC 11 1 X 100G SOC 10 1 X 100G SOC 9 1 X 100G SOC 8 1 X 100G SOC 7 1 X 100G SOC 6 1 X 100G SOC 5 1 X 100G SOC 4 1 X 100G SOC 3 1 X 100G SOC 2 Fabric ASIC Fabric ASIC EOBC LC Inband to ARB x 12… To Central Arbiters …x 12 to FSA CPU FSA CPU 1G switch Arbitration Aggregator
  • 29.
    F3 Module 40Gand 100G Flows • Virtual Queuing Index (VQI) sustains 10G, 40G, or 100G traffic flow based on destination interface type • No single-flow limit – full 40G/100G flow support Ingress Modules 10G 10G 40G 40G 100G Spines Spines Spines Spines Fabrics Egress Interfaces Destination VQIs 1 VQI 1 VQI 1 VQI 1 VQI 1 VQI
  • 30.
    • Introduction toNexus 7000 / Nexus 7700 • Nexus 7000 / Nexus 7700 Architecture • Chassis, Supervisor Engines and NX-OS software, I/O modules (M2/F2E/F3) • I/O Module Interoperability • Fabric Architecture • Hardware Forwarding • Generic Designs with Nexus 7000 / Nexus 7700 • STP/VPC, L4-7 services integration, VDCs, VRF/MPLS VPNs, OTV Agenda
  • 31.
    I/O Module Interoperability •General module interoperability rule is: “+/-1 generation” in same Virtual Device Context (VDC) • Layer 3 forwarding behaviour is key difference between interop models: • “Proxy Forwarding” • “Ingress Forwarding” with Lowest Common Denominator
  • 32.
    L2 Host A 10.1.10.100 vlan 10 HostB 10.1.20.100 vlan 20 L2 Proxy Forwarding Model • F2E modules run in pure Layer 2 mode – all L3 functions disabled • M2 modules host SVIs and other L3 functions • From F2E perspective, Router MAC reachable via M2 modules • All packets destined to Router MAC forwarded through fabric toward one M2 module, selected via port-channel hash • M2 modules(s) perform all L3 forwarding and policy, pass packets back over fabric to output port • Key consideration: M-series L3 routing capacity versus F-series front-panel port count – How much Layer 3 routing is required? M2 + F2E VDC F2Emodules M2modules interface vlan 10 ip address 10.1.10.1/24 ! interface vlan 20 ip address 10.1.20.1/24 Router MAC reached via M2 modules M2 modules host SVIs MAC Table rtr-mac → M2 modules Port-channel hash selects M2 module
  • 33.
    Ingress Forwarding withLowest Common Denominator Model • F3 module interoperability always “Ingress Forwarding” – NO proxy forwarding Ingress module receiving packet makes all forwarding decisions for that packet • Supported feature set and scale based on Lowest Common Denominator Feature available if all modules support the feature Table sizes based on lowest capacity F3 + M2 VDC -or- F3 + F2E VDC Module Types in VDC Layer 2 Layer 3 VPC MPLS OTV Fabric Path VXLAN Table Sizes F3 ✓ ✓ ✓ ✓ ✓ ✓ ✓ F3 size F3 + M2 ✓ ✓ ✓ ✓ ✓ ✗ ✗ F3 size F3 + F2E ✓ ✓ ✓ ✗ ✗ ✓ ✗ F2E size M2 + F2E + F3 Not supported Not all features supported by software today…
  • 34.
    Module Interoperability UseCases • M2 + F2E VDC • Provide higher-density 1G/10G while supporting M2 features and L3 functions • Full internet routes, MPLS VPNs • FabricPath with increased MAC address scale (proxy L2 learning) • F2E + F3 VDC • Introduction of 40G/100G into existing 10G environments • Migration to larger table sizes • Transition to additional features/functionality (OTV, MPLS, VXLAN, etc.) • M2 + F3 VDC • Introduce higher 1G/10G/40G/100G port-density while maintaining feature-set • Avoid proxy-forwarding model for module interoperability • Migrate to 40G/100G interfaces with full-rate flow capability M2 F2E F2E F3 M2 F3
  • 35.
    • Introduction toNexus 7000 / Nexus 7700 • Nexus 7000 / Nexus 7700 Architecture • Chassis, Supervisor Engines and NX-OS software, I/O modules (M2/F2E/F3) • I/O Module Interoperability • Fabric Architecture • Hardware Forwarding • Generic Designs with Nexus 7000 / Nexus 7700 • STP/VPC, L4-7 services integration, VDCs, VRF/MPLS VPNs, OTV Agenda
  • 36.
    Crossbar Switch FabricModules • Provide interconnection of I/O modules • Nexus 7000 and Nexus 7700 fabrics based on Fabric 2 ASIC • Each installed fabric increases available per-payload slot bandwidth • Different I/O modules leverage different amount of available fabric bandwidth • Access to fabric bandwidth controlled using QOS-aware central arbitration with VOQ N7K-C7018-FAB-2 N7K-C7010-FAB-2 N7K-C7009-FAB-2 Fabric Module Supported Chassis Per-fabric module bandwidth Max fabric modules Total bandwidth per slot Nexus 7000 Fabric 2 7009 / 7010 / 7018 110Gbps per slot 5 550Gbps per slot Nexus 7700 Fabric 2 7706 / 7710 / 7718 220Gbps per slot 6 1.32Tbps per slot N77-C7718-FAB-2 N77-C7710-FAB-2 N77-C7706-FAB-2
  • 37.
    Nexus 7000 /Nexus 7700 implement 3-stage crossbar switch fabric • Stages 1 and 3 on I/O modules • Stage 2 on fabric modules Ingress ModuleEgress Module Ingress Module 220G (4 x 55G) Egress Module Multistage Crossbar 1st stage 2nd stage 3rd stage Fabric ASIC Fabric ASIC Fabric ASICFabric ASIC 1.32T 1st stage 3rd stage 550G 110G (2 x 55G) 1 Fabric ASIC 2 3 4 5Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC Fabric Modules Nexus 7000 Nexus 7700 Fabric ASIC Fabric ASIC Fabric Modules Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC 1 Fabric ASIC 2 3 4 5 Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC 6 Fabric ASIC
  • 38.
    110Gbps220Gbps330Gbps440Gbps550Gbps Local Fabric (480G) Local Fabric (240G) I/OModule Capacity – Nexus 7000 One fabric: • Any port can pass traffic to any other port in VDC Three fabrics: • 240G M2 module has maximum bandwidth Five fabrics: • 480G F2E/F3 module has maximum bandwidth Fabric 2 Modules 1Fabric ASIC 2Fabric ASIC 3Fabric ASIC 4Fabric ASIC 5Fabric ASIC per slot bandwidth
  • 39.
    What About Nexus7004? • Nexus 7004 has no fabric modules • Each I/O module has local fabric with 10 available fabric channels • I/O modules connect “back-to-back” via 8 fabric channels • Two fabric channels “borrowed” to connect supervisor engines Sup Slot 2Sup Slot 1 M2/F2E/F3 Module 4 M2/F2E/F3 Module 3 Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC 2 * 55G fabric channels 8 * 55G local fabric channels interconnect I/O modules (440G)
  • 40.
    220Gbps440Gbps660Gbps880Gbps1100Gbps1320Gbps Local Fabric #1(480G) Local Fabric #1 (960G) Local Fabric #1 (1.2T) Fabric #2 Fabric #2 Fabric #2 I/O Module Capacity – Nexus 7700 One fabric: • Any port can pass traffic to any other port in VDC Three fabrics: • 480G F2E/F3 10G module has maximum bandwidth Five fabrics: • 960G F3 40G module has maximum bandwidth Six fabrics: • 1.2T F3 100G module has maximum bandwidth per slot bandwidth Fabric 2 Modules 1 Fabric ASICs 2 Fabric ASICs 3 Fabric ASICs 4 Fabric ASICs 5 Fabric ASICs 6 Fabric ASICs
  • 41.
    • Nexus 7702has no fabric modules • Single I/O module – all traffic locally switched • Two fabric channels connect to supervisor engine F3 Module Supervisor What About Nexus 7702? 1* 55G fabric channel Fabric ASIC Fabric ASIC Fabric ASIC
  • 42.
    • Introduction toNexus 7000 / Nexus 7700 • Nexus 7000 / Nexus 7700 Architecture • Chassis, Supervisor Engines and NX-OS software, I/O modules (M2/F2E/F3) • I/O Module Interoperability • Fabric Architecture • Hardware Forwarding • Generic Designs with Nexus 7000 / Nexus 7700 • STP/VPC, L4-7 services integration, VDCs, VRF/MPLS VPNs, OTV Agenda
  • 43.
    Hardware Forwarding Lookups •Layer 2 and Layer 3 packet flow virtually identical in hardware • Forwarding engine / decision engine pipeline provides consistent L2 and L3 lookup performance • Pipelined architecture also performs ingress and egress ACL, QOS, and Netflow lookups, affecting final forwarding result
  • 44.
    M2 Forwarding EngineHardware • Two hardware forwarding engines integrated on every M2 I/O module • Layer 2 switching (with hardware MAC learning) • Layer 3 IPv4/IPv6 unicast and multicast • MPLS/VPLS/EoMPLS • OTV / GRE • RACL/VACL/PACL • QOS remarking and policing policies • Ingress and egress Netflow (full and sampled) Hardware Table M-Series Modules without Scale License M-Series Modules with Scale License MAC Address Table 128K 128K FIB TCAM 128K IPv4 / 64K IPv6 900K IPv4 / 350K IPv6 Classification TCAM (ACL/QOS) 64K 128K Netflow Table 1M 1M
  • 45.
    FE Daughter Card L2Engine L3 Engine Classification (ACL/QOS) Netflow Layer 3 FIB FIB TCAM CL TCAM Policing Netflow Table From I/O Module Replication Engines To I/O Module Replication Engines Ingress Parser MAC Table L2 Lookup (pre-L3) L2 Lookup (post-L3) Final Results M-Series Forwarding Engine Architecture FE Daughter Card Ingress lookup pipeline Egress lookup pipeline • Ingress MAC table lookups • Port-channel hash result • Ingress ACL/QOS classification • Ingress Netflow collection • FIB TCAM and adjacency table lookups for Layer 3 forwarding • ECMP hashing • Ingress policing HDR • Egress ACL/QOS classification • Egress Netflow collection • Egress policing • Egress MAC lookups • Receive packet header for lookup from Replication Engine • Return final result (destination + priority) to Replication Engine
  • 46.
    F2E Forwarding EngineHardware • 4 x 10G SOC with decision engine • Layer 2 switching (with hardware MAC learning) • Layer 3 IPv4/ IPv6 unicast and multicast • FabricPath forwarding • RACL/VACL/PACL • QOS remarking and policing policies • Ingress sampled Netflow Hardware Table F2E Capacity MAC Address Table 16K FIB TCAM 32K IPv4/16K IPv6 Classification TCAM (ACL/QOS) 16K Per F2E Module
  • 47.
    F3 Forwarding EngineHardware • 8 x 10G, 2 x 40G, or 1 x 100G SOC with decision engine • Layer 2 switching (with hardware MAC learning) • Layer 3 IPv4/ IPv6 unicast and multicast • FabricPath forwarding • RACL/VACL/PACL • QOS remarking and policing policies • Ingress sampled Netflow • MPLS/VPLS/EoMPLS • OTV / GRE tunnels • LISP • VXLAN Hardware Table F3 Capacity MAC Address Table 64K FIB TCAM 64K IPv4/32K IPv6 Classification TCAM (ACL/QOS) 16K
  • 48.
    MAC Table FIB TCAM CL TCAM Layer 3 FIB Classification (ACL/QOS/SNF) Policing FromIngress Port Logic To Ingress Buffer L2 Lookup (pre-L3) L2 Lookup (post-L3) Final Results Ingress Parser F2E/F3 Decision Engine Decision Engine F2E/F3 SOC HDR • Ingress MAC table lookups • Port-channel hash result • Ingress ACL/QOS/SNF classification • FIB TCAM and adjacency table lookups for Layer 3 forwarding • ECMP hashing • Ingress policing • Egress ACL/QOS classification • Egress policing • Egress MAC lookups • Receive packet from Port Logic block • Send payload to Ingress Buffer • Send header to Decision Engine • Return final result (destination + priority) to Ingress Buffer Ingress lookup pipeline Egress lookup pipeline PKT
  • 49.
    • Introduction toNexus 7000 / Nexus 7700 • Nexus 7000 / Nexus 7700 Architecture • Chassis, Supervisor Engines and NX-OS software, I/O modules (M2/F2E/F3) • I/O Module Interoperability • Fabric Architecture • Hardware Forwarding • Generic Designs with Nexus 7000 / Nexus 7700 • STP/VPC, L4-7 services integration, VDCs, VRF/MPLS VPNs, OTV Agenda
  • 50.
    Nexus 7000 /Nexus 7700 Design Building Blocks Foundational: • Spanning Tree (RSTP+/MST) • Virtual Port Channel (VPC) • Virtual Routing and Forwarding (VRF) and MPLS VPNs Innovative: • Remote Integrated Service Engine (RISE) • Virtual Device Context (VDC) • Overlay Transport Virtualisation (OTV)
  • 51.
    STP → VirtualPort Channel (VPC) • Eliminates STP blocked ports, leveraging all available uplink bandwidth and minimising reliance on STP • Provides active-active HSRP • Works seamlessly with current network designs/topologies • Works with any module type (M2/F2E/F3) • Most customers have taken this step No blocking ports STP blocks one uplink Topology without VPC L2 L3 Topology with VPC L2 L3
  • 52.
    …L2 L3 L3 Collapsed Core/Aggregation • Nexus7000 / Nexus 7700 as Data Centre collapsed core/aggregation • Consolidate multiple aggregation building blocks into single switch pair • Reduce number of managed devices • Simplify East-West communication path • M-series or F-series I/O modules, depending on: • Port density, feature-set, and scale requirements • Desired level of oversubscription L2 L3 VPC Domain L3
  • 53.
    Traditional 3-Tier HierarchicalDesign … core1 core2 agg1 agg2 aggX aggY • Extremely wide customer-deployment footprint • Nexus 7000 / Nexus 7700 in both Data Centre aggregation and core • Provides high-density, high-performance 10G / 40G / 100G • Same module-type considerations as collapsed core – density, features, scale • Scales well, but scoping of failure domains imposes some restrictions • VLAN extension / workload mobility options limited L2 L3 L3
  • 54.
    L4-7 Services Integration– VPC Connected • VPC designs well-suited for L4-7 services integration – pair of aggregation devices makes service appliance connections simple • Multiple service types possible – transparent services, appliance as gateway, active-standby or active-active models • VPC-connected appliances preferred: • Ensures that all traffic – data plane, fault-tolerance, and management – sent direct via VPC port-channels • Minimises VPC peer link utilisation in steady state • Use orphan ports with “vpc orphan-port suspend” when services appliance does not support port- channels or Layer 3 peering to VPC peer required L2 L3 VPC Primary VPC Secondary L3 Active Service Standby Service
  • 55.
    L4-7 Services Integration– RISE • Logical integration of external services appliance with Nexus 7000 / Nexus 7700 Citrix NetScaler and Cisco Prime NAM appliance supported today • Enables tight services integration between services appliance and Nexus 7000 / Nexus 7700 switches, including: Discovery and bootstrap Automated Policy Based Routing (APBR) Route Health Injection (RHI) (future) Remote Integrated Service Engine (RISE) Physical Topology Logical Topology with RISE
  • 56.
    RISE Auto-PBR • Userconfigures new service in NetScaler • NetScaler sends server list and next-hop interface to Nexus 7000/7700 switch over RISE control channel • Switch automatically generates PBR route-maps and applies PBR rules in data- plane hardware to redirect target traffic – no manual configuration on switch • Client traffic destined to VIP redirected to NetScaler for processing, destination rewritten to Real server IP NetScaler MPX APBR APBR APBR rules Configure new service Client → VIP Client → Real
  • 57.
    RISE Auto-PBR • Userconfigures new service in NetScaler • NetScaler sends server list and next-hop interface to Nexus 7000/7700 switch over RISE control channel • Switch automatically generates PBR route-maps and applies PBR rules in data- plane hardware to redirect target traffic – no manual configuration on switch • Client traffic destined to VIP redirected to NetScaler for processing, destination rewritten to Real server IP • Return traffic redirected to rewrite Real IP to VIP NetScaler MPX APBR APBR VIP → Client Real → Client APBR rules
  • 58.
    VDC Details Virtual DeviceContexts • Create multiple logical devices out of one physical device • Provide data-plane, control-plane, and management-plane separation • Fault isolation and reduced fate sharing Infrastructure Kernel VDC 1 VDC 1 Layer 2 Protocols VLAN STP VPC CDP LACP CTS Network Stack (L2 / IPv4 / IPv6) Layer 3 Protocols OSPF BGP VRRP SNMP PIM RIB VDC 2 VDC n VDC n VDC 2 Layer 2 Protocols VLAN STP VPC CDP LACP CTS Network Stack (L2 / IPv4 / IPv6) Layer 3 Protocols OSPF BGP VRRP SNMP PIM RIB Note: VDCs do not provide a hypervisor capability, or ability to run different OS versions in each VDC
  • 59.
    VDC Interface Allocation •Physical interfaces assigned on per VDC basis, from default/admin VDC • All subsequent interface configuration performed within the assigned VDC • A single interface cannot be shared across multiple VDCs • VDC type (“limit-resource module-type”) determines types of interfaces allowed in VDC • VDC type driven by operational goals and/or hardware restrictions, e.g.: • Mix M2 and F2E in same VDC to increase MAC scale in FabricPath • Restrict VDC to F3 only to avoid lowest common denominator • Cannot mix M1 and F3 in same VDC
  • 60.
    VDC Interface Allocation– M2 • Allocate any interface to any VDC • But, be aware of shared hardware resources – backend ASICs may be shared by several VDCs • Best practice: allocate entire module to one VDC to minimise shared hardware resources VDC 1 VDC 2 VDC 3 VDC 4 M2-10G M2-40G
  • 61.
    VDC Interface Allocation– F2E / F3 Modules • Allocation on port-group boundaries – aligns ASIC resources to VDCs • Port-group size varies depending on module type F3-10G 8-port port-group VDC 1 VDC 2 VDC 3 VDC 4 F2E 4-port port-group F3-40G 2-port port-group F3-100G 1-port port-group
  • 62.
    Communicating Between VDCs •Must use front-panel ports to communicate between VDCs • No backplane inter-VDC communication • No restrictions on L2/L3 configuration, module types, or physical media type – just like interconnecting two physical switches • Copper Twinax cables (CX-1) or 40G bidi optics provide low-cost interconnect options VDC 1 VDC 2
  • 63.
    … core1 core2 agg1 agg2aggX aggY L2 L3 Admin Zone 1 Admin Zone 2 Admin Zone 3 L2 L3 core1 core2 Admin Zone 1 (VDC 1) Admin Zone 3 (VDC 3) Admin Zone 2 (VDC 2) Collapsed Core Design with VDCs • Maintain administrative segmentation while consolidating network infrastructure • Maintain fault isolation between zones (independent L2, routing processes per zone) • Firewalling between zones facilitated by VDC port membership model
  • 64.
    VRF / MPLSVPNs • Provides network virtualisation – One physical network supporting multiple virtual networks • While maintaining security/segmentation and access to shared services • VRF-lite segmentation for simple/limited virtualisation environments • MPLS L3VPN for larger-scale, more flexible deployments
  • 65.
    Agg CampusInternet MPLS Layer 3VPN – Secure Multi-Tenant Data Centre Requirement: • Secure segmentation for hosted / enterprise data centre Solution: • MPLS Layer 3 VPNs for segmentation • MPLS PE boundary in Pod aggregation layer with VRF membership on SVIs • Direct PE-PE or PE-P-PE interconnections in core • Layer 2 with VLANs below MPLS boundary MPLS Access Pod 1 Pod 2 PE PP PE Core Agg L2 L2 L3
  • 66.
    Any Transport! –L2, L3, MPLS OTV for Multi-Site VLAN Extension • Overlay Transport Virtualisation (OTV) provides multi-site Layer 2 Data Centre Interconnect (DCI) • Dynamic “MAC in IP” encapsulation with forwarding based on MAC “routing” table • No pseudo-wire or tunnel state maintained Site 2 Site 3 OTV – Virtual Layer 2 Interconnect Site 1 VLAN x VLAN x VLAN x Layer 2 adjacent
  • 67.
    MAC2 MAC3MAC1 OTV ata Glance • MAC addresses advertised in routing protocol (control plane learning) between Data Centre sites • Ethernet traffic between sites encapsulated in IP: “MAC in IP” IP A IP B Ethernet Frame MAC1→MAC2 IP packet IP A→IP B Ethernet Frame MAC1→MAC2 Encap Decap OTV edge OTV edge OTV Ethernet Frame MAC1→MAC2 MAC IF MAC1 IP A MAC2 po1 MAC3 po1 MAC IF MAC1 po1 MAC2 IP B MAC3 IP B Site 2Site 1
  • 68.
    • Current limitation– SVI (for VLAN termination at L3) and OTV overlay interface (for VLAN extension over OTV) cannot exist in same VDC • Typical designs move OTV to separate VDC, or separate switch (e.g. Nexus 7702) OTV VDC Requirement Nexus 7000/7700 SVI L2 VLAN Nexus 7000/7700 OTV L2 VLAN SVI L2 VLAN OTV L2 VLAN Nexus 7000/7700 VDC w/SVI VDC w/OTV OTVSVI L2 VLAN Nexus 7000/7700  
  • 69.
    Key Takeaways • Nexus7000 / Nexus 7700 switching architecture provides foundation for flexible and scalable Enterprise network designs • Nexus 7000 / Nexus 7700 design building blocks interwork and complement each other to solve customer challenges • Nexus 7000 / Nexus 7700 platform continues to evolve to support next-generation/emerging technologies and architectures
  • 70.
  • 71.
    Complete Your OnlineSession Evaluation Learn online with Cisco Live! Visit us online after the conference for full access to session videos and presentations. www.CiscoLiveAPAC.com Give us your feedback and receive a Cisco 2016 T-Shirt by completing the Overall Event Survey and 5 Session Evaluations. – Directly from your mobile device on the Cisco Live Mobile App – By visiting the Cisco Live Mobile Site http://showcase.genie-connect.com/ciscolivemelbourne2016/ – Visit any Cisco Live Internet Station located throughout the venue T-Shirts can be collected Friday 11 March at Registration
  • 72.