Your SlideShare is downloading. ×
0
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
DC Architecture Palma
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

DC Architecture Palma

839

Published on

DataCenter Infrastructure architecture (VXLAN/Nexus architecture) (Dave Palma)

DataCenter Infrastructure architecture (VXLAN/Nexus architecture) (Dave Palma)

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
839
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
66
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Separate location from identityVirtualization
  • VxLAN can be thought of a stateless tunneling scheme to overlay Layer 2 networks on top of Layer 3 networks. The endpoints of the tunnel (which originates and/or terminates VxLAN tunnels) are called VTEP and could be located either on the hypervisor on a physical switch/router or physical server implemented in software or hardware. They are also interchangeably referred to as Network Virtualization Endpoint (NVE).A VxLAN Segment ID is an identifier for the Layer 2 overlay network over which VMs communicate.An entity which forwards traffic between VxLAN and L2/VLAN environments. The gateway strips out the VXLAN header and forwards to a physical port based on the destination MAC address of the inner Ethernet frame.Remote switch or hypervisor VTEP reachable via core portRefers to underlay group address associated with a VNI. Multiple VNIs can share a DGVxLAN BUD-node scenarios are not supported. Device cannot be a transit and VTEP gateway at the same time.
  • There are other platforms like ASR9K, ASR 1K, CSR 1K, N1Kv.
  • Merchant Silicon significant limitation is its inability to route the frames based on the DMAC of the inner frame after stripping the VxLAN header. This means that the VxLAN gateway based on single chip in merchant silicon cannot easily support routing in the VxLAN networks.This will be required when a same customer/tenant has multiple segments and wishes to route across them.
  • Label Server a Bare Metal server connecting on a VLAN network
  • Transcript

    • 1. DataCenter Infrastructure - VXLAN and Nexus 7K Architecture David Palma Solutions Architect SLED North East davpalma@cisco.com
    • 2. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Agenda  Overlays  Introduction to VXLAN  VXLAN Design  Deployment Steps  Key Takeaways  References
    • 3. Overlays
    • 4. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Why Overlays? Flexible Overlay Virtual Network • Mobility – Track end-point attach at edges • Scale – Reduce core state – Distribute and partition state to network edge • Multi-tenancy – Share Network resources • Flexibility/Programmability – Reduced number of touch points Robust Underlay/Fabric • High Capacity Resilient Fabric • Intelligent Packet Handling • Programmable & Manageable
    • 5. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Types of Overlay Service • Emulate a LAN segment • Transport Ethernet Frames (IP and non-IP) • Single subnet mobility (L2 domain) • Exposure to open L2 flooding • Useful in emulating physical topologies • Abstract IP based connectivity • Transport IP Packets • Full mobility regardless of subnets • Contain network related failures (floods) • Useful in abstracting connectivity and policy Layer 2 Overlays Layer 3 Overlays
    • 6. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Types of Overlay Edge Devices • Virtual end-points only • Single admin domain • VXLAN, NVGRE, STT Tunnel End-points • Physical and Virtual • Resiliency + Scale • x-organizations/federation • Open Standards Network Overlays Integrated Overlays A p pO S A p pO S Virtual Physical Fabric DB V M O S V M O S Virtual Virtual V M O S V M O S Host Overlays Physical Physical • Router/switch end-points • Protocols for resiliency/loops • Traditional VPNs • OTV, VXLAN, VPLS, LISP
    • 7. Introduction to VXLAN
    • 8. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public VXLAN Overview Challenges VXLAN addresses: • VLAN Scalability (4K) – VXLAN extends the L2 Segment ID field to 24- bits, potentially allowing up to 16 million unique L2 Segments over the same network • VM mobility restricted within a VLAN – VXLAN encapsulates L2 frame in IP- UDP header allowing L2 adjacency across router boundaries VXLAN Technology Overview: • MAC-in-UDP encapsulation • Leverages multicast in the transport network to simulate flooding behavior for broadcast, unknown unicast and multicast in the layer 2 segment • Leverage ECMP to achieve optimal path usage over the transport network
    • 9. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public VXLAN Overview VLAN 10 VLAN 20 vSwitch VTEP vSwitch VTEP Switch VTEP VNI 1000 VNI 2000 VXLAN can be implemented on both Hypervisor-based Virtual Switches to allow for scalable VM deployments, as well as on Physical switches, which provides the ability to bridge VXLAN segments back into VLAN segments. In these cases, the Physical Switch instantiates a VTEP, and function as a VXLAN Gateway…
    • 10. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Virtual eXtensible LAN (VXLAN) • Virtual eXtensible LAN (VXLAN) is a Layer 2 overlay scheme over a Layer 3 network. • A 24-bit VXLAN Segment ID or VXLAN Network Identifier (VNI) is included in the encapsulation to provide up to 16M VXLAN segments for traffic isolation / segmentation, in contrast to 4K segments achievable with VLANs. • Each of these segments represents a unique Layer 2 broadcast domain, and can be administered in such a way that it can uniquely identify a given tenants address space or subnet. Ethernet Header Payload FCS Outer IP Outer UDP VXLAN Outer Ethernet Inner Ethernet Payload New FCS Instance ID 1 ReservedReservedFlags Rsvd Rsvd 8 Bytes 1 Byte Outer UDP Destination Port = VXLAN (originally 8472, recently updated to 4789) Outer UDP Source Port = Hash of Inner Frame Headers (optional)
    • 11. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public VXLAN Frame Format A Layer-2 Gateway bridges traffic to VLAN based on MAC DA A Layer-3 Gateway routes traffic to VLAN based on IP DA Original Ethernet Frame Outer MAC DA Outer MAC SA Outer 802.1Q Outer IP DA Outer IP SA Outer UDP VXLAN ID (24 bits) Inner MAC DA Inner MAC SA Optional Inner 802.1Q Original IP Payload CRC Allows for possible 16M segments IP header, allowing transport across any IP network Identifies packet as a VXLAN packet Transport VLAN Inner IP SA Inner IP DA
    • 12. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public VXLAN Key Terminology VTEP NVE (Network Virtual Endpoint) VNI (VXLAN Network Identifier or VXLAN Segment ID) VXLAN Gateway Transit Remote VTEP Delivery Group (DG) BUM
    • 13. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public VXLAN VTEP VXLAN terminates its tunnels on VTEPs (Virtual Tunnel End Point). Each VTEP has two interfaces - one to provide bridging function for local hosts, the other has an IP identification in the core network for VxLAN encapsulation/de- encapsulation. Local LAN Segment IP Interface End SystemEnd System VTEP Transport IP Network Local LAN Segment IP Interface End SystemEnd System VTEP
    • 14. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Handling of Multi-Destination Traffic VTEP-1 End System A MAC-A IP-A VTEP-3 End System End System VTEP-2 End System B MAC-B IP-B Mcast Group IP Network VTEP 1 IP-1 VTEP 2 IP-2 VTEP3 IP-3 Since a control/signaling protocol has not been defined, emulation of Multi-Destination traffic (Broadcast, Multicast, Unknown Unicast) is handled through the VXLAN IP underlay through the use of segment control multicast groups…
    • 15. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public VXLAN Overview VTEP Discovery & Address Learning VTEP-2 End System B MAC-B IP-B VTEP 1 IP-1 MAC-1 VTEP 2 IP-2 MAC-2 VTEP3 IP-3 VTEP-1 VTEP- 3 End System End System Mcast Group 239.1.1.1 VXLAN VNID: 10 Outer S-IP: IP-1 Outer D-IP: 239.1.1.1 S-MAC: MAC-1 D-MAC: 00:01:5E:01:01:01 ARP Request for IP B Src MAC: MAC-A Dst MAC: FF:FF:FF:FF:FF:FF UDP 2 2 ARP Request for IP B Src MAC: MAC-A Dst MAC: FF:FF:FF:FF:FF:FF 3 3 ARP Request for IP B Src MAC: MAC-A Dst MAC: FF:FF:FF:FF:FF:FF ARP Response from IP B Src MAC: MAC-B Dst MAC: MAC-A 4 VXLAN VNID: 10 Outer S-IP: IP-2 Outer D-IP: IP-1 S-MAC: MAC-2 D-MAC: MAC-1 ARP Response from IP B Src MAC: MAC-B Dst MAC: MAC- A UDP 5 MAC Address VxLAN ID Remote VTEP MAC-A 10 IP-1 MAC Address VxLAN ID Remote VTEP MAC-A 10 IP-1 MAC Address VxLAN ID Remote VTEP MAC-B 10 IP-2 6 ARP Response from IP B Src MAC: MAC-B Dst MAC: MAC-A 7 End System A MAC-A IP-A ARP Request for IP B Src MAC: MAC-A Dst MAC: FF:FF:FF:FF:FF:FF 1
    • 16. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public VXLAN Overview Unicast Forwarding Packet Flow Router-1 Host-A Host-B MAC-A IP-A: 10.1.1.100 MAC-B IP-B: 10.1.1.101 MAC-1 IP-1: 165.123.1.1 MAC-4 IP-4: 140.123.1.1 MAC-2 IP-2: 165.123.1.2 MAC-3 IP-3: 140.123.1.2 VXLAN VNID: 10 Outer S-IP: IP-1 Outer D-IP: IP-4 Outer S-MAC: MAC-1 Outer D-MAC: MAC-2 S-IP: IP-A D-IP: IP-B S-MAC: MAC-A D-MAC: MAC-B UDP UDP VXLAN VNID: 10 Outer S-IP: IP-1 Outer D-IP: IP-4 Outer S-MAC: MAC-3 Outer D-MAC: MAC-4 S-IP: IP-A D-IP: IP-B S-MAC: MAC-A D-MAC: MAC-B IP Network VXLAN VNID 10 (Tenant Blue) Router-2 VTEP-1 VTEP-2 S-IP: IP-A D-IP: IP-B S-MAC: MAC-A D-MAC: MAC-B 1 2 3 S-IP: IP-A D-IP: IP-B S-MAC: MAC-A D-MAC: MAC-B 5 Routed Based on Outer IP header 4
    • 17. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Nexus 9000 Series VXLAN Support VXLAN is supported across the Nexus 9000 series platforms. The VXLAN Gateway functionality is supported across all form factors and line cards. Integrated routing functionality is only supported on ACI-enabled Modules… Nexus 9500 SeriesNexus 9300 Series
    • 18. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Supported Platforms Platform NX-OS version Minimum Recommended Nexus 9500 6.1.2I3.1.x 6.1.2.I3.1.x Nexus 9300 6.1.2I2.1.x 6.1.2.I2.1.x Nexus 3100 (3132/3172) 6.x Q2 CY14 Nexus 6000 7.0(0)N1x) Q3 CY14 Nexus 7000 with F3 7.0.x Q4 CY14 * There is no licensing cost for VXLAN – Enhance Layer 3
    • 19. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Nexus 9000 Series VXLAN Gateway VXLAN gateway bridges traffic between VXLAN segment and another physical / logical layer 2 domain (such as a VLAN)… L3 Network VNI 1010 VNI 1020 VLAN 10 VLAN 20 VxLAN VTEP VxLAN VTEP (VxLAN Gateway) VxLAN VTEP VLAN ID VXLAN ID 10 1010 20 1020
    • 20. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Nexus 9000 Series VXLAN Gateway VLAN ID VXLAN ID 10 1010 20 1020 VLAN to VXLAN Mapping VXLAN Forwarding Table Ethernet/ 802.1Q VXLAN EncapVTEP MAC Address VxLAN ID Remote VTEP AA:AA:AA:AA:AA:AA 1010 10.1.1.2 BB:BB:BB:BB:BB:BB 1020 10.1.1.3 feature nv overlay feature vn-segment-vlan-based interface et4/13 switchport switchport access vlan 10 no shut interface nve1 no shutdown source-interface loopback0 overlay-encapsulation vxlan member vni 1010 mcast-group 230.1.1.1 vlan 10 vn-segment 1010 switch# show nve vni Interface VNI Multicast-group VNI State ---------------- -------- --------------- --------- nve1 1010 230.1.1.1 up switch# show nve peers Interface Peer-IP VNI Up Time ------------- ---------------- -------- ------- nve1 10.1.1.2 1010 00:52:24 switch# The Nexus 9000 series supports VXLAN Gateway function, allowing VLANs to be bridged/mapped to VXLAN Segments and vice versa…
    • 21. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Nexus 9000 Series VXLAN Bridging VXLAN “Bridging” bridges traffic between VXLAN segments L3 Network VNI 1010 VNI 1020 VLAN 10 VLAN 20 VxLAN VTEP VxLAN VTEP (VxLAN Bridging) VxLAN VTEP VLAN ID VXLAN ID 10 1010 20 1020
    • 22. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Nexus 9000 Series VXLAN Routed Mode VXLAN routed mode „routes‟ traffic between VXLAN segments and between VXLAN another physical / logical layer 2 domain (such as a VLAN)… L3 Network VNI 1010 VNI 1020 VLAN 10VLAN 20 VxLAN VTEP (VxLAN Routed Mode) VxLAN VTEP VxLAN VTEP
    • 23. VXLAN Design
    • 24. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public VXLAN Designs High-level design options considered in this presentation are in the following areas:  Routed Access + IP Mobility  L2 extension across Pod / Multi-tenancy  Datacenter Interconnect (DCI)
    • 25. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public VXLAN Gateway – Routed Access + IP Mobility VXLAN Enabled Hypervisor VTE P VXLAN Enabled Hypervisor VTEP • VXLAN Gateway defined at access layer (leaf) – Nexus 9000 • Multicast needs to be enabled for VXLAN to work on the source interface • Next hop of VTEP needs to be Layer 3 • vPC needs peer gateway • Only 1:1 mapping is allowed for VXLAN to VLAN • Recommended N9K to be configured as STP root switch in each L2 network • Link discovery protocols like CDP, LLDP will not discover neighbors on the remote VTEPs • Virtual to physical migration (P2V)
    • 26. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public VXLAN Forwarding Design Considerations VXLAN VTEP downstream of a Nexus 2000 FEX is not supported VTEP VXLAN VLAN When VXLAN is being routed the next hop for VXLAN encapsulated frames needs to be over an L3 interface Alternatively, all SVIs from a VXLAN Gateway must point to the same physical next hop [same VXLAN header MAC DA for all VXLAN encapsulated packets sent from the same physical port]
    • 27. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public VXLAN Design with VXLAN Bridging only L2 Extension across Pods L3 Core Pod 1 Pod 2 VXLAN Overlay (VLAN Extension) Layer-2 VLAN Domain Layer-2 VLAN Domain IP GW IP GW VTEP (Layer-2 only) VTEP (Layer-2 only) L2 Link L3 Link
    • 28. VXLAN Deployment Steps
    • 29. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public VXLAN Sample Topology - VTEP Host A (VLAN 10) Host B (VLAN 10) L3 Transport Network (OSPF and IP PIM) Lpbk0: 100.100.100.1/32 Nexus 9000 VTEP-1 e1/1 Mcast grp: 230.1.1.1 RP: 10.1.1.1 Lpbk0: 100.100.100.2/32 .1 e2/1 20.1.1.0/30 .2 30.1.1.0/30 .1 e2/1 .2 Nexus 9000 VTEP-2 e1/1
    • 30. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public VXLAN Sample topology with vPC Host A (VLAN 10) Host B (VLAN 10) Nexus 9000 VTEP-1 e1/1 L3 Transport Network (OSPF and IP PIM) Multicast group: 230.1.1.1 Loop backk0: 100.100.100.2/32.1 e2/1 20.1.1.0/30 .6 30.1.1.0/30 .1 e2/1 .2 Nexus 9000 VTEP-2 Nexus 9000 VTEP-3 .2 20.1.1.4/30 .5 e1/1 Loop back 0: 200.200.200.2/32 100.100.100.1/32 (Secondary) Loop back 0: 200.200.200.1/32 100.100.100.1/32 (Secondary) vPC e1/1
    • 31. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Key Takeaways – VXLAN Technology  VXLAN is simple – Keeps the attractive aspects of Layer 2 – No re-addressing, simple configuration and deployment – Integrates stability and scale of Layer 3  VXLAN is efficient – Proper utilization of ECMP – Optimal path between any two nodes  VXLAN is scalable – Can extend a bridged domain without extending the risks generally associated with Layer 2 and beyond 4K VLAN limit  VXLAN Control Plane (Future) – BGP and LISP
    • 32. Cisco Nexus 7000 / 7700 Switch Architecture BRKARC-3470
    • 33. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public What Is Nexus 7000? Data-center class Ethernet switch designed to deliver high performance, high availability, system scale, and investment protection Nexus 7000 designed for general-purpose Data Center deployments, focused on 10G density plus 40G/100G I/O Modules Supervisor Engines Fabrics Chassis
    • 34. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public What Is Nexus 7700? Data-center class Ethernet switch designed to deliver high performance, high availability, system scale, and investment protection Nexus 7700 designed for SP and MSDC Data Center deployments, focused on high- density 40G/100G I/O Modules Supervisor Engine Fabrics Chassis
    • 35. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Agenda  Chassis Architecture  Supervisor Engine and I/O Module Architecture  Forwarding Engine Architecture  Fabric Architecture  I/O Module Queuing  NetFlow  Conclusion
    • 36. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Nexus 7700 Chassis Family Front Rear 26RU N77-C7718 Nexus 7718 Front Rear 14RU N77-C7710 Nexus 7710 Front Rear 9RU N77-C7706 Nexus 7706 NX-OS 6.2(6) and later NX-OS 6.2(2) and later NX-OS 6.2(2) and later Front Back Front Back Front Back
    • 37. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Agenda  Chassis Architecture  Supervisor Engine and I/O Module Architecture  Forwarding Engine Architecture  Fabric Architecture  I/O Module Queuing  NetFlow  Conclusion
    • 38. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public  Next generation supervisors providing control plane and management functions  Connects to fabric via 1G inband interface  Interfaces with I/O modules via 1G switched EOBC  Second-generation dedicated central arbiter ASIC – Controls access to fabric bandwidth via dedicated arbitration path to I/O modules Supervisor Engine 2 / 2E Console Port Management Ethernet N7K-SUP2/N7K-SUP2E USB Host Ports ID and Status LEDs Supervisor Engine 2 (Nexus 7000) Supervisor Engine 2E (Nexus 7000 / Nexus 7700) Base performance High performance One quad-core 2.1GHz CPU with 12GB DRAM Two quad-core 2.1GHz CPU with 32GB DRAM USB Log Flash USB Expansion Flash N77-SUP2E ID and Status LEDs Console Port Management Ethernet USB Expansion Flash
    • 39. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public  10G / 40G / 100G M2 I/O modules  Share common hardware architecture  Two integrated forwarding engines (120Mpps) – Support for “XL” forwarding tables (licensed)  Distributed L3 multicast replication  802.1AE LinkSec on all ports  Supports Nexus 2000 (FEX) connections (10G) N7K-M224XP-23L Nexus 7000 M2 I/O Modules N7K-M224XP-23L / N7K-M206FQ-23L / N7K-M202CF-22L Supported in NX-OS release 6.1(1) and later N7K-M206FQ-23L N7K-M202CF-22L Module Port Density Optics Bandwidth M2 10G 24 x 10G (plus Nexus 2000 FEX support) SFP+ 240G M2 40G 6 x 40G (or up to 24 x 10G via breakout) QSFP+ 240G M2 100G 2 x 100G CFP 200G
    • 40. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Nexus 7000 / 7700 F2E I/O Modules N7K-F248XP-25E / N7K-F248XT-25E / N77-F248XP-23E 7000: Supported in NX-OS release 6.1(2) and later 7700: Supported in NX-OS release 6.2(2) and later N7K-F248XP-25E N7K-F248XT-25E 48-port 1G/10G with SFP/SFP+ transceivers  480G full-duplex fabric connectivity  System-on-chip (SoC) forwarding engine design – 12 independent SoC ASICs  Layer 2/Layer 3 forwarding with L3/L4 services (ACL/QoS)  Interoperability with M1/M2, in Layer 2 mode on Nexus 7000 – Proxy routing for inter-VLAN/L3 traffic  LinkSec support* – Last 8 ports (SFP+) – All 48 ports (Copper) * Roadmap item N77-F248XP-23E
    • 41. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public DC EDGELEAF SPINE Integrated AND rich for Core, Spine, Leaf, DCI, SAN Deployments MOST COMPREHENSIVE Multi-tenancy and Virtualization Capable Hardware READY FOR Energy Efficient ENVIRONMENTAL UNPRECEDENTED Investment Protection on Nexus 7000 Nexus 7700 24 port 40GE Nexus 7700 12 port 100GE Q4 CY13 Nexus 7000 12 port 40GE Nexus 7000 6 port 100GE Q4 CY13 Q4 CY13 Q1 CY14 F3-Series Modules Nexus 7000 / 7700 F3 I/O Modules
    • 42. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Nexus 7700 F3 12-Port 100G Module Architecture Front Panel Ports (CPAK) To Fabric Modules To Central Arbiters Arbitration Aggregator 1 X 100G SoC 2 2 1 X 100G SoC 3 3 1 X 100G SoC 4 4 1 X 100G SoC 5 5 1 X 100G SoC 6 6 1 X 100G SoC 7 1 X 100G SoC 8 1 X 100G SoC 9 1 X 100G SoC 10 1 X 100G SoC 11 Fabric ASIC Fabric ASIC 7 8 9 10 11 1 X 100G SoC 12 12 1 X 100G SoC 1 1 FSA CPU EOBC LC Inband 1G switch … …x 12 to FSA CPUto ARB x 12 x 6
    • 43. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Agenda  Chassis Architecture  Supervisor Engine and I/O Module Architecture  Forwarding Engine Architecture  Fabric Architecture  I/O Module Queuing  NetFlow  Conclusion
    • 44. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public M-Series Forwarding Engine Hardware  Two hardware forwarding engines integrated on every M2 I/O module  120Mpps (60Mpps per forwarding engine) Layer 2 bridging with hardware MAC learning  120 Mpps (60Mpps per forwarding engine) Layer 3 IPv4  60Mpps (30Mpps per forwarding engine) Layer 3 IPv6 unicast  Layer 3 IPv4 and IPv6 multicast support (SM, SSM, Bidir)  MPLS/VPLS/EoMPLS  OTV  RACL/VACL/PACL  QoS remarking and policing policies  Policy-based routing (PBR)  Unicast RPF check and IP source guard  IGMP snooping  Ingress and egress NetFlow (full and sampled)Hardware Table M-Series Modules without Scale License M-Series Modules with Scale License MAC Address Table 128K 128K FIB TCAM 128K IPv4 / 64K IPv6 900K IPv4 / 350K IPv6 Classification TCAM (ACL/QoS) 64K 128K NetFlow Table 1M 1M
    • 45. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public F3 Forwarding Engine Hardware  Each SoC forwarding engine services: – 8 front-panel 10G ports – 2 front-panel 40G ports – 1 front-panel 100G port  148Mpps per SoC Layer 2 bridging with hardware MAC learning  148Mpps per forwarding engine Layer 3 IPv4/ IPv6 unicast  Layer 3 IPv4 and IPv6 multicast support (SM, SSM, Bidir*)  RACL/VACL/PACL  QoS remarking and policing policies  Policy-based routing (PBR)  Unicast RPF check and IP source guard  IGMP snooping  FabricPath forwarding  Overlay Transport Virtualization (OTV)  MPLS/VPLS/EoMPLS, LISP, VXLAN, GRE, FCoE*  Ingress/egress* sampled NetFlowHardware Table Per F3 SoC Per F3 Module MAC Address Table 64K 384K/768K** FIB TCAM 64K IPv4/32K IPv6 64K IPv4/32K IPv6 Classification TCAM (ACL/QoS) 16K 96K/192K** ** Assumes specific configuration to scale SoC resources * Roadmap items
    • 46. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Agenda  Chassis Architecture  Supervisor Engine and I/O Module Architecture  Forwarding Engine Architecture  Fabric Architecture  I/O Module Queuing  NetFlow  Conclusion
    • 47. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Crossbar Switch Fabric Modules  Provide interconnection of I/O modules  Each installed fabric increases available per-payload slot bandwidth  Nexus 7000 and Nexus 7700 fabrics based on Fabric 2 ASIC  Different I/O modules leverage different amount of available fabric bandwidth  Access to fabric bandwidth controlled using QoS-aware central arbitration with VOQ N7K-C7018-FAB-2 N7K-C7010-FAB-2 N7K-C7009-FAB-2 Fabric Module Supported Chassis Per-fabric module bandwidth Max fabric modules Total bandwidth per slot Nexus 7000 Fabric 2 7009 / 7010 / 7018 110Gbps per slot 5 550Gbps per slot Nexus 7700 Fabric 2 7706 / 7710 / 7718 220Gbps per slot 6 1.32Tbps per slot N77-C7718-FAB-2 N77-C7710-FAB-2 N77-C7706-FAB-2
    • 48. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public 110G (2 x 55G) Ingress Module Egress Module Multistage Crossbar Nexus 7000 / Nexus 7700 implement 3-stage crossbar switch fabric  Stages 1 and 3 on I/O modules  Stage 2 on fabric modules 1st stage Egress Module 2nd stage Ingress Module 3rd stage Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASICFabric ASIC Fabric ASIC Fabric Modules Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC 1 Fabric ASIC 2 3 4 5 Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC 6 Fabric ASIC 1.32T 1st stage 3rd stage 550G 110G (2 x 55G) 1 Fabric ASIC 2 3 4 5Fabric ASIC Fabric ASIC Fabric ASIC Fabric ASIC Fabric Modules Nexus 7000 Nexus 7700
    • 49. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public 220Gbps440Gbps660Gbps880Gbps1100Gbps1320Gbps Local Fab2 #1 (480G) Local Fab2 #1 (960G) Local Fab2 #1 (1.2T) Fab2 #2 Fab2 #2 Fab2 #2 I/O Module Capacity – Nexus 7700 One fabric:  Any port can pass traffic to any other port in VDC Three fabrics:  480G F2E/F3 10G module has maximum bandwidth Five fabrics:  960G F3 40G module has maximum bandwidth Six fabrics:  1.2T F3 100G module has maximum bandwidth per slot bandwidth Fabric 2 Modules 1 Fabric 2 ASICs 2 Fabric 2 ASICs 3 Fabric 2 ASICs 4 Fabric 2 ASICs 5 Fabric 2 ASICs 6 Fabric 2 ASICs
    • 50. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Fabric, VOQ, and Arbitration  Crossbar fabric – Provides dedicated, high-bandwidth interconnects between ingress and egress I/O modules  Virtual Output Queues (VOQs) – Provide buffering and queuing for ingress- buffered switch architecture  Central arbitration – Controls scheduling of traffic into fabric based on fairness, priority, and bandwidth availability at egress ports  Fabric, VOQ, and arbitration combine to provide all necessary infrastructure for packet transport inside switch
    • 51. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Agenda  Chassis Architecture  Supervisor Engine and I/O Module Architecture  Forwarding Engine Architecture  Fabric Architecture  I/O Module Queuing  NetFlow  Conclusion
    • 52. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Buffering, Queuing, and Scheduling  Buffering – storing packets in memory – Needed to absorb bursts, manage congestion  Queuing – buffering packets according to traffic class – Provides dedicated buffer for packets of different priority  Scheduling – controlling the order of transmission of buffered packets – Ensures preferential treatment for packets of higher priority and fair treatment for packets of equal priority  Nexus 7000 / Nexus 7700 use queuing policies and network-QoS policies to define buffering, queuing, and scheduling behavior  Default queuing and network-QoS policies always in effect in absence of any user configuration
    • 53. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Agenda  Chassis Architecture  Supervisor Engine and I/O Module Architecture  Forwarding Engine Architecture  Fabric Architecture  I/O Module Queuing  NetFlow  Conclusion
    • 54. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Full vs. Sampled NetFlow  NetFlow collects full or sampled flow data  Full NetFlow: Accounts for every packet of every flow on interface – Available on M-Series modules only – Flow data collection up to capacity of hardware NetFlow table  Sampled NetFlow: Accounts for M in N packets on interface – Available on both M2 (ingress/egress) and F2E/F3 (ingress only) – M2: Flow data collection up to capacity of hardware NetFlow table – F2E/F3: Flow data collection for up to ~1000pps per module – F3 (future): Increased per-module sampling rate leveraging on-board Fabric Services Accelerator (FSA) complex
    • 55. © 2014 Cisco and/or its affiliates. All rights reserved.BRKDCT-1301 Cisco Public Nexus 7000 / Nexus 7700 Architecture Summary I/O Modules Supervisor Engines Fabrics Chassis

    ×