3. Session Abstract
This session will discuss the foundations of the Nexus 7000 and 7700 series
switches, including chassis, I/O modules, and NX-OS software. Examples will
show common use-cases for different module types and considerations for
module interoperability. The focus will then shift to key platform capabilities and
features – including VPC,OTV, VDCs, and others – along with real-world designs
and deployment models.
4. Session Goals
• To provide an understanding of the Nexus 7000 / Nexus 7700 switching
architecture, which provides the foundation for flexible, scalable Data Centre
designs
• To examine key Nexus 7000 / Nexus 7700 design building blocks and illustrate
common design alternatives leveraging those features and functionalities
• To see how the Nexus 7000 / Nexus 7700 platform plays in emerging
technologies and architectures
6. Introduction to Nexus 7000 / Nexus 7700 Platform
Data-centre class Ethernet switches designed to deliver high performance, high
availability, system scale, and investment protection
Designed for wide range of Data Centre deployments, focused on feature-rich
10G/40G/100G density and performance
I/O Modules
Supervisor Engines
Fabrics
Chassis
7. Nexus 7000
General purpose DC switching w/10/40/100G
Nexus 7700
Targeted at Dense 40G/100G deployments
CommonFoundation
• Same release vehicles, versioning, feature-sets
• Common configuration model
• Common operational model
• Common fabric ASICs (Fab2) and architecture
• Same central arbitration model
• Same VOQ/QOS model
• Identical forwarding ASICs (F2E, F3)
• Consistent hardware feature sets
• Consistent hardware scale
Nexus 7000 / Nexus 7700 – Common Foundation
9. Nexus 7000 Chassis Family
Front Rear
21RU
N7K-C7010
25RU
Front RearN7K-C7018
Front RearN7K-C7009
14RU
Nexus 7010 Nexus 7018
Nexus 7009
Front N7K-C7004
7RU
Rear
Nexus 7004
Front
Back
Side Side
Side Side Side
Back
10. Nexus 7700 Chassis Family
Front Rear
26RU
N77-C7718
Nexus 7718
Front Rear
14RU
N77-C7710
Nexus 7710
Front Rear
9RU
N77-C7706
Nexus 7706
Front
Back
Front
Back
Front
Back
11. Nexus 7702 Chassis
NX-OS 7.2 and later
Front RearN77-C7702
Front
Back
3RU
One Supervisor
Engine
Two Power Supplies
One F3 Series
I/O Module
One Fan Tray
(3 Fans)
No Fabric Modules!
12. Supervisor Engine 2 / 2E
• Provides all control plane and management functions
• Connects to fabric via 1G inband interface
• Interfaces with I/O modules via 1G switched EOBC
• Onboard central arbiter ASIC
Controls access to fabric bandwidth via dedicated arbitration path to I/O modules
Console Port
Management
Ethernet
N7K-SUP2/N7K-SUP2E
USB Host
Ports
ID and Status
LEDs
Supervisor Engine 2 (Nexus 7000) Supervisor Engine 2E (Nexus 7000 / Nexus 7700)
Base performance High performance
One quad-core 2.1GHz CPU with 12GB DRAM Two quad-core 2.1GHz CPU with 32GB DRAM
USB Log
Flash
USB Expansion
Flash
N77-SUP2E
ID and Status
LEDs
Console Port Management
Ethernet
USB Expansion
Flash
13. Supervisor Engine 2 / 2E Architecture
Fabric ASIC
VOQs
I/O Controller
Main CPU Main CPU
NVRAM
DRAM
Bootflash
(eUSB)
USB expansion
USB logflash USB device port
Console Mgmt0
Central
ArbiterSwitched
EOBC
12GB/32GB2.1GHz
Quad-Core
32MB
2.1GHz
Quad-Core
Sup2E
Only
To Fabric Modules To Module VOQs
Dedicated
Arbitration
Path
Dedicated
Arbitration
Path
To Module CPUs
Switched
1GE EOBC
1GE Inband
2GB
15. Nexus 7000 M2 I/O Modules
• 10G / 40G / 100G M2 I/O modules
• Share common hardware architecture – multi-chipset
• Two integrated forwarding engines (120Mpps)
• Layer 2/Layer 3 forwarding with L3/L4
services (ACL/QOS) and advanced
features (MPLS/OTV/GRE etc.)
• Large forwarding tables (900K FIB/
128K ACL)
N7K-M224XP-23L / N7K-M206FQ-23L / N7K-M202CF-22L
N7K-M224XP-23L
N7K-M206FQ-23L
N7K-M202CF-22L
Module Port Density Optics Bandwidth
M2 10G 24 x 10G (plus Nexus 2000 FEX support) SFP+ 240G
M2 40G 6 x 40G (or up to 24 x 10G via breakout) QSFP+ 240G
M2 100G 2 x 100G CFP 200G
16. Fabric ASIC
Nexus 7000 M2 I/O Module Architecture
N7K-M224XP-23L / N7K-M206FQ-23L / N7K-M202CF-22L
To Fabric Modules
Front Panel Ports
VOQsVOQsForwarding
Engine
Replication
Engine
Replication
Engine
LinkSec +
12 X 10G MAC -or-
3 X 40G MAC -or-
1 X 100G MAC
VOQs VOQs Forwarding
Engine
Replication
Engine
Replication
Engine
LinkSec +
12 X 10G MAC -or-
3 X 40G MAC -or-
1 X 100G MAC
LC
CPU
EOBC
Arbitration
Aggregator
To Central Arbiters
17. Nexus 7000 / Nexus 7700 F2E I/O Modules
• 48-port 1G/10G with SFP/SFP+ transceivers
• 480G full-duplex fabric connectivity
• System-on-chip (SOC) forwarding engine design
12 independent SOC ASICs
• Layer 2/Layer 3 forwarding with L3/L4 services
(ACL/QOS)
• Interoperability with M1/M2, in Layer 2 mode on Nexus
7000
Proxy routing for inter-VLAN/L3 traffic
N7K-F248XP-25E / N7K-F248XT-25E / N77-F248XP-23E
7000: Supported in NX-OS release 6.1(2) and later
7700: Supported in NX-OS release 6.2(2) and later
Nexus 7000
N7K-F248XP-25E
Nexus 7000
N7K-F248XT-25E
Nexus 7700
N77-F248XP-23E
18. 4 X 10G
SOC 1
Nexus 7000 F2E Module Architecture
N7K-F248XP-25E / N7K-F248XT-25E
To Fabric Modules
1
Front Panel Ports (SFP/SFP+)
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
4 X 10G
SOC 2
4 X 10G
SOC 3
4 X 10G
SOC 4
4 X 10G
SOC 5
4 X 10G
SOC 6
4 X 10G
SOC 7
4 X 10G
SOC 8
4 X 10G
SOC 9
4 X 10G
SOC 10
4 X 10G
SOC 11
4 X 10G
SOC 12
Fabric ASIC
LC
CPU
EOBC
Arbitration
Aggregator
To Central Arbiters
to ARB
to LC
CPU
LC Inband
19. 4 X 10G
SOC 1
Nexus 7700 F2E Module Architecture
N77-F248XP-23E
1
Front Panel Ports (SFP/SFP+)
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
4 X 10G
SOC 2
4 X 10G
SOC 3
4 X 10G
SOC 4
4 X 10G
SOC 5
4 X 10G
SOC 6
4 X 10G
SOC 7
4 X 10G
SOC 8
4 X 10G
SOC 9
4 X 10G
SOC 10
4 X 10G
SOC 11
4 X 10G
SOC 12
To Fabric Modules
Fabric ASIC Fabric ASIC
LC
CPU
EOBC
Arbitration
Aggregator
To Central Arbiters
to ARB
to LC
CPU
LC Inband
20. Nexus 7000 F3 I/O Modules
• 10G / 40G / 100G F3 I/O modules
• Share common hardware architecture
• SOC-based forwarding engine design
6 independent SOC ASICs per module
• Layer 2/Layer 3 forwarding with L3/L4 services (ACL/QOS)
and advanced features (MPLS/OTV/GRE/VXLAN etc.)
• Require Supervisor Engine 2 / 2E
N7K-F348XP-25 / N7K-F312FQ-25 / N7K-F306CK-25
Module Port Density Optics Bandwidth
F3 10G 48 x 1/10G (plus Nexus 2000 FEX support) SFP+ 480G
F3 40G 12 x 40G (or up to 48 x 10G via breakout) QSFP+ 480G
F3 100G 6 x 100G CPAK 550G
N7K-F348XP-25
N7K-F312FQ-25
N7K-F306CK-25
21. Nexus 7700 F3 I/O Modules
• 10G / 40G / 100G F3 I/O modules
• Share common hardware architecture
• SOC-based forwarding engine design
6 independent SOC ASICs per 10G module
12 independent SOC ASICs per 40G/100G module
• Layer 2/Layer 3 forwarding with L3/L4 services (ACL/QOS)
and advanced features (MPLS/OTV/GRE/VXLAN etc.)
N7K-F348XP-25 / N7K-F312FQ-25 / N7K-F306CK-25
Module Port Density Optics Bandwidth
F3 10G 48 x 1/10G (plus Nexus 2000 FEX support) SFP+ 480G
F3 40G 24 x 40G (or up to 76 x 10G + 5 x 40G via
breakout)
QSFP+ 960G
F3 100G 12 x 100G CPAK 1.2T
N77-F348XP-23
N77-F324FQ-25
N77-F312CK-26
22. 8 X 10G
SOC 1
Nexus 7000 F3 48-Port 1G/10G Module Architecture
N7K-F348XP-25
8 X 10G
SOC 6
8 X 10G
SOC 5
8 X 10G
SOC 4
8 X 10G
SOC 3
8 X 10G
SOC 2
1
Front Panel Ports (SFP/SFP+)
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
To Fabric Modules
Fabric ASIC
EOBC
LC Inband
to ARB
x 6…
FSA
CPU
1G switch
To Central Arbiters
…x 6
to FSA
CPU
Arbitration
Aggregator
23. FSA CPU
Fabric Services Accelerator (FSA) for F3
• High-performance module CPU with on-board
acceleration engines
• 6Gbps inband connectivity from SOCs to FSA
• Multi-Mpps packet processing
• 2 X 2GB dedicated DRAM
• Performance/scale boost for distributed fabric
services, including sampled Netflow and BFD
(roadmap)
• Other potential applications include distributed
ARP/ping processing, data plane packet
analysis (wireshark), network probing, etc.
6 x 1Gbps
Module Inband
I/O
2GB
DRAM
Dual-Core
LC CPU
Acceleration
Engines
2GB
DRAM
EOBC
24. Nexus 7000 F3 12-Port 40G Module Architecture
N7K-F312FQ-25 To Fabric Modules
2 X 40G
SOC 6
2 X 40G
SOC 5
2 X 40G
SOC 4
2 X 40G
SOC 3
2 X 40G
SOC 2
2 X 40G
SOC 1
Fabric ASIC
Front Panel Ports (QSFP+)
EOBC
LC Inband
to ARB
x 6…
FSA
CPU
1G switch
To Central Arbiters
…x 6
to FSA
CPU
Arbitration
Aggregator
1 2 3 4 5 6 7 8 9 10 11 12
25. Nexus 7000 F3 6-Port 100G Module Architecture
N7K-F306CK-25 To Fabric Modules
1 X 100G
SOC 6
1 X 100G
SOC 5
1 X 100G
SOC 4
1 X 100G
SOC 3
1 X 100G
SOC 2
1 X 100G
SOC 1
Fabric ASIC
2 3 4 5 61
Front Panel Ports (CPAK)
EOBC
LC Inband
to ARB
x 6…
FSA
CPU
1G switch
To Central Arbiters
…x 6
to FSA
CPU
Arbitration
Aggregator
26. 8 X 10G
SOC 1
Nexus 7700 F3 48-Port 1G/10G Module Architecture
N77-F348XP-23 To Fabric Modules
8 X 10G
SOC 6
8 X 10G
SOC 5
8 X 10G
SOC 4
8 X 10G
SOC 3
8 X 10G
SOC 2
1
Front Panel Ports (SFP/SFP+)
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
Fabric ASIC Fabric ASIC
EOBC
LC Inband
to ARB
x 6…
FSA
CPU
1G switch
To Central Arbiters
…x 6
to FSA
CPU
Arbitration
Aggregator
27. 2 X 40G
SOC 1
Nexus 7700 F3 24-Port 40G Module Architecture
N77-F324FQ-25
Front Panel Ports (QSFP+)
To Fabric Modules
2 X 40G
SOC 12
2 X 40G
SOC 11
2 X 40G
SOC 10
2 X 40G
SOC 9
2 X 40G
SOC 8
2 X 40G
SOC 7
2 X 40G
SOC 6
2 X 40G
SOC 5
2 X 40G
SOC 4
2 X 40G
SOC 3
2 X 40G
SOC 2
Fabric ASIC Fabric ASIC
EOBC
LC Inband
to ARB
x 12…
To Central Arbiters
…x 12
to FSA
CPU
FSA
CPU
1G switch
Arbitration
Aggregator
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
28. 1 X 100G
SOC 1
Nexus 7700 F3 12-Port 100G Module Architecture
N77-F312CK-26
Front Panel Ports (CPAK)
2 3 4 5 6 7 8 9 10 11 121
To Fabric Modules
1 X 100G
SOC 12
1 X 100G
SOC 11
1 X 100G
SOC 10
1 X 100G
SOC 9
1 X 100G
SOC 8
1 X 100G
SOC 7
1 X 100G
SOC 6
1 X 100G
SOC 5
1 X 100G
SOC 4
1 X 100G
SOC 3
1 X 100G
SOC 2
Fabric ASIC Fabric ASIC
EOBC
LC Inband
to ARB
x 12…
To Central Arbiters
…x 12
to FSA
CPU
FSA
CPU
1G switch
Arbitration
Aggregator
29. F3 Module 40G and 100G Flows
• Virtual Queuing Index (VQI)
sustains 10G, 40G, or 100G traffic
flow based on destination interface
type
• No single-flow limit – full 40G/100G
flow support
Ingress Modules
10G 10G 40G 40G 100G
Spines
Spines
Spines
Spines
Fabrics
Egress Interfaces
Destination
VQIs
1 VQI 1 VQI 1 VQI 1 VQI 1 VQI
31. I/O Module Interoperability
• General module interoperability rule is: “+/-1 generation” in same Virtual Device
Context (VDC)
• Layer 3 forwarding behaviour is key difference between interop models:
• “Proxy Forwarding”
• “Ingress Forwarding” with Lowest Common Denominator
32. L2
Host A
10.1.10.100
vlan 10
Host B
10.1.20.100
vlan 20
L2
Proxy Forwarding Model
• F2E modules run in pure Layer 2 mode – all L3
functions disabled
• M2 modules host SVIs and other L3 functions
• From F2E perspective, Router MAC reachable
via M2 modules
• All packets destined to Router MAC forwarded
through fabric toward one M2 module, selected
via port-channel hash
• M2 modules(s) perform all L3 forwarding and
policy, pass packets back over fabric to output
port
• Key consideration: M-series L3 routing
capacity versus F-series front-panel port count –
How much Layer 3 routing is required?
M2 + F2E VDC
F2Emodules
M2modules
interface vlan 10
ip address 10.1.10.1/24
!
interface vlan 20
ip address 10.1.20.1/24
Router MAC
reached via M2
modules
M2 modules
host SVIs
MAC Table
rtr-mac → M2 modules
Port-channel
hash selects
M2 module
33. Ingress Forwarding with Lowest Common
Denominator Model
• F3 module interoperability always “Ingress Forwarding” – NO proxy forwarding
Ingress module receiving packet makes all forwarding decisions for that packet
• Supported feature set and scale based on Lowest Common Denominator
Feature available if all modules support the feature
Table sizes based on lowest capacity
F3 + M2 VDC -or- F3 + F2E VDC
Module Types
in VDC
Layer 2 Layer 3 VPC MPLS OTV
Fabric
Path
VXLAN Table Sizes
F3 ✓ ✓ ✓ ✓ ✓ ✓ ✓ F3 size
F3 + M2 ✓ ✓ ✓ ✓ ✓ ✗ ✗ F3 size
F3 + F2E ✓ ✓ ✓ ✗ ✗ ✓ ✗ F2E size
M2 + F2E + F3 Not supported
Not all features
supported by
software today…
34. Module Interoperability Use Cases
• M2 + F2E VDC
• Provide higher-density 1G/10G while supporting M2 features and L3 functions
• Full internet routes, MPLS VPNs
• FabricPath with increased MAC address scale (proxy L2 learning)
• F2E + F3 VDC
• Introduction of 40G/100G into existing 10G environments
• Migration to larger table sizes
• Transition to additional features/functionality (OTV, MPLS, VXLAN, etc.)
• M2 + F3 VDC
• Introduce higher 1G/10G/40G/100G port-density while maintaining feature-set
• Avoid proxy-forwarding model for module interoperability
• Migrate to 40G/100G interfaces with full-rate flow capability
M2
F2E
F2E
F3
M2
F3
36. Crossbar Switch Fabric Modules
• Provide interconnection of I/O modules
• Nexus 7000 and Nexus 7700 fabrics based on Fabric 2 ASIC
• Each installed fabric increases available per-payload slot bandwidth
• Different I/O modules leverage different amount of available fabric bandwidth
• Access to fabric bandwidth controlled using QOS-aware central arbitration with
VOQ
N7K-C7018-FAB-2
N7K-C7010-FAB-2
N7K-C7009-FAB-2
Fabric Module Supported Chassis
Per-fabric module
bandwidth
Max fabric
modules
Total bandwidth per
slot
Nexus 7000 Fabric 2 7009 / 7010 / 7018 110Gbps per slot 5 550Gbps per slot
Nexus 7700 Fabric 2 7706 / 7710 / 7718 220Gbps per slot 6 1.32Tbps per slot
N77-C7718-FAB-2
N77-C7710-FAB-2
N77-C7706-FAB-2
38. 110Gbps220Gbps330Gbps440Gbps550Gbps
Local Fabric
(480G)
Local Fabric
(240G)
I/O Module Capacity – Nexus 7000
One fabric:
• Any port can pass traffic to any
other port in VDC
Three fabrics:
• 240G M2 module has maximum
bandwidth
Five fabrics:
• 480G F2E/F3 module has maximum
bandwidth
Fabric 2 Modules
1Fabric
ASIC
2Fabric
ASIC
3Fabric
ASIC
4Fabric
ASIC
5Fabric
ASIC
per slot bandwidth
39. What About Nexus 7004?
• Nexus 7004 has no fabric modules
• Each I/O module has local fabric with 10 available fabric channels
• I/O modules connect “back-to-back” via 8 fabric channels
• Two fabric channels “borrowed” to connect supervisor engines
Sup Slot 2Sup Slot 1
M2/F2E/F3
Module 4
M2/F2E/F3
Module 3
Fabric
ASIC
Fabric
ASIC
Fabric
ASIC
Fabric
ASIC
2 * 55G
fabric channels
8 * 55G local fabric channels
interconnect I/O modules (440G)
40. 220Gbps440Gbps660Gbps880Gbps1100Gbps1320Gbps Local Fabric
#1 (480G)
Local Fabric
#1 (960G)
Local Fabric
#1 (1.2T)
Fabric
#2
Fabric
#2
Fabric
#2
I/O Module Capacity – Nexus 7700
One fabric:
• Any port can pass traffic to any other port
in VDC
Three fabrics:
• 480G F2E/F3 10G module has maximum
bandwidth
Five fabrics:
• 960G F3 40G module has maximum
bandwidth
Six fabrics:
• 1.2T F3 100G module has maximum
bandwidth
per slot bandwidth
Fabric 2 Modules
1
Fabric
ASICs
2
Fabric
ASICs
3
Fabric
ASICs
4
Fabric
ASICs
5
Fabric
ASICs
6
Fabric
ASICs
41. • Nexus 7702 has no fabric modules
• Single I/O module – all traffic locally switched
• Two fabric channels connect to supervisor engine
F3 Module
Supervisor
What About Nexus 7702?
1* 55G
fabric channel
Fabric
ASIC
Fabric
ASIC
Fabric ASIC
50. Nexus 7000 / Nexus 7700 Design Building Blocks
Foundational:
• Spanning Tree (RSTP+/MST)
• Virtual Port Channel (VPC)
• Virtual Routing and Forwarding
(VRF) and MPLS VPNs
Innovative:
• Remote Integrated Service Engine
(RISE)
• Virtual Device Context (VDC)
• Overlay Transport Virtualisation
(OTV)
51. STP → Virtual Port Channel (VPC)
• Eliminates STP blocked ports, leveraging
all available uplink bandwidth and
minimising reliance on STP
• Provides active-active HSRP
• Works seamlessly with current network
designs/topologies
• Works with any module type (M2/F2E/F3)
• Most customers have taken this step
No blocking
ports
STP blocks
one uplink
Topology without VPC
L2
L3
Topology with VPC
L2
L3
52. …L2
L3
L3
Collapsed Core/Aggregation
• Nexus 7000 / Nexus 7700 as Data Centre
collapsed core/aggregation
• Consolidate multiple aggregation building
blocks into single switch pair
• Reduce number of managed devices
• Simplify East-West communication path
• M-series or F-series I/O modules,
depending on:
• Port density, feature-set, and scale
requirements
• Desired level of oversubscription
L2
L3
VPC Domain
L3
53. Traditional 3-Tier Hierarchical Design
…
core1 core2
agg1 agg2 aggX aggY
• Extremely wide customer-deployment
footprint
• Nexus 7000 / Nexus 7700 in both
Data Centre aggregation and core
• Provides high-density, high-performance
10G / 40G / 100G
• Same module-type considerations as
collapsed core – density, features, scale
• Scales well, but scoping of failure
domains imposes some restrictions
• VLAN extension / workload mobility
options limited
L2
L3
L3
54. L4-7 Services Integration – VPC Connected
• VPC designs well-suited for L4-7 services
integration – pair of aggregation devices makes
service appliance connections simple
• Multiple service types possible – transparent
services, appliance as gateway, active-standby or
active-active models
• VPC-connected appliances preferred:
• Ensures that all traffic – data plane, fault-tolerance, and
management – sent direct via VPC port-channels
• Minimises VPC peer link utilisation in steady state
• Use orphan ports with “vpc orphan-port suspend”
when services appliance does not support port-
channels or Layer 3 peering to VPC peer required
L2
L3
VPC
Primary
VPC
Secondary
L3
Active
Service
Standby
Service
55. L4-7 Services Integration – RISE
• Logical integration of external services
appliance with Nexus 7000 / Nexus 7700
Citrix NetScaler and Cisco Prime NAM
appliance supported today
• Enables tight services integration
between services appliance and Nexus
7000 / Nexus 7700 switches, including:
Discovery and bootstrap
Automated Policy Based Routing (APBR)
Route Health Injection (RHI) (future)
Remote Integrated Service Engine (RISE)
Physical
Topology
Logical Topology
with RISE
56. RISE Auto-PBR
• User configures new service in NetScaler
• NetScaler sends server list and next-hop
interface to Nexus 7000/7700 switch over
RISE control channel
• Switch automatically generates PBR
route-maps and applies PBR rules in data-
plane hardware to redirect target traffic –
no manual configuration on switch
• Client traffic destined to VIP redirected to
NetScaler for processing, destination
rewritten to Real server IP
NetScaler MPX
APBR APBR
APBR rules
Configure new service
Client → VIP
Client → Real
57. RISE Auto-PBR
• User configures new service in NetScaler
• NetScaler sends server list and next-hop
interface to Nexus 7000/7700 switch over
RISE control channel
• Switch automatically generates PBR
route-maps and applies PBR rules in data-
plane hardware to redirect target traffic –
no manual configuration on switch
• Client traffic destined to VIP redirected to
NetScaler for processing, destination
rewritten to Real server IP
• Return traffic redirected to rewrite Real IP
to VIP
NetScaler MPX
APBR APBR
VIP → Client
Real → Client
APBR rules
58. VDC Details
Virtual Device Contexts
• Create multiple logical devices out of one physical device
• Provide data-plane, control-plane, and management-plane separation
• Fault isolation and reduced fate sharing
Infrastructure
Kernel
VDC 1
VDC 1
Layer 2 Protocols
VLAN
STP
VPC
CDP
LACP CTS
Network Stack (L2 / IPv4 / IPv6)
Layer 3 Protocols
OSPF
BGP
VRRP
SNMP
PIM RIB
VDC 2
VDC n
VDC n
VDC 2
Layer 2 Protocols
VLAN
STP
VPC
CDP
LACP CTS
Network Stack (L2 / IPv4 / IPv6)
Layer 3 Protocols
OSPF
BGP
VRRP
SNMP
PIM RIB
Note: VDCs do not provide a hypervisor
capability, or ability to run different OS
versions in each VDC
59. VDC Interface Allocation
• Physical interfaces assigned on per VDC basis, from default/admin VDC
• All subsequent interface configuration performed within the assigned VDC
• A single interface cannot be shared across multiple VDCs
• VDC type (“limit-resource module-type”) determines types of interfaces allowed
in VDC
• VDC type driven by operational goals and/or hardware restrictions, e.g.:
• Mix M2 and F2E in same VDC to increase MAC scale in FabricPath
• Restrict VDC to F3 only to avoid lowest common denominator
• Cannot mix M1 and F3 in same VDC
60. VDC Interface Allocation – M2
• Allocate any interface to any VDC
• But, be aware of shared hardware resources – backend ASICs may be shared by several
VDCs
• Best practice: allocate entire module to one VDC to minimise shared hardware resources
VDC 1
VDC 2
VDC 3
VDC 4
M2-10G
M2-40G
62. Communicating Between VDCs
• Must use front-panel ports to communicate between VDCs
• No backplane inter-VDC communication
• No restrictions on L2/L3 configuration, module types, or
physical media type – just like interconnecting two physical
switches
• Copper Twinax cables (CX-1) or 40G bidi optics provide low-cost
interconnect options
VDC 1
VDC 2
63. …
core1 core2
agg1 agg2 aggX aggY
L2
L3
Admin Zone 1
Admin Zone 2 Admin Zone 3
L2
L3
core1 core2
Admin Zone 1
(VDC 1)
Admin Zone 3
(VDC 3)
Admin Zone 2
(VDC 2)
Collapsed Core Design with VDCs
• Maintain administrative
segmentation while
consolidating network
infrastructure
• Maintain fault isolation
between zones (independent
L2, routing processes per
zone)
• Firewalling between zones
facilitated by VDC port
membership model
64. VRF / MPLS VPNs
• Provides network virtualisation – One physical network
supporting multiple virtual networks
• While maintaining security/segmentation and access to shared
services
• VRF-lite segmentation for simple/limited virtualisation
environments
• MPLS L3VPN for larger-scale, more flexible deployments
65. Agg
CampusInternet
MPLS Layer 3 VPN – Secure Multi-Tenant Data
Centre
Requirement:
• Secure segmentation for hosted / enterprise
data centre
Solution:
• MPLS Layer 3 VPNs for segmentation
• MPLS PE boundary in Pod aggregation layer
with VRF membership on SVIs
• Direct PE-PE or PE-P-PE interconnections in
core
• Layer 2 with VLANs below MPLS boundary
MPLS
Access
Pod 1 Pod 2
PE
PP
PE
Core
Agg
L2
L2
L3
66. Any Transport! – L2, L3, MPLS
OTV for Multi-Site VLAN Extension
• Overlay Transport Virtualisation (OTV) provides multi-site Layer 2 Data Centre Interconnect (DCI)
• Dynamic “MAC in IP” encapsulation with forwarding based on MAC “routing” table
• No pseudo-wire or tunnel state maintained
Site 2 Site 3
OTV – Virtual Layer 2 Interconnect
Site 1
VLAN x VLAN x VLAN x
Layer 2 adjacent
67. MAC2 MAC3MAC1
OTV at a Glance
• MAC addresses advertised in routing protocol (control plane learning) between Data
Centre sites
• Ethernet traffic between sites encapsulated in IP: “MAC in IP”
IP A IP B
Ethernet Frame
MAC1→MAC2
IP packet
IP A→IP B
Ethernet Frame
MAC1→MAC2
Encap Decap
OTV edge OTV edge
OTV
Ethernet Frame
MAC1→MAC2
MAC IF
MAC1 IP A
MAC2 po1
MAC3 po1
MAC IF
MAC1 po1
MAC2 IP B
MAC3 IP B
Site 2Site 1
68. • Current limitation – SVI (for VLAN termination at L3) and OTV overlay interface (for VLAN
extension over OTV) cannot exist in same VDC
• Typical designs move OTV to separate VDC, or separate switch (e.g. Nexus 7702)
OTV VDC Requirement
Nexus 7000/7700
SVI
L2 VLAN
Nexus 7000/7700
OTV
L2 VLAN
SVI
L2 VLAN
OTV
L2 VLAN
Nexus 7000/7700
VDC w/SVI VDC w/OTV
OTVSVI
L2 VLAN
Nexus 7000/7700
69. Key Takeaways
• Nexus 7000 / Nexus 7700 switching architecture provides
foundation for flexible and scalable Enterprise network designs
• Nexus 7000 / Nexus 7700 design building blocks interwork and
complement each other to solve customer challenges
• Nexus 7000 / Nexus 7700 platform continues to evolve to support
next-generation/emerging technologies and architectures
71. Complete Your Online Session Evaluation
Learn online with Cisco Live!
Visit us online after the conference
for full access to session videos and
presentations.
www.CiscoLiveAPAC.com
Give us your feedback and receive a
Cisco 2016 T-Shirt by completing the
Overall Event Survey and 5 Session
Evaluations.
– Directly from your mobile device on the Cisco Live
Mobile App
– By visiting the Cisco Live Mobile Site
http://showcase.genie-connect.com/ciscolivemelbourne2016/
– Visit any Cisco Live Internet Station located
throughout the venue
T-Shirts can be collected Friday 11 March
at Registration
