Partner Technical Connect: Cisco Nexus Updates

Partner Technical Connect
Mai 2010, Data Center
1Cisco Confidential© 2010 Cisco Systems, Inc. All rights reserved.Cisco UCS Positioning 18.01.2010
Christian Hasse, chhasse@cisco.com

Agenda
13:00-13:15 Begrüßung & Registrierung
13:15-15:00 Teil1: News & Produktupdates
Nexus 1010
Nexus 22xx
MDS9148
DCI mit OTV
Nexus 7000 new Linecards
Nexus 7000 new Linecards
15:00-15:30 Kaffee & Kuchen
15:30-17:00 Teil2: Data Center Design
Cisco FCoE Designs
NetApp im virtualisierten Rechenzentrum

Cisco Nexus 1010

Cisco Nexus 1000V
Faster VM Deployment
ServerServer
VM
#1
VM
#4
VM
#3
VM
#2
VM
#5
VM
#8
VM
#7
VM
#6
Cisco VN-Link—Virtual Network Link
Policy-Based
VM Connectivity
Non-Disruptive
Operational Model
Mobility of Network
& Security Properties
VMW ESXVMW ESX
Cisco Nexus 1000V
#1 #4#3#2 #5 #8#7#6
VM Connection Policy
Defined in the network
Applied in Virtual Center
Linked to VM UUID
Defined Policies
WEB Apps
HR
DB
Compliance
Virtual
Center

Cisco Nexus 1000V
Richer Network Services
ServerServer
VMVMVMVM
Policy-Based
VM Connectivity
Non-Disruptive
Operational Model
Mobility of Network
VN-Link: Virtualizing the Network Domain
VMW ESXVMW ESX
Cisco Nexus 1000V
VM
#5
VM
#8
VM
#7
VM
#6
VM
#4
VM
#3
VM
#2
VM
#1
VM
#4
VM
#3
VM
#2
VM
#1
VN-Link Property Mobility
VMotion for the network
Ensures VM security
Maintains connection stateVirtual
Center
VMs Need to Move
VMotion
DRS
SW Upgrade/Patch
Hardware Failure

Cisco Nexus 1000V
Increase Operational Efficiency
ServerServer
VM
#5
VM
#8
VM
#7
VM
#6
VM
#4
VM
#3
VM
#2
VM
#1
Policy-Based
VM Connectivity
Non-Disruptive
Operational Model
Mobility of Network
VN-Link: Virtualizing the Network Domain
VMW ESXVMW ESX
Cisco Nexus 1000V
#5 #8#7#6#4#3#2#1
Network Benefits
Unifies network mgmt and ops
Improves operational security
Enhances VM network features
Ensures policy persistence
Enables VM-level visibility
Virtual
Center
Server Benefits
Maintains existing VM mgmt
Reduces deployment time
Improves scalability
Reduces operational workload
Enables VM-level visibility

Cisco Nexus 1000V Components
Virtual Ethernet Module(VEM)
Replaces Vmware’s virtual switch
Enables advanced switching capability
on the hypervisor
Provides each VM with dedicated
“switch ports”
Virtual Supervisor Module(VSM)
CLI interface into the Nexus 1000V
Leverages NX-OS
Controls multiple VEMs as a single
network device
AA CC DD FFAA BB CC
vCenter Server

Virtual Supervisor Modules Options
CC DD FFAA BB CC
VSM - Virtual Appliance
ESX Virtual Appliance
Supports 64 VEMs
Installable via GUI, OVA or ISO file
vCenter Server
Nexus 1010 - Physical Appliance
Cisco Physical Server
Hosts 4 VSM Virtual Appliance + Additional
virt. Service Blades
Deployed in pairs for redundancy

Nexus 1010 Hardware Configuration
Based on the UCS C200 M1 Physical Appliance
2 * Intel X5650- 2.66GHz, 6 core
4 * 4 GB RDIMMs RAM
2 * 500GB SATA-II HDD
1 * Broadcom Quadport GbE 5709 NIC Card
1 * Serial Port
1 * Serial Port
1 * Rail-Kit

Nexus 1010 Software
Network Analysis
Module*
10Cisco Confidential© 2010 Cisco Systems, Inc. All rights reserved.Cisco UCS Positioning 18.01.2010 1010
Nexus 1010 Manager: Cisco management experience
Manages Virtual Service Blades
Nexus 1000V VSM Nexus 1000V VSM Nexus 1000V VSMNexus 1000V VSM
Nexus 1010 Manager
* Optional virtual service blade add-on

Nexus 1010 Design Option
There are 4 options to connect the Nexus 1010 to the Network
This will influence how the control, packet, management and data
information will be connected to rest of the network

Nexus 1010 Connectivity
2x 1G ports Lan on Motherboard
4x 1G PCI Card
Serial Access for initial installation
if not using Serial Over LAN

There are 4 options to connect the Nexus 1010 to the
physical Network using the 6x 1G interfaces available
Within the Nexus 1010 CLI they are called network
option and will influence on which interface the different
traffic will be configured on.
The four type of traffic available on the system are:
Mgmt Control Packet Data

Network Option 1
Management, Control, Packet and Data traffic go over
the 2 LoM NIC in HA mode
Mgmt Control Packet Data
Really easy to deploy Less bandwidth available for the
Virtual Service Blade

Network Option 2
Management and Control are over the 2 LoM uplink
Data Traffic of the 4 remaining ports
Data
Mgmt Control Packet
Really easy to deploy
Most of the bandwidth available
for the Service Blades
Control traffic and management
share the same uplink

Network Option 3
Management over the 2 LoM uplink
Control, Packet and Data Traffic of the 4 remaining ports
Control Packet Data
Mgmt
Allows for outbound
management network
Less bandwidth available for the

Network Option 4
Management over the 2 LoM uplink
Control, Packet over port 3 and 4
Data over port 5 and 6
Control Packet Data
Mgmt
Clear separation of all the
different type of traffic
More configuration required upstream
Less Bandwidth available for the

Option Pros Cons
Option 1 Really easy to deploy Less bandwidth available for the
Option 2 Really easy to deploy
Most of the bandwidth
available for the Service
Control traffic and management
share the same uplink
available for the Service
Blades
Option 3 Allows for outbound
management network
Less bandwidth available for the
Option 4 Clear separation of all the
different type of traffic
More configuration required
upstream
Less Bandwidth available for the

Nexus 1010 HA
A pair of Nexus 1010 needs to be deploy for HA.
The HA pair will be formed based as soon as the information
match:
Control VLAN and Domain ID
Similarly the VSMs on both Nexus 1010 should back up each
other so one primary VSM should be created on one Nexus
1010. The secondary VSM should be created on the other Nexus
1010

Nexus 1010 Virtual Service Blade
The Nexus 1010 comes with a new concept of Virtual
Services called virtual service blade
Shipping with the product will be:
-VSM virtual service blade
-VSM virtual service blade
-NAM virtual service blade
And way more to come
Nexus 1000V VSM
Nexus 1010 Manager
Network Analysis
Module*
* Optional virtual service blade add-on

Nexus 1010 Virtual Service Blade
The Network Admin now has total control over the
virtual service blade deployment
pe-nexus1010-1# sh virtual-service-blade
virtual-service-blade VSM-AV.1-1
virtual-service-blade VSM-AV.1-1
Description:
Slot id: 1
Host Name: pe-nexus1010-VSM-1
Management IP: 172.25.203.182
VSB Type Name : VSM-1.0
Interface: control vlan: 20
Interface: management vlan: 1
Interface: packet vlan: 20
Interface: internal vlan: NA
<SNIP>
virtual-service-blade:
HA Oper role: ACTIVE
Status: VSB POWERED ON
Location: PRIMARY
SW version: 4.0(4)SV1(3)
He can power off and
power down the VSM
without the help of the
Server Administrator

Q and A

Cisco Nexus 2232 / 2248
Fabric Extender

Nexus 5000 + FEX Single Access Layer
=+
Nexus 5000 Parent Switch
Cisco Nexus® 2000 FEX
Nexus 5000 + FEX combines logically as a Virtual Modular System
Nexus 2000 FEX is a Virtual Line Card to the Nexus 5000
Nexus 5000 maintains all management & configuration
No Spanning Tree between FEX & Nexus 5000
Virtual Modular System

Aggregation
Layer
Core
Layer
L3
L2
VSS/vPC
Nexus 2000
Physically ToR
Logically, a linecard of parent
switch
Overview: Nexus 5000 und 2000
25Cisco Confidential© 2010 Cisco Systems, Inc. All rights reserved.Cisco UCS Positioning 18.01.20105/21/2010 Cisco Systems Confidential, Non-Disclosure Required 25
Rack-1 Rack-2 Rack-3
Access
Layer
Servers
Rack-N
Nexus 2000
Fabric
Extender
Nexus
5000

GE Fabric Extender Nexus 2248
GE Fabric Extender
48x 100/1000M host interfaces; 4x 10GE on network interfaces
48 100/1000 RJ45 Downlinks
4 10GE SFP+ Uplinks
48x 100/1000M host interfaces; 4x 10GE on network interfaces
Can mix-and-match with existing GE and next-gen GE FEX in network topologies
Host port-channel support
ACL classification
SPAN source/destination support

32x 1/10GE host interfaces; 8x 10GE on network interfaces
10GE Fabric Extender Highlights Nexus 2232
32 10GE/FCoE SFP+ Downlinks 8 10GE/FCoE SFP+ Uplinks
32x 1/10GE host interfaces; 8x 10GE on network interfaces
10GE interfaces support FCoE
HW supports 1G but SW support in a post-FCS release
Can mix-and-match with existing GE and next-gen GE FEX in network topologies
Host port-channel support
ACL classification
SPAN source/destination support

Cisco Nexus 2000 Fabric Extender (FEX)
Model Nexus 2148T Nexus 2248 TP Nexus 2232-10G PP
Form Factor 1 RU 1 RU 1 RU
Uplink Ports 4 x 10GbE SFP+ 2248TP: 4 x 10GbE SFP+ 2232PP: 8 x 10GbE SFP+
Uplink SFP+ Transceivers Copper CX-1 (passive): 1m, 3m, 5m
Uplink SFP+ Transceivers
Supported
Copper CX-1 (passive): 1m, 3m, 5m
Optical: SR, LR [distance limited to 300m]
Host Facing Ports 48 x 1GbE RJ45
(note: 1000BaseT only)
48 x 100/1000Base-T RJ45 2232PP: 32 x SFP+ (10G)
Local Classification No Yes Yes
FCoE No No Yes
Power Consumption 165W maximum 110 W 270 W
Buffering per port 90KB/port within FEX 195 KB -> Network to Host (N2H) 147.50 KB/port -> for( N2H)
Multiple PortChannel
member ports on a FEX
Not Supported Yes Yes
Latency ~ 7us (LIFO) ~ 4.5 us 4.6 us

Nexus 2148T versus 2232 versus 2248
Features 2148T 2232 2248
Host ports 48 32 48
Fabric Ports 4 8 4
Port speed 1 GigE only 10Gbps/1Gbps (1 Gbps not in
1st release). Each port
1Gbps/100Mb Each port
independent
1 release). Each port
independent
independent
Etherchannels
on Fabric Links
Yes
Hash up to L3 fields
Yes
8 ports maximum
Hash up to L4 ports
Yes
8 ports maximum
Hash up to L4 ports
Etherchannels
on Host
Interfaces
Not Supported Yes
8 ports maximum
Hash up to L4 ports
Yes
8 ports maximum
Hash up to L4 ports
#Etherchannels N/A Max 16 Port Channels / FEX
2232
Max 24 Port-Channels /
FEX 2248

N5K
N5K01 N5K02
Nexus 2232 and 2248 with 4.2(1)N1(1) aka
Dee Why
Dee Why with 2248/2232
Dee Why + 2248/2232

FEX Static Pinning mode and Host Port
Channels
In Static Pinning mode, all members of a HIFPC must be part of
the same pinning group. This is enforced at configuration time.

Cisco Nexus 2000 Series
Straight-Through vPC
Cisco Nexus 2000
Active-Active
vPC
Primary
vPC
Secondary
Peer Keepalive
Peer Link
vPC Member Port
vPC
Primary
vPC
Secondary
Legacy Designs with FEX 2148T
With Cisco Nexus 2148T, 2 Gigabit
Ethernet Ports Host PortChannel
vPC
FEX120FEX100
vPC 1 vPC 2
FEX120FEX100
HIF HIF
HIFHIF
Fabric Links Fabric Links

Cisco Nexus 2000
Active-Active
vPC
Primary
vPC
Secondary
Peer Keepalive
Peer Link
vPC Member Port
vPC
Primary
vPC
Secondary
Designs with FEX 2248
FEX120FEX100
vPC 1 vPC 2
FEX120FEX100
HIF HIF
HIFHIF
up to 8 ports
up to 8 ports
up to 8 ports up to 8 ports
up to 24 PC
per FEX up to 24 PC
per FEX

Cisco Nexus 2000
Active-Active
vPC
Primary
vPC
Secondary
Peer Keepalive
Peer Link
vPC Member Port
vPC
Primary
vPC
Secondary
Designs with FEX 2232
FEX120FEX100
vPC 1 vPC 2
FEX120FEX100
HIF HIF
HIFHIF
up to 8 ports
up to 8 ports
up to 16 PC
per FEX up to 16 PC
per FEX

Port-channel Hashing on the FEX
• Each Portchannel on the FEX is modeled as a SINGLE VIF (i.e. a
single port as seen on the Nexus 5000)
• The FEX Hardware does a hash on the PortChannel members
(upto 8) to select a hif member port to send the frame out.
• Hashing fields
Mac (src + dest or none)
Mac (src + dest or none)
IP (src + dest or none)
Port (src + dest or none)
• In the Woodside and Portola ASIC, source and destination fields
cannot be separately selected for hashing.
For example if mac based load balancing is selected, hashing
would be done based on both the source and destination macs.

FEX 2232 and FCoE
vfc can bind to the
physical port (A)
vfc can bind to a Port-
channel with 1 single
port per FEX (B)
FCF
port per FEX (B)
vfc cannot bind to the
Port-channel if > 1 port
per FEX (C)
FIP enabled CNAs
FEX 2232PP
A B C
NOYESFCoE
LLDP & DCBX run
locally on the FEX
vfc binds to vfc binds to

FCoE support on FEX2232
FCoE configuration not supported on A-A fex topology
Only FIP enabled CNAs supported on fex
No Gen -1 Menlo
Binding to port channel limited to one member port
Binding to port channel limited to one member port
channel only
Bind check entries limited to fabric ports limits

Multi-hop Topologies with FEX
Servers connection to the
Nexus 4000 is Active/Standby
Servers connect to Nexus 4000
over 10Gig FCoE
PFC support at every hop to
achieve flow-control
Ethernet/LAN Core
SAN A SAN B
achieve flow-control
Support for up to 640
10Gig/FCoE attached hosts
managed by a single Nexus
5000
FEX 2232 is single homed to
upstream Nexus 5000 with
single links or a port-channel
Nexus 5000
FCF
Nexus 5000
FCF
FCoE over an
STP Cloud
FEX-2232 FEX-2232
Nexus 4000: FIP
Snooping Bridge
or pass-through
CNA mezzanine
cards Enhanced Ethernet and FCoE
Ethernet LAN
Native Fibre Channel

Sample Topology
MDS1 MDS2
N5K-1
(SAN-10)
N5K-2
(SAN-11)
FEX-101 FEX-101
CNA
VPC 10

Sample Configuration with FCoE
Configure the VSAN
# vsan database
# vsan 10
Configure the necessary VLANs
# vlan 2 (A non FCoE VLAN that will be used as a native VLAN)
# vlan 10
# fcoe vsan 10
Configure the FEX port with the right port mode
Configure the FEX port with the right port mode
# interface Ethernet 101/1/1
# switchport mode trunk
# switchport trunk native vlan 2 (Configuring native VLAN to a no FCoE VLAN)
# spanning-tree port type edge trunk
Bind the FEX port to the VFC:
# interface vfc 1
# bind interface Ethernet 101/1/1
# no shutdown
Put the VFC in the right VSAN
# vsan database
# vsan 10 interface VFC 1

Sample Configuration with FCoE and
vPC
On N5K-1
# feature vpc
On N5K-2
# feature vpc
Configure the VPC domain and peer keepalive
On N5K-1
switch(config-if)# vpc domain 100
switch(config-vpc-domain)# peer-keepalive destination 10.193.51.95 (IP of the peer)
On N5K-2
On N5K-2
switch(config-if)# vpc domain 100
switch(config-vpc-domain)# peer-keepalive destination 10.193.51.94 (IP of the peer)
Configure the MCT on both switches
switch(config)# feature vpc
switch(config)# interface port-channel 1
switch(config-if)# interface ethernet 1/3
switch(config-if)# channel-group 1 mode on
switch(config-if)# interface ethernet 1/4
switch(config-if)# channel-group 1 mode on
switch(config-vpc-domain)# interface port-channel 1
switch(config-if)# vpc peer-link

4+ Ports vPCs
2-Ports vPCs
Scalability for Port-Channels
5k01 5k02
As many as the number of ports on the 5k
does NOT consume HW resourcesConsumes 1 HW Port-channel
of the 16 available
Peer-link
primary
mgmt0 mgmt0
secondary
5k01 5k02
does NOT consume HW resources
CASE A CASE B
CASE C
5k01
eth2/1,2/2 eth2/3,2/4 eth2/1 eth2/2
5k02
vPC
5k01 5k02
Max 16 HW-Port Channel
vPC
2 ports
FEX120FEX100
HIFHIF
“fabric links”
5k01 5k02
Regular Switch Regular Switch

n5k02n5k01
Nexus 2000 straight-through with vPC
Scalability for “Host” vPC in 4.1(3)N1 Cronulla
max 24 FEXes = 1152
max 480 vPCs (each vPC has 2 ports)
FEX 2148T

vPC Primary
vPC Secondary
Po10
5k01 5k02
Nexus 2000 dual-homed scalability with
4.1(3)N1 Cronulla
max 12 FEXes: 576 ports
Caveat: no CDP
FEX 2148T

Scalability for Port-Channels with FEX
2248 and FEX 2232
Peer-link
primary
mgmt0 mgmt0
secondary
5k01 5k02
does NOT consume HW resources on the 5k
vPC
Primary
vPC
Secondary
Fabric Links
does NOT consume HW resources on the 5k
vPC
2 ports
FEX120FEX100
HIFHIF
“fabric links”
FEX120FEX100
HIFHIF
Fabric Links
up to 8 ports
up to 8 ports
FEX 2248TP or FEX 2232PP
The port-channel on the FEX, is modeled as a
“single” port on the Nexus5k so it categorizes
as case B or case C on slide 15

n5k02n5k01
Scalability for “Host” vPC in 4.2(1)N1(1) (aka Dee Why)
max 24 FEXes = 1152
max 576 vPCs with FEX 2148T
max 576 vPCs with FEX 2248
max 384 vPCs with FEX 2232

n5k02n5k01
Scalability for “Host” vPC in 4.2(1)N1(1) (aka Dee Why)
max 24 FEXes = 1152
Still:
theoretical max 576 vPCs
BUT limited by port count:
48 ports x 12 / 2 ports =
288 vPCs with 2248
32 ports x 12 / 2 =
192 vPC with 2232
N/A to FEX 2148T

vPC Primary
vPC Secondary
Po10
5k01 5k02
Nexus 2000 dual-homed scalability with
4.2(1)N1(1) (aka Dee Why)
max 12 FEXes
+ CDP
+LACP
e.g. FEX 2148T e.g. FEX 2248TP

What made this possible?
Protocol offload on
FEX 2148T
FEX 2248TP
FEX 2232PP
Protocol Offloads offload some of the CP processing to
Protocol Offloads offload some of the CP processing to
the FEX CPU.
Protocols being offloaded include:
LACP
CDP (particuarly useful for FEX A/A which didn’t support CDP)
LLDB (see FCoE on FEX2232)
DCBX (see FCoE on FEX2232)

Q and A

MDS 9148 Technical
Overview
Overview

MDS-9000 Series Overview
MDSFabricManager
MDS-9200 Series
Multi-Purpose Fabric Switches
MDS-9200/MDS-9500 Modules
MDS-9124 MDS-9134
MDS 4 / 8 Gb Fabric Switches
Blade Switches
für IBM/HP MDS-9148
New
MDSFabricManager
NX--OS
Storage Media Encryption,
iSCSI, FCIP; Applications
Virtualization, DMM
4/44Gb Host Optimized
Fibre Channel ports
MDS-9222i
Storage Services Module
Four 10 Gb
Fibre Channel ports
MDS-9500 Series Director
MDS-9506 MDS-9509 MDS-9513
24, 48 8Gb
Fibre Channel ports
18/4 Module

Agenda
Cisco MDS 9148 Technical Overview
Hardware
Software
Cisco MDS 9148 Highlights
Performance
Ease of Use
Flexibility
Enhanced Availability
Enhanced Availability
Security
Traffic Management
Diagnostics
Network Design
Competitive Comparison

Cisco MDS 9148 Fabric Switch
48 line rate 8-Gbps Fibre Channel
MDS 9148 Fabric Switch
2 hot-swappable Power Supplies with
48 line rate 8-Gbps Fibre Channel
ports
128 buffer-to-buffer credits per group of
four ports
1/2/4 and 2/4/8 Gbps shortwave Fibre
Channel SFPs
16-port base configuration
32 and 48 port configurations optional
8-port incremental licensing
Management through 10/100 Base-T
Ethernet and RS232 Console Port
2 hot-swappable Power Supplies with
integrated fans
2 hot-swappable Fan Assemblies
Back (Fan) to Front (Ports) Airflow
Complete NX-OS 5.x feature set
Non-disruptive software upgrade - ISSU
Support for 32 VSANs
IVR Capable – future release
1 SPAN session
Full MIB and SMI-S support

Consistent NX-OS across MDS platforms
Consistent with all MDS 9500, MDS 9200 and MDS 9100
Port Based licensing
Ports licensed in incremental groups of 8 ports
Pay as you grow
VSAN Support
Up to 32 VSANs per MDS 9148
Isolate fabric disruption
Cisco MDS 9148 Software Features
NX-OS Supported Features
Isolate fabric disruption
Port level granularity
Port-Channeling
Up to 16 physical links
Up to 128 Gbps of bandwidth
Non-disruptive firmware upgrade
No Application disruption during upgrade process
Easy to upgrade
NX-OS upgrade wizard
Single upgrade command from the CLI

Supported Port Types – F, FL, E, TE, SD & Auto
All ports are auto-sensing for port type and speed
Translated Loop (TL) devices not supported
Buffer Credits
Default - 32 Buffer to Buffer Credits per port
Twelve 128 buffer pools
Maximum 125 Buffer-to-Buffer Credits on any port in pool
Traffic Engineering
NX-OS Supported Features (continued)
Traffic Engineering
Quality of Service tagging for critical applications
Inter-VSAN Routing (IVR) – Future Release
No Egress Fibre Channel Congestion Control (FCC)

Security
Centralized secure management
TACACS+, RADIUS, SNMP v3, SSH, VSAN RBAC
Secure firmware transfers
SFTP and SCP
Fibre Channel Security Protocol (FCSP)
Utilizes DH-CHAP authentication
Restricts unauthorized server to switch access
NX-OS Supported Features (continued)
Restricts unauthorized switch-to-switch access
Advance Diagnostic Tools
Integrated Call Home
Integrates to other management call home tools
Troubleshooting tools
FCPing & FCTraceRoute
SPAN session
Non-disruptive redirection of trouble port to SPAN port to assist in diagnosis
FC Analyzer
Command line interface to debug FC control data

Cisco MDS 9148
Highlights
Highlights

Cisco MDS 9148 - Ease of Use
Simple Dash-Board for Device Status1) Run Switch CLI Setup
2) Download DM to Management Server
3) Quick Configuration Wizard
Enhancement to Device Manager
Port Properties
Port Enabling
VSAN membership
View port description
Switch-Port zoning
Filter window by VSAN
Use Wizard locally or remotely
Recommended for single switch fabrics
Point-and-Click Provisioning
Intuitive
No need to use WWNs
Recommended for single switch fabrics
Easy Setup Script

Cisco MDS 9148 – Flexibility
Switch Flexibility
8-port group licensing
Use initial 16 ports on switch
Add additional ports with 8-port licenses as
demand increases
VSANs
Create virtual fabrics for separate applications
VSANs done on per port basis
Port Flexibility
Supported Port Types
F, FL, E, TE, SD & Auto
Buffer-to-Buffer Credits
Default 32 buffers per port
125 buffer credits can be allocated to a single
port within a 4-port group
E-Mail
Start with 16-ports and a
VSAN for E-Mail
DatabaseE-Mail
Add 8 more ports and a VSAN
for Database
DatabaseE-Mail
Add 8 more ports and a VSAN
for Backup
Backup

Cisco MDS 9148 – High Availability
High Availability
VSANs
Isolates fabric disruption
Online non-disruptive
software upgrade
Dual Power Supplies

Cisco MDS 9148 – Security
Secure Switch Access
SNMPv3, SSH, SCP and SFTP
Device Security
Fibre Channel Secure Protocol (FCSP)
DH-CHAP authentication
Host-to-Switch Authentication
Unauthorized
Server
Host-to-Switch Authentication
Port Security (WWN)
Switch-to-Switch Authentication
Fabric binding
Standard AAA: RADIUS and
TACACS+
Roles Base Access Control (RBAC)
Rogue Switch
Storage

Cisco MDS 9148 – Traffic Management
Port-Channeling
Up to 16 Physical Links
Up to 128 Gbps of bandwidth
Exchange Based Load-
Balancing
Virtual Output Queues
MDS 9148
MDS 9148 MDS 9148
MDS 9148
Port Channel
QoS
Virtual Output Queues
(VOQ)
Non-blocking architecture
VOQ per destination
Quality of Service (QoS)
4 different queues
High priority for critical
applications
Zone based or QoS based
DWRR Weight
Priority Queue Absolute
Queue 2 60
Queue 3 10
Queue 4 30
QoS
PQ
DWRR 2
DWRR 3
DWRR 4
Transmit
Queue

Cisco MDS 9148
Network Design
Network Design

MDS 9148 Network Design
Consolidation of SAN
Islands
Minimizes number of
SAN switches
Centralized
Application Consolidation E-mail Database Backup
Centralized
Management
E-mail Database Backup
E-mail VSAN
Database VSAN
Backup VSAN MDS 9148

Highest density 1RU
switch for 8 Gbps server
connections
MDS 9148 in NPV
mode reduces number
Top of Rack for Virtualized Servers
Eight x 8 Gbps
Uplinks
mode reduces number
of switches in fabric
One Switch Domain ID in
Fabric “A” and Fabric “B”
Up to 40 servers per rack @ 4:1 over-subscription
MDS 9148
SAN “A”
MDS 9148
SAN “B”

Extending the Fabric
Shared buffer-to-buffer
pool per 4-port port-
group
128 buffer-to-buffer
credits per port-group
Business Continuity / Disaster Recovery
Main Data Center
MDS 9500
Satellite DC 1
IVR used for
remote VSANs
to shared
Backup VSAN
Production VSAN
Shared Backup VSAN
Replication VSANs
credits per port-group
Default – 16 buffer-to-
buffer credits per port
Maximum of 125 buffer-
to-buffer credits per any
single interface in port-
group
MAN
MDS 9148
MDS 9148
Satellite DC 2
Local Production VSAN
Backup/Replication VSAN
Local Exchange VSAN
Backup/Replication VSAN
Local Engineering VSAN

Q and A

Nexus 7000 Platform Overview
Next Generation Modular
Nexus 7000 and NX-OS
• 10 & 18 Slot versions
• 15+ Terabit System
• Unified Fabric Ready
• Modern, Modular OS
• Device Virtualization
• Cisco TrustSec
• Continuous Operations
Linecard
Modules
• Continuous Operations
Supervisor
Cisco NX-OS Multi-protocol Operating System
Data Center Network Manager (DCNM)
10G Ethernet
• 32 Port SFP+ 10G
• 8 Port X2 10G - XL
1G Ethernet
• 48 Port 10/100/1000
• 48 Port 1G - XL

8-Port 10GE XL I/O Module, 80G Fabric
8 ports of Line Rate 10GE
80G full duplex fabric connectivity
Dual M1 Forwarding Engines
for 120Mpps
64 byte packet line rate
performance on all 8 ports
Up to 1M FIB entries, 128K
ACL/QoS TCAM
IEEE802.1AE MACSec on
every port
Buffering:
Dedicated mode: 65MB ingress,
NEW
Flexible XL Option with
Feature License
Dedicated mode: 65MB ingress,
80MB egress
Queues: 8q2t ingress,
1p7q4t egress

Optics for M1 8-Port 10G XL I/O Module
X2 optics enhance capability to support long reach optics not
available today as SFP+
Initial support for SR, LR, LRM, ER and DWDM
Post FCS will add support for ZR and CX4
SR LR ER LRM DWDM ZR CX4
Now Now Now Now Now July 10 July 10
300m 10km 40km 220m 80km 80km 15m
MM SM SM MM SM SM Cu

48-Port 1G –XL I/O Module - Fiber
48 1G SFP ports
SX, LX, ZX, T, CWDM, DWDM
Supports Digital Optical Monitor
46G full duplex fabric
connectivity
Integrated 60Mpps forwarding
engine
48 ports wire-rate L3 multicast
replication
Flexible XL Option with
Feature License
NEW
Line rate on 48-ports
With ~10% local switching
Feature License
802.1AE MacSec on every
port

Addresses Rapid growth of both IPv4 and IPv6 table size – up to 1M routes
Internet Peering requires more than 300K entries today and growing rapidly
IPv4 Table grows ~ 18% per year so will pass 512K in ~2013
Growth of IPv6 may accelerate tables beyond 512K in ~2011
Use of VRF’s can scale an Enterprise beyond 100K very quickly
Expanded TCAM to 128K enhances the scalability of security and QoS ACLs
Enhance Network Scale with XL Modules
Why are “XL” Modules Required
Expanded TCAM to 128K enhances the scalability of security and QoS ACLs
All future M1 modules are designed for XL scale tables
ISP-1 ISP-2 ISP-3
Dense Internet Peering
Capability
M1 Series
M1-XL w/o
License
M1-XL w/
License
FIB (IPv4 / IPv6) 128K 128K Up to 1M
Security ACL / QoS TCAM 64K 64K 128K
Adjacencies 1M 1M 1M
Netflow 512K 512K 512K

Integrated Forwarding Engine
Advanced hardware forwarding engine
Up to 60Mpps IPv4 unicast, 30Mpps IPv6 unicast
throughput
M1 Series Forwarding Engine
NEW
M1 Series Forwarding Engine
Equal to Cat 6K EARL 8
Integrated on every I/O module
(NOT a FRU)
New dual personality modules operate in Standard
or XL mode

Nexus 7000 Scalable XL I/O Modules
System
Feature
License
Investment Protection
XL System
Ease of Sparing
New dual personality modules operate in Standard or Large mode
Addition of the System Features License enables the larger tables
Modules
Investment Protection
Cost Effective
Easy Upgrades
Complete Portfolio
Ease of Sparing
Simpler Ordering
System Licensing
Tiered Pricing Model
Module Description Availability
N7K-M108X2-12L 8 Port 10GE Module with X2 Q1 CY2010
N7K-M148GS-11L 48 Port 1G Module with SFP 1G Q1 CY2010

Challenges with LAN Extensions
Extensions over any transport
(IP, MPLS, DF)
Failure Boundary Preservation
Site independence / Isolation
Optimal BW utilization
(no head-end replication)
North
Data
CenterFault
Domain
Fault
Domain
LAN Extension
Resiliency/multi-homing
Built-in end-to-end Loop Prevention
Multi-site connectivity (inter and intra DC)
Scalability
VLANs, Sites, MACs
ARP, Broadcasts/Floods
Operations Simplicity
South
Data
Center
Fault
Domain
Fault
Domain
LAN Extension

Traditional Layer 2 VPNs
EoMPLS
VPLS
Dark Fiber

Flooding Behavior
x2
Traditional Layer 2 VPN technologies rely on flooding to propagate MAC
reachability.
The flooding behavior causes failures to propagate to every site in the L2-VPN.
Site A
Site B
Site C
MAC 1
propagationMAC 1
A solution that provides layer 2 connectivity, yet restricts the reach of the
flood domain, is necessary in order to contain failures & preserve resiliency.

Pseudo-wires Maintenance
Before any learning can happen a full mesh of pseudo-wires/tunnels must be in place.
For N sites, there will be N*(N-1)/2 pseudo-wires. Complex to add/remove sites.
Head-end replication for multicast and broadcast Sub-optimal BW utilization.
A simple overlay protocol with built-in functionality and point-to-cloud
provisioning is key to reducing the cost of providing this connectivity

Multi-Homing
Active Active
Requires additional protocols to support Multi-homing.
STP is often extended across the sites of the Layer 2 VPN.
Very difficult to manage as the number of sites grows.
Malfunctions on one site will likely impact all sites on the VPN.
L2 SiteL2 Site L2 VPNL2 VPN
A solution that natively provides automatic detection of multi-homing
without the need to extend the STP domains is key.

What can be improved
Data Plane Learning Control Plane Learning
Moving to a Control Plane protocol that proactively advertises MAC
addresses and their reachability instead of the current flooding
mechanism.
Pseudo-wires and Tunnels Dynamic Encapsulation
No static tunnel or pseudo-wire configuration required.
No static tunnel or pseudo-wire configuration required.
Optimal replication of traffic done closer to the destination, which
translates into much more efficient bandwidth utilization in the core.
Multi-Homing Native Built-in Multi-homing
Ideally a multi-homed solution should allow load balancing of flows
within a single VLAN across the active devices in the same site,
while preserving the independence of the sites.
STP confined within the site (each site with its own STP Root bridge)

Overlay Transport Virtualization
Technology Pillars
OTV is a “MAC in IP” technique for
supporting Layer 2 VPNs
OVER ANY TRANSPORT.
Protocol Learning
Built-in Loop Prevention
Preserve Failure
Boundary
Seamless Site
Addition/Removal
Automated Multi-homing
Dynamic Encapsulation
No Pseudo-Wire State
Maintenance
Optimal Multicast
Replication
Multi-point Connectivity
Point-to-Cloud Model

OTV at a Glance
Ethernet traffic between sites is encapsulated in IP: “MAC in IP”
Dynamic encapsulation based on MAC routing table
No Pseudo-Wire or Tunnel state maintained
Encap Decap
Ethernet Frame IP packetEthernet Frame Ethernet Frame
West
Site
East
Site
OTV OTV
MAC IF
MAC1 Eth1
MAC2 IP B
MAC3 IP B
IP A IP B
MAC IF
MAC1 IP A
MAC2 Eth 1
MAC3 Eth 2
Communication between
MAC1 (West) and MAC2 (East)

MAC TABLE
VLAN MAC IF
100 MAC 1 Eth 2
100 MAC 2 Eth 1
100 MAC 3 IP B
100 MAC 4 IP B
OTV Data Plane: Unicast
OTV Inter-Site Traffic
MAC Table contains
MAC addresses reachable through
IP addresses
Layer 2
Lookup
1
MAC TABLE
VLAN MAC IF
100 MAC 1 IP A
100 MAC 2 IP A
100 MAC 3 Eth 3
100 MAC 4 Eth 4
Layer 2
Lookup
5
Eth 4
Eth 3
100 MAC 4 IP B
MAC 2
MAC 1
Core
MAC 4
MAC 3
OTVOTV
External
IP A
External
IP B
West East
L2 L3 L3 L2
OTVOTV
Encap
2
3 Decap
4 MAC 1 MAC 3
6
100 MAC 4 Eth 4
Eth 1
Eth 2
MAC 1 MAC 3
IP A IP BMAC 1 MAC 3 MAC 1 MAC 3IP A IP BMAC 1 MAC 3
No Pseudo-Wire state is maintained.
The encapsulation is done based on a Layer 2 destination lookup.
The encapsulation is done in hardware by the Forwarding Engine.

Building the MAC tables
The OTV Control Plane
The OTV control plane proactively advertises MAC reachability (control-
plane learning).
The MAC addresses are advertised in the background once OTV has
been configured.
No protocol specific configuration is required.
MAC Addresses
Cor
e
IP A IP B
IP C
West East
South
MAC Addresses
Reachability

OTV Control Plane
MAC address advertisements – Multicast Core
Every time an Edge Device learns a new MAC address, the OTV
control plane will advertise it together with its associated VLAN IDs
and IP next hop.
The IP next hops are the addresses of the Edge Devices through
which these MACs are reachable in the core.
A single update reaches all neighbors.
OTV update is replicated
by the core
Core
IP A
West
East
3 New MACs are
learned on VLAN 100
Vlan 100 MAC A
Vlan 100 MAC B
Vlan 100 MAC C
South-East
VLAN MAC IF
100 MAC A IP A
100 MAC B IP A
100 MAC C IP A
4
by the core
3
3
2
VLAN MAC IF
100 MAC A IP A
100 MAC B IP A
100 MAC C IP A
4
3 New MACs are
learned on VLAN 100
1

Multicast Groups in the Core
OTV will leverage the multicast capabilities of the core.
This is the summary of the Multicast groups used by OTV:
An ASM/Bidir group to exchange MAC reachability.
An SSM group range for the multicast data generated by the site.
Summary of the Multicast groups used by OTV

STP BPDU Handling
When STP is configured at a site, an Edge Device will send and
receive BPDUs on the internal interfaces.
An OTV Edge Device will not originate or forward BPDUs on the
overlay network.
An OTV Edge Device can become (but it is not required to) a root of
one or more spanning trees within the site.
one or more spanning trees within the site.
An OTV Edge Device will take the typical action when receiving
Topology Change Notification (TCNs) messages.
OTVOTV
Core
The BPDUs
stop here

Unknown Unicast Packet Handling
Flooding of unknown unicast over the overlay is not required and is
therefore suppressed.
Any unknown unicasts that reach the OTV edge device will not be
forwarded onto the overlay.
The assumption here is that the end-points connected to the network
are not silent or uni-directional.
MAC addresses for uni-directional host are learnt and advertised by
MAC addresses for uni-directional host are learnt and advertised by
snooping the host’s ARP reply
OTVOTV
Core
No MAC 3 in the
MAC Table
MAC 1 MAC 3
MAC TABLE
VLAN MAC IF
100 MAC 1 Eth1
100 MAC 2 IP B

Controlling ARP traffic
Proxy ARP
OTV Edge Devices can proxy ARP replies on behalf of remote hosts
ARP traffic spanning multiple sites can thus be significantly reduced
An ARP cache is maintained by every OTV edge device
The ARP cache is populated by snooping ARP replies
Initial ARP requests are broadcasted to all sites
Subsequent ARP requests are suppressed and answered locally
The ARP cache could also be populated at MAC learning time, this would
allow the suppression of all ARP related broadcast
Core
OTVOTV
AED
OTVOTV
ARP Cache
MAC 1 IP 1
MAC 2 IP 2
ARP
reply
2
First
ARP
request
(IP A)
1
Snoop &
cache
ARP
reply
3
Subsequent
ARP requests
(IP A)
4
Proxy ARP
reply (IP A)
5
One time traffic

OTV solves Layer 2 fault propagation
Summary
STP Isolation: BPDUs are not forwarded over the
overlay
Unknown unicasts are not flooded across sites
Selective flooding is optional
Selective flooding is optional
Cross site ARP traffic is reduced with Proxy ARP
Broadcast can be controlled based on a white list as
well as a rate limiting profile

Design Example (5)
Layer 2 Link
Layer 3 Link
OTV Virtual Link
WAN
OTV VDC as an appliance at the
Aggregation Layer.
Medium-to-Large Site:
3-Tier Design
Combined L2 and L3 WAN/MAN
PIM from the WAN-core reaching
to the Aggregation Layer.
Access
Agg
Core
OTV Virtual Link
Pod A
… Pod N
DCI DCI DCI DCI
Aggregation Layer.
OTV VDC joins the multicast
core groups at the Agg. Layer.

Design Example (6)
Layer 2 Link
Layer 3 Link
OTV Virtual Link
OTV VDC as an appliance at the
Aggregation Layer.
Leverage the Adjacency Server
Medium-to-Large Site:
Dedicated DCI Connection
Sites directly connected with p2p links
No core hops, no-multicast needed
Access
Agg
OTV Virtual Link
Pod A
… Pod N
DCI DCI DCI DCI
Leverage the Adjacency Server
for discovery

Configuration
OTV CLI configuration
interface Overlay0
description otv-demo
otv join-interface Ethernet1/1
Connects to the core. Used to join the Overlay network.
Its IP address is used as source IP for the OTV encap
ASM/Bidir group in the core used for the
OTV Control Plane.
SSM group range used to carry the site’s
mcast traffic data.
otv join-interface Ethernet1/1
otv control-group 239.1.1.1
otv data-group 232.192.1.2/32
otv extend-vlan 100-150
otv site-vlan 100
Site VLANs being extended by OTV
VLAN used within the Site for communication
between the site’s Edge Devices

Summary
Layer 2 extension requires a lot of effort to build a
robust interconnect between multiple sites
OTV meets those needs whilst preserving site isolation
OTV is easy to configure and manage
OTV is easy to configure and manage
OTV will be available early in Q2 on the Nexus 7000
OTV will require a Transport Services license

Q and A

Partner Technical Connect: Cisco Nexus Updates

Recommended

Recommended

More Related Content

What's hot

What's hot (17)

Similar to Partner Technical Connect: Cisco Nexus Updates

Similar to Partner Technical Connect: Cisco Nexus Updates (20)

More from xKinAnx

More from xKinAnx (20)

Recently uploaded

Recently uploaded (20)

Partner Technical Connect: Cisco Nexus Updates