Go to www.esgjrconsultinginc.com to learn more about Software/Network Engineering Solutions.

1. March 16, 2017
Learning@Cisco
Cisco CCNA Data Center
Data Center Networking Fundamentals

2. Today’s Speakers
Ozden Karakok
Technical Leader from the Data Center Products and Technologies team in the
Technical Assistant Center (TAC). She has been with Cisco Systems for 17 years and
specializes in storage area and data center networks. She is a CCIE R&S, SNA/IP,
and storage. A frequent speaker at Cisco and data center events, Ozden holds a
degree in computer engineering from Istanbul Bogazici University. Currently, she is
focused on application centric infrastructure (ACI) and software-defined storage
(SDS). She is also the lead for Cisco Live Europe Instructor Led and Walk-in Self-
Paced Labs. Connect with Ozden on Twitter via: @okarakok
Matt Saunders
Community Manager for Cisco Learning Network Data Center and Security

3. Agenda
Overview of Cisco Data Center’s
Physical Infrastructure Technology
Data Center Networking Concepts
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Live Q&A
Study Materials and Next Steps

4. Overview of Cisco Data
Center’s Physical
Infrastructure Technology

5. c
Data Centre Infrastructure (3 Layers)
Network
Services
FC
SAN A
FC
SAN B
vPC+
FabricPath
Core
End-of-Row End-of-Row End-of-Row Top-of-Rack UCS FCoE Top-of-Rack
L3
L2
SAN
Director
FC
SAN A
FC
SAN B
Aggregation
& Services
Layer
DC Edge
Layer
(LAN & SAN)
Access Layer
SAN Edge
WAN Edge
Layer

6. “The number of transistors
incorporated into a chip
will approximately double
every 24 months …”
“Moore’s Law” - 1975

7. Moore’s Law
http://en.wikipedia.org/wiki/Semiconductor_device_fabrication
The new generation of Nexus 9000 is
leveraging 16nm FF+ (FinFet)
It’s all about the Economics
• Increased function, efficiency
• Reduced costs, power
• ~ 1.6 x increase in gates between
process nodes
Intel 14nm - 2016
Cisco 16FF+ - 2016
BCOM 28nm - 2016
BCOM 40nm - 2013
Cisco 28nm - 2014

8. SerDes: Serializer + Deserializer
• SerDes Clocking Increases
• 10.3125G (40G, 10G)
• 25.78125(25G/50G/100G) - 2016

9.  40GE/100GE interfaces have multiple lanes (coax cables, fibers, wavelengths)
 MLD provides a simple (common) way to map 40G/100G to physical interfaces of different
lane widths
MLD (Multi Lane Distribution)
Multi Lane Distribution (MLD)

10. Parallel Lanes
4 x10 = 40G shifts to 4 x 25 = 100G
Backed by 10G SerDes Backed by 25G SerDes
100-GbE

11. Metcalfe, Moore and ASIC Pin I/O Rates
The Switch Architectural ChallengeTechnologyImpactsonSwitchDesigns
Pin (I/O) Speed
Capacity from ASIC to
external components
The rate of change for overall network bandwidth is growing
faster than Moore’s Law which in turn is faster than the rate
of change for I/O from the ASIC to off chip components
Pressure from the disparity in rates of change has required a
new architectural balance
Moores’ Law
Transistor Density
Metcalfe's Law
Network Bandwidth
Time - t
Factor
Year 1990 2000 2010 2016
Switch BW 1 67 2,667 30,000
Moore’s Law 1 32 1,024 8,129
DRAM 1 5.6 32 90.5

12. Switching Architecture Changes
Shifting of Internal Architecture
DBUS
RBUS
EOBC
Linecard Linecard Linecard
CROSSBAR
Linecard Linecard
SOC SOC SOC SOC
SOC SOC SOC SOC
Design Shifts Resulting from Increasing Gate Density and Bandwidth
10/100M 100M/1G 1G/10G 10G/100G

13. Switching Architecture Changes
Consolidation of Functions onto fewer components
Design Shifts Resulting from Increasing Gate Density and Bandwidth
Distributed
Forwarding Card
40Gbps Fabric
Channel
40Gbps Fabric
Channel
L2 FWD
L3 FWD
Linecard
FIRE
ASIC
FIRE
ASIC
PO
RT
ASI
C
PO
RT
ASI
C
PO
RT
ASI
C
PO
RT
ASI
C
PO
RT
ASI
C
PO
RT
ASI
C
PO
RT
ASI
C
PO
RT
ASI
C
CT
S
ASI
C
CT
S
ASI
C
CT
S
ASI
C
CT
S
ASI
C
CT
S
ASI
C
CT
S
ASI
C
CT
S
ASI
C
CT
S
ASI
C
FIRE
ASIC
FIRE
ASIC
FABRIC INTERFACE
EoBC
4 X 10G
SOC 1
4 X 10G
SOC 2
4 X 10G
SOC 3
4 X 10G
SOC 4
4 X 10G
SOC 5
4 X 10G
SOC 6
4 X 10G
SOC 7
4 X 10G
SOC 8
4 X 10G
SOC 9
4 X 10G
SOC 10
4 X 10G
SOC 11
4 X 10G
SOC 12
Fabric ASIC
LC
CPU
Arbitration
Aggregator
to ARB
to LC
CPU
LC Inband
32 x 10G
Ports
48 x 10G
Ports
64 x 100G
Ports
ASE-4

14. Switch On Chip (SOC)
It is a full multi-stage switch on an ASIC
SOC ASIC Architecture
Cross Connect
Network
Slice 1 Slice 3Slice 2
Slice N
Central
Statistic
Global
Component
Slice
Component
IO
Component

15. Modular Nexus 9500
A CLOS Based SOC Architecture
SOC SOC SOC SOC
SOC SOC SOC SOC
Non Blocking Leaf and Spine based CLOS Network inside the Switch
Leverage
Switch on Chip
(SOC) Based
components

16. Responding to Fast Market Changes
Sharing Platforms Among Different Architectures
DB DB
Web Web App Web App
VTS
Creation Expansion
Fault MgmtReporting
Connection
• Common hardware platforms for ACI and NX-OS fabric
• Sharing platform with UCS FI
• 3rd Generation FI is based on first gen 9300
• 4th Generation FI will be based on 2nd
Generation 9300EX

17. QSFPSFP
Pluggable Options
• 1G SFP
• 10G SFP+, Twinax, AOC
• 25G SFP+, Twinax, AOC
Pluggable Options
• 1G SFP (via QSA)
• 10G SFP+, Twinax, AOC (via QSA)
• 25G SFP+, Twinax, AOC (via SLIC)
• 40G QSFP, Twinax, AOC
• 50G Twinax, AOC (via SLIC)
• 100G QSFP, Twinax, AOC
Optics Pluggable Multispeed Interfaces
SFP & QSFP

18. MM Fiber Plant
MMF LC
Patch cord
MMF LC
Patch cord
SFP-10G-SR SFP-10G-SR
Used Fiber Pair
QSFP-40G-SR-BD
MM Fiber PlantMMF LC
Patch cord
MMF LC
Patch cord
Used Fiber Pair
Distance up to 125m with OM4
QSFP-40G-SR-BD
MM Fiber Plant
MPO
MPO
Used Fiber Pair
Used Fiber Pair
Used Fiber Pair
Used Fiber Pair
QSFP-40G-SR4 QSFP-40G-SR4
40G BiDi Optics Preserve Existing MM 10G
Cabling

19. Data Center Networking
Concepts
Beyond STP, from Networks to Fabrics

20. • Spanning Tree introduced around 1985,
prevents loops
• 32 years ago, we also saw:
• Windows 1.0
• DNS come out of academia
• First Nintendo Entertainment System
• Successfully deployed for some time
• But since a few years, most DC Designs
built to work around STP
History Lesson: Spanning tree
Host or
Switch

21. Data Center “Fabric” Journey (Standalone)
Layer-2 Layer-2 Layer-2 Layer-2 Layer-2 Layer-2 Layer-2
Hypervisor HypervisorHypervisor HypervisorBaremet al Baremet al Baremet alBaremet al Hypervisor Hypervisor
Spanning-Tree
Layer-3
Layer-2
HSRP HSRP

22. Data Center “Fabric” Journey (Standalone)
Layer-2 Layer-2 Layer-2 Layer-2 Layer-2 Layer-2 Layer-2
Hypervisor HypervisorHypervisor HypervisorBaremet al Baremet al Baremet alBaremet al Hypervisor Hypervisor
Spanning-Tree
Layer-3
Layer-2
HSRP HSRP

23. Data Center “Fabric” Journey (Standalone)
Layer-2 Layer-2 Layer-2 Layer-2 Layer-2 Layer-2 Layer-2
Hypervisor HypervisorHypervisor HypervisorBaremet al Baremet al Baremet alBaremet al Hypervisor Hypervisor
Spanning-Tree
Layer-3
Layer-2
HSRP HSRP

24. Data Center “Fabric” Journey (Standalone)
Layer-2 Layer-2 Layer-2 Layer-2 Layer-2 Layer-2 Layer-2
Hypervisor HypervisorHypervisor HypervisorBaremet al Baremet al Baremet alBaremet al Hypervisor Hypervisor
Spanning-Tree
Layer-3
Layer-2
HSRP HSRP

25. Data Center “Fabric” Journey (Standalone)
Layer-2 Layer-2 Layer-2 Layer-2 Layer-2 Layer-2 Layer-2
Hypervisor HypervisorHypervisor HypervisorBaremet al Baremet al Baremet alBaremet al Hypervisor Hypervisor
Spanning-Tree
Layer-3
Layer-2
HSRP HSRP

26. Data Center “Fabric” Journey (Standalone)
Layer-2 Layer-2 Layer-2 Layer-2 Layer-2 Layer-2 Layer-2
Hypervisor HypervisorHypervisor HypervisorBaremet al Baremet al Baremet alBaremet al Hypervisor Hypervisor
Spanning-Tree
Layer-3
Layer-2
HSRP HSRP

27. • VPC invented to overcome STP limitations
• IEEE standard in 2000 (802.3ad)
• Not perfect, but a good workaround
• STP is still there on every link
• Human error, misconfiguration can still
cause issues
Virtual Port Channel (VPC)
Host or
Switch
vPC Domain

28. • VPC Northbound & Southbound
• More efficient than native STP
• STP is still running
• Another good workaround
• Configuration can become complex as
switch counts grow
• vPC makes two switches look as
one….but what about 4 switches?
Virtual Port Channel (VPC) “Fabric”
Host or
Switch
vPC Domain 1
vPC Domain 2
Dual sided vPC
Back-to-Back vPC
“mini-fabric”

29. L2
L3
• Natural migration from vPC
• MAC in MAC encapsulation
• Easy to turn on (Nexus 5/6/7K)
• No STP within Fabric; BPDUs don’t
traverse fabric
• Distributed L3 gateway to edge, VLAN
anywhere notion
• TRILL=Standard based, limited
capabilities
• FabricPath = Cisco proprietary features
FP
MAN/WAN Border
Multi-path Fabric Based Designs - FabricPath

30. The Leaf / Spine Topology (Clos* Network)
• Wide ECMP: Unicast or Multicast
• Uniform Reachability
• Deterministic Latency
• High Redundancy
• On Node or Link Failure
SpineSpine Spine Spine
Leaf LeafLeaf LeafLeaf Leaf Leaf
*Clos, Charles (1953) "A study of non-blocking switching networks"

31. SpineSpine Spine Spine
Leaf LeafLeaf LeafLeaf Leaf Leaf
A Scale Out Architecture
• Leaf
• Smallest Operational Entity
• Spines
• Wide vs. Big
• Uplinks
• Symmetric to all Spines or Pods
• SAYG: Scale as You Grow
More Spine – More Bandwidth – More Resiliency
More Leaf – More Ports – More Capacity

32. The Super-Spine
SpineSpine Spine Spine
Leaf LeafLeaf LeafLeaf Leaf Leaf
POD 2
SpineSpine Spine Spine
Leaf LeafLeaf LeafLeaf Leaf Leaf
POD 1
SuperSpine SuperSpine
SuperSpine

33. Data Center Fabric Properties
SpineSpine Spine Spine
Leaf LeafLeaf LeafLeaf Leaf Leaf
• Any Subnet, Anywhere, Rapidly
• Any Network on Any Leaf
• Reduced Failure Domain
• Any Default Gateway on Any Leaf
- Distributed
• Extensible Scale and Resiliency

34. Overlay Based Data Center: Fabrics
• Mobility
• Segmentation
• Scale
• Automated and Programmable
• Abstracted Consumption Model
• Layer-2 and Layer-3 Service
• Physical and Virtual Workloads
SpineSpine Spine Spine
Overlay
VTEP VTEPVTEP VTEPVTEP VTEP VTEP

35. • Router/Switch End-Points
• Protocols for Resiliency/Loops
• Traditional VPNs
• VXLAN, OTV, VPLS, LISP, FP
BRKDCT-3378
Overlay Based Data Center: Edge Devices
Overlay
VTEP VTEP
Baremetal Baremetal
VTEP VTEP
Baremetal Baremetal
Network Overlays
• Virtual End-Points only
• Single Admin Domain
• VXLAN, NVGRE, STT
Overlay
- -
Host Overlays
-
Hypervisor
VTEP
-
Hypervisor
VTEP
Hypervisor
VTEP
Hypervisor
VTEP
• Physical and Virtual
• Resiliency and Scale
• Cross-Organizations/Federation
• Open Standards
Overlay
VTEP VTEP
Baremetal Baremetal
Hybrid Overlays
-
Hypervisor
VTEP
-
Hypervisor
VTEP

36. Overlay Taxonomy - Underlay
Underlay
SpineSpine Spine Spine
Leaf LeafLeaf LeafLeaf Leaf Leaf
HypervisorHypervisor HypervisorHypervisor BaremetalBaremetal BaremetalBaremetal
Virtual
Server Physical
Server
Edge Device
Layer-3
Interface
Peering
LAN
Segment

37. Overlay Taxonomy - Overlay
SpineSpine Spine Spine
Overlay
VTEP VTEPVTEP VTEPVTEP VTEP VTEP
HypervisorHypervisor HypervisorHypervisor BaremetalBaremetal BaremetalBaremetal
Virtual
Server Physical
Server
LAN
Segment
VTEP
VTEP: VXLAN Tunnel End-Point
VNI/VNID: VXLAN Network Identifier

38. Overlay Taxonomy - Overlay
SpineSpine Spine Spine
Overlay
VTEP VTEPVTEP VTEPVTEP VTEP VTEP
HypervisorHypervisor HypervisorHypervisor BaremetalBaremetal BaremetalBaremetal
Tunnel Encapsulation
(VNI Namespace)
Virtual
Server Physical
Server
LAN
Segment
VTEP
VTEP: VXLAN Tunnel End-Point
VNI/VNID: VXLAN Network Identifier

39. Application Centric Infrastructure Components
External
Network
App DBWeb
QoS
Filter
QoS
Service
QoS
Filter
ACI Fabric
Consider the Interaction between the endpoints
Non-Blocking Penalty Free Overlay
APIC
APIC
APIC

40. Enter Stateless Application Policies
EPG App EPG DBEPG Web
Application
Profile
There is stateless filtering between End Point Groups (EPGs) that may be
able to eliminate the need for some firewalls within the datacenter. Contracts
define what an EPG exposes to other app tiers and how. In other words,
any communication not explicitly allowed, is denied.
End Points
Single or Device Groups
Virtual / Physical
Single/Multiple Subnets
Health Monitoring
Network & Security
Quality of Service (QoS)
Contracts & Filters (TCP/UDP)
Redirection
SPAN & Monitoring
L4 – L7 Services
Firewalls
Load Balancers
Orchestration & Management
Network Analysis
QoS
Service
Filter
QoS
Service
Filter
QoS
Service
Filter

41. Programmable NetworkProgrammable Fabric
Application Centric
Infrastructure
DB DB
Web Web App Web App
VxLAN-BGP EVPN
standard-based
3rd party controller support
Modern NX-OS with enhanced NX-
APIs
DevOps toolset used for Network
Management
(Puppet, Chef, Ansible etc.)
Turnkey integrated solution with security,
centralized management, compliance and
scale
Automated application centric-policy
model with embedded security
Broad and deep ecosystem
Cisco Data Centre Networking Strategy:
Providing Choice in Automation and Programmability
Cisco Controller for software overlay
provisioning and management
across N2K-N9K
VTS
Creation Expansion
Fault MgmtReporting
Connection

42. Certification and Training
Resources

43. Get Started Today
• Join the Cisco Learning Network Data Center
community
• Pick your preferred method of training:
• Instructor-led training: DCICN and DCICT
• CCNA Data Center Official Cert Guides
• Cisco Learning Network certification
resources (see slide 23)
• Get certified

44. Cisco Press CCNA Data Center Official
Certification Guides
Launch special: Save 35% (plus, free U.S. Shipping)
CISCOPRESS.COM | Code: CCNADC35
See CISCOPRESS.COM for the latest specials

45. CCNA Data Center Training Courses
• Instructor-led training
• DCICN and DCICT
• Extensive hands-on learning: configuration, usage
• Taught by certified Cisco Learning Partners specializing in data center
• Good option for focused learning with instructor expertise
Acronym Version Course Name
DCICN 6.0 Introducing Cisco Data Center Networking
DCICT 6.0 Introducing Cisco Data Center Technologies

47. Q&A Session…