The document discusses successfully interconnecting data centers. It outlines some of the challenges with using layer 2 interconnects including spanning tree issues that can cause network flooding and split brains. Layer 3 interconnects require IP readdressing. The document recommends using a virtual data center approach with dual redundant physically diverse paths to avoid shared fate domains. It provides best practices for avoiding spanning tree, containing fault domains, ensuring symmetric routing, optimal routing, latency for synchronization, and using jumbo frames.

2. Successfully Interconnecting
Data Centers
Ronnie Scott
Technology Solutions Architect
Sept 2014

3. Cisco Confidential 4© 2013-2014 Cisco and/or its affiliates. All rights reserved.
 The Intent
 The Reality
 Delivering On The Promise
Agenda

4. Cisco Confidential 5© 2013-2014 Cisco and/or its affiliates. All rights reserved.

5. Cisco Confidential 6© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Data Center BData Center A
Layer 2 vs Layer 3 Interconnects
Layer 3
Subnet A
10.0.1.0
Subnet B
10.0.2.0
IP
Readdressing
Required

6. Cisco Confidential 7© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Virtual Data Center
Layer 2 vs Layer 3 Interconnects
Layer 2
Subnet A
10.0.1.0
Subnet A
10.0.1.0
NO IP
Readdressing
Required
Tunnel

7. Cisco Confidential 8© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Layer 1, 2, or 3 Transport
• Dark Fiber
• Lambda (CWDM / DWDM)
Layer 1 Transport
8
• Cost
• Distance
• Optics
• Administration

8. Cisco Confidential 9© 2013-2014 Cisco and/or its affiliates. All rights reserved.
• MPLS (EoMPLS)
• Metro Ethernet
• Wireless
Layer 1, 2, or 3 Transport
Layer 2 Transport
9
• Latency
• Bandwidth
• Frame Size
• Spanning Tree

9. Cisco Confidential 10© 2013-2014 Cisco and/or its affiliates. All rights reserved.
• Any WAN Service
Layer 1, 2, or 3 Transport
Layer 3 Transport
10
• Latency
• Bandwidth
• Frame Size
• Tunneling

10. Cisco Confidential 11© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Virtual Data Center
Multiple Site Meshes
11

11. Cisco Confidential 12© 2013-2014 Cisco and/or its affiliates. All rights reserved.

12. Cisco Confidential 13© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Spanning Tree
• Loop Flooding
• TCN Flooding
• Broadcast/ARP Flooding
• Slow Convergence Time
• Interface Blocking
13

13. Cisco Confidential 14© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Virtual Data Center
Spanning Tree Example
Network Flooding
14

14. Cisco Confidential 15© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Split Brain
• Potential Data Corruption
• Incorrect DR Take Over
• Loss Of Network State
• Loss Of Application Tiers
15

15. Cisco Confidential 16© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Virtual Data Center
Split Brain Example
Data Inconsistency
16
Branch 1 Branch 2

16. vPC
DC Edge
Internal DC Zone(s)
DC Core VDC (Routed)
DC Aggregation Layer VDC
Compute Access Layer
FW
CLUSTER
Entanglement
HOT vMotion!
in
in
out

17. Cisco Confidential 18© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Asymmetric Routing
• Traffic Management
• Network Service State Issues
• Troubleshooting
• Latency/Performance
18

18. Cisco Confidential 19© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Virtual Data Center
Assymetric Routing Example
Loss Of State
19
Branch

19. 20

20. 21

21. Cisco Confidential 22© 2013-2014 Cisco and/or its affiliates. All rights reserved.

22. 23

23. Cisco Confidential 24© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Virtual Data Center
• Physically Diverse Paths
• Physically Separate Devices
• No Shared Fate Domain
Best Practice
Dual Redundant and Diverse Paths
24

24. Cisco Confidential 25© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Virtual Data Center
• VPC
• OTV
• VXLAN
Best Practice
Avoid Spanning Tree
25
• FabricPath
• NVGRE
• MPLS

25. Cisco Confidential 26© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Virtual Data Center
• Broadcast Suppression
• ARP Filtering
Best Practice
Contain Fault Domain
26
• L2 Routing (ISIS)
• Gateway Management

26. Cisco Confidential 27© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Symmetric Routing - Inbound
• Explicit Routes
• NAT / Gateway Devices
• LISP
• DNS
• IP Readdressing
2727

27. Cisco Confidential 28© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Symmetric Routing - Outbound
28
• Weighted Routing
• Default Gateway Prioritization
• NAT / Gateway Devices

28. Cisco Confidential 29© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Optimal Routing
• GSLB
• SLB Clustering
• FW Clustering
• LISP
2929

29. Cisco Confidential 30© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Virtual Data Center
Latency
Ensuring Synchronization Completion
30
• Storage Replication
• Application Synchronization
• Virtual Machine Migration

30. Cisco Confidential 31© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Virtual Data Center
Jumbo Frames
Ensuring Packet Delivery
31
• No PMTUD
• No “Could Not Fragment”
• Dramatic IP Performance Degadation
101100110…

31. www.cisco.com/go/vmdc
32

32. Cisco Confidential 33© 2013-2014 Cisco and/or its affiliates. All rights reserved.

33. Cisco Confidential 34© 2013-2014 Cisco and/or its affiliates. All rights reserved.

34. Cisco Confidential 35© 2013-2014 Cisco and/or its affiliates. All rights reserved.
35

35. Cisco Confidential 36© 2013-2014 Cisco and/or its affiliates. All rights reserved.
36

36. Cisco Confidential 37© 2013-2014 Cisco and/or its affiliates. All rights reserved.
37

37. Thank you.