Creating the Network at Cisco Live Orlando: Technical Case Study TechAdvantage Webinar

2,098 views
1,905 views

Published on

Have you ever wondered what is involved building and tearing down a network in the span of a week? We will give you an inside view into the Cisco Live Orlando event network, the technologies used, and the Network Operations Center or NOC that managed it.

The Cisco Live network is one of the most critical elements of the conference. It requires hundreds of access switches (wired), access points (wireless), and provides network services like load balancing, IPv6, and network monitoring to meet the constant needs of attendees.

Cisco engineers and experts Joe Clarke, Jason Davis, Ryan S D'Souza, and Patrick Warichet will share their area of expertise and role they played from the overall architecture and design to routing, switching, wireless, network management and automation.

In addition, we will cover what worked, best practices followed and statistics from the event. You will benefit from hearing what we did, how we did it and what we learned, all over the course of the week!
Download the WebEx Replay at: https://cisco.webex.com/ciscosales/lsr.php?AT=pb&SP=EC&rID=72318727&rKey=349eabd73f199d04

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,098
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
44
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Creating the Network at Cisco Live Orlando: Technical Case Study TechAdvantage Webinar

  1. 1. Cisco TechAdvantage Webinars Creating the Network at Cisco Live Orlando: Technical Case Study Patrick Warichet: Core Design and Switching Backbone Ryan D’Souza: Wireless Design Joe Clarke: Network Management Jason Davis: Network Management and Automation We’ll get started a few minutes past the top of the hour. Note: you may not hear any audio until we get started.
  2. 2. Housekeeping •  Submit questions in Q&A panel and send to “All Panelists” Avoid CHAT window for better access to panelists •  For WebEx audio, select COMMUNICATE > Join Audio Broadcast •  For WebEx call back, click ALLOW phone button at the bottom of participants side panel •  Where can I get the presentation? Or send email to: ask_techadvantage@cisco.com •  Please complete the post-event survey •  Join us for upcoming TechAdvantage Webinars: www.cisco.com/go/techadvantage PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public
  3. 3. Introductions Patrick Warichet Ryan D’Souza Joe Clarke Jason Davis Technical Marketing Engineer pwariche@cisco.com Network Consulting Engineer rydsouza@cisco.com Twitter: @ryanstevedsouza Distinguished Services Engineer jclarke@cisco.com Distinguished Services Engineer jadavis@cisco.com Twitter: @SNMPguy PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public
  4. 4. The CiscoLive! Network Our Team PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
  5. 5. CiscoLive! Network – Core / Data Center Design Orlando, June 23 - 27, 2013 Patrick Warichet, CCIE #14218 Technical Marketing Engineer – Network Operating Systems Technology Group
  6. 6. Backbone Architecture IPv4 Address Range - 63.231.216.0/21 IPv6 Address Range – 2001:428:CE01::/48 Higher Weight for all Local addresses Orlando AS 64726 63.231.216/21 2001:428:CE01::/48 Sunnyvale Denver Bandwidth Provided by CenturyLink AS 209 PNLNMS-3000 AS 209 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
  7. 7. BGP Route Validation with SoBGP RPKI Validator Cache Location: Switzerland 4 RIPE based servers were installed RPKI IPv4 an Route Colo Sunnyvale d IPv6 r-Serv er Pro tocol ASR1k Orlando Cisco Live 2013 Colo Denver Cat6509 Quad Sup SSO ASR1k PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
  8. 8. SoBGP Prefix verification IPv4 and IPv6 JSV-ASR#show bgp ipv4 unicast summary BGP router identifier 66.77.8.142, local AS number 64726 Path RPKI states:5268 valid, 1742953 not found, 7516 invalid <SNIP> BGP activity 706942/244807 prefixes, 5078283/3255957 paths, scan interval 60 secs Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 63.231.216.9 4 64726 65062 65067 7695624 0 0 3d20h 3 66.77.8.129 4 209 702344 17230 7695607 0 0 1w3d 449040 66.77.8.130 4 209 706159 17211 7695607 0 0 1w3d 449039 66.77.8.150 4 64726 697436 750500 7695624 0 0 1w1d 3 JSV-ASR#show bgp ipv6 unicast summary BGP router identifier 66.77.8.142, local AS number 64726 Path RPKI states: 1370 valid, 24658 not found, 130 invalid <SNIP> BGP activity 707057/244951 prefixes, 5079057/3256907 paths, scan interval 60 secs Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 2001:428:CE00:1:66:77:8:129 4 209 108934 17232 115745 0 0 1w3d 13078 2001:428:CE00:1:66:77:8:130 4 209 109133 17229 115745 0 0 1w3d 13078 2001:428:CE01:0:63:231:216:9 4 64726 65155 98282 115746 0 0 3d20h 1 PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 8 8
  9. 9. Sun Mon Tue Wed Thu Internet Statistics Link To Denver Colo 940 Mbps Traffic Link To Sunnyvale Colo 615 Mbps Traffic PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
  10. 10. IPv4 Traffic 220.2K PPS Link To Denver Colo 74.1K PPS Link To Sunnyvale Colo PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
  11. 11. IPv6 Traffic 12.7K PPS Approximately 8 % of the traffic was IPv6 PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
  12. 12. Multi-Hop FCOE UCS-B with Director Class FCOE Nexus 7000 §  UCS release (2.1.1a) provides support for multi-hop FCOE capability –  “FCOE uplink port” supported on both FI 6100 and 6200 (native ports + extension module ports for both models) §  Nexus 7000 is a Director Class FCOE platform: supports FCOE with F1 (NX-OS 5.2), F2 (NX-OS 6.1.1) and F2E (NX-OS 6.1.2) series line cards: –  F1 with SUP1 or SUP2/SUP2E –  F2/F2E with SUP2 or SUP2E (SUP1 not supported in this case) –  ETH or FCOE ports on F1/F2/F2E (no FC port) PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
  13. 13. FCoE with N7004 and UCS-B – Problem Statement Cisco Live 2012 San Diego Design Netapp-A Cisco Live 2013 Orlando Design Netapp-A Netapp-B Unified FCoE in VPC Unified FCoE in VPC N5K-1 Netapp-B N5K-2 N7K-1 N7K-2 IP traffic in VPC FCoE traffic in a Port-Channel Design Can’t work! VPC requires PortChannel interfaces in a Port-Channel that cannot be shared IP traffic in VPC FCoE traffic in a Port-Channel PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
  14. 14. FCoE with N7000 and UCS-B – Design Solution Create a storage VDC on Nexus 7000 Storage VDC = SAN VDC allocated to run as a virtual MDS FC switch Separate Storage and IP Networking completely Unified FCoE in VPC Storage VDC Storage VDC IP traffic in VPC FCoE traffic in a Port-Channel Downside: Not Unified Extra cabling PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
  15. 15. CiscoLive! Network – Switching Backbone Orlando, June 23 - 27, 2013
  16. 16. Network Backbone at Previous Events Routing, Switching and Services in the Cisco Live Network : design and topology has evolved through various events 2011 2013 InteropNet, Las Vegas CiscoLive!, London CiscoLive!, Orlando CiscoLive!, Las Vegas CiscoLive!, San Diego InteropNet, New York PNLNMS-3000 2012 CiscoLive!, Mexico © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
  17. 17. Catalyst 6500 Equipment Inventory - Core §  2 x 6509-E §  4 x Supervisor 2T §  VS-SUP2T-10G with VS-F6K-PFC4XL §  40 Gigabit Ethernet §  2 x WS-X6904-40G §  10 Gigabit Ethernet §  2 x WS-X6908-10G §  1 Gigabit Ethernet Fiber §  2 x WS-X6848-SFP PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
  18. 18. Catalyst 6500 Equipment Inventory - Distribution §  2 x 6513-E and 2 x 6504-E §  4 x Supervisor 2T §  VS-SUP2T-10G with VS-F6K-PFC4 §  40 Gigabit Ethernet §  4 x WS-X6904-40G §  10 Gigabit Ethernet §  6 x WS-X6908-10G §  1 Gigabit Ethernet Fiber §  2 x WS-X6848-SFP §  1 Gigabit Ethernet Copper §  2 x WS-X6848-GE-TX PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
  19. 19. Staging PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
  20. 20. Cisco Live Orlando 2013 Topology 18 – 3750-X Switches OCCC – SmartCity Distribution Meeting Rooms Event Center Floors 1,2,3,4 75 - 3560CG Switches 177 – 3502P Access Points Halls A - D OCCC IDF 12 – 3750G Switches OCCC – World of Solutions 110 - 3560CG Switches 150 – 3602E Access Points OCCC IDF Floors 1,2,3,4 OCCC – Cisco Live Distribution Event Center 55 - 3560CG Switches 125 – 3602E Access Points OCCC IDF Peabody IDF Meeting Rooms Peabody Hotel Orlando 12 – 3750-X Switches - 3560CG Switches 130 - 3602E Access Points Floors 1,2,3,4 35 - 3560-E Switches Wireless Control (10 x 5508) 4510R+E (Peabody) DISTRIBUTION 6504-E SUP2T VSS 6513-E SUP2T VSS 6513-E (SmartCity) 2 x NetApp DS2246 2 x NetApp FAS3240 4 x 40GE 2 x UCS 5108 Service Provider OCCC MDF Peabody MDF 2 x ASA 5585-X NAT64 CORE 2 x Nexus 7004 ASA 5585-X Internet Firewall San Jose / Cisco CoLo (CenturyLink) 2 x ASA 5585-X Data Center Firewall Denver / Cisco CoLo (CenturyLink) ASR 1006 PNLNMS-3000 2 x UCS 6248UP 6509-E SUP2T QUAD SUP VSS © 2013 Cisco and/or its affiliates. All rights reserved. ASR 1006 Cisco Public 21
  21. 21. Core & Distribution - Features and Highlights §  VSS Quad-Sup SSO and Multi-chassis Etherchannel (10 / 40 GE links) §  Routing protocols - OSPF and BGP for IPv4 and IPv6 §  Network Management - SNMPv3, Control Plane Policing, Syslog, etc. §  Flexible NetFlow on Sup2T for IPv4 & IPv6 traffic §  Multicast 1 Terabyte of Multicast traffic during the show Vlan361 is up, line protocol is up   <SNIP>   L3 in Switched: ucast: 0 pkt, 0 bytes - mcast: 741428621 pkt, 1009825781802 bytes L3 out Switched: ucast: 5 pkt, 590 bytes - mcast: 0 pkt, 0 bytes 741429788 packets input, 1006860549402 bytes, 0 no buffer PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
  22. 22. 40 Gigabit Ethernet with a 160GE Backbone §  Multi-chassis Etherchannel between Core and Distribution VSS §  WS-X6904-40G-2T / XL in the Network Snapshot from ‘show version’ on the Core switch: 4 Virtual Ethernet interfaces 108 Gigabit Ethernet interfaces 116 Ten Gigabit Ethernet interfaces 4 Forty Gigabit Ethernet interfaces DIST DIST 6509E SUP2T VSS 10GE / 40GE 6509E SUP2T VSS 10GE 4x40GE 4x10GE CORE SWITCH 6509E SUP2T-XL QUAD SUP VSS §  Split-brain mode on Distribution switch (10GE/40GE mixed mode) 40GE ports to the Core, and 10GE to the Access on the same module hw-module switch 1 slot 1 operation-mode port-group 2 TenGigabitEthernet   hw-module switch 2 slot 1 operation-mode port-group 2 TenGigabitEthernet   PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
  23. 23. Core – Catalyst 6509-E Quad Supervisor VSS Support Sup2T YES Sup720 NO 10GE Si Si LACP or PagP Si Si LACP Access Switch or ToR or Blades Monitoring Server Simplified Network Design 10GE LACP or PagP LACP Access Switch or ToR or Blades SSO Sync SSO Sync SSO Sync VSS Quad SUP SSO Double Bandwidth Utilization •  Spanning Tree and first-hop redundancy •  With active-active multi-chassis protocols eliminated •  Single touchpoint manageability •  1+1 Supervisor redundancy for EtherChannel (LACP/ PagP) dual-attached devices SSO Sync 15.1(1)SY1 Traditional VSS Monitoring Server Deterministic and Automated Recovery •  Maximize network throughput with VSS quad Sup SSO •  1:1 (active/standby) supervisor redundancy for single and dual attached devices PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
  24. 24. VSS Quad-Sup SSO Redundancy Domains In Chassis Redundancy Domain VSS Active In Chassis Redundancy Domain VSS Domain Default Redundancy Domain VSS Standby SSO In Chassis Active (ICA) In Chassis Active (ICA) SSO SSO In Chassis Standby (ICS) In Chassis Standby (ICS) PNLNMS-3000 VS Switch ID 1 © 2013 Cisco and/or its affiliates. All rights reserved. VS Switch ID 2 Cisco Public
  25. 25. VSS Quad-Sup SSO Redundancy Domains In Chassis Redundancy Domain In Chassis Active (ICA) In Chassis Redundancy Domain VSS Domain Default Redundancy Domain VSS Active (Active) SSO VSS Standby Hot (Switchover Target) In Chassis Active (ICA) SSO SSO In Chassis Standby (ICS) (Standby Hot Chassis) In Chassis Standby (ICS) (Standby Hot Chassis) PNLNMS-3000 VS Switch ID 1 © 2013 Cisco and/or its affiliates. All rights reserved. VS Switch ID 2 Cisco Public
  26. 26. IPv4 and IPv6 Statistics Sun Mon Tue Wed Thu 15.7K Unique IPv4 Mac Address 7.81K Unique IPv6 Mac Addresses PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
  27. 27. Goodies Catalyst 6500 Twitter Script Tweet information from IOS using EEM and Twitter's API Follow it CiscoLive NOC @CiscoLive2013 Download the code from https:// supportforums.cisco.com/ docs/DOC-19363 PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
  28. 28. CiscoLive! Network – Wireless Design Orlando, June 23 - 27, 2013 Ryan D’Souza Network Consulting Engineer – Mobility and Wireless Practice, Advanced Services
  29. 29. Key Statistics •  Convention Coverage Area: ~ 2.5 million sq.ft •  Most Total Attendance ever : 20,000+ registered attendees •  Highest Total unique associations: ~19,500 •  Largest simultaneous associations ever for Cisco Live: ~13,100 •  ~5500 concurrent associations during keynote •  Highest ever recorded Aggregate Wi-Fi traffic: ~3.0 TB •  Peak Wi-Fi throughput:~900 Mbps in, ~200 Mbps out •  Client Distribution by Band: 43.7% 5 GHz (11n) 53.5% 2.4 GHz (11n) PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
  30. 30. CiscoLive! Wireless Network •  10 x 5508 Controllers to manage a total of 673 Access Points across the venue •  180 x 3502P Access Points with Air-ANT25137NP-R stadium antennas to cover Attendee Dining, Keynote and World of Solutions. •  300 x 3602 Access Points with AIR-ANT2566P4W-R to cover the hallways and session rooms in OCCC PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public
  31. 31. CiscoLive! Wireless Network •  4 x MSE 7.5 for Location Analytics, Ekahau RFID tags and show statistics •  1 x CPI 1.4 to manage the 7.5 MSE •  110 x 3602 APs to cover the hallways and session rooms in Peabody •  83 x in house APs were used for some session room coverage in OCCC PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public
  32. 32. Client Statistics PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public
  33. 33. PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public
  34. 34. NOC Network Management Orlando, June 23 - 27, 2013 Joe Clarke, CCIE #5384 Distinguished Services Engineer – Global Technical Center
  35. 35. Built For Management §  All devices managed with SNMPv3 authPriv (SHA / DES) §  Prime NAM used to traffic volume and utilization §  Flexible NetFlow sent from the 6500 core and distribution switches to FreeBSD VM “exploder” which forwarded to other collectors (Prime Infrastructure, Prime NAM, StealthWatch, and Plixer [WoS]) §  Syslog sent from all devices to FreeBSD then forwarded to interested parties Netflow Syslog PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public
  36. 36. Cisco Prime LMS - Provisioning PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
  37. 37. Embedded Automation Using EEM To Set Port Descriptions EEM dynamically sets descriptions on ports based on last connected CDP neighbor. Applet available from https://supportforums.cisco.com/ docs/DOC-24529 PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
  38. 38. Embedded Automation Converting Access Switches From Stage to Production Switch from stage to production VLAN Ping production default GW, and verify reachability If reachable, reconfigure switch for production… …If not, then revert to stage. PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
  39. 39. Traffic Breakdown Courtesy of Cisco Prime Network Analysis Module Video Streaming Wins! PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public
  40. 40. NOC Network Management and Automation Orlando, June 23 - 27, 2013 Jason Davis Distinguished Services Engineer – Advanced Services
  41. 41. Intelligent Automation Use at CiscoLive! NOC Cisco Service Portal – Cisco Process Orchestrator – Cisco Prime PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
  42. 42. Intelligent Automation to Monitor Network Device and Server Reachability PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
  43. 43. Prime Infrastructure and NCCM Monitoring for Faults and Config PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
  44. 44. 1 Regional 1 Authoritative Cisco Prime Network Registrar 2 DHCP DNS / DHCP Statistics – Monday 3pm to Thursday Noon 2 cDNS 2 DNS Secondaries Max Active DHCP Leases 33959 @ Wed 14:50 Total – 690M requests Total – 122M DNS lookups PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 45
  45. 45. How Reliable Was It? How Many Nines Are Enough? PNLNMS-3000 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public
  46. 46. What Will You Connect?
  47. 47. •  Thank you! •  Please complete the post-event survey •  Join us for upcoming webinars: Register: www.cisco.com/go/techadvantage Follow us PNLNMS-3000 @GetYourBuildOn © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

×