Your SlideShare is downloading. ×
0
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Networking is NOT Free: Lessons in Network Design
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Networking is NOT Free: Lessons in Network Design

5,001

Published on

An in-depth critique of the existing OpenStack networking approach, with a focus on how the Nova network controller is more of a hindrance than a help. Discusses the gap in Quantum's functionality …

An in-depth critique of the existing OpenStack networking approach, with a focus on how the Nova network controller is more of a hindrance than a help. Discusses the gap in Quantum's functionality required to close the gap, and alternative solutions. How can we make networking in OpenStack robust, high performance, and fault tolerant? What do typical large scale networks look like and what lessons can we learn from them? Is there an approach to networking we can take that is the same with a handful of servers as it is with hundreds of racks?

Published in: Technology
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,001
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
93
Comments
0
Likes
7
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Networking is NOT Free:Lessons In Network Design Dan Sneddon Member Technical Staff Twitter: @dxs Download: http://engineering.cloudscaling.com/portland13 CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution* * All unlicensed or borrowed works retain their original licenses
  • 2. Presenter Bio• 20 years of network engineering and systems design• Lead Global Network Engineer for Apple• Network Security Architect for SLAC National Laboratory• IT Architect for division of Schneider Electric• Financial sector networking (banks and trading floors)• Major startups, including Twitter Dan Sneddon Member Technical Staff Twitter: @dxs 2
  • 3. Our Journey Today1. Datacenter Networking: Historical Perspective2. Rise and Fall Of The VLANs3. Networking At Cloud Scale4. OpenStack Networking Models5. Room For Improvement In OpenStack Networking 3
  • 4. Datacenter Networks:Historical Perspective
  • 5. Datacenter Networking Timeline 198 199 200 201 0s 0s 0s 0+•Client/Server •100 Mb •1 Gb+ Servers •10 Gb+ Servers•10 Mb Ethernet •Switched Ethernet •10 Gb Uplinks •40/100 Gb Uplinks•Token Ring •Bonded Interfaces •VLANs •Virtual Networks•Serial Cables •Spanning-Tree •Virtual Machines •SDN 5
  • 6. 1980’s: Shared Media and Serial Token Ring 10 M Hub 10M Hub Serial Link User 6
  • 7. 1990‘s: 100 Megabits Switched! Database User Switch User User 7
  • 8. 2000’s: Rise Of the Gigabit VLANs! Database VLAN 10 Server VLAN VLAN 20 Administration VLAN 30 Accounting Etc... Everyone Else 8
  • 9. 2010’s: Everything Gets Simple!User 9
  • 10. Rise And FallOf the VLANS
  • 11. Datacenter VLAN Segregation Layer 2/3 Boundary VLAN 10 VLAN 20 VLAN 30 11
  • 12. VLAN Physical Separation 12
  • 13. VLAN Pros and ConsPros:• Provide a level of isolation• Reduction in size of broadcast domain• Manageable, up to a certain size (especially with VTP, etc)Cons:• Each VLAN can only reach other VLANs through routers• Spanning-tree (when it breaks, everything breaks)• 4096 VLAN limit--assigning in blocks uses this up faster 13
  • 14. Death Of the VLANsVLANs Only Scale So Far• In the late 2000’s, high-density (1U) servers become standard• There is no way to make spanned VLANs work for many thousands of servers• A new model takes over: small layer 2 domains with layer 3 routing 14
  • 15. Breaking Through The Scale BarrierVLANs Only Scale So Far VLAN Locally, Route Globally 15
  • 16. Hierarchical Internetworking ModelScale-Out Networking Core Distribution Access Hosts 16
  • 17. Networking At the Scale of Cloud
  • 18. Two Cloud Infrastructure Models 1 2 Enterprise Elastic Virtualization Infrastructure New Legacy Apps Dynamic Apps 18
  • 19. Elastic Cloud vs.Enterprise Virtualization Enterprise Virtualization Elastic Cloud Applications Traditional & Legacy Dynamic Scaling Architecture Managed Silos Horizontal Technology Stack Heavy & Proprietary Distributed & Open Price/Performance Low High (4-7x better) Failure Domains Large Small Provisioning Slower & Manual Faster & 100% API Server consolidation and lower On-demand, scale-out Best For: datacenter mgmt costs infrastructure for new apps 19
  • 20. Nova-NetworkClassic OpenStack Networking, With That Old-Timey Feel FlatDHCP4 Modes: Flat Flat DHCP VlanManager Multi-host HA• Flat/Flat DHCP only support a single VLAN for everything• VlanManager is the most feature-rich for multi-tenant• VlanManager requires trunking all VLANs down to each host• In a public cloud, max of 4096 VLANs limits tenants 20
  • 21. OCS Nova-Networking L3 PluginCloudscaling Exclusive Solution• Layer 3 networking for VMs, with DHCP and NAT service• Each VM is on its own Linux bridge, no shared layer 2• Quantum not required• DHCP service is local to each compute host• AWS-like: floating IPs, elastic netblocks, and now VPC 21
  • 22. Brokerless Messaging With ZeroMQAvoiding RabbitMQ’s Single Point Of Failure Nova-Compute Nova-Compute Single Point Of Failure RabbitMQ Broker Nova-Scheduler Nova-API Nova-Scheduler Nova-API RabbitMQ vs. ZeroMQ (Brokered) (Peer To Peer) 22
  • 23. OpenStack Networking
  • 24. OpenStack NetworkingAPIs For All Your Networking Things• “Quantum” is now known as “OpenStack Networking”• Pluggable architecture, with APIs for all network functionality• Basic L3 plugin (finally!), but designed for L3 on flat L2 network• nova-network process still performs some very basic functions• Some plugins are more complete/stable than others 24
  • 25. OpenStack Networking Horizon REST over HTTP(S) REST Nova Quantum REST (Quantum API Service Plugin) RPC DHCP Agent Ceilometer OPENSTACK Notifi- NETWORK cations SERVICE Virtual Keystone Network REST Plugin Quantum Agent(s) SDN Provider Varies Solution Network Plugin Varies Physical Varies Hardware SQL Hypervisor Quantum DB compute node 25
  • 26. OpenStack Networking Modes• VLAN networks are supported using provider network plugins• Layer 3 plugin• GRE tunnel support using virtual network plugins• May be used with Linux Namespaces to isolate tenants from one another within a hypervisor• Many commercial vendor plugins 26
  • 27. Quantum CompatibilityLots Of Choices For Virtual Network/SDN Providers•Open vSwitch. http://www.openvswitch.org/openstack/documentation•Nicira NVP. quantum/plugins/nicira/nicira_nvp_plugin/README and http://www.nicira.com/support.•Midokura. http://www.midokura.com/midonet/openstack/•BigSwitch. http://www.bigswitch.com/sites/default/files/sdn_resources/openstack_aag.pdf•Cisco. quantum/plugins/cisco/README and http://wiki.openstack.org/cisco-quantum•Linux Bridge. quantum/plugins/linuxbridge/README and http://wiki.openstack.org/Quantum-Linux-Bridge-Plugin  •Ryu. quantum/plugins/ryu/README and http://www.osrg.net/ryu/using_with_openstack.html•NEC OpenFlow. http://wiki.openstack.org/Quantum-NEC-OpenFlow-Plugin 27
  • 28. Room ForImprovement
  • 29. Default Layer 3 DesignOpenStack Networking Won’t Magically Configure RoutingVLANs * Diagram taken from OpenStack Networking official documentation 29
  • 30. Gaps In Functionality• VLAN networks are still problematic, Quantum doesn’t fix that• Layer 3 network plugin still gets deployed on shared layer 2• Dynamic routing protocols are not supported by L3 plugin• Overlay networks are great, unless something goes wrong-- GRE tunnels hard to troubleshoot, we need tooling, diagnostics• Load-balancer-, firewall-, and VPN-as-a-service still in design phase, may not be production-ready until I or J release 30
  • 31. How Can We Make Things Better?There Are Plenty Of Ways To Contribute• Further work needed on the “metaplugin” that allows more than one plugin simultaneously• ZeroMQ support (there are known problems with DHCP, etc.)• Better high-availability, including active-active DHCP• Better support for custom tenant networks with overlapping IPs 31
  • 32. Questions 32
  • 33. Networking is NOT Free:Lessons In Network Design Dan Sneddon Member Technical Staff Twitter: @dxs Download: http://engineering.cloudscaling.com/portland13 CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution* * All unlicensed or borrowed works retain their original licenses

×