Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

SRECon-Europe-2017: Networks for SREs

149 views

Published on

All of us depend on the underlying network to be stable whether in the datacenter or in the cloud. We all have a basic knowledge of how traditional networks run, however in the past 10 years, we’ve moved to building redundant physical topologies in our networks, optimized the routing methodologies accordingly, moved into the cloud and gotten greater visibility and tuneables in the Linux kernel network stack. A lot has changed!

However, the way we troubleshoot the network in relation to the applications we support hasn’t adapted. In this session, we’ll review the progress that network infrastructure has made look at specific examples where traditional troubleshooting responses fail us and demonstrate our need to rethink our approach to making applications and the network interact harmoniously.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

SRECon-Europe-2017: Networks for SREs

  1. 1. Networks for SREs: What do I need to know Michael Kehoe Staff SRE
  2. 2. Introduction
  3. 3. Michael Kehoe $ WHOAMI • Staff Site Reliability Engineer @ LinkedIn • Production-SRE Team • Funny accent = Australian + 3 years American • Former Network Engineer at the University of Queensland
  4. 4. Agenda and Vision
  5. 5. Today’s agenda 1 Introductions 2 Problem Statement 3 Basics of Networks 4 Advances in networks 5 Clos Networks 6 Advances in Network Speeds 7 IPv6 8 Summary
  6. 6. Networks just work right?
  7. 7. Probably…
  8. 8. Probably…Not…
  9. 9. What are we trying to solve Problem Statement • Network Design – Has evolved • Network software/ hardware – Has advanced • Learning – The average SRE may not necessarily understand the ramifications • Tooling – Has been left behind
  10. 10. What this talk is • Tale into potential pitfalls of modern day networks
  11. 11. What this talk isn’t • How to make the network do all the things…quickly & reliably…
  12. 12. What this talk isn’t • How to make the network do all the things…quickly & reliably… • Sorry
  13. 13. Basics of Networks
  14. 14. Basics of Networks Peering Facility Tier 1 ISP’s Tier 3 ISP Tier 2 ISP Tier 2 ISP Tier 2 Cable ISP
  15. 15. Basics of Networks
  16. 16. Advances in Network Design
  17. 17. Advances in Network Design • Clos Networks • Advancement of network speeds • IPv6 Implementation (Finally) • Multi-homed internet connections • Moving away from traditional internal routing protocols
  18. 18. Clos Networks
  19. 19. Clos Networks
  20. 20. Clos Networks
  21. 21. Clos Networks Credit: Facebook
  22. 22. Clos Networks Credit: Facebook
  23. 23. Advancement of Network Speeds
  24. 24. Advancement of Network Speeds Speed Name Standard Year 10Mb 10BASE-T 802.3i 1990 100Mb 100BASE-TX 802.3u 1995 1000Mb = 1Gb 1000BASE-T 802.3ab 1999 10Gb 10GBASE 802.3ae 2002 40/100Gb 40GbE/ 100GbE 802.3ba 2010
  25. 25. Advancement of Network Speeds • What this gives us • Better transfer bulk speeds • The ability to have higher concurrency services (1M connection problem) • Run multiple high-concurrency applications (LPS)
  26. 26. Networks just work right?
  27. 27. Probably…
  28. 28. Probably…Not…
  29. 29. Optimizations Required
  30. 30. Advancement of Network Speeds NIC Linux Kernel Network Switches
  31. 31. Advancement of Network Speeds • Network Interface Cards • Various RX/ TX queue size limits/ defaults • Various interrupt schemes • Plethora of tunables that vary wildly • LITTLE TO NO DOCUMENTATION! • How do you monitor/ tune it???
  32. 32. Advancement of Network Speeds • Linux Kernel • Lots of network tunables • Some defaults assume year ~2000 era hardware • E.g. net.ipv4.tcp_max_syn_backlog • Important to understand the type of application you run and cater your tunables to that.
  33. 33. Advancement of Network Speeds • Network switches • Similarly to interfaces and Linux software, there’s a lot of options • Deep Buffers • DSCP marking • Switching latency • DCTCP
  34. 34. Adoption of IPv6
  35. 35. IPv6 Features Address Space Better Performance Simplified Header No-NAT Auto- Configuratio n
  36. 36. IPv6: Address Space • Moving from a 32-bit address space to 128-bit. • 4B  340TTT • Read up on IPv6 addressing representation • RFC-5952
  37. 37. IPv6: Address Space A SINGLE ADDRESS CAN BE REPRESENTED MANY WAYS 2001:db8:0:0:1:0:0:1 2001:0db8:0:0:1:0:0:1 2001:db8::1:0:0:1 2001:db8::0:1:0:0:1 2001:0db8::1:0:0:1 2001:db8:0:0:1::1 2001:db8:0000:0:1::1 2001:DB8:0:0:1::1
  38. 38. IPv6: Address Space YOU CAN MAKE FUN PHRASES • :cafe:beef • :feed:f00d: • :bad:f00d: • :bad:beef: • :bad:d00d: • :f00d:cafe: • :bad:fa11:
  39. 39. IPv6: Address Space OR CLEVER ADVERTISING [mkehoe@mkehoe ~]$ host -6 www.facebook.com www.facebook.com is an alias for star-mini.c10r.facebook.com. star-mini.c10r.facebook.com has IPv6 address 2a03:2880:f113:8083:face:b00c:0:25de
  40. 40. IPv6: Address Space SPECIAL ADDRESSES: IPV4 RFC IP Block Use 1918 10.0.0.0/8 172.16.0.0/16 192.168.0.0/16 Private IP Addressing 6890/ 3927 169.254.0.0/16 Link-Local 5771 2365 224.0.0.0/4 Multicast
  41. 41. IPv6: Address Space SPECIAL ADDRESSES: IPV6 IP Block Use ::/128 Unspecified Address ::1/128 Loopback address ::ffff:0:0/96 IPv4 mapped addresses 64:ff9b::/96 IPv4/ V6 translation fc00:::/7 Unique Local Address fe80::/10 Link-Local address ff00::/8 Multicast addresses
  42. 42. IPv6: Address Space OR CLEVER ADVERTISING [mkehoe@mkehoe ~]$ host -6 www.facebook.com www.facebook.com is an alias for star-mini.c10r.facebook.com. star-mini.c10r.facebook.com has IPv6 address 2a03:2880:f113:8083:face:b00c:0:25de
  43. 43. IPv6: Simplified Header
  44. 44. IPv6: No NAT • No need for NAT anymore • Simplified Configuration • Less points-of-failure • Potential for better performance • NAT is slow • Harder for abusers to hide behind NAT
  45. 45. IPv6: Auto-Configuration • Stateless = Auto-Configured • Stateful = DHCP/ Statically assigned
  46. 46. IPv6: Better Performance • The elimination of NAT is a significant factor • Generally less hops across the internet for IPv6 vs IPv4 • Simplified Header gives small amount of optimization
  47. 47. Summary
  48. 48. Summary • Don’t implicitly trust the network! • Understand where your packets flow • End-to-End monitoring of your network. It is the lifeblood of your infrastructure • For any network infrastructure changes, ensure you understand how to benchmark and monitor it!
  49. 49. Networks just work right?
  50. 50. Q&A

×