Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

L2/L3 für Fortgeschrittene - Helle und dunkle Magie im Linux-Netzwerkstack

73 views

Published on

Der Switch in meinem Linux-Rechner - was ist eine Bridge und wie benutze ich sie? Was sind VLANs und gar vlan-aware-Bridges? Fesselspiele mit Netzwerkkabeln - Bonding/Channel/Trunks mit und ohne LACP.

Auf Layer 3 tauchen wir ab in die Routingtabellen jedes Linux-Systems (derer gibt’s immer mindestens 3) sowie fortschrittlichere Magie wie policy-based Routing, VRFs und Network Namespaces; Beispiele aus dem echten Leben zeigen, wozu das alles gut ist und wie man damit arbeitet.

Published in: Internet
  • Be the first to comment

  • Be the first to like this

L2/L3 für Fortgeschrittene - Helle und dunkle Magie im Linux-Netzwerkstack

  1. 1. L2/L3 für Fortgeschrittene Helle und dunkle Magie imLinux-Netzwerkstack FrOSCon 13 Network Track Falk Stern, Maximilian Wilhelm 1 / 36
  2. 2. Agenda 1. Who are we 2. Layer 2 1. Link Aggregation 2. VLANs 3. Bridges 3. Layer 3 1. Policy based routing 2. VRFs 3. NetNS 2 / 36
  3. 3. Who's who Falk Stern Full Stack Infrastructure Engineer IPv6 fanboy Runs his own Kubernetes cluster in his basement Consultant @ Profi Engineering Systems AG Contact @wrf42 falk@fourecks.de 3 / 36
  4. 4. Who's who Maximilian Wilhelm Networker OpenSource Hacker Fanboy of (Debian) Linux ifupdown2 Occupation: By day: Senior Infrastructure Architect, Uni Paderborn By night: Infrastructure Archmage, Freifunk Hochstift In between: Freelance Solution Architect for hire Contact @BarbarossaTM max@sdn.clinic 4 / 36
  5. 5. Who's who Layer 2 Layer 2 / LAGs 5 / 36
  6. 6. Who's who Layer 2 LAGs Link Aggregation Combine one or more physical links between two peers to one virtual link, to increase over-all bandwidth create a redundant Layer 2 link both Also know as: LAG Bonding (Linux) Aggregated Ethernet (Juniper) Port-Channel (Cisco) Trunk (3Com, HP?) NIC-Teaming 6 / 36
  7. 7. Who's who Layer 2 LAGs Link Aggregation - Simple Linux bonding Just use multiple links and hope the peer does, too. Drawbacks: If media converters are involved a link-down event may not propagate No way to tell it the peer is configured the same way 7 / 36
  8. 8. Who's who Layer 2 LAGs Link Aggregation - LACP Link Aggregation Control Protocol (802.3ad / 802.1AX) De-facto standard within networking world Use LACP signalling to set up LAG with peer Maximum of 8 interface per LAG Keep alive every 1s (fast) or every 30s (slow) An interface can be on one of two modes: active: send out LACP packets to activly form the LAG passive: wait for and only then reply to LACP packets 8 / 36
  9. 9. Who's who Layer 2 LAGs Multi-Chassis Link Aggregation Groups Link Aggregation between more than two peers At least on peer as to do magic to make this work Also know as: MC-LAG MLAG Virtual Port-Channel (vPC) Source: Wikipedia 9 / 36
  10. 10. Who's who Layer 2 LAGs Loadbalancing Tra c over LAGs Round-Robin One packet on link 1, one on link 2, ..., and repeat Hashing of header elds Layer 2 (src MAC + dst MAC) Only useful if communication is to multiple stations within local subnet Layer 2+3 (src MAC + dst MAC + src IP + dst IP) Might be more useful for communication without local subnet Layer 3+4 (src IP + dst IP + src Port + dst Port) Probably most useful when communicating with multiple peers 10 / 36
  11. 11. Who's who Layer 2 LAGs Layer 2 / VLANs 11 / 36
  12. 12. Who's who Layer 2 LAGs VLANs Virtual Local Area Networks (VLANs) Used to separate broadcast domains in LANs VLAN transport between switches standardized as IEEE 802.1q after proprietary standards from Cisco, 3COM 12 bit VLAN Identifier only 4096 possible VLANs, ~100 reserved for internal switch functions 1 2 3 4 5 6 Destination MAC 1 2 3 4 5 6 Source MAC 1 2 3 4 802.1Q Header TPID=0x8100 PCP/DEI/VID 1 . . . n Payload 1 2 EtherType/ Size 1 2 3 4 CRC / FCS 1 2 3 4 5 6 Inter Frame Gap 7 8 9 10 11 12 1 2 3 4 5 6 Destination MAC 1 2 3 4 5 6 Source MAC 1 . . . n Payload 1 2 EtherType/ Size 1 2 3 4 CRC / FCS 1 2 3 4 5 6 Inter Frame Gap 7 8 9 10 11 12 n = 42–1500 n = 46–1500 1 2 3 4 5 6 Preamble 7 8 SFD 1 2 3 4 5 6 Preamble 7 8 SFD 12 / 36
  13. 13. Who's who Layer 2 LAGs VLANs QinQ - 802.1ad Double Tag Of course we can put VLANs into VLANs 1 2 3 4 5 6 Destination MAC 1 2 3 4 5 6 Source MAC 1 2 3 4 802.1Q Header TPID=0x 8100 PCP/DEI/VID 1 . . . n Payload 1 2 EtherType/ Size 1 2 3 4 CRC / FCS 1 2 3 4 5 6 Inter Frame Gap 7 8 9 10 11 12 1 2 3 4 5 6 Destination MAC 1 2 3 4 5 6 Source MAC 1 . . . n Payload 1 2 EtherType/ Size 1 2 3 4 CRC / FCS 1 2 3 4 5 6 Inter Frame Gap 7 8 9 10 11 12 n = 42–1500 n = 46–1500 1 2 3 4 5 6 Preamble 7 8 SFD 1 2 3 4 5 6 Preamble 7 8 SFD 1 2 3 4 5 6 Destination MAC 1 2 3 4 5 6 Source MAC 1 2 3 4 802.1Q Header TPID=0x 88A8 PCP/DEI/VID 1 . . . n Payload 1 2 EtherType/ Size 1 2 3 4 CRC / FCS 1 2 3 4 5 6 Inter Frame Gap 7 8 9 10 11 12 n = 38–1500 1 2 3 4 5 6 Preamble 7 8 SFD 1 2 3 4 802.1Q Header TPID=0x 8100 PCP/DEI/VID 13 / 36
  14. 14. Who's who Layer 2 LAGs VLANs Bridges Layer 2 / Bridges 14 / 36
  15. 15. Who's who Layer 2 LAGs VLANs Bridges Bridges The switch(es) within your Linux box Usage: ... bridge [ forward_delay FORWARD_DELAY ] [ hello_time HELLO_TIME ] [ max_age MAX_AGE ] [ ageing_time AGEING_TIME ] [ stp_state STP_STATE ] [ vlan_filtering VLAN_FILTERING ] [ vlan_default_pvid VLAN_DEFAULT_PVID ] [ mcast_snooping MULTICAST_SNOOPING ] [...] [ nf_call_iptables NF_CALL_IPTABLES ] [ nf_call_ip6tables NF_CALL_IP6TABLES ] [ nf_call_arptables NF_CALL_ARPTABLES ] ip link add br0 type bridge ip link set br0 up ip link set eth0 master br0 15 / 36
  16. 16. Who's who Layer 2 LAGs VLANs Bridges VLANs and Bridges Two options, both suck External trunk as bridge member External interface is part of the bridge All VLANs transported within the bridge All VLANs forwarded on any port External trunk with many bridges One interface per VLAN on trunk (e.g. bond0.2342) One bridge per VLAN (e.g. br2342) 16 / 36
  17. 17. Who's who Layer 2 LAGs VLANs Bridges VXLAN and Bridges One bridge per VNI Possibly multiple physical or virtual NICs within bridge, too VLAN interfaces VM interfaces (e.g. on KVM host) 17 / 36
  18. 18. Who's who Layer 2 LAGs VLANs Bridges Vlan-aware Bridges VLANs and bridges have been a challenge That ain't true no more Now it's a “regular switch” Configured with bridge utility from iproute Real World Use Case: Simple KVM/Qemu hook for VLAN assignment https://github.com/FreifunkHochstift/ffho-salt-public/blob/master/kvm/qemu-hook 18 / 36
  19. 19. Who's who Layer 2 LAGs VLANs Bridges Vlan-aware Bridges Port VLAN management bridge vlan { add | del } vid VLAN_ID dev DEV [ pvid ] [ untagged ] [ self ] [ master ] bridge vlan show [ dev DEV ] [ vid VLAN_ID ] Forwarding database bridge fdb [...] 19 / 36
  20. 20. Who's who Layer 2 LAGs VLANs Bridges Vlan-aware Bridges with ifupdown2 iface br0 bridge-ports bond0 bridge-vlan-aware yes bridge-vids 1013 4002 iface bond0 bridge-vids 100 101 200 201 1013 2000 [..] iface cr02_eth1 bridge-vids 1013 2000 2004 2006 3002 iface br0.1013 address 10.132.252.22/28 20 / 36
  21. 21. Who's who Layer 2 LAGs VLANs Bridges Vlan-aware Bridges and VXLAN Extending 24 bit to 36 bit 21 / 36
  22. 22. Who's who Layer 2 Layer 3 Layer 3 / Routing tables 22 / 36
  23. 23. Who's who Layer 2 Layer 3 Routing tables Every Linux box has a number of routing tables $ ip route help Usage: ip route { list | flush } SELECTOR ... SELECTOR := ... [ table TABLE_ID ] ... TABLE_ID := [ local | main | default | all | NUMBER ] By default routing table main is used So ip route show and ip route show table main show the same thing 23 / 36
  24. 24. Who's who Layer 2 Layer 3 Routing tables Table local Contains all routes to Locally connected IPs Broadcast addresses Table main Contains "usual" routes Locally connected subnets Routes to remote subnets Table default Usually empty 24 / 36
  25. 25. Who's who Layer 2 Layer 3 PBR Policy based routing Available since Linux 2.2 (1999) Defaut routing policy on every Linux box: $ ip rule 0: from all lookup local 32766: from all lookup main 32767: from all lookup default Drawbacks No mechanism for persitancy available Beware to close every loophole Rule for IPv4 Rule for IPv6 Rule for incoming interface 25 / 36
  26. 26. Who's who Layer 2 Layer 3 PBR VRFs Virtual Routing and Forwarding (VRFs) Independent routing instances L3-VPNs Usually in combination with MPLS Layer 3 separation VRF interface is master for “real” interfaces Defines routing table for VRF Since Kernel 4.[345] (use >= 4.9) 26 / 36
  27. 27. Who's who Layer 2 Layer 3 PBR VRFs Virtual Routing and Forwarding (VRFs) By foot ip link add vrf_external type vrf table 1023 ip link set eth0 master vrf_external ifupdown2 auto eth0 iface eth0 address 2002:db8:23:42::2/64 gateway 2001:db8:23:42::1/64 vrf vrf_external auto vrf_external iface vrf_external vrf-table 1023 Device routes move from table main and local to table 1023 27 / 36
  28. 28. Who's who Layer 2 Layer 3 PBR VRFs Connecting VRFs Requires vEth pair Like a virtual network cable within the box A end in main VRF, Z end in VRF “foo” Usual routing Static Bird talking BGP to itself 28 / 36
  29. 29. Who's who Layer 2 Layer 3 PBR VRFs Connecting VRFs By foot ip link add VETH_END1 type veth peer name VETH_END2 ifupdown2* iface veth_ext2int link-type veth veth-peer-name veth_int2ext vrf vrf_external iface veth_int2ext link-type veth veth-peer-name veth_ext2int * veth-peer-name not merged upstream yet (PR25) 29 / 36
  30. 30. Who's who Layer 2 Layer 3 PBR VRFs Real World Applications for VRFs External interface in VRF External interface is part of vrf_external GRE / OpenVPN tunnel sent / receive encapsulated packets over VRF Local tunnel endpoint is in main VRF Helpful sysctl /proc/sys/net/ipv4/tcp_l3mdev_accept l3mdev == Layer3 Master Device VRF info is added to socket Replies send out in VRF where request originated 30 / 36
  31. 31. Who's who Layer 2 Layer 3 PBR VRFs Real World Applications - Tunnels / GRE Outer and/or inner side of tunnel can be part of a VRF Send ip link add DEVICE type gre remote ADDR local ADDR dev PHYS_DEV If PHYS_DEV is within a VRF, all encapsulated packets are send/received in VRF That's how your internet access is built right now :) Pushing the inner side of a tunnel into a VRF is equally simple: ip link set DEVICE master VRF 31 / 36
  32. 32. Who's who Layer 2 Layer 3 PBR VRFs Real World Applications - Tunnel / OpenVPN Pushing the inner side of an OpenVPN tunnel into a VRF is a simple as before. Sending/receiving encapsulated packets into/from a VRF is more complicated But there's a patch since October 2016 https://github.com/OpenVPN/openvpn/pull/65 Used to glue remote POPs from Freifunk Hochstift together openvpn --config your_config.cfg --bind-dev VRF Now go and motivate Gert - Hi Gert! - to merge it, so we all can us it :) 32 / 36
  33. 33. Who's who Layer 2 Layer 3 PBR VRFs NetNS Network Namespaces (NetNS) Layer 1 separation Since Kernel 2.6.29 Own set of routing tables VRFs and PBR available within NetNS Own set of netfilter rules A process can be run in a special NetNS Two NetNS can be connected by vETH, too. 33 / 36
  34. 34. Who's who Layer 2 Layer 3 Takeaways Key takeaways Linux networking has evolved A LOT Linux today is a first class citizen wrt networking Vlan-aware bridges are great for virtualization hosts VRFs can help separte layer 3 domains nicely Tunneling technologies integrate accordingly 34 / 36
  35. 35. Who's who Layer 2 Layer 3 Takeaways Links Further Reading Contemporary Linux Networking - DENOG9 (2017) https://www.slideshare.net/BarbarossaTM/contemporary-linux-networking VRFs https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/ networking/vrf.txt https://cumulusnetworks.com/blog/vrf-for-linux/ https://de.slideshare.net/CumulusNetworks/operationalizing-vrf-in-the-data-center 35 / 36
  36. 36. Who's who Layer 2 Layer 3 Takeaways Links Questions Question? 36 / 36

×