Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Operationalizing EVPN in the Data Center: Part 2

1,001 views

Published on

In the second of our two-part series on EVPN, Cumulus Networks Chief Scientist Dinesh Dutt dives into more technical details of network routing, EVPN use cases, and best practices for operationalizing EVPN in the data center.

To view the recording of this webinar, visit http://go.cumulusnetworks.com/l/32472/2017-09-23/95t7xh

Published in: Technology
  • Be the first to comment

Operationalizing EVPN in the Data Center: Part 2

  1. 1. 1 Nov 1, 2017 Dinesh G Dutt, Vivek Venkataraman | Cumulus Networks Part 2: Routing, Deployment Use Cases & Best Practices Operationalizing EVPN in the DC
  2. 2. 2Cumulus Networks EVPN Summary Routing Models Configuring Routing Troubleshooting EVPN Deployment Models and Recommendations Agenda
  3. 3. 3Cumulus Networks Key Takeaways • EVPN supports routing as well as bridging • Since L2 is no longer behind a single rack, multiple routing models are possible ▪ VRF is supported in all models • Pick right routing model based on use case • FRR/Cumulus continues the simple configuration model even with EVPN routing
  4. 4. 4Cumulus Networks The Story So Far • Designed to address the twin issues of: ▪ Multi-tenancy over an L3 network ▪ Allow disjointed L2 segments over an L3 network • Dataplane: ▪ Supports multiple encapsulations: MPLS, VxLAN, NVGRE… ▪ VxLAN is the common choice within the data center • Control plane is BGP • Standards-based ▪ IETF original draft for MPLS: RFC 7432 ▪ IETF draft for support with VxLAN: draft-ietf-bess-evpn-overlay
  5. 5. 5Cumulus Networks Why Now ? • Adoption of leaf-spine based IP fabrics to build data centers • Rise of switching silicon that supports VxLAN routing • Multi-vendor support for EVPN ▪ Lack of widespread adoption of controller-based overlays
  6. 6. 6Cumulus Networks The Next Chapter • EVPN is more than just multi-tenancy L2: ▪ supports routing, multicast handling, MAC/VM mobility etc. • This part will cover these other aspects • Plus, deployment models
  7. 7. 7Cumulus Networks VXLAN Summary • UDP/IP based encapsulation carrying L2 payloads ▪ RFC 7438 • Source port hashing allows fine-grained traffic spreading of overlay traffic without requiring deep packet parsing • 24-bit Virtual Network Identifier (VNI) identifies the VPN • Tunnel ingress and egress are called VTEP (VXLAN Tunnel Endpoint)
  8. 8. 8Cumulus Networks • Protocol aspects based on BGP-based MPLS VPNs: ▪ Routes of a tenant kept separate with Route Distinguisher (RD) ▪ Routes contain Route Targets (RTs) to identify the VPN (L2 and/or L3 ) ▪ Uses MP-BGP AFI L2VPN (25) SAFI EVPN (70) ▪ Various new BGP attributes (extended communities) - MAC Mobility, Default Gateway, Encapsulation, Router MAC etc. • Multiple pieces of information exchanged in EVPN: ▪ Another level of encoding, called route types, to identify the information carried EVPN Summary: Protocol
  9. 9. 9Cumulus Networks EVPN Summary - key route types Route Type Name Usage RT-2 MAC/IP Advertisement Route Advertise MACs and/or MACIPs RT-3 Inclusive Multicast Ethernet Tag Route Advertise VNI membership (primarily to prune recipients of BUM traffic) RT-5 IP Prefix Route Advertise routes to subnet prefixes RT-1 Ethernet AutoDiscovery (A-D) Route For multi-homing, used to let remote VTEPs know about connectivity to an Ethernet Segment and VLANs reachable on it. RT-4 Ethernet Segment Route For designated forwarder (DF) election for BUM traffic handling in multi-homing scenarios. RT-6 Selective Multicast Ethernet Tag Route To carry IGMP multicast group membership information for a tenant using EVPN. Route/VNI info Dual attach support Multicast Info
  10. 10. 10Cumulus Networks H11 -> H41: VXLAN Bridging (Packet Forwarding Level Set) 50.1.1.11 (VL 100) 50.1.1.41 (VL 100) L1 L2 L3 L4 S1 S2 H11 H41 Unencapsulated packet: DMAC is H41 Encapsulated packet: Routed from L1 -> S1 Encapsulated packet: Routed from S1 -> L4 Unencapsulated packet: DMAC is H41
  11. 11. 11Cumulus Networks H11 -> H41: VXLAN Bridging (Packet Forwarding Level Set) 50.1.1.11 50.1.1.41 L1 L2 L3 L4 S1 S2 H11 H41 Unencapsulated packet: DMAC is H41 Encapsulated packet: Routed from L1 -> S1 Encapsulated packet: Routed from S1 -> L4 ● Spines use only the VXLAN Header to route the packet ● Inner packet is carried practically unmodified ● L1 maps brown VLAN to brown VNI, L4 does the opposite Unencapsulated packet: DMAC is H42 DMAC: H41 SMAC: H11 DstP: H41 SrcIP: H11 DMAC: H41 SMAC: H11 DstP: H41 SrcIP: H11 DMAC: H41 SMAC: H11 DstP: H41 SrcIP: H11 DMAC: H41 SMAC: H11 DstP: H41 SrcIP: H11 Data Data Data Data DMAC: S1 SMAC: L1 DstIP: L4 SrcIP: L1 VNI: Brown DMAC: S1 SMAC: L1 DstIP: L4 SrcIP: L1 VNI: Brown VXLAN Header
  12. 12. 12 Routing Models
  13. 13. 13Cumulus Networks Regular Routing (H11 -> H12), No VxLAN: Case 1 50.1.1.11 (VLAN 100) 50.1.2.22 (VLAN 110) L1 L2 L3 L4 S1 S2 H11 H12 1. H11 bridges to L1, default gateway 2. L1: a. routes to Blue subnet b. L1 identifies Blue subnet as being local c. L1 does neighbor lookup on H12 3. L1 bridges to H12 1 2
  14. 14. 14Cumulus Networks Regular Routing (H11 -> H42), No VxLAN: Case 2 50.1.1.11 (VLAN 100) 50.1.2.42 (VLAN 110) L1 L2 L3 L4 S1 S2 H11 H42 L1 and L4 have exchanged subnet routes 1. H11 bridges to L1, default gateway 2. L1 routes to next hop S1 (or S2) 3. S1 (or S2) routes to L4 4. On L4, destination is on a local subnet. L4 does neighbor lookup and bridges to H42 1 2 3 4
  15. 15. 15Cumulus Networks Routing (H11 -> H42) with VxLAN • Where is H11’s (and H42’s) default router ? • If L1 is the default router, what happens after initial routing? ▪ Bridge to H42 (case 1) ? ▪ Routing at next hop L4 (case 2)? • L1 and L4 always encapsulate and decapsulate VXLAN packet • Spines only route encapsulated packets 50.1.1.11 (VL 100) 50.1.2.42 (VL 110) L1 L2 L3 L4 S1 S2 H11 H42
  16. 16. 16Cumulus Networks The Rise of the Routing Models • Where is H11’s (and H42’s) default router ? ▪ Specific per-VNI (or all VNI) gateways (Centralized routing) ▪ All ingress VTEPs are gateways (Distributed routing) • So, what happens after the initial routing ? ▪ Bridge (case 1): Asymmetric Routing ▪ Route (case 2): Symmetric Routing
  17. 17. 17Cumulus Networks Asymmetric vs Symmetric: Observations • Asymmetric Model assumes all subnets are locally attached • Symmetric model assumes all subnets are NOT locally attached • This choice plays a role in what’s suitable for what deployment
  18. 18. 18Cumulus Networks Asymmetric Routing H11 -> H42: Step by Step 50.1.1.11 (VLAN 100) 50.1.2.42 (VLAN 110) L1 L2 L3 L4 S1 S2 H11 H42 1. H11 sends unencapsulated to L1 a. DMAC = L1’s MAC, DIP = H42’s IP 1
  19. 19. 19Cumulus Networks Asymmetric Routing H11 -> H42: Step by Step 50.1.1.11 (VLAN 100) 50.1.2.42 (VLAN 110) L1 L2 L3 L4 S1 S2 H11 H42 1. H11 sends unencapsulated to L1 a. DMAC = L1’s MAC, DIP = H42’s IP 2. L1: a. routes the packet (in tenant’s VRF) to blue subnet b. identifies it is a local subnet and does a neighbor lookup to get H42’s MAC* c. Determines H42’s MAC is behind L4 d. L1 encapsulates the packet with VxLAN header: i. Payload: DMAC = H42’s MAC. SMAC = L1’s MAC ii. DIP = L4’s VTEP, SIP = L1’s VTEP, VNI = Blue iii. DMAC = S1’s MAC, SMAC = L1’s MAC 1 2
  20. 20. 20Cumulus Networks Asymmetric Routing H11 -> H42: Step by Step 50.1.1.11 (VLAN 100) 50.1.2.42 (VLAN 110) L1 L2 L3 L4 S1 S2 H11 H42 1. H11 sends unencapsulated to L1 a. DMAC = L1’s MAC, DIP = H42’s IP 2. L1: a. routes the packet (in tenant’s VRF) to blue subnet b. identifies it is a local subnet and does a neighbor lookup to get H42’s MAC* c. Determines H42’s MAC is behind L4 d. L1 encapsulates the packet with VxLAN header: i. Payload: DMAC = H42’s MAC. SMAC = L1’s MAC ii. DIP = L4’s VTEP, SIP = L1’s VTEP, VNI = Blue iii. DMAC = S1’s MAC, SMAC = L1’s MAC 3. S1 routes to L4 4. L4: a. decapsulates the packet; VNI = Blue b. Looks up DMAC of H42 on corresponding VLAN, bridges out port 1 2 3 4
  21. 21. 21Cumulus Networks Asymmetric Routing: Putting It All Together 1. Host sends packet to gateway router 2. Ingress VTEP (GW): a. Routes b. Bridges c. Encapsulates 3. Spine switches (underlay) route 4. Egress VTEP: a. Decapsulates b. Bridges to end host Packets are transported through the fabric in the final destination VNI 50.1.1.11 (VLAN 100) 50.1.2.42 (VLAN 110) L1 L2 L3 L4 S1 S2 H11 H42 1 2 3 4
  22. 22. 22Cumulus Networks Symmetric Routing H11 -> H42: Step by Step 50.1.1.11 (VLAN 100) 50.1.2.42 (VLAN 110) L1 L2 L3 L4 S1 S2 H11 H42 1. H11 sends unencapsulated to L1 a. DMAC = L1’s MAC, DstIP = H42 1
  23. 23. 23Cumulus Networks Symmetric Routing H11 -> H42: Step by Step 50.1.1.11 (VLAN 100) 50.1.2.42 (VLAN 110) L1 L2 L3 L4 S1 S2 H11 H42 1. H11 sends unencapsulated to L1 a. DMAC = L1’s MAC, DIP = H42’s IP 2. L1: a. routes the packet (/32 route) to next hop L4 - DMAC is L4’s Router MAC b. L1 encapsulates the packet with VxLAN header: i. Payload: DMAC = L4’s Router MAC. SMAC = L1’s MAC ii. DIP = L4’s VTEP, SIP = L1’s VTEP, VNI = ?? iii. DMAC = S1’s MAC, SMAC = L1’s MAC 1 2
  24. 24. 24Cumulus Networks Symmetric Routing H11 -> H42: Step by Step 50.1.1.11 (VLAN 100) 50.1.2.42 (VLAN 110) L1 L2 L3 L4 S1 S2 H11 H42 1. H11 sends unencapsulated to L1 a. DMAC = L1’s MAC, DIP = H42’s IP 2. L1: a. routes the packet (/32 route) to next hop L4 - DMAC is L4’s Router MAC b. L1 encapsulates the packet with VxLAN header: i. Payload: DMAC = L4’s Router MAC. SMAC = L1’s MAC ii. DIP = L4’s VTEP, SIP = L1’s VTEP, VNI = ?? iii. DMAC = S1’s MAC, SMAC = L1’s MAC Question: What VNI to use to transport the frame to L4 ? 1. Brown (ingress VNI) 2. Blue (egress VNI, but how do I know ?) 3. Some other VNI 1 2
  25. 25. 25Cumulus Networks Symmetric Routing H11 -> H42: Step by Step 50.1.1.11 (VLAN 100) 50.1.2.42 (VLAN 110) L1 L2 L3 L4 S1 S2 H11 H42 1. H11 sends unencapsulated to L1 a. DMAC = L1’s MAC, DIP = H42’s IP 2. L1: a. routes the packet (/32 route) to next hop L4 - DMAC is L4’s Router MAC b. L1 encapsulates the packet with VxLAN header: i. Payload: DMAC = L4’s Router MAC. SMAC = L1’s MAC ii. DIP = L4’s VTEP, SIP = L1’s VTEP, VNI = per-tenant L3 transport VNI iii. DMAC = S1’s MAC, SMAC = L1’s MAC 3. S1 routes to L4 4. L4: a. decapsulates the packet. VNI is the L3 VNI - identifies the VRF. b. Looks up the DIP in VRF and routes to local subnet c. Looks up neighbor table for H42 d. Bridges to H42 1 2 3 4
  26. 26. 26Cumulus Networks Symmetric Routing: Putting It All Together 50.1.1.11 (VLAN 100) 50.1.2.42 (VLAN 110) L1 L2 L3 L4 S1 S2 H11 H42 1. Host sends packet to gateway router 2. Ingress VTEP (GW): a. Routes to egress VTEP b. Encapsulates 3. Spine switches (underlay) route 4. Egress VTEP: a. Decapsulates b. Routes to local subnet c. Bridges to end host Packets are transported through the fabric in a per-tenant L3 VNI. 1 2 3 4
  27. 27. 27Cumulus Networks • L3 VNI - configured and exchanged in control plane and carried in routed packets. ▪ Additional configuration ▪ Corresponds to VRF associated with the L2 VNI(s) ▪ Different number space from L2 VNI • Router MAC - Automatically derived (in Cumulus Linux/FRR) and exchanged in the control plane. Used in routed packets to indicate packet should be routed by egress VTEP (next hop) Symmetric routing - L3 Transport VNI and Router MAC
  28. 28. 28Cumulus Networks Asymmetric vs Symmetric: Packet Header View 50.1.1.11 50.1.2.42 L1 L2 L3 L4 S1 S2 H11 H42 DMAC: L1 SMAC: H11 DstP: H42 SrcIP: H11 Data DMAC: H42 SMAC: L1 DstP: H42 SrcIP: H11 Data DMAC: S1 SMAC: L1 DstIP: L4 SrcIP: L1 VNI: L3 VNI DMAC: L4 SMAC: S1 DstIP: L4 SrcIP: L1 VNI: L3 VNI DMAC: L4 SMAC: L1 DstP: H42 SrcIP: H11 Data DMAC: L4 SMAC: L1 DstP: H42 SrcIP: H11 Data DMAC: H42 SMAC: L1 DstP: H42 SrcIP: H11 Data DMAC: S1 SMAC: L1 DstIP: L4 SrcIP: L1 VNI: Blue DMAC: L4 SMAC: S1 DstIP: L4 SrcIP: L1 VNI: Blue ASYMM SYMM ASYMMSYMM DMAC: H42 SMAC: L1 DstP: H42 SrcIP: H11
  29. 29. 29Cumulus Networks Asymmetric vs Symmetric: Forwarding Tables View Asymmetric Symmetric MAC Table All end stations End stations in all locally known subnets plus remote VTEPs Neighbor Table All end stations End stations in all locally known subnets* plus remote VTEPs Route Table Locally attached prefixes All end stations plus local subnets VNIs All VNIs in fabric Locally attached VNIs plus L3 transport VNIs * - Needed for ARP Suppression
  30. 30. 30Cumulus Networks Asymmetric vs Symmetric: Configuration View Asymmetric Symmetric Uniform configuration Yes No, since not all VNIs are present everywhere Need Orchestrator No Most likely, since VNIs and their VLAN mappings will need to be configured or torn down as hosts/VMs move Scaling Yes, breaking mobility up into pods Yes Miscellaneous Need configuring and mapping additional L3 transport VNIs
  31. 31. 31Cumulus Networks Asymmetric vs Symmetric: Vendor Interop View Aymmetric Symmetric Arista X Cisco X Juniper X Cumulus/FRR X X* * - Supported in upcoming 3.5 release of Cumulus Linux
  32. 32. 32Cumulus Networks Distributed Routing Model • Since end station IP/MAC is spread throughout the network, no specific first hop router can be first hop router • Distributed model assumes every ToR switch is the first hop router for all locally attached subnets ▪ Anycast IP and anycast MAC model ▪ Similar to VRR used today (VARP in Arista lingo) • Most common deployed: when used to replace existing VLAN-based access-agg-core networks with VXLAN-based Clos networks
  33. 33. 33Cumulus Networks Centralized Routing Model • Encapsulated packets bridged to a designated first hop router • Packets are routed by this router • Encapsulated packets bridged to final destination by this router • Primary switching silicon requirement: ▪ To decapsulate, route, bridge, encapsulate, route on underlay header • Most commonly deployed: when EVPN is used for multi-tenancy in cloud-like environments
  34. 34. 34Cumulus Networks Centralized Routing H11 -> H42: Sample Packet Flow 50.1.1.11 (VLAN 100) 50.1.2.42 (VLAN 110) L1 L2 L3 L4 S1 S2 H11 H42 1. Host sends packet to gateway router (L2) 2. Ingress VTEP (GW): a. Bridges to egress VTEP/router L2 b. Encapsulates packet & sends out 3. Spine switches (underlay) route 4. Gateway VTEP: a. Decapsulates b. Routes to local subnet c. Bridges to end host d. Encapsulates packet & sends out 5. Spine switches (underlay) route 6. Egress VTEP: a. Decapsulates b. Bridges to end host Packets are transported through the fabric in the bridge VNI. 1 2 3 4 5 6
  35. 35. 35Cumulus Networks How do I talk to the outside world? • Routing/Packet Forwarding was all based on /32 routes or neighbor entries. • To route to external networks, we need to route to prefixes. ▪ Enter EVPN type-5 routes (RT-5). • RT-5 allows an IP prefix to be advertised, not just MAC+IP. ▪ For the common scenario of connecting to another subnet or external network, the advertising VTEP is itself the next hop. RT-5 contains the Router MAC of this VTEP. ▪ Specified in draft-ietf-bess-evpn-prefix-advertisement
  36. 36. 36Cumulus Networks Control Plane Illustration for External Routing L1 L2 L3 L4 S1 S2 ● Per-tenant VRF peering between Border Leaf BL1 and WAN edge router R1 ● R1 advertises prefixes relevant to a tenant (e.g., default route) on corresponding peering. ● BLs are typically deployed in pairs for redundancy. ● For internal destinations to be reachable, BLs will advertise corresponding subnets to R1. BL1 R1 WAN
  37. 37. 37Cumulus Networks Control Plane Illustration for External Routing L1 L2 L3 L4 S1 S2 ● Per-tenant VRF peering between Border Leaf BL1 and WAN edge router R1 ● R1 advertises prefixes relevant to a tenant (e.g., default route) on corresponding peering. ● BLs are typically deployed in pairs for redundancy. ● For internal destinations to be reachable, BLs will advertise corresponding subnets to R1. BL1 ● BL1 installs routes in VRF routing table ● BL1 exports these routes into EVPN as RT-5. ● RT-5 advertised to other VTEPs with L3 VNI of associated VRF. Next hop is BL1.. R1 WAN
  38. 38. 38Cumulus Networks Control Plane Illustration for External Routing L1 L2 L3 L4 S1 S2 Receiving VTEPs (L1, …) install routes into VRF routing table - next hop is BL1, MAC is BL1’s RMAC ● Per-tenant VRF peering between Border Leaf BL1 and WAN edge router R1 ● R1 advertises prefixes relevant to a tenant (e.g., default route) on corresponding peering. ● BLs are typically deployed in pairs for redundancy. ● For internal destinations to be reachable, BLs will advertise corresponding subnets to R1. BL1 ● BL1 installs routes in VRF routing table ● BL1 exports these routes into EVPN as RT-5. ● RT-5 advertised to other VTEPs with L3 VNI of associated VRF. Next hop is BL1.. R1 WAN Note: This is for illustration purposes, a real deployment is likely to have NAT, FW etc.
  39. 39. 39Cumulus Networks External Routing: Packet Flow L1 L2 L3 L4 S1 S2 BL1 R1 WAN 50.1.1.11 (VL 100) H11 201.11.1.45 H100 H11 sends the packet for H100 to L1 - its default GW
  40. 40. 40Cumulus Networks External Routing: Packet Flow L1 L2 L3 L4 S1 S2 BL1 R1 WAN 50.1.1.11 (VL 100) H11 201.11.1.45 H100 H11 sends the packet for H100 to L1 - its default GW ● L1 matches packet against external route (default or prefix advertised by BL1) and routes to next hop VTEP BL1. ● Packet routed over core with DMAC = BL1’s Router MAC. VNI is the L3 VNI for this VRF.
  41. 41. 41Cumulus Networks External Routing: Packet Flow L1 L2 L3 L4 S1 S2 BL1 terminates the VxLAN tunnel and routes the packet in the tenant VRF - on to R1. . BL1 R1 WAN 50.1.1.11 (VL 100) H11 201.11.1.45 H100 H11 sends the packet for H100 to L1 - its default GW ● L1 matches packet against external route (default or prefix advertised by BL1) and routes to next hop VTEP BL1. ● Packet routed over core with DMAC = BL1’s Router MAC. VNI is the L3 VNI for this VRF.
  42. 42. 42Cumulus Networks Wait...Is RT-5 used only for external connectivity? • No! RT-5 can also be used for inter-POD and inter-DC communication. • It really depends on how the subnets have been provisioned i.e., contained within a POD or DC. • Cumulus Linux (and FRR) supports RT-5 for external and inter-POD/inter-DC communication - available in upcoming release.
  43. 43. 43 Configuration Example
  44. 44. 44Cumulus Networks Configuration Steps: Asymmetric Routing • Provision VLANs and VNIs on all leaves • Provision subnets for all relevant VLANs (SVIs) • Map SVIs to appropriate VRF • Configure eBGP between leaf and spine • Activate and advertise information about all locally active VNIs
  45. 45. 45Cumulus Networks Configuration Steps: Symmetric Routing • Provision relevant locally attached VLANs and VNIs on the leaves (dynamic, non-uniform compared to asymmetric) • Provision subnets for all locally attached VLANs (SVIs) • Map SVIs to appropriate VRF • For each VRF, provision an L3 VNI (additional step compared to asymmetric) • Configure eBGP between leaf and spine • Activate and advertise information about all locally active VNIs
  46. 46. 46Cumulus Networks Asymmetric vs Symmetric Routing: FRR Configuration # BGP/EVPN configuration router bgp 65456 bgp router-id 110.0.0.1 neighbor fabric peer-group neighbor fabric remote-as external neighbor uplink-1 interface peer-group fabric neighbor uplink-2 interface peer-group fabric address-family ipv4 unicast neighbor fabric activate redistribute connected address-family l2vpn evpn neighbor fabric activate advertise-all-vni # L3 VNI configuration for tenant VRF vrf vrf-tenant1 vni 104001 # BGP/EVPN configuration router bgp 65456 bgp router-id 110.0.0.1 neighbor fabric peer-group neighbor fabric remote-as external neighbor uplink-1 interface peer-group fabric neighbor uplink-2 interface peer-group fabric address-family ipv4 unicast neighbor fabric activate redistribute connected address-family l2vpn evpn neighbor fabric activate advertise-all-vni
  47. 47. 47Cumulus Networks Centralized routing • Fundamental configuration on Gateway VTEP(s) is same as in the distributed case. • Gateway VTEP(s) need to be configured to advertise their own MACIP. # BGP/EVPN configuration router bgp 65456 bgp router-id 110.0.0.5 neighbor fabric peer-group neighbor fabric remote-as external neighbor uplink-1 interface peer-group fabric neighbor uplink-2 interface peer-group fabric address-family ipv4 unicast neighbor fabric activate redistribute connected address-family l2vpn evpn neighbor fabric activate advertise-all-vni advertise-default-gw
  48. 48. 48Cumulus Networks Switching Silicon Support • Considering only native, single-pass support for VxLAN routing • Cavium and Barefoot chipsets are supposed to have support for all modes T2 T2+ T3 Tomahawk family Spectrum/ A0 Spectrum /A1 Spectrum2 Asymmetric - X X - X X X Symmetric - X X - X X X Centralized - X X - - X X
  49. 49. 49 What about multicast?
  50. 50. 50Cumulus Networks The jury is still out • Multicast routing in EVPN is still evolving. • There are at least two key aspects: ▪ Optimized intra-subnet multicast (only to VTEPs behind which interested receivers are present) ▪ Optimized inter-subnet multicast - local/distributed routing wherever possible • There are multiple proposals being discussed - including leveraging MVPN and VPLS Multicast. • Stay tuned for a future update on this topic!
  51. 51. 51Cumulus Networks Summary • EVPN supports routing besides bridging • Due to the distributed nature of L2 in EVPN, several routing models are possible • Choose the right model based on deployment use case ▪ Choose wisely • Cumulus/FRR supports (or will shortly support) all of the routing models, including interop with other vendors ▪ Most other vendors support only a subset of these • Cumulus/FRR provides a radically simplified config for EVPN routing
  52. 52. 52 Thank you! Visit us at cumulusnetworks.com or follow us @cumulusnetworks or slack.cumulusnetworks.com © 2017 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc. or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.
  53. 53. 53Cumulus Networks Flood multicast only where there are receivers • Basic BUM handling will flood to all remote VTEPs. • What if there is real multicast traffic (i.e., non link-local) for a tenant - e.g., system monitoring, discovery, data dissemination using Pub/Sub etc? Receivers may be dispersed in the DC. ▪ Enter Selective Multicast and EVPN Type-6 (RT-6) routes ▪ IGMP/MLD state on attachment circuits (ACs) conveyed using EVPN RT-6 to remote VTEPs ▪ Receiving VTEPs generate proxy reports on their ACs ▪ Receiving VTEPs also build state indicating which VTEPs need traffic for a particular (C-*, C-G) or (C-S, C-G)
  54. 54. 54Cumulus Networks Distributed multicast routing • When multicast sources and receivers are on different subnets, the (inter-subnet) multicast routing can get hairy: ▪ Only one VTEP can be the Designated Router (DR) on a subnet, so even for local receivers on a different subnet from source, packet may have to be routed by a remote VTEP. ▪ A VTEP could get multiple copies of the packet, one for each subnet • Distributed multicast routing is the solution. In one proposal: ▪ Each VTEP routes to local receivers on all subnets. ▪ Only one copy sent to remote VTEPs - on source subnet ▪ Receivers will receive on a special broadcast domain if they don’t have the source subnet.
  55. 55. 55Cumulus Networks Symmetric routing - sample topology 50.1.1.11 (VL 100) 50.1.4.44 (VL 130) L1 L2 L3 L4 S1 S2 H11 50.1.2.12 (VL 110) H12 50.1.3.43 (VL 120) H43 VL 130 H44 ● Tenant has 4 VLANs: ○ VL 100 - 50.1.1.x/24 ○ VL 110 - 50.1.2.x/24 ○ VL 120 - 50.1.3.x/24 ○ VL 130 - 50.1.4.x/24 ● VLANs 100 and 110 (and corresponding SVIs) are provisioned on {L1, L2} and VLANs 120 and 130 on {L3, L4} ● Anycast GW IP is 50.1.x.250 - provisioned on all Leafs. ● VLAN - VNI mappings: ○ VL 100 - VNI 10100 ○ VL 110 - VNI 10110 ○ VL 120 - VNI 10120 ○ VL 130 - VNI 10130 ● L3 VLAN and VNI for tenant are 4001 and 104001 respectively
  56. 56. 56Cumulus Networks Symmetric routing - sample interface configuration (L1) # VxLAN interfaces and VLAN-VNI mappings (local ones) auto vxlan100 iface vxlan100 vxlan-id 10100 vxlan-local-tunnelip 110.0.0.1 bridge-learning off bridge-access 100 bridge-arp-nd-suppress on # VxLAN interface and VLAN-VNI mapping for the L3VNI auto vxlan4001 iface vxlan4001 vxlan-id 104001 vxlan-local-tunnelip 110.0.0.1 bridge-learning off bridge-access 4001 # Bridge with member ports (VLAN-aware) auto br0 iface br0 bridge-vlan-aware yes bridge-ports swp3 swp4 swp5 swp6 vxlan100 vxlan110 vxlan4001 bridge-stp on bridge-vids 100 110 4001 # Tenant VRF configuration - if multiple tenants exist auto vrf-tenant1 iface vrf-tenant1 vrf-table auto # SVI with anycast GW IP (for local tenant subnets) auto vlan100 iface vlan100 address 50.1.1.1/24 vlan-id 100 vlan-raw-device br0 address-virtual 00:00:5e:00:01:01 50.1.1.250/24 vrf vrf-tenant1 # L3 VLAN interface per tenant (for L3 VNI) auto vlan4001 iface vlan4001 vlan-id 4001 vlan-raw-device br0 vrf vrf-tenant1
  57. 57. 57Cumulus Networks Symmetric routing - sample FRR configuration (L1) # L3 VNI configuration for tenant VRF vrf vrf-tenant1 vni 104001 # BGP/EVPN configuration router bgp 65456 bgp router-id 110.0.0.1 neighbor fabric peer-group neighbor fabric remote-as external neighbor uplink-1 interface peer-group fabric neighbor uplink-2 interface peer-group fabric address-family ipv4 unicast neighbor fabric activate redistribute connected address-family l2vpn evpn neighbor fabric activate advertise-all-vni

×