Download to read offline
![[2018] 오픈스택 5년 운영의 경험](https://cdn.slidesharecdn.com/ss_thumbnails/cloudinfra05-190131073350-thumbnail.jpg?width=640&height=640&fit=bounds)
Logical_Routing_NSX_T_2.4.pptx.pptx
- 1. Confidential │ ©2019VMware, Inc. NSX-T Data Center 2.4 Logical Routing Amit Aneja Senior Technical Product Manager, NSBU May 2019
- 2. Terminology: Introducing theLogical Router(LR) 2 Confidential │ ©2019 VMware, Inc. Logical Router • Provides E-W routing between different Logical Switches. • Peers with the physical infrastructure for N-S Routing. • Can provides network services like Network Address Translation(NAT), Load-Balancing, Perimeter Firewall, VPN etc. Logical Switch 1 O R Segment 1 Phys ical Router Downlink/ Connected to Segment Uplink/ External 10.1.1.0/ 24 Logical Switch 2 O R Segment 2 10.2.2.2.0/2 4 10.2.2.1/ 24 Log ical Router 10.1.1.1/ 24
- 3. • Runs locallyin the transport nodes participating in the NSX fabric. • Typically runs as kernel module in the hypervisor. • Provides distributed E-W routing • Traffic between different subnets on same hypervisor doesn’t leave the hypervisor • Responsible for providing on/off ramp gateway services including N/S routing. • Provides centralized services like NAT, BGP, LB, Edge Firewall, Connectivity to the physical • The SR is instantiated as a service on an appliance called the Edge Node. Distributed Router (DR) Services Router (SR) Terminology: Logical Router Components Distributed Router (DR) and Services Router (SR) D R S R 3 Confidential │ ©2019 VMware, Inc.
- 4. Logical Router: DistributedComponent(DR) Distributed Routing Tier-0 Log ical Router ES Xi- ES Xi- 2 K V M 1 D R D R D R 10.1.1.0/ 24 10.2.2.0/24 10.1.1.10 10.2.2.10/2 10.2.2.20/2 10.1.1.20/ /24 4 4 24 4 Confidential │ ©2019 VMware, Inc.
- 5. Logical Router: ServicesComponent(SR) Services Router (SR) Load Balancing Pto V Gateway Router Perimeter Firewall VPN Whenever a service which cannot be distributed is enabled on a Logical Router, an SR or Services Component is instantiated. SR is instantiated for the following services: SR is instantiated on an appliance called the Edge Node. NAT DHCP 5 Confidential │ ©2019 VMware, Inc.
- 6. • Edge-nodes areappliances with pools of capacity for hosting any services which are not distributed. • Form Factor Choice - Virtual Machine or Bare Metal Both OVA and ISO both flavors available • Sizing Choice - 3 sizes available (Small, Medium, Large) • Built for resiliency - A/A and A/S models available • Leverages DPDK technology for fast packet processing. NSX-T Terminology: Edge Node Introducing the “Edge Node” BM *Edge Node in NSX-T = ! Edge service Gateway (ESG) in N S X for vSphere 6 Confidential │ ©2019 VMware, Inc.
- 7. Compute Hypervisors (vSphere /K V M ) Confidential │ ©2019 VMware, Inc. 7 Logical Routing Topology Spin e WAN Infrastructure Clusters: Edge Nodes, Management Nodes Lea f Edge Node hosting S R D R on every hypervisor (in kernel)
- 8. Confidential │ ©2019VMware, Inc. 8 Topology View : Putting it all together DR/SR Interaction Phys ical Router 10.1.1.10/24 ES X i Transport Network 10.1.1.10 W eb 1 External Interface 169.254.0.1 169.254.0.2 169.254.0.1 DR SR W eb 1 W eb S egmen t Tier-0 G atewa y 10.1.1.1 Web Segment Transit Segment DR 10.1.1.1 Create External interface N S X Management plane auto-plumbs this link (internal VNI) and routing between D R and S R P hys ical Router Edge Node NSX user configuration What happened behind the scenes?
- 9. Logical Routing Packet Walk(South-North traffic) Edge Node ES X i Transport Network External Segment WEB Segment Transit Segment SR DR Physical Router DR 10.1.1.10, MAC1 W eb1 192.168.100.0/24 192.168.240.1 192.168.240.3 • TEP 10.10.10.1 0 DR Routing table Network 0.0.0.0/0 10.1.1.0/24 Gateway 169.254.0.2 0.0.0.0 DR ARP Table Network Mac 169.254.0.2 02:50:56:56:53:00 Logical Switch MAC Table Outer IP Inner MAC 02:50:56:56:53:00 30.30.30.30 169.254.0.2 Payload Src= 10.1.1.10 D s t = 192.168.100.1 • TEP 30.30.30.30 SR Routing table Next hop Type Network t0s 0.0.0.0/0 t0c 10.1.1.0/24 192.168.240.1 0.0.0.0 GENEVE Src=10.10.10. 10 Dst=30.30.30. 30 9 Confidential │ ©2019 VMware, Inc.
- 10. Logical Routing Packet Walk(North–South traffic) Edge Node ES X i Transport Network External Segment WEB Segment Transit Segment SR DR Physical Router DR 10.1.1.10, MAC1 Web1 192.168.100.0/24 192.168.240.1 • TEP 10.10.10.1 0 • TEP 30.30.30.30 SR Routing table Type Network Next hop t0s 0.0.0.0/0 192.168.240.1 t0c 10.1.1.0/24 0.0.0.0 IP ARP Table : 10.1.1.10 MAC : 00:50:56:b7:2c:79 MAC Table of Web-Segment MAC : 00:50:56:b7:2c:79 LOCAL : 30.30.30.30 REMOTE: 10.10.10.10 ENCAP : GENEVE Payload Src= 192.168.100.1 D s t = 10.1.1.10 GENEVE Src=30.30.30. 30 Dst=10.10.10. 10 10 Confidential │ ©2019 VMware, Inc.
- 11. Multi-Tier Routing 11 Confidential │©2019 VMware, Inc.
- 12. • • Tenant Isolation Separate controlfor Infra and Tenant admin Eliminates dependency on physical infrastructure when a new tenant is provisioned • • Role- Connects to physical infra • Manual Management Tier-0 Logical Router Benefit Logical Routing- Multi Tier Topology Tier-0 Logical Router Phys ical Router Tier-1 Log ical Router 12 Confidential │ ©2019 VMware, Inc. Tier-1 Logical Router • • Role- Per tenant first hop router Cloud Management Platform (CMP) driven Management Tier0-Tier1 Transit link (100.64.0.0/31) Uplink Tier-1 Logical Router D ownlink Tenant-1 Tenant-2
- 13. Logical Routing- MultiTier Topology Interface and Route types Interface Types Uplink/External ▪ Used to connect to physical infrastructure Tier0-Tier1 Transit link ▪ Used to interconnect Tier0 and Tier1 Logical Routers. DownLink ▪ Used to connect overlay Logical Switches Service Port ▪ Interface connecting Vlan backed logical switch to provide connectivity to vlan backed physical or virtual workloads. Tier-0 Log ical Router P hys ica l Router Tier-1 Logical Router 13 Confidential │ ©2019 VMware, Inc. 20.20.20.0/2 4 30.30.30.0/24 100.64.224.0/31 100.64.224.1/31
- 14. Logical Routing- MultiTier Topology Route Advertisement and Route Redistribution- Auto Plumbing Phys ica l 14 Confidential │ ©2019 VMware, Inc. Router 20.20.20.0/2 4 30.30.30.0/24 100.64.224.0/31 100.64.224.1/31 Tier-1 Logical Router 20.20.20.0/24 & 30.30.30.0/24 are seen a s Tier-1 Connected routes * with next hop IP a s 100.64.224.1/31 Tier-0 redistributes Tier-1 Connected routes * (20.20.20.0/24 & 30.30.30.0/24) Default route with next hop IP a s 100.64.224.0/31 20.20.20.0/24 & 30.30.30.0/24 are seen a s eBGP routes with next hop IP a s 192.168.240.3/24 24 Tier-0 Log ical Router 192.168.240.1/ 24 192.168.240.3/ * NSX-static –NSX-T 2.3 or earlier
- 15. Logical Routing- MultiTier Topology Multi Tier Distributed Routing ESXi-1 Tier-0 D R Tenant 1 Tier-1 D R Tenant 2 Tier-1 D R ESXi- 2 Tier-0 D R Tenant 1 Tier-1 D R Tenant 2 Tier-1 D R • Tier0 and Tier1 routers are also instantiated on the hypervisors in order to prevent hair- pinning • Fully distributed architecture : as much routing as possible is performed upfront at the source 15 Confidential │ ©2019 VMware, Inc. 100.64.224.0/31 100.64.224.2/31 100.64.224.1/31 100.64.224.3/31 100.64.224.0/31 100.64.224.2/31 100.64.224.1/31 100.64.224.3/31
- 16. Edge Node Architecture 16 Confidential│ ©2019 VMware, Inc.
- 17. Edge Node Architecture Basics • •Edge Node needs to encapsulate/decapsulate traffic to/from Compute Transport nodes. -Overlay Transport Zone Edge Node needs to send/receive traffic to/from Physical infrastructure. -Vlan Transport Zone • Edge Node is a TN and has to be prepared • Edge Node can belong to a single Overlay Transport Zone and multiple VLAN Transport Zone. Uplink Profile- The Uplink Profile is a global constructs that defines how traffic will leave a Transport Node. The Uplink Profile is applied to a Transport Node when it joins a Transport Zone. -One Uplink Profile per Transport Node -One Uplink Profile per N-VDS (default mode) 17 Confidential │ ©2019 VMware, Inc.
- 18. Edge Node Architecture VMForm Factor Edge Node Edge Node V M • • 4 vNIC VM Leverages portgroups to send/receive traffic Supported on ESXi host only Supported on VSS, VDS and N-VDS* Internal Architecture remains the same as Baremetal Edge Same N-VDS can carry both Tunnel and Uplink traffic or you could use a separate N- VDS for Tunnel and Uplink traffic • • • • ESXi Host vNIC1 Edge Node V M vNIC2 vNIC3 vNIC4 Uplink1 N-VDS Overlay N-VDS TEP-IP TEP-IP Uplink2 N-VDS Edg e-M g m t Vlan 72 Edg e-Transport Vlan 75 Edg e-Uplink1 Vlan 76 Edg e-Uplink2 Vlan 77 VS S /VD S Uplink2 Uplink1 18 Confidential │ ©2019 VMware, Inc. Portgroups
- 19. Edge Node Architecture VMForm Factor - Three N-VDS deployment V D S O verla y N- V D S Ext-1 N -VD S M g m t P G Transport PGExt1 PG Ext2 PG Vlan 100 Vlan 200 Vlan 300 Vlan 400 vNIC2 fp-eth0 ESXi Port Groups VDS-Uplink1 VD S-Uplink2 P2 P1 Edg e V M 1 TOR-Left Mgmt VLAN : 100 Overlay VLAN : 200 Mgmt VLAN: 100 Overlay VLAN 200 External VLAN 1:300 Mgmt VLAN : 100 Overlay VLAN 200 External VLAN 2:400 TOR-Right Ext-2 N -VD S TEP-IP Uplink1 Uplink1 vNIC3 fp-eth1 vNIC4 fp-eth2 Uplink1 vN IC1 eth0 Each N-VDS has its own Uplink i.e. Uplink1 as defined by the transport node profile below. S a m e profile is used for all three N- VDS. Overlay N-VDS Uplink1 = vNIC2 of Edge V M Ext1 N-VDS Uplink1 = vNIC3 of Edge V M Ext2 N-VDS Uplink1 = vNIC4 of Edge V M VDS Uplinks 19 Confidential │ ©2019 VMware, Inc. Fastpath interfaces (fp-eth)
- 20. Edge Node Architecture EdgeVM configuration 20 Confidential │ ©2019 VMware, Inc. Edge V M configuration • Portgropus leveraged by Edge VM can be access or trunk PG should be configured as Trunk when Edge VM sends 802.1Q tag. - - Vlan defined at the vlan backed LS used to connect Tier-0 to physical Uplink profile has a transport vlan defined
- 21. Edge Cluster Basics andSingle Edge Cluster • • • Edge Nodes are pooled in clusters for scale out and redundancy Maximum of 10 Edge nodes in an Edge Cluster There is total flexibility in assigning Logical routers to Edge Nodes and clusters Design 1: All Edge nodes in a single Edge cluster. - ECMP layer at Tier-0 uses all nodes (if uplinks are present) - Simplicity EN1 EN2 EN 3 Tier-0 Tier-0 Tier-0 Tier-0 EN 4 Edge Cluster1 Tier-1 Tier-1 Tier-1 Tier-1 Tier-1 21 Confidential │ ©2019 VMware, Inc. Tier-1 Tier-1 Tier-1
- 22. High-Availability 22 Confidential │ ©2019VMware, Inc.
- 23. Confidential │ ©2019VMware, Inc. 23 High Availability Active/Active and Active/Standby Edge Cluster NSX-T Edge nodes are pooled in edge- cluster to provide scale out and High- Availability for Services. Gateway in Active/Active HA mode • • • Scale out HA ECMP Stateless Services (Reflexive NAT) Gateway in Active/Standby HA mode Stateful Services SNAT/DNAT Load Balancer Edge Firewall DHCP server VPN • • • • • Edge Cluster Tier-1 Tier-1 Tier-0 Tier-0 Tier-1 Tier-1 Edge Node1 Edge Node2
- 24. Confidential │ ©2019VMware, Inc. 24 High Availability Failure Triggers Failure Triggers • BFD sessions on Management and Tunnel interfaces of Edge node are down. • All GENEVE Tunnels down • Northbound routing state is Down (Applicable to Tier-0 SR only) Edge Cluster Active Tier-0 S R S Ttia en r- d 0 by S R EN 1 EN 2 eBGP Mgmt Network Tunnel Network eBGP eB GP • TEP • TEP
- 25. Confidential │ ©2019VMware, Inc. Thank You