SlideShare a Scribd company logo
1 of 42
Confidential │ ©2021 VMware, Inc.
Service Interface
Guidelines and Design
Nicolas Michel
Technical Product Manager | NSBU
November 2021
Confidential │ ©2021 VMware, Inc. 2
Agenda Service Interface introduction
Support and Topologies
Lab Topology
Diagram and Description
vSphere Configuration
dVPG TOR-01 / dVPG TOR-02 / dVPG
NSX-T Configuration
UI Configuration
Confidential │ ©2021 VMware, Inc. 3
Service Interface Introduction
Confidential │ ©2021 VMware, Inc. 4
Service Interface
Support in NSX-T 3.2
Connectivity
Originally developed to connect vlan
backed segments.
Overlay Segments also supported.
On a Tier0, Uplink are recommended.
On a Tier1, Service Interfaces are
mandatory to provide connectivity to
vlan backed segment (no Uplink on
Tier0).
Service interfaces using the same
vlan backed segment can not be
instantiated on different logical
routers hosted on the same edge
node.
Topologies
Can be used to interconnect Tier-1
gateways together (static routing
manually configured).
Service Interfaces are supported in
Active/Standby topologies only.
Networking and Services
Dynamic Routing protocols are not
supported (OSPF, BGP). Except for
EVPN Route Server Mode
Static Routing is supported. IPsec not
supported
A Service Interface supports the
following services:
• DHCP relay.
• DHCP Server
• NAT
• Gateway Firewall
• Native Load Balancer
Confidential │ ©2021 VMware, Inc. 5
Service Interface
Introduction
Service Interface*: Interface
connecting Vlan backed logical
switch to provide connectivity to
vlan backed physical or virtual
workloads.
*Referred as Centralized Service Port (CSP) in NSX-T
2.3
Baremetal
Servers
Tier-1
Tier-1
Gateway
Tier-0
Gateway
Vlan Segment
SI-2
Baremetal
Servers
Confidential │ ©2021 VMware, Inc. 6
Service Interface
Introduction
Service Interface: Can also be
connected to Overlay Segment for
Load balancer use cases.
Tier-1
Gateway
SI-2
Overlay or Vlan
Segment
SI-1
Tier-1
Gateway
Tier-1
Gateway
Standalone
Physical
Router
Vlan Segment
Baremetal
Servers
Tier-0
Gateway
Confidential │ ©2021 VMware, Inc. 7
Service Interface Topology
Interconnect Tier1 gateways
Service interface can be used to connect 2
Tier1 gateway together.
A service interface is used on the Org VDC
Tier1
A Linked Segment is used on the vApp Tier1
Static routing must be configured manually on
both Tier1 gateways:
• Org VDC Tier1 should have static routes for
the vApp tier1 segments :
• Static route to 10.2.2.0/24 with a next hop of
172.16.0.2
• Static route to 10.2.3.0/24 with a next hop of
172.16.0.2
• vApp Tier1 should have a default route
pointing to the Service Port Tier1 interfaces
hosted on the Org VDC:
• Static route to 0.0.0.0/0 with a next hop of
172.16.0.1
Confidential │ ©2021 VMware, Inc. 8
Service Interface
Supported Topology
Identical Overlay backed
segment used on different Tier-1
Service Interface Gateways.
10.1.1.1/24 on Tier-1 Tenant-01
10.1.1.2/24 on Tier-1 Tenant-02
Supported Topology
Tier-1
Gateway
Tenant-01
Segment – Vlan 10
Active: Edge-01
Tier-1
Gateway
Tenant-02
Tier-0
Gateway
Same Active Edge node
OVERLAY BACKED
Active: Edge-01
Confidential │ ©2021 VMware, Inc. 9
Service Interface
NOT Supported Topology
Identical VLAN backed segment
used on different Tier-1 Service
Interface Gateways.
10.1.1.1/24 on Tier-1 Tenant-01
10.1.1.2/24 on Tier-1 Tenant-02
Topology not supported.
Tier-1 must be in different edge
clusters if they share interfaces
on the same segment.
Tier-1
Gateway
Tenant-01
Segment – Vlan 10
Active: Edge-01
Tier-1
Gateway
Tenant-02
Tier-0
Gateway
Same Active Edge node
VLAN BACKED
Active: Edge-01
Confidential │ ©2021 VMware, Inc. 10
Topology
Confidential │ ©2021 VMware, Inc. 11
Topology Used
Edge Node VM Design - 2 pNICs Design
NSX-T Edge 2 PNIC Design
Single N-VDS to simplify design and deployment. Multi-TEP on
Edge is supported.
Single VLAN per uplink is recommended (BGP / OSPF / Static).
Easier to troubleshoot.
Best Practice in the network industry.
Named Teaming Policy is used to map each vNIC to NSX Segment
to pNIC to ToR (Detailed in the next slides).
Single VLAN and Overlay Transport Zone.
VLAN used
VLAN 110 is the vlan used for the TEP and is shared
between TOR-01 and TOR-02
VLAN 10 is the vlan used to establish the BGP peering
between TOR-01 and T0-SR on EDGE-NODE-01
VLAN 13 is the vlan used to establish the BGP peering
between TOR-02 and T0-SR on EDGE-NODE-01
VLAN 11 is the vlan used to establish the BGP peering
between TOR-01 and T0-SR on EDGE-NODE-02
VLAN 12 is the vlan used to establish the BGP peering
between TOR-02 and T0-SR on EDGE-NODE-02
VLAN 300 is the vlan used to connect by the Service
Interface Segment on NSX-T
N-VDS-01
Mgmt IP
Uplink-1 Uplink-2
Edge-01
Mgmt IP
Trunk-TOR-LEFT
A/S Failover Order
Trunk-TOR-RIGHT
A/S Failover Order
ToR-01 ToR-02
Management-PG
VDS
Uplink 1 Uplink 2
VLAN10
TEP-IP-1 TEP-IP-2
ESXi Host
N-VDS-01
Uplink-1 Uplink-2
TEP-IP-1 TEP-IP-2
Tier-0 (A)
eth0 fp-eth2 eth0 fp-eth2
fp-eth0 fp-eth1 fp-eth1
fp-eth0
VLAN 110 -TEP
VLAN 10 and 11 - BGP
Uplink configured as a trunk
VLAN 110 - TEP
VLAN 12 and 13 - BGP
Uplink configured as a trunk
VLAN13
Edge-02
VLAN11
Tier-0 (S)
VLAN12
VLAN 300 – Service Interface VLAN 300 – Service Interface
VLAN300 VLAN300
Confidential │ ©2021 VMware, Inc. 12
Topology Used
Layer 3 Topology
Layer 3 Design
Separate VLANs are used for each uplink between the TORs and
the T0-SR on EN01 and EN02
VLANs / IP Addressing plan
VLAN 300 is the vlan used for the service interface on the
T0.
Virtual Machine on a standard DVPG (non NSX-T
overlay)
Confidential │ ©2021 VMware, Inc. 13
vSphere Configuration – vNIC - DVPG
- Edge Node Virtual Machine vNIC
- DVPG TOR-Left
- DVPG TOR-Right
- DVPG Service Interface (VM vNIC)
Confidential │ ©2021 VMware, Inc. 14
vSphere Configuration – vNIC - DVPG
Edge Node Virtual Machine vNIC
Network Adapter 2
This vSphere vNIC is considered as fp-eth0 for NSX-T
The adapter type is VMXNet3
Network Adapter 3
This vSphere vNIC is considered as fp-eth1 for NSX-T
The adapter type is VMXNet3
Network Adapter 4 is not connected to anything.
Confidential │ ©2021 VMware, Inc. 15
vSphere Configuration – vNIC - DVPG
Distributed Port Group Configuration – TOR LEFT (01)
Confidential │ ©2021 VMware, Inc. 16
vSphere Configuration – vNIC - DVPG
Distributed Port Group Configuration – TOR RIGHT (02)
Confidential │ ©2021 VMware, Inc. 17
vSphere Configuration – vNIC - DVPG
Distributed Port Group Configuration – Service Interface DPG (VM vNIC)
 Teaming and Failover policy doesn’t really matter in our example
as the VM could be hosted anywhere in the DC
Confidential │ ©2021 VMware, Inc. 18
NSX-T Configuration
- Uplink Profile – Teaming Policies
- Service Interface Segment – TOR01
- Service Interface on T0
- Verification
Confidential │ ©2021 VMware, Inc. 19
NSX-T Configuration
Uplink Profile
An Uplink profile defines the way N-VDS
operates
Transport VLAN: TEP Vlan
MTU: N-VDS MTU
Teaming Policies:
- Default Teaming Policy: Multi TEP (fp-eth0
and fp-eth1)
- TOR-1: Use uplink-1 only (fp-eth0)
- TOR-2: Use uplink-2 only (fp-eth1)
Confidential │ ©2021 VMware, Inc. 20
NSX-T Configuration
Teaming policies
Teaming Policies Configuration:
- Default Teaming Policy:
- Multi TEP – Load Balance Source
- Uplink-1 maps to fp-eth0
- Uplink-2 maps to fp-eth1
- Named Teaming Policy TOR-1:
- Uplink-1 maps to fp-eth0 only
- Not supported to have a standby uplink
- Name Teaming Policy TOR-2:
- Uplink-2 maps to fp-eth1 only
- Not supported to have a standby uplink
Confidential │ ©2021 VMware, Inc. 21
NSX-T Configuration
Segment for the Service Interface – TOR 01
Creation of a VLAN Backed Segment
This segment is not attached to a specific T0
or T1.
The Transport zone must be VLAN Based
The Subnet is not set. It will be configured on
the Service interface itself (T0 construct)
Transport VLAN must match dvPG VLAN
connecting the virtual machine.
Uplink Teaming Policy:
- Supports a single Teaming Policy
- TOR-1:
- Normal Behavior:
- Use fp-eth0 connected to dvPG TOR-LEFT
- dvPG Config (Active/Standby) : dvUPLINK1
- Failover Behavior:
- Use fp-eth0 connected to dvPG TOR-LEFT
- dvPG Config (Active/Standby) : dvUPLINK2
Confidential │ ©2021 VMware, Inc. 22
NSX-T Configuration
Service Interface on the T0
Interface type must be “Service”
IP Address in the same range as the Virtual
Machine. 172.31.210.1 will be the VM’s default
gateway.
Connected to the Segment created earlier.
Confidential │ ©2021 VMware, Inc. 23
NSX-T Configuration
SRV-EDGE-01(tier0_sr)> get interfaces
Fri May 21 2021 UTC 05:14:33.012
Logical Router
UUID VRF LR-ID Name Type
a76ffe3e-8ed8-4509-a65b-5e52e43cda15 13 46 SR-Tier0-Tenant01 SERVICE_ROUTER_TIER0
Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable)
Interface : ddb58170-00eb-4651-90e8-ac3dec6fccd7
Ifuid : 399
Name : Service-Interface
Fwd-mode : IPV4_AND_IPV6
Internal name : service-399
Mode : lif
Port-type : service
IP/Mask : 172.31.210.1/24
MAC : 00:50:56:ab:ce:ac
VLAN : 300
Access-VLAN : untagged
LS port : 531af945-0f5a-43c9-9313-843e622d8027
Urpf-mode : STRICT_MODE
DAD-mode : LOOSE
RA-mode : SLAAC_DNS_TRHOUGH_RA(M=0, O=0)
Admin : up
Op_state : up
MTU : 9000
arp_proxy :
EDGE-01 Get Interfaces - ACTIVE
Confidential │ ©2021 VMware, Inc. 24
NSX-T Configuration
SRV-EDGE-02(tier0_sr)> get interfaces
Fri May 21 2021 UTC 05:14:36.985
Logical Router
UUID VRF LR-ID Name Type
126bf0b3-1c60-466b-ae09-f89562e33634 13 43 SR-Tier0-Tenant01 SERVICE_ROUTER_TIER0
Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable)
Interface : ddb58170-00eb-4651-90e8-ac3dec6fccd7
Ifuid : 397
Name : Service-Interface
Fwd-mode : IPV4_AND_IPV6
Internal name : service-397
Mode : lif
Port-type : service
IP/Mask : 172.31.210.1/24
MAC : 00:50:56:ab:84:94
VLAN : 300
Access-VLAN : untagged
LS port : 531af945-0f5a-43c9-9313-843e622d8027
Urpf-mode : STRICT_MODE
DAD-mode : LOOSE
RA-mode : SLAAC_DNS_TRHOUGH_RA(M=0, O=0)
Admin : up
Op_state : down
MTU : 9000
arp_proxy :
EDGE-02 Get Interfaces - STANDBY
Confidential │ ©2021 VMware, Inc. 25
NSX-T Configuration
Representation within vSphere and NSX-T Edge VM
Only Service Interface traffic is represented in
this diagram for simplicity reason
Uplink Teaming Policy configured on the
segment: TOR-01
If multiple Edge Node are hosted by the same
ESXi hypervisor, all traffic will be pinned to the
same uplink. In this case, all edges have been
deployed with the same uplink profile.
Traffic for VLAN 300 (Service Interface) will be
pinned to fp-eth0 from an NSX-T standpoint
and to Uplink-1 from a vSphere perspective.
N-VDS-01
Mgmt IP
Uplink-1 Uplink-2
Edge-01
Mgmt IP
Trunk-TOR-LEFT
A/S Failover Order
Trunk-TOR-RIGHT
A/S Failover Order
ToR-01 ToR-02
Management-PG
VDS
Uplink 1 Uplink 2
VLAN10
TEP-IP-1 TEP-IP-2
ESXi Host
N-VDS-01
Uplink-1 Uplink-2
TEP-IP-1 TEP-IP-2
Tier-0 (A)
eth0 fp-eth2 eth0 fp-eth2
fp-eth0 fp-eth1 fp-eth1
fp-eth0
Uplink configured as a trunk Uplink configured as a trunk
VLAN13
Edge-02
VLAN11
Tier-0 (S)
VLAN12
VLAN 300 – Service Interface VLAN 300 – Service Interface
NORMAL MODE
VLAN300 VLAN300
Confidential │ ©2021 VMware, Inc. 26
NSX-T Configuration
packer@tenant01-service-interface-01:~$ arp -a
_gateway (172.31.210.1) at 00:50:56:ab:ce:ac [ether] on ens192
packer@tenant01-service-interface-01:~$ ping 10.1.1.10 -c 1
PING 10.1.1.10 (10.1.1.10) 56(84) bytes of data.
64 bytes from 10.1.1.10: icmp_seq=1 ttl=62 time=0.717 ms
packer@tenant01-service-interface-01:~$ ping 172.31.100.30 -c 1
PING 172.31.100.30 (172.31.100.30) 56(84) bytes of data.
64 bytes from 172.31.100.30: icmp_seq=1 ttl=60 time=1.16 ms
Connectivity on the Virtual machine on the dvPG
T0-SR EDGE 01 MAC Interface
Successful ping to Northbound VM
Successful ping to Overlay VM
Confidential │ ©2021 VMware, Inc. 27
NSX-T Configuration
[root@srv-esxi-01:~] esxtop
PORT-ID USED-BY TEAM-PNIC DNAME PKTTX/s MbTX/s PSZTX PKTRX/s MbRX/s PSZRX %DRPTX %DRPRX
67108959 2147032:SRV-EDGE-01.eth2 vmnic1 DvsPortset-0 3.39 0.00 66.00 6.78 0.00 91.00 0.00 0.00
67108960 2147032:SRV-EDGE-01.eth1 vmnic0 DvsPortset-0 3.39 0.00 116.00 6.78 0.00 66.00 0.00 0.00
67108961 2147032:SRV-EDGE-01.eth0 vmnic0 DvsPortset-0 3.39 0.00 66.00 0.00 0.00 0.00 0.00 0.00
[root@srv-esxi-01:~] pktcap-uw --switchport 67108960 --dir 2 --vlan 300 -o /tmp/capture.pcap
Verification – ESXTOP (N)
 ICMP Traffic is being received on fp-eth0
 Packet 64 – 68 : ICMP with Overlay VM
 Packet 146 – 149: ICMP with Northbound VM
Confidential │ ©2021 VMware, Inc. 28
NSX-T Configuration
Representation within vSphere and NSX-T Edge VM
Only Service Interface traffic is represented in
this diagram for simplicity reason
If multiple Edge Node are hosted by the same
ESXi hypervisor, all traffic will be pinned to the
same uplink. In this case, all edges have been
deployed with the same uplink profile.
Traffic for VLAN 300 (Service Interface) will be
pinned to fp-eth0 from an NSX-T standpoint
and to Uplink-1 from a vSphere perspective.
In this case, the T0-SR hosted on the Edge
Node 02, will send a Gratuitous ARP on VLAN
300 so that the devices can update their ARP
entry for 172.31.210.1 . (see next slide)
N-VDS-01
Mgmt IP
Uplink-1 Uplink-2
Edge-01
Mgmt IP
Trunk-TOR-LEFT
A/S Failover Order
Trunk-TOR-RIGHT
A/S Failover Order
ToR-01 ToR-02
Management-PG
VDS
Uplink 1 Uplink 2
VLAN10
TEP-IP-1 TEP-IP-2
ESXi Host
N-VDS-01
Uplink-1 Uplink-2
TEP-IP-1 TEP-IP-2
Tier-0 (DOWN)
eth0 fp-eth2 eth0 fp-eth2
fp-eth0 fp-eth1 fp-eth1
fp-eth0
VLAN13
Edge-02
VLAN11
Tier-0 (A)
VLAN12
EDGE FAILURE
Uplink configured as a trunk
VLAN 300 – Service Interface
Uplink configured as a trunk
VLAN 300 – Service Interface
VLAN300 VLAN300
Confidential │ ©2021 VMware, Inc. 29
NSX-T Configuration
packer@tenant01-service-interface-01:~$ arp -a
_gateway (172.31.210.1) at 00:50:56:ab:84:94 [ether] on ens192
packer@tenant01-service-interface-01:~$ ping 10.1.1.10 -c 1
PING 10.1.1.10 (10.1.1.10) 56(84) bytes of data.
64 bytes from 10.1.1.10: icmp_seq=1 ttl=62 time=0.674 ms
packer@tenant01-service-interface-01:~$ ping 172.31.100.30 -c 1
PING 172.31.100.30 (172.31.100.30) 56(84) bytes of data.
64 bytes from 172.31.100.30: icmp_seq=1 ttl=60 time=0.920 ms
Connectivity on the Virtual machine on the dvPG
T0-SR EDGE 02 MAC Interface
Successful ping to Northbound VM
Successful ping to Overlay VM
Confidential │ ©2021 VMware, Inc. 30
NSX-T Configuration
Representation within vSphere and NSX-T Edge VM
Only Service Interface traffic is represented in
this diagram for simplicity reason
If multiple Edge Node are hosted by the same
ESXi hypervisor, all traffic will be pinned to the
same uplink. In this case, all edges have been
deployed with the same uplink profile.
Since Uplink-1 is down, Network traffic for
VLAN 300 (Service Interface) will be pinned to
Uplink-2 (dvPG configuration)
From an NSX-T standpoint, Traffic is still pinned
to fp-eth0
vSphere is handling the Failover (not NSX-T)
N-VDS-01
Mgmt IP
Uplink-1 Uplink-2
Edge-01
Mgmt IP
Trunk-TOR-LEFT
A/S Failover Order
Trunk-TOR-RIGHT
A/S Failover Order
ToR-01 ToR-02
Management-PG
VDS
Uplink 1 Uplink 2
VLAN10
TEP-IP-1 TEP-IP-2
ESXi Host
N-VDS-01
Uplink-1 Uplink-2
TEP-IP-1 TEP-IP-2
Tier-0 (A)
eth0 fp-eth2 eth0 fp-eth2
fp-eth0 fp-eth1 fp-eth1
fp-eth0
VLAN13
Edge-02
VLAN11
Tier-0 (S)
VLAN12
TOR FAILURE
Uplink configured as a trunk
VLAN 300 – Service Interface
Uplink configured as a trunk
VLAN 300 – Service Interface
VLAN300 VLAN300
Confidential │ ©2021 VMware, Inc. 31
NSX-T Configuration
packer@tenant01-service-interface-01:~$ arp -a
_gateway (172.31.210.1) at 00:50:56:ab:ce:ac [ether] on ens192
packer@tenant01-service-interface-01:~$ ping 10.1.1.10 -c 1
PING 10.1.1.10 (10.1.1.10) 56(84) bytes of data.
64 bytes from 10.1.1.10: icmp_seq=1 ttl=62 time=1.07 ms
packer@tenant01-service-interface-01:~$ ping 172.31.100.30 -c 1
PING 172.31.100.30 (172.31.100.30) 56(84) bytes of data.
64 bytes from 172.31.100.30: icmp_seq=1 ttl=60 time=1.27 ms
Connectivity on the Virtual machine on the dvPG
T0-SR EDGE 01 MAC Interface
Successful ping to Northbound VM
Successful ping to Overlay VM
Confidential │ ©2021 VMware, Inc. 32
NSX-T Configuration
- Service Interface Segment using another
uplink teaming policy for testing purpose
(ToR02)
- Verification
Confidential │ ©2021 VMware, Inc. 33
NSX-T Configuration
Segment for the Service Interface – TOR 02
Teaming Policy has been changed to TOR-
02.
Uplink Teaming Policy:
- Supports a single Teaming Policy
- TOR-2:
- Normal Behavior:
- Use fp-eth1 connected to dvPG TOR-RIGHT
- dvPG Config (Active/Standby) : dvUPLINK2
- Failover Behavior:
- Use fp-eth1 connected to dvPG TOR-RIGHT
- dvPG Config (Active/Standby) : dvUPLINK1
Confidential │ ©2021 VMware, Inc. 34
NSX-T Configuration
SRV-EDGE-01(tier0_sr)> get interfaces
Fri May 21 2021 UTC 05:14:33.012
Logical Router
UUID VRF LR-ID Name Type
a76ffe3e-8ed8-4509-a65b-5e52e43cda15 13 46 SR-Tier0-Tenant01 SERVICE_ROUTER_TIER0
Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable)
Interface : ddb58170-00eb-4651-90e8-ac3dec6fccd7
Ifuid : 292
Name : Service-Interface
Fwd-mode : IPV4_AND_IPV6
Internal name : service-292
Mode : lif
Port-type : service
IP/Mask : 172.31.210.1/24
MAC : 00:50:56:ab:03:e1
VLAN : 300
Access-VLAN : untagged
LS port : 531af945-0f5a-43c9-9313-843e622d8027
Urpf-mode : STRICT_MODE
DAD-mode : LOOSE
RA-mode : SLAAC_DNS_TRHOUGH_RA(M=0, O=0)
Admin : up
Op_state : up
MTU : 9000
arp_proxy :
EDGE-01 Get Interfaces - ACTIVE
Confidential │ ©2021 VMware, Inc. 35
NSX-T Configuration
SRV-EDGE-02(tier0_sr)> get interfaces
Fri May 21 2021 UTC 07:31:32.497
Logical Router
UUID VRF LR-ID Name Type
126bf0b3-1c60-466b-ae09-f89562e33634 13 43 SR-Tier0-Tenant01 SERVICE_ROUTER_TIER0
Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable)
Interface : ddb58170-00eb-4651-90e8-ac3dec6fccd7
Ifuid : 397
Name : Service-Interface
Fwd-mode : IPV4_AND_IPV6
Internal name : service-397
Mode : lif
Port-type : service
IP/Mask : 172.31.210.1/24
MAC : 00:50:56:ab:f6:7d
VLAN : 300
Access-VLAN : untagged
LS port : 531af945-0f5a-43c9-9313-843e622d8027
Urpf-mode : STRICT_MODE
DAD-mode : LOOSE
RA-mode : SLAAC_DNS_TRHOUGH_RA(M=0, O=0)
Admin : up
Op_state : down
MTU : 9000
arp_proxy :
EDGE-02 Get Interfaces - STANDBY
Confidential │ ©2021 VMware, Inc. 36
NSX-T Configuration
Representation within vSphere and NSX-T Edge VM
Only Service Interface traffic is represented in
this diagram for simplicity reason
Uplink Teaming Policy configured on the
segment: TOR-02
If multiple Edge Node are hosted by the same
ESXi hypervisor, all traffic will be pinned to the
same uplink. In this case, all edge nodes have
been deployed with the same uplink profile.
Traffic for VLAN 300 (Service Interface) will be
pinned to fp-eth1 from an NSX-T standpoint
and to Uplink-2 from a vSphere perspective.
N-VDS-01
Mgmt IP
Uplink-1 Uplink-2
Edge-01
Mgmt IP
Trunk-TOR-LEFT
A/S Failover Order
Trunk-TOR-RIGHT
A/S Failover Order
ToR-01 ToR-02
Management-PG
VDS
Uplink 1 Uplink 2
VLAN10
TEP-IP-1 TEP-IP-2
ESXi Host
N-VDS-01
Uplink-1 Uplink-2
TEP-IP-1 TEP-IP-2
Tier-0 (A)
eth0 fp-eth2 eth0 fp-eth2
fp-eth0 fp-eth1 fp-eth1
fp-eth0
Uplink configured as a trunk Uplink configured as a trunk
VLAN13
Edge-02
VLAN11
Tier-0 (S)
VLAN12
VLAN 300 – Service Interface VLAN 300 – Service Interface
NORMAL MODE
VLAN300 VLAN300
Confidential │ ©2021 VMware, Inc. 37
NSX-T Configuration
packer@tenant01-service-interface-01:~$ arp -a
_gateway (172.31.210.1) at 00:50:56:ab:03:e1 [ether] on ens192
packer@tenant01-service-interface-01:~$ ping 10.1.1.10 -c 1
PING 10.1.1.10 (10.1.1.10) 56(84) bytes of data.
64 bytes from 10.1.1.10: icmp_seq=1 ttl=62 time=0.987 ms
packer@tenant01-service-interface-01:~$ ping 172.31.100.30 -c 1
PING 172.31.100.30 (172.31.100.30) 56(84) bytes of data.
64 bytes from 172.31.100.30: icmp_seq=1 ttl=60 time=0.912 ms
Connectivity on the Virtual machine on the dvPG
T0-SR EDGE 01 MAC Interface
Successful ping to Northbound VM
Successful ping to Overlay VM
Confidential │ ©2021 VMware, Inc. 38
NSX-T Configuration
[root@srv-esxi-01:~] esxtop
PORT-ID USED-BY TEAM-PNIC DNAME PKTTX/s MbTX/s PSZTX PKTRX/s MbRX/s PSZRX %DRPTX %DRPRX
67108959 2147032:SRV-EDGE-01.eth2 vmnic1 DvsPortset-0 3.39 0.00 66.00 6.78 0.00 91.00 0.00 0.00
67108960 2147032:SRV-EDGE-01.eth1 vmnic0 DvsPortset-0 3.39 0.00 116.00 6.78 0.00 66.00 0.00 0.00
67108961 2147032:SRV-EDGE-01.eth0 vmnic0 DvsPortset-0 3.39 0.00 66.00 0.00 0.00 0.00 0.00 0.00
[root@srv-esxi-01:~] pktcap-uw --switchport 67108959 --dir 2 --vlan 300 -o /tmp/capture.pcap
Verification – ESXTOP (N)
 ICMP Traffic is being received on fp-eth1
 Packet 64 – 68 : ICMP with Overlay VM
 Packet 146 – 149: ICMP with Northbound VM
Confidential │ ©2021 VMware, Inc. 39
NSX-T Configuration
Representation within vSphere and NSX-T Edge VM
Only Service Interface traffic is represented in
this diagram for simplicity reason
Uplink Teaming Policy configured on the
segment: TOR-02
If multiple Edge Node are hosted by the same
ESXi hypervisor, all traffic will be pinned to the
same uplink. In this case, all edge nodes have
been deployed with the same uplink profile.
Traffic for VLAN 300 (Service Interface) will be
pinned to fp-eth1 from an NSX-T standpoint
and to Uplink-2 from a vSphere perspective.
In this case, the T0-SR hosted on the Edge
Node 02, will send a Gratuitous ARP on VLAN
so that the devices can update their ARP entry
for 172.31.210.1 . (see next slide)
N-VDS-01
Mgmt IP
Uplink-1 Uplink-2
Edge-01
Mgmt IP
Trunk-TOR-LEFT
A/S Failover Order
Trunk-TOR-RIGHT
A/S Failover Order
ToR-01 ToR-02
Management-PG
VDS
Uplink 1 Uplink 2
VLAN10
TEP-IP-1 TEP-IP-2
ESXi Host
N-VDS-01
Uplink-1 Uplink-2
TEP-IP-1 TEP-IP-2
Tier-0 (DOWN)
eth0 fp-eth2 eth0 fp-eth2
fp-eth0 fp-eth1 fp-eth1
fp-eth0
Uplink configured as a trunk Uplink configured as a trunk
VLAN13
Edge-02
VLAN11
Tier-0 (A)
VLAN12
VLAN 300 – Service Interface VLAN 300 – Service Interface
VLAN300 VLAN300
EDGE FAILURE
Confidential │ ©2021 VMware, Inc. 40
NSX-T Configuration
packer@tenant01-service-interface-01:~$ arp -a
_gateway (172.31.210.1) at 00:50:56:ab:f6:7d [ether] on ens192
packer@tenant01-service-interface-01:~$ ping 10.1.1.10 -c 1
PING 10.1.1.10 (10.1.1.10) 56(84) bytes of data.
64 bytes from 10.1.1.10: icmp_seq=1 ttl=62 time=0.127 ms
packer@tenant01-service-interface-01:~$ ping 172.31.100.30 -c 1
PING 172.31.100.30 (172.31.100.30) 56(84) bytes of data.
64 bytes from 172.31.100.30: icmp_seq=1 ttl=60 time=0.100 ms
Connectivity on the Virtual machine on the dvPG
T0-SR EDGE 01 MAC Interface
Successful ping to Northbound VM
Successful ping to Overlay VM
Confidential │ ©2021 VMware, Inc. 41
NSX-T Configuration
Representation within vSphere and NSX-T Edge VM
Only Service Interface traffic is represented in
this diagram for simplicity reason
Uplink Teaming Policy configured on the
segment: TOR-02
If multiple Edge Node are hosted by the same
ESXi hypervisor, all traffic will be pinned to the
same uplink. In this case, all edge nodes have
been deployed with the same uplink profile.
Traffic for VLAN 300 (Service Interface) will be
pinned to fp-eth1 from an NSX-T standpoint
and to Uplink-2 from a vSphere perspective.
N-VDS-01
Mgmt IP
Uplink-1 Uplink-2
Edge-01
Mgmt IP
Trunk-TOR-LEFT
A/S Failover Order
Trunk-TOR-RIGHT
A/S Failover Order
ToR-01 ToR-02
Management-PG
VDS
Uplink 1 Uplink 2
VLAN10
TEP-IP-1 TEP-IP-2
ESXi Host
N-VDS-01
Uplink-1 Uplink-2
TEP-IP-1 TEP-IP-2
Tier-0 (A)
eth0 fp-eth2 eth0 fp-eth2
fp-eth0 fp-eth1 fp-eth1
fp-eth0
Uplink configured as a trunk Uplink configured as a trunk
VLAN13
Edge-02
VLAN11
Tier-0 (S)
VLAN12
VLAN 300 – Service Interface VLAN 300 – Service Interface
VLAN300 VLAN300
TOR FAILURE
Thank You
Confidential │ ©2021 VMware, Inc.

More Related Content

Similar to NSX-T and Service Interfaces presentation

Deploying Carrier Ethernet Features on Cisco ASR 9000
Deploying Carrier Ethernet Features on Cisco ASR 9000Deploying Carrier Ethernet Features on Cisco ASR 9000
Deploying Carrier Ethernet Features on Cisco ASR 9000Vinod Kumar Balasubramanyam
 
Overview of Distributed Virtual Router (DVR) in Openstack/Neutron
Overview of Distributed Virtual Router (DVR) in Openstack/NeutronOverview of Distributed Virtual Router (DVR) in Openstack/Neutron
Overview of Distributed Virtual Router (DVR) in Openstack/Neutronvivekkonnect
 
CCNA R&S-13-Spanning Tree Protocol Implementation
CCNA R&S-13-Spanning Tree Protocol ImplementationCCNA R&S-13-Spanning Tree Protocol Implementation
CCNA R&S-13-Spanning Tree Protocol ImplementationAmir Jafari
 
vPC techonology for full ha from dc core to baremetel server.
vPC techonology for full ha from dc core to baremetel server.vPC techonology for full ha from dc core to baremetel server.
vPC techonology for full ha from dc core to baremetel server.Ajeet Singh
 
EYWA Presentation v0.1.27
EYWA Presentation v0.1.27EYWA Presentation v0.1.27
EYWA Presentation v0.1.27JungIn Jung
 
Technical introduction to MidoNet
Technical introduction to MidoNetTechnical introduction to MidoNet
Technical introduction to MidoNetMidoNet
 
Presentation dc design for small and mid-size data center
Presentation   dc design for small and mid-size data centerPresentation   dc design for small and mid-size data center
Presentation dc design for small and mid-size data centerxKinAnx
 
VMware NSX-T Design for Small to Mid-Sized Data Centers v1.0 EN.pptx
VMware NSX-T Design for Small to Mid-Sized Data Centers v1.0 EN.pptxVMware NSX-T Design for Small to Mid-Sized Data Centers v1.0 EN.pptx
VMware NSX-T Design for Small to Mid-Sized Data Centers v1.0 EN.pptxHythamsaadeh
 
Ccna 3 v 4.0 final-exam-17-07-2010
Ccna 3 v 4.0 final-exam-17-07-2010Ccna 3 v 4.0 final-exam-17-07-2010
Ccna 3 v 4.0 final-exam-17-07-2010irbas
 
Ccna 3 v4.0 final-exam-17-07-2010
Ccna 3 v4.0  final-exam-17-07-2010Ccna 3 v4.0  final-exam-17-07-2010
Ccna 3 v4.0 final-exam-17-07-2010irbas
 
Day 20.1 configuringframerelay
Day 20.1 configuringframerelayDay 20.1 configuringframerelay
Day 20.1 configuringframerelayCYBERINTELLIGENTS
 
Fttx configuration-577 k-_ver_31072011
Fttx configuration-577 k-_ver_31072011Fttx configuration-577 k-_ver_31072011
Fttx configuration-577 k-_ver_31072011Igors Cardoso
 
SR-IOV ixgbe Driver Limitations and Improvement
SR-IOV ixgbe Driver Limitations and ImprovementSR-IOV ixgbe Driver Limitations and Improvement
SR-IOV ixgbe Driver Limitations and ImprovementLF Events
 
Ovs dpdk hwoffload way to full offload
Ovs dpdk hwoffload way to full offloadOvs dpdk hwoffload way to full offload
Ovs dpdk hwoffload way to full offloadKevin Traynor
 

Similar to NSX-T and Service Interfaces presentation (20)

Deploying Carrier Ethernet features on ASR 9000
Deploying Carrier Ethernet features on ASR 9000Deploying Carrier Ethernet features on ASR 9000
Deploying Carrier Ethernet features on ASR 9000
 
Deploying Carrier Ethernet Features on Cisco ASR 9000
Deploying Carrier Ethernet Features on Cisco ASR 9000Deploying Carrier Ethernet Features on Cisco ASR 9000
Deploying Carrier Ethernet Features on Cisco ASR 9000
 
Overview of Distributed Virtual Router (DVR) in Openstack/Neutron
Overview of Distributed Virtual Router (DVR) in Openstack/NeutronOverview of Distributed Virtual Router (DVR) in Openstack/Neutron
Overview of Distributed Virtual Router (DVR) in Openstack/Neutron
 
CCNA R&S-13-Spanning Tree Protocol Implementation
CCNA R&S-13-Spanning Tree Protocol ImplementationCCNA R&S-13-Spanning Tree Protocol Implementation
CCNA R&S-13-Spanning Tree Protocol Implementation
 
vPC techonology for full ha from dc core to baremetel server.
vPC techonology for full ha from dc core to baremetel server.vPC techonology for full ha from dc core to baremetel server.
vPC techonology for full ha from dc core to baremetel server.
 
EYWA Presentation v0.1.27
EYWA Presentation v0.1.27EYWA Presentation v0.1.27
EYWA Presentation v0.1.27
 
Frame Relay
Frame RelayFrame Relay
Frame Relay
 
Frame Relay
Frame RelayFrame Relay
Frame Relay
 
Technical introduction to MidoNet
Technical introduction to MidoNetTechnical introduction to MidoNet
Technical introduction to MidoNet
 
Presentation dc design for small and mid-size data center
Presentation   dc design for small and mid-size data centerPresentation   dc design for small and mid-size data center
Presentation dc design for small and mid-size data center
 
Day 13.1..1 catalyst switch
Day 13.1..1 catalyst switchDay 13.1..1 catalyst switch
Day 13.1..1 catalyst switch
 
VMware NSX-T Design for Small to Mid-Sized Data Centers v1.0 EN.pptx
VMware NSX-T Design for Small to Mid-Sized Data Centers v1.0 EN.pptxVMware NSX-T Design for Small to Mid-Sized Data Centers v1.0 EN.pptx
VMware NSX-T Design for Small to Mid-Sized Data Centers v1.0 EN.pptx
 
Ccna 3 v 4.0 final-exam-17-07-2010
Ccna 3 v 4.0 final-exam-17-07-2010Ccna 3 v 4.0 final-exam-17-07-2010
Ccna 3 v 4.0 final-exam-17-07-2010
 
Ccna 3 v4.0 final-exam-17-07-2010
Ccna 3 v4.0  final-exam-17-07-2010Ccna 3 v4.0  final-exam-17-07-2010
Ccna 3 v4.0 final-exam-17-07-2010
 
Day 20.1 configuringframerelay
Day 20.1 configuringframerelayDay 20.1 configuringframerelay
Day 20.1 configuringframerelay
 
Fttx configuration-577 k-_ver_31072011
Fttx configuration-577 k-_ver_31072011Fttx configuration-577 k-_ver_31072011
Fttx configuration-577 k-_ver_31072011
 
SR-IOV ixgbe Driver Limitations and Improvement
SR-IOV ixgbe Driver Limitations and ImprovementSR-IOV ixgbe Driver Limitations and Improvement
SR-IOV ixgbe Driver Limitations and Improvement
 
Ies5000 usg
Ies5000 usgIes5000 usg
Ies5000 usg
 
Day 20.3 frame relay
Day 20.3 frame relay Day 20.3 frame relay
Day 20.3 frame relay
 
Ovs dpdk hwoffload way to full offload
Ovs dpdk hwoffload way to full offloadOvs dpdk hwoffload way to full offload
Ovs dpdk hwoffload way to full offload
 

Recently uploaded

Thank You Luv I’ll Never Walk Alone Again T shirts
Thank You Luv I’ll Never Walk Alone Again T shirtsThank You Luv I’ll Never Walk Alone Again T shirts
Thank You Luv I’ll Never Walk Alone Again T shirtsrahman018755
 
一比一原版(Design毕业证书)新加坡科技设计大学毕业证原件一模一样
一比一原版(Design毕业证书)新加坡科技设计大学毕业证原件一模一样一比一原版(Design毕业证书)新加坡科技设计大学毕业证原件一模一样
一比一原版(Design毕业证书)新加坡科技设计大学毕业证原件一模一样AS
 
APNIC Policy Roundup presented by Sunny Chendi at TWNOG 5.0
APNIC Policy Roundup presented by Sunny Chendi at TWNOG 5.0APNIC Policy Roundup presented by Sunny Chendi at TWNOG 5.0
APNIC Policy Roundup presented by Sunny Chendi at TWNOG 5.0APNIC
 
SOC Analyst Guide For Beginners SOC analysts work as members of a managed sec...
SOC Analyst Guide For Beginners SOC analysts work as members of a managed sec...SOC Analyst Guide For Beginners SOC analysts work as members of a managed sec...
SOC Analyst Guide For Beginners SOC analysts work as members of a managed sec...Varun Mithran
 
一比一原版布兰迪斯大学毕业证如何办理
一比一原版布兰迪斯大学毕业证如何办理一比一原版布兰迪斯大学毕业证如何办理
一比一原版布兰迪斯大学毕业证如何办理A
 
一比一原版(Wintec毕业证书)新西兰怀卡托理工学院毕业证原件一模一样
一比一原版(Wintec毕业证书)新西兰怀卡托理工学院毕业证原件一模一样一比一原版(Wintec毕业证书)新西兰怀卡托理工学院毕业证原件一模一样
一比一原版(Wintec毕业证书)新西兰怀卡托理工学院毕业证原件一模一样AS
 
iThome_CYBERSEC2024_Drive_Into_the_DarkWeb
iThome_CYBERSEC2024_Drive_Into_the_DarkWebiThome_CYBERSEC2024_Drive_Into_the_DarkWeb
iThome_CYBERSEC2024_Drive_Into_the_DarkWebJie Liau
 
一比一原版(Soton毕业证书)南安普顿大学毕业证原件一模一样
一比一原版(Soton毕业证书)南安普顿大学毕业证原件一模一样一比一原版(Soton毕业证书)南安普顿大学毕业证原件一模一样
一比一原版(Soton毕业证书)南安普顿大学毕业证原件一模一样Fi
 
一比一原版(毕业证书)新西兰怀特克利夫艺术设计学院毕业证原件一模一样
一比一原版(毕业证书)新西兰怀特克利夫艺术设计学院毕业证原件一模一样一比一原版(毕业证书)新西兰怀特克利夫艺术设计学院毕业证原件一模一样
一比一原版(毕业证书)新西兰怀特克利夫艺术设计学院毕业证原件一模一样AS
 
100^%)( POLOKWANE))(*((+27838792658))*))௹ )Abortion Pills for Sale in Sibasa,...
100^%)( POLOKWANE))(*((+27838792658))*))௹ )Abortion Pills for Sale in Sibasa,...100^%)( POLOKWANE))(*((+27838792658))*))௹ )Abortion Pills for Sale in Sibasa,...
100^%)( POLOKWANE))(*((+27838792658))*))௹ )Abortion Pills for Sale in Sibasa,...musaddumba454
 
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样Fi
 
I’ll See Y’All Motherfuckers In Game 7 Shirt
I’ll See Y’All Motherfuckers In Game 7 ShirtI’ll See Y’All Motherfuckers In Game 7 Shirt
I’ll See Y’All Motherfuckers In Game 7 Shirtrahman018755
 
一比一原版(NYU毕业证书)美国纽约大学毕业证学位证书
一比一原版(NYU毕业证书)美国纽约大学毕业证学位证书一比一原版(NYU毕业证书)美国纽约大学毕业证学位证书
一比一原版(NYU毕业证书)美国纽约大学毕业证学位证书c6eb683559b3
 
如何办理(UCLA毕业证)加州大学洛杉矶分校毕业证成绩单本科硕士学位证留信学历认证
如何办理(UCLA毕业证)加州大学洛杉矶分校毕业证成绩单本科硕士学位证留信学历认证如何办理(UCLA毕业证)加州大学洛杉矶分校毕业证成绩单本科硕士学位证留信学历认证
如何办理(UCLA毕业证)加州大学洛杉矶分校毕业证成绩单本科硕士学位证留信学历认证hfkmxufye
 
Dan Quinn Commanders Feather Dad Hat Hoodie
Dan Quinn Commanders Feather Dad Hat HoodieDan Quinn Commanders Feather Dad Hat Hoodie
Dan Quinn Commanders Feather Dad Hat Hoodierahman018755
 
一比一原版英国创意艺术大学毕业证如何办理
一比一原版英国创意艺术大学毕业证如何办理一比一原版英国创意艺术大学毕业证如何办理
一比一原版英国创意艺术大学毕业证如何办理AS
 
Washington Football Commanders Redskins Feathers Shirt
Washington Football Commanders Redskins Feathers ShirtWashington Football Commanders Redskins Feathers Shirt
Washington Football Commanders Redskins Feathers Shirtrahman018755
 
Free on Wednesdays T Shirts Free on Wednesdays Sweatshirts
Free on Wednesdays T Shirts Free on Wednesdays SweatshirtsFree on Wednesdays T Shirts Free on Wednesdays Sweatshirts
Free on Wednesdays T Shirts Free on Wednesdays Sweatshirtsrahman018755
 
[Hackersuli] Élő szövet a fémvázon: Python és gépi tanulás a Zeek platformon
[Hackersuli] Élő szövet a fémvázon: Python és gépi tanulás a Zeek platformon[Hackersuli] Élő szövet a fémvázon: Python és gépi tanulás a Zeek platformon
[Hackersuli] Élő szövet a fémvázon: Python és gépi tanulás a Zeek platformonhackersuli
 

Recently uploaded (20)

Thank You Luv I’ll Never Walk Alone Again T shirts
Thank You Luv I’ll Never Walk Alone Again T shirtsThank You Luv I’ll Never Walk Alone Again T shirts
Thank You Luv I’ll Never Walk Alone Again T shirts
 
GOOGLE Io 2024 At takes center stage.pdf
GOOGLE Io 2024 At takes center stage.pdfGOOGLE Io 2024 At takes center stage.pdf
GOOGLE Io 2024 At takes center stage.pdf
 
一比一原版(Design毕业证书)新加坡科技设计大学毕业证原件一模一样
一比一原版(Design毕业证书)新加坡科技设计大学毕业证原件一模一样一比一原版(Design毕业证书)新加坡科技设计大学毕业证原件一模一样
一比一原版(Design毕业证书)新加坡科技设计大学毕业证原件一模一样
 
APNIC Policy Roundup presented by Sunny Chendi at TWNOG 5.0
APNIC Policy Roundup presented by Sunny Chendi at TWNOG 5.0APNIC Policy Roundup presented by Sunny Chendi at TWNOG 5.0
APNIC Policy Roundup presented by Sunny Chendi at TWNOG 5.0
 
SOC Analyst Guide For Beginners SOC analysts work as members of a managed sec...
SOC Analyst Guide For Beginners SOC analysts work as members of a managed sec...SOC Analyst Guide For Beginners SOC analysts work as members of a managed sec...
SOC Analyst Guide For Beginners SOC analysts work as members of a managed sec...
 
一比一原版布兰迪斯大学毕业证如何办理
一比一原版布兰迪斯大学毕业证如何办理一比一原版布兰迪斯大学毕业证如何办理
一比一原版布兰迪斯大学毕业证如何办理
 
一比一原版(Wintec毕业证书)新西兰怀卡托理工学院毕业证原件一模一样
一比一原版(Wintec毕业证书)新西兰怀卡托理工学院毕业证原件一模一样一比一原版(Wintec毕业证书)新西兰怀卡托理工学院毕业证原件一模一样
一比一原版(Wintec毕业证书)新西兰怀卡托理工学院毕业证原件一模一样
 
iThome_CYBERSEC2024_Drive_Into_the_DarkWeb
iThome_CYBERSEC2024_Drive_Into_the_DarkWebiThome_CYBERSEC2024_Drive_Into_the_DarkWeb
iThome_CYBERSEC2024_Drive_Into_the_DarkWeb
 
一比一原版(Soton毕业证书)南安普顿大学毕业证原件一模一样
一比一原版(Soton毕业证书)南安普顿大学毕业证原件一模一样一比一原版(Soton毕业证书)南安普顿大学毕业证原件一模一样
一比一原版(Soton毕业证书)南安普顿大学毕业证原件一模一样
 
一比一原版(毕业证书)新西兰怀特克利夫艺术设计学院毕业证原件一模一样
一比一原版(毕业证书)新西兰怀特克利夫艺术设计学院毕业证原件一模一样一比一原版(毕业证书)新西兰怀特克利夫艺术设计学院毕业证原件一模一样
一比一原版(毕业证书)新西兰怀特克利夫艺术设计学院毕业证原件一模一样
 
100^%)( POLOKWANE))(*((+27838792658))*))௹ )Abortion Pills for Sale in Sibasa,...
100^%)( POLOKWANE))(*((+27838792658))*))௹ )Abortion Pills for Sale in Sibasa,...100^%)( POLOKWANE))(*((+27838792658))*))௹ )Abortion Pills for Sale in Sibasa,...
100^%)( POLOKWANE))(*((+27838792658))*))௹ )Abortion Pills for Sale in Sibasa,...
 
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样
一比一原版(UWE毕业证书)西英格兰大学毕业证原件一模一样
 
I’ll See Y’All Motherfuckers In Game 7 Shirt
I’ll See Y’All Motherfuckers In Game 7 ShirtI’ll See Y’All Motherfuckers In Game 7 Shirt
I’ll See Y’All Motherfuckers In Game 7 Shirt
 
一比一原版(NYU毕业证书)美国纽约大学毕业证学位证书
一比一原版(NYU毕业证书)美国纽约大学毕业证学位证书一比一原版(NYU毕业证书)美国纽约大学毕业证学位证书
一比一原版(NYU毕业证书)美国纽约大学毕业证学位证书
 
如何办理(UCLA毕业证)加州大学洛杉矶分校毕业证成绩单本科硕士学位证留信学历认证
如何办理(UCLA毕业证)加州大学洛杉矶分校毕业证成绩单本科硕士学位证留信学历认证如何办理(UCLA毕业证)加州大学洛杉矶分校毕业证成绩单本科硕士学位证留信学历认证
如何办理(UCLA毕业证)加州大学洛杉矶分校毕业证成绩单本科硕士学位证留信学历认证
 
Dan Quinn Commanders Feather Dad Hat Hoodie
Dan Quinn Commanders Feather Dad Hat HoodieDan Quinn Commanders Feather Dad Hat Hoodie
Dan Quinn Commanders Feather Dad Hat Hoodie
 
一比一原版英国创意艺术大学毕业证如何办理
一比一原版英国创意艺术大学毕业证如何办理一比一原版英国创意艺术大学毕业证如何办理
一比一原版英国创意艺术大学毕业证如何办理
 
Washington Football Commanders Redskins Feathers Shirt
Washington Football Commanders Redskins Feathers ShirtWashington Football Commanders Redskins Feathers Shirt
Washington Football Commanders Redskins Feathers Shirt
 
Free on Wednesdays T Shirts Free on Wednesdays Sweatshirts
Free on Wednesdays T Shirts Free on Wednesdays SweatshirtsFree on Wednesdays T Shirts Free on Wednesdays Sweatshirts
Free on Wednesdays T Shirts Free on Wednesdays Sweatshirts
 
[Hackersuli] Élő szövet a fémvázon: Python és gépi tanulás a Zeek platformon
[Hackersuli] Élő szövet a fémvázon: Python és gépi tanulás a Zeek platformon[Hackersuli] Élő szövet a fémvázon: Python és gépi tanulás a Zeek platformon
[Hackersuli] Élő szövet a fémvázon: Python és gépi tanulás a Zeek platformon
 

NSX-T and Service Interfaces presentation

  • 1. Confidential │ ©2021 VMware, Inc. Service Interface Guidelines and Design Nicolas Michel Technical Product Manager | NSBU November 2021
  • 2. Confidential │ ©2021 VMware, Inc. 2 Agenda Service Interface introduction Support and Topologies Lab Topology Diagram and Description vSphere Configuration dVPG TOR-01 / dVPG TOR-02 / dVPG NSX-T Configuration UI Configuration
  • 3. Confidential │ ©2021 VMware, Inc. 3 Service Interface Introduction
  • 4. Confidential │ ©2021 VMware, Inc. 4 Service Interface Support in NSX-T 3.2 Connectivity Originally developed to connect vlan backed segments. Overlay Segments also supported. On a Tier0, Uplink are recommended. On a Tier1, Service Interfaces are mandatory to provide connectivity to vlan backed segment (no Uplink on Tier0). Service interfaces using the same vlan backed segment can not be instantiated on different logical routers hosted on the same edge node. Topologies Can be used to interconnect Tier-1 gateways together (static routing manually configured). Service Interfaces are supported in Active/Standby topologies only. Networking and Services Dynamic Routing protocols are not supported (OSPF, BGP). Except for EVPN Route Server Mode Static Routing is supported. IPsec not supported A Service Interface supports the following services: • DHCP relay. • DHCP Server • NAT • Gateway Firewall • Native Load Balancer
  • 5. Confidential │ ©2021 VMware, Inc. 5 Service Interface Introduction Service Interface*: Interface connecting Vlan backed logical switch to provide connectivity to vlan backed physical or virtual workloads. *Referred as Centralized Service Port (CSP) in NSX-T 2.3 Baremetal Servers Tier-1 Tier-1 Gateway Tier-0 Gateway Vlan Segment SI-2 Baremetal Servers
  • 6. Confidential │ ©2021 VMware, Inc. 6 Service Interface Introduction Service Interface: Can also be connected to Overlay Segment for Load balancer use cases. Tier-1 Gateway SI-2 Overlay or Vlan Segment SI-1 Tier-1 Gateway Tier-1 Gateway Standalone Physical Router Vlan Segment Baremetal Servers Tier-0 Gateway
  • 7. Confidential │ ©2021 VMware, Inc. 7 Service Interface Topology Interconnect Tier1 gateways Service interface can be used to connect 2 Tier1 gateway together. A service interface is used on the Org VDC Tier1 A Linked Segment is used on the vApp Tier1 Static routing must be configured manually on both Tier1 gateways: • Org VDC Tier1 should have static routes for the vApp tier1 segments : • Static route to 10.2.2.0/24 with a next hop of 172.16.0.2 • Static route to 10.2.3.0/24 with a next hop of 172.16.0.2 • vApp Tier1 should have a default route pointing to the Service Port Tier1 interfaces hosted on the Org VDC: • Static route to 0.0.0.0/0 with a next hop of 172.16.0.1
  • 8. Confidential │ ©2021 VMware, Inc. 8 Service Interface Supported Topology Identical Overlay backed segment used on different Tier-1 Service Interface Gateways. 10.1.1.1/24 on Tier-1 Tenant-01 10.1.1.2/24 on Tier-1 Tenant-02 Supported Topology Tier-1 Gateway Tenant-01 Segment – Vlan 10 Active: Edge-01 Tier-1 Gateway Tenant-02 Tier-0 Gateway Same Active Edge node OVERLAY BACKED Active: Edge-01
  • 9. Confidential │ ©2021 VMware, Inc. 9 Service Interface NOT Supported Topology Identical VLAN backed segment used on different Tier-1 Service Interface Gateways. 10.1.1.1/24 on Tier-1 Tenant-01 10.1.1.2/24 on Tier-1 Tenant-02 Topology not supported. Tier-1 must be in different edge clusters if they share interfaces on the same segment. Tier-1 Gateway Tenant-01 Segment – Vlan 10 Active: Edge-01 Tier-1 Gateway Tenant-02 Tier-0 Gateway Same Active Edge node VLAN BACKED Active: Edge-01
  • 10. Confidential │ ©2021 VMware, Inc. 10 Topology
  • 11. Confidential │ ©2021 VMware, Inc. 11 Topology Used Edge Node VM Design - 2 pNICs Design NSX-T Edge 2 PNIC Design Single N-VDS to simplify design and deployment. Multi-TEP on Edge is supported. Single VLAN per uplink is recommended (BGP / OSPF / Static). Easier to troubleshoot. Best Practice in the network industry. Named Teaming Policy is used to map each vNIC to NSX Segment to pNIC to ToR (Detailed in the next slides). Single VLAN and Overlay Transport Zone. VLAN used VLAN 110 is the vlan used for the TEP and is shared between TOR-01 and TOR-02 VLAN 10 is the vlan used to establish the BGP peering between TOR-01 and T0-SR on EDGE-NODE-01 VLAN 13 is the vlan used to establish the BGP peering between TOR-02 and T0-SR on EDGE-NODE-01 VLAN 11 is the vlan used to establish the BGP peering between TOR-01 and T0-SR on EDGE-NODE-02 VLAN 12 is the vlan used to establish the BGP peering between TOR-02 and T0-SR on EDGE-NODE-02 VLAN 300 is the vlan used to connect by the Service Interface Segment on NSX-T N-VDS-01 Mgmt IP Uplink-1 Uplink-2 Edge-01 Mgmt IP Trunk-TOR-LEFT A/S Failover Order Trunk-TOR-RIGHT A/S Failover Order ToR-01 ToR-02 Management-PG VDS Uplink 1 Uplink 2 VLAN10 TEP-IP-1 TEP-IP-2 ESXi Host N-VDS-01 Uplink-1 Uplink-2 TEP-IP-1 TEP-IP-2 Tier-0 (A) eth0 fp-eth2 eth0 fp-eth2 fp-eth0 fp-eth1 fp-eth1 fp-eth0 VLAN 110 -TEP VLAN 10 and 11 - BGP Uplink configured as a trunk VLAN 110 - TEP VLAN 12 and 13 - BGP Uplink configured as a trunk VLAN13 Edge-02 VLAN11 Tier-0 (S) VLAN12 VLAN 300 – Service Interface VLAN 300 – Service Interface VLAN300 VLAN300
  • 12. Confidential │ ©2021 VMware, Inc. 12 Topology Used Layer 3 Topology Layer 3 Design Separate VLANs are used for each uplink between the TORs and the T0-SR on EN01 and EN02 VLANs / IP Addressing plan VLAN 300 is the vlan used for the service interface on the T0. Virtual Machine on a standard DVPG (non NSX-T overlay)
  • 13. Confidential │ ©2021 VMware, Inc. 13 vSphere Configuration – vNIC - DVPG - Edge Node Virtual Machine vNIC - DVPG TOR-Left - DVPG TOR-Right - DVPG Service Interface (VM vNIC)
  • 14. Confidential │ ©2021 VMware, Inc. 14 vSphere Configuration – vNIC - DVPG Edge Node Virtual Machine vNIC Network Adapter 2 This vSphere vNIC is considered as fp-eth0 for NSX-T The adapter type is VMXNet3 Network Adapter 3 This vSphere vNIC is considered as fp-eth1 for NSX-T The adapter type is VMXNet3 Network Adapter 4 is not connected to anything.
  • 15. Confidential │ ©2021 VMware, Inc. 15 vSphere Configuration – vNIC - DVPG Distributed Port Group Configuration – TOR LEFT (01)
  • 16. Confidential │ ©2021 VMware, Inc. 16 vSphere Configuration – vNIC - DVPG Distributed Port Group Configuration – TOR RIGHT (02)
  • 17. Confidential │ ©2021 VMware, Inc. 17 vSphere Configuration – vNIC - DVPG Distributed Port Group Configuration – Service Interface DPG (VM vNIC)  Teaming and Failover policy doesn’t really matter in our example as the VM could be hosted anywhere in the DC
  • 18. Confidential │ ©2021 VMware, Inc. 18 NSX-T Configuration - Uplink Profile – Teaming Policies - Service Interface Segment – TOR01 - Service Interface on T0 - Verification
  • 19. Confidential │ ©2021 VMware, Inc. 19 NSX-T Configuration Uplink Profile An Uplink profile defines the way N-VDS operates Transport VLAN: TEP Vlan MTU: N-VDS MTU Teaming Policies: - Default Teaming Policy: Multi TEP (fp-eth0 and fp-eth1) - TOR-1: Use uplink-1 only (fp-eth0) - TOR-2: Use uplink-2 only (fp-eth1)
  • 20. Confidential │ ©2021 VMware, Inc. 20 NSX-T Configuration Teaming policies Teaming Policies Configuration: - Default Teaming Policy: - Multi TEP – Load Balance Source - Uplink-1 maps to fp-eth0 - Uplink-2 maps to fp-eth1 - Named Teaming Policy TOR-1: - Uplink-1 maps to fp-eth0 only - Not supported to have a standby uplink - Name Teaming Policy TOR-2: - Uplink-2 maps to fp-eth1 only - Not supported to have a standby uplink
  • 21. Confidential │ ©2021 VMware, Inc. 21 NSX-T Configuration Segment for the Service Interface – TOR 01 Creation of a VLAN Backed Segment This segment is not attached to a specific T0 or T1. The Transport zone must be VLAN Based The Subnet is not set. It will be configured on the Service interface itself (T0 construct) Transport VLAN must match dvPG VLAN connecting the virtual machine. Uplink Teaming Policy: - Supports a single Teaming Policy - TOR-1: - Normal Behavior: - Use fp-eth0 connected to dvPG TOR-LEFT - dvPG Config (Active/Standby) : dvUPLINK1 - Failover Behavior: - Use fp-eth0 connected to dvPG TOR-LEFT - dvPG Config (Active/Standby) : dvUPLINK2
  • 22. Confidential │ ©2021 VMware, Inc. 22 NSX-T Configuration Service Interface on the T0 Interface type must be “Service” IP Address in the same range as the Virtual Machine. 172.31.210.1 will be the VM’s default gateway. Connected to the Segment created earlier.
  • 23. Confidential │ ©2021 VMware, Inc. 23 NSX-T Configuration SRV-EDGE-01(tier0_sr)> get interfaces Fri May 21 2021 UTC 05:14:33.012 Logical Router UUID VRF LR-ID Name Type a76ffe3e-8ed8-4509-a65b-5e52e43cda15 13 46 SR-Tier0-Tenant01 SERVICE_ROUTER_TIER0 Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable) Interface : ddb58170-00eb-4651-90e8-ac3dec6fccd7 Ifuid : 399 Name : Service-Interface Fwd-mode : IPV4_AND_IPV6 Internal name : service-399 Mode : lif Port-type : service IP/Mask : 172.31.210.1/24 MAC : 00:50:56:ab:ce:ac VLAN : 300 Access-VLAN : untagged LS port : 531af945-0f5a-43c9-9313-843e622d8027 Urpf-mode : STRICT_MODE DAD-mode : LOOSE RA-mode : SLAAC_DNS_TRHOUGH_RA(M=0, O=0) Admin : up Op_state : up MTU : 9000 arp_proxy : EDGE-01 Get Interfaces - ACTIVE
  • 24. Confidential │ ©2021 VMware, Inc. 24 NSX-T Configuration SRV-EDGE-02(tier0_sr)> get interfaces Fri May 21 2021 UTC 05:14:36.985 Logical Router UUID VRF LR-ID Name Type 126bf0b3-1c60-466b-ae09-f89562e33634 13 43 SR-Tier0-Tenant01 SERVICE_ROUTER_TIER0 Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable) Interface : ddb58170-00eb-4651-90e8-ac3dec6fccd7 Ifuid : 397 Name : Service-Interface Fwd-mode : IPV4_AND_IPV6 Internal name : service-397 Mode : lif Port-type : service IP/Mask : 172.31.210.1/24 MAC : 00:50:56:ab:84:94 VLAN : 300 Access-VLAN : untagged LS port : 531af945-0f5a-43c9-9313-843e622d8027 Urpf-mode : STRICT_MODE DAD-mode : LOOSE RA-mode : SLAAC_DNS_TRHOUGH_RA(M=0, O=0) Admin : up Op_state : down MTU : 9000 arp_proxy : EDGE-02 Get Interfaces - STANDBY
  • 25. Confidential │ ©2021 VMware, Inc. 25 NSX-T Configuration Representation within vSphere and NSX-T Edge VM Only Service Interface traffic is represented in this diagram for simplicity reason Uplink Teaming Policy configured on the segment: TOR-01 If multiple Edge Node are hosted by the same ESXi hypervisor, all traffic will be pinned to the same uplink. In this case, all edges have been deployed with the same uplink profile. Traffic for VLAN 300 (Service Interface) will be pinned to fp-eth0 from an NSX-T standpoint and to Uplink-1 from a vSphere perspective. N-VDS-01 Mgmt IP Uplink-1 Uplink-2 Edge-01 Mgmt IP Trunk-TOR-LEFT A/S Failover Order Trunk-TOR-RIGHT A/S Failover Order ToR-01 ToR-02 Management-PG VDS Uplink 1 Uplink 2 VLAN10 TEP-IP-1 TEP-IP-2 ESXi Host N-VDS-01 Uplink-1 Uplink-2 TEP-IP-1 TEP-IP-2 Tier-0 (A) eth0 fp-eth2 eth0 fp-eth2 fp-eth0 fp-eth1 fp-eth1 fp-eth0 Uplink configured as a trunk Uplink configured as a trunk VLAN13 Edge-02 VLAN11 Tier-0 (S) VLAN12 VLAN 300 – Service Interface VLAN 300 – Service Interface NORMAL MODE VLAN300 VLAN300
  • 26. Confidential │ ©2021 VMware, Inc. 26 NSX-T Configuration packer@tenant01-service-interface-01:~$ arp -a _gateway (172.31.210.1) at 00:50:56:ab:ce:ac [ether] on ens192 packer@tenant01-service-interface-01:~$ ping 10.1.1.10 -c 1 PING 10.1.1.10 (10.1.1.10) 56(84) bytes of data. 64 bytes from 10.1.1.10: icmp_seq=1 ttl=62 time=0.717 ms packer@tenant01-service-interface-01:~$ ping 172.31.100.30 -c 1 PING 172.31.100.30 (172.31.100.30) 56(84) bytes of data. 64 bytes from 172.31.100.30: icmp_seq=1 ttl=60 time=1.16 ms Connectivity on the Virtual machine on the dvPG T0-SR EDGE 01 MAC Interface Successful ping to Northbound VM Successful ping to Overlay VM
  • 27. Confidential │ ©2021 VMware, Inc. 27 NSX-T Configuration [root@srv-esxi-01:~] esxtop PORT-ID USED-BY TEAM-PNIC DNAME PKTTX/s MbTX/s PSZTX PKTRX/s MbRX/s PSZRX %DRPTX %DRPRX 67108959 2147032:SRV-EDGE-01.eth2 vmnic1 DvsPortset-0 3.39 0.00 66.00 6.78 0.00 91.00 0.00 0.00 67108960 2147032:SRV-EDGE-01.eth1 vmnic0 DvsPortset-0 3.39 0.00 116.00 6.78 0.00 66.00 0.00 0.00 67108961 2147032:SRV-EDGE-01.eth0 vmnic0 DvsPortset-0 3.39 0.00 66.00 0.00 0.00 0.00 0.00 0.00 [root@srv-esxi-01:~] pktcap-uw --switchport 67108960 --dir 2 --vlan 300 -o /tmp/capture.pcap Verification – ESXTOP (N)  ICMP Traffic is being received on fp-eth0  Packet 64 – 68 : ICMP with Overlay VM  Packet 146 – 149: ICMP with Northbound VM
  • 28. Confidential │ ©2021 VMware, Inc. 28 NSX-T Configuration Representation within vSphere and NSX-T Edge VM Only Service Interface traffic is represented in this diagram for simplicity reason If multiple Edge Node are hosted by the same ESXi hypervisor, all traffic will be pinned to the same uplink. In this case, all edges have been deployed with the same uplink profile. Traffic for VLAN 300 (Service Interface) will be pinned to fp-eth0 from an NSX-T standpoint and to Uplink-1 from a vSphere perspective. In this case, the T0-SR hosted on the Edge Node 02, will send a Gratuitous ARP on VLAN 300 so that the devices can update their ARP entry for 172.31.210.1 . (see next slide) N-VDS-01 Mgmt IP Uplink-1 Uplink-2 Edge-01 Mgmt IP Trunk-TOR-LEFT A/S Failover Order Trunk-TOR-RIGHT A/S Failover Order ToR-01 ToR-02 Management-PG VDS Uplink 1 Uplink 2 VLAN10 TEP-IP-1 TEP-IP-2 ESXi Host N-VDS-01 Uplink-1 Uplink-2 TEP-IP-1 TEP-IP-2 Tier-0 (DOWN) eth0 fp-eth2 eth0 fp-eth2 fp-eth0 fp-eth1 fp-eth1 fp-eth0 VLAN13 Edge-02 VLAN11 Tier-0 (A) VLAN12 EDGE FAILURE Uplink configured as a trunk VLAN 300 – Service Interface Uplink configured as a trunk VLAN 300 – Service Interface VLAN300 VLAN300
  • 29. Confidential │ ©2021 VMware, Inc. 29 NSX-T Configuration packer@tenant01-service-interface-01:~$ arp -a _gateway (172.31.210.1) at 00:50:56:ab:84:94 [ether] on ens192 packer@tenant01-service-interface-01:~$ ping 10.1.1.10 -c 1 PING 10.1.1.10 (10.1.1.10) 56(84) bytes of data. 64 bytes from 10.1.1.10: icmp_seq=1 ttl=62 time=0.674 ms packer@tenant01-service-interface-01:~$ ping 172.31.100.30 -c 1 PING 172.31.100.30 (172.31.100.30) 56(84) bytes of data. 64 bytes from 172.31.100.30: icmp_seq=1 ttl=60 time=0.920 ms Connectivity on the Virtual machine on the dvPG T0-SR EDGE 02 MAC Interface Successful ping to Northbound VM Successful ping to Overlay VM
  • 30. Confidential │ ©2021 VMware, Inc. 30 NSX-T Configuration Representation within vSphere and NSX-T Edge VM Only Service Interface traffic is represented in this diagram for simplicity reason If multiple Edge Node are hosted by the same ESXi hypervisor, all traffic will be pinned to the same uplink. In this case, all edges have been deployed with the same uplink profile. Since Uplink-1 is down, Network traffic for VLAN 300 (Service Interface) will be pinned to Uplink-2 (dvPG configuration) From an NSX-T standpoint, Traffic is still pinned to fp-eth0 vSphere is handling the Failover (not NSX-T) N-VDS-01 Mgmt IP Uplink-1 Uplink-2 Edge-01 Mgmt IP Trunk-TOR-LEFT A/S Failover Order Trunk-TOR-RIGHT A/S Failover Order ToR-01 ToR-02 Management-PG VDS Uplink 1 Uplink 2 VLAN10 TEP-IP-1 TEP-IP-2 ESXi Host N-VDS-01 Uplink-1 Uplink-2 TEP-IP-1 TEP-IP-2 Tier-0 (A) eth0 fp-eth2 eth0 fp-eth2 fp-eth0 fp-eth1 fp-eth1 fp-eth0 VLAN13 Edge-02 VLAN11 Tier-0 (S) VLAN12 TOR FAILURE Uplink configured as a trunk VLAN 300 – Service Interface Uplink configured as a trunk VLAN 300 – Service Interface VLAN300 VLAN300
  • 31. Confidential │ ©2021 VMware, Inc. 31 NSX-T Configuration packer@tenant01-service-interface-01:~$ arp -a _gateway (172.31.210.1) at 00:50:56:ab:ce:ac [ether] on ens192 packer@tenant01-service-interface-01:~$ ping 10.1.1.10 -c 1 PING 10.1.1.10 (10.1.1.10) 56(84) bytes of data. 64 bytes from 10.1.1.10: icmp_seq=1 ttl=62 time=1.07 ms packer@tenant01-service-interface-01:~$ ping 172.31.100.30 -c 1 PING 172.31.100.30 (172.31.100.30) 56(84) bytes of data. 64 bytes from 172.31.100.30: icmp_seq=1 ttl=60 time=1.27 ms Connectivity on the Virtual machine on the dvPG T0-SR EDGE 01 MAC Interface Successful ping to Northbound VM Successful ping to Overlay VM
  • 32. Confidential │ ©2021 VMware, Inc. 32 NSX-T Configuration - Service Interface Segment using another uplink teaming policy for testing purpose (ToR02) - Verification
  • 33. Confidential │ ©2021 VMware, Inc. 33 NSX-T Configuration Segment for the Service Interface – TOR 02 Teaming Policy has been changed to TOR- 02. Uplink Teaming Policy: - Supports a single Teaming Policy - TOR-2: - Normal Behavior: - Use fp-eth1 connected to dvPG TOR-RIGHT - dvPG Config (Active/Standby) : dvUPLINK2 - Failover Behavior: - Use fp-eth1 connected to dvPG TOR-RIGHT - dvPG Config (Active/Standby) : dvUPLINK1
  • 34. Confidential │ ©2021 VMware, Inc. 34 NSX-T Configuration SRV-EDGE-01(tier0_sr)> get interfaces Fri May 21 2021 UTC 05:14:33.012 Logical Router UUID VRF LR-ID Name Type a76ffe3e-8ed8-4509-a65b-5e52e43cda15 13 46 SR-Tier0-Tenant01 SERVICE_ROUTER_TIER0 Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable) Interface : ddb58170-00eb-4651-90e8-ac3dec6fccd7 Ifuid : 292 Name : Service-Interface Fwd-mode : IPV4_AND_IPV6 Internal name : service-292 Mode : lif Port-type : service IP/Mask : 172.31.210.1/24 MAC : 00:50:56:ab:03:e1 VLAN : 300 Access-VLAN : untagged LS port : 531af945-0f5a-43c9-9313-843e622d8027 Urpf-mode : STRICT_MODE DAD-mode : LOOSE RA-mode : SLAAC_DNS_TRHOUGH_RA(M=0, O=0) Admin : up Op_state : up MTU : 9000 arp_proxy : EDGE-01 Get Interfaces - ACTIVE
  • 35. Confidential │ ©2021 VMware, Inc. 35 NSX-T Configuration SRV-EDGE-02(tier0_sr)> get interfaces Fri May 21 2021 UTC 07:31:32.497 Logical Router UUID VRF LR-ID Name Type 126bf0b3-1c60-466b-ae09-f89562e33634 13 43 SR-Tier0-Tenant01 SERVICE_ROUTER_TIER0 Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable) Interface : ddb58170-00eb-4651-90e8-ac3dec6fccd7 Ifuid : 397 Name : Service-Interface Fwd-mode : IPV4_AND_IPV6 Internal name : service-397 Mode : lif Port-type : service IP/Mask : 172.31.210.1/24 MAC : 00:50:56:ab:f6:7d VLAN : 300 Access-VLAN : untagged LS port : 531af945-0f5a-43c9-9313-843e622d8027 Urpf-mode : STRICT_MODE DAD-mode : LOOSE RA-mode : SLAAC_DNS_TRHOUGH_RA(M=0, O=0) Admin : up Op_state : down MTU : 9000 arp_proxy : EDGE-02 Get Interfaces - STANDBY
  • 36. Confidential │ ©2021 VMware, Inc. 36 NSX-T Configuration Representation within vSphere and NSX-T Edge VM Only Service Interface traffic is represented in this diagram for simplicity reason Uplink Teaming Policy configured on the segment: TOR-02 If multiple Edge Node are hosted by the same ESXi hypervisor, all traffic will be pinned to the same uplink. In this case, all edge nodes have been deployed with the same uplink profile. Traffic for VLAN 300 (Service Interface) will be pinned to fp-eth1 from an NSX-T standpoint and to Uplink-2 from a vSphere perspective. N-VDS-01 Mgmt IP Uplink-1 Uplink-2 Edge-01 Mgmt IP Trunk-TOR-LEFT A/S Failover Order Trunk-TOR-RIGHT A/S Failover Order ToR-01 ToR-02 Management-PG VDS Uplink 1 Uplink 2 VLAN10 TEP-IP-1 TEP-IP-2 ESXi Host N-VDS-01 Uplink-1 Uplink-2 TEP-IP-1 TEP-IP-2 Tier-0 (A) eth0 fp-eth2 eth0 fp-eth2 fp-eth0 fp-eth1 fp-eth1 fp-eth0 Uplink configured as a trunk Uplink configured as a trunk VLAN13 Edge-02 VLAN11 Tier-0 (S) VLAN12 VLAN 300 – Service Interface VLAN 300 – Service Interface NORMAL MODE VLAN300 VLAN300
  • 37. Confidential │ ©2021 VMware, Inc. 37 NSX-T Configuration packer@tenant01-service-interface-01:~$ arp -a _gateway (172.31.210.1) at 00:50:56:ab:03:e1 [ether] on ens192 packer@tenant01-service-interface-01:~$ ping 10.1.1.10 -c 1 PING 10.1.1.10 (10.1.1.10) 56(84) bytes of data. 64 bytes from 10.1.1.10: icmp_seq=1 ttl=62 time=0.987 ms packer@tenant01-service-interface-01:~$ ping 172.31.100.30 -c 1 PING 172.31.100.30 (172.31.100.30) 56(84) bytes of data. 64 bytes from 172.31.100.30: icmp_seq=1 ttl=60 time=0.912 ms Connectivity on the Virtual machine on the dvPG T0-SR EDGE 01 MAC Interface Successful ping to Northbound VM Successful ping to Overlay VM
  • 38. Confidential │ ©2021 VMware, Inc. 38 NSX-T Configuration [root@srv-esxi-01:~] esxtop PORT-ID USED-BY TEAM-PNIC DNAME PKTTX/s MbTX/s PSZTX PKTRX/s MbRX/s PSZRX %DRPTX %DRPRX 67108959 2147032:SRV-EDGE-01.eth2 vmnic1 DvsPortset-0 3.39 0.00 66.00 6.78 0.00 91.00 0.00 0.00 67108960 2147032:SRV-EDGE-01.eth1 vmnic0 DvsPortset-0 3.39 0.00 116.00 6.78 0.00 66.00 0.00 0.00 67108961 2147032:SRV-EDGE-01.eth0 vmnic0 DvsPortset-0 3.39 0.00 66.00 0.00 0.00 0.00 0.00 0.00 [root@srv-esxi-01:~] pktcap-uw --switchport 67108959 --dir 2 --vlan 300 -o /tmp/capture.pcap Verification – ESXTOP (N)  ICMP Traffic is being received on fp-eth1  Packet 64 – 68 : ICMP with Overlay VM  Packet 146 – 149: ICMP with Northbound VM
  • 39. Confidential │ ©2021 VMware, Inc. 39 NSX-T Configuration Representation within vSphere and NSX-T Edge VM Only Service Interface traffic is represented in this diagram for simplicity reason Uplink Teaming Policy configured on the segment: TOR-02 If multiple Edge Node are hosted by the same ESXi hypervisor, all traffic will be pinned to the same uplink. In this case, all edge nodes have been deployed with the same uplink profile. Traffic for VLAN 300 (Service Interface) will be pinned to fp-eth1 from an NSX-T standpoint and to Uplink-2 from a vSphere perspective. In this case, the T0-SR hosted on the Edge Node 02, will send a Gratuitous ARP on VLAN so that the devices can update their ARP entry for 172.31.210.1 . (see next slide) N-VDS-01 Mgmt IP Uplink-1 Uplink-2 Edge-01 Mgmt IP Trunk-TOR-LEFT A/S Failover Order Trunk-TOR-RIGHT A/S Failover Order ToR-01 ToR-02 Management-PG VDS Uplink 1 Uplink 2 VLAN10 TEP-IP-1 TEP-IP-2 ESXi Host N-VDS-01 Uplink-1 Uplink-2 TEP-IP-1 TEP-IP-2 Tier-0 (DOWN) eth0 fp-eth2 eth0 fp-eth2 fp-eth0 fp-eth1 fp-eth1 fp-eth0 Uplink configured as a trunk Uplink configured as a trunk VLAN13 Edge-02 VLAN11 Tier-0 (A) VLAN12 VLAN 300 – Service Interface VLAN 300 – Service Interface VLAN300 VLAN300 EDGE FAILURE
  • 40. Confidential │ ©2021 VMware, Inc. 40 NSX-T Configuration packer@tenant01-service-interface-01:~$ arp -a _gateway (172.31.210.1) at 00:50:56:ab:f6:7d [ether] on ens192 packer@tenant01-service-interface-01:~$ ping 10.1.1.10 -c 1 PING 10.1.1.10 (10.1.1.10) 56(84) bytes of data. 64 bytes from 10.1.1.10: icmp_seq=1 ttl=62 time=0.127 ms packer@tenant01-service-interface-01:~$ ping 172.31.100.30 -c 1 PING 172.31.100.30 (172.31.100.30) 56(84) bytes of data. 64 bytes from 172.31.100.30: icmp_seq=1 ttl=60 time=0.100 ms Connectivity on the Virtual machine on the dvPG T0-SR EDGE 01 MAC Interface Successful ping to Northbound VM Successful ping to Overlay VM
  • 41. Confidential │ ©2021 VMware, Inc. 41 NSX-T Configuration Representation within vSphere and NSX-T Edge VM Only Service Interface traffic is represented in this diagram for simplicity reason Uplink Teaming Policy configured on the segment: TOR-02 If multiple Edge Node are hosted by the same ESXi hypervisor, all traffic will be pinned to the same uplink. In this case, all edge nodes have been deployed with the same uplink profile. Traffic for VLAN 300 (Service Interface) will be pinned to fp-eth1 from an NSX-T standpoint and to Uplink-2 from a vSphere perspective. N-VDS-01 Mgmt IP Uplink-1 Uplink-2 Edge-01 Mgmt IP Trunk-TOR-LEFT A/S Failover Order Trunk-TOR-RIGHT A/S Failover Order ToR-01 ToR-02 Management-PG VDS Uplink 1 Uplink 2 VLAN10 TEP-IP-1 TEP-IP-2 ESXi Host N-VDS-01 Uplink-1 Uplink-2 TEP-IP-1 TEP-IP-2 Tier-0 (A) eth0 fp-eth2 eth0 fp-eth2 fp-eth0 fp-eth1 fp-eth1 fp-eth0 Uplink configured as a trunk Uplink configured as a trunk VLAN13 Edge-02 VLAN11 Tier-0 (S) VLAN12 VLAN 300 – Service Interface VLAN 300 – Service Interface VLAN300 VLAN300 TOR FAILURE
  • 42. Thank You Confidential │ ©2021 VMware, Inc.