NSX-T and Service Interfaces presentation

Confidential │ ©2021 VMware, Inc.
Service Interface
Guidelines and Design
Nicolas Michel
Technical Product Manager | NSBU
November 2021

Confidential │ ©2021 VMware, Inc. 2
Agenda Service Interface introduction
Support and Topologies
Lab Topology
Diagram and Description
vSphere Configuration
dVPG TOR-01 / dVPG TOR-02 / dVPG
NSX-T Configuration
UI Configuration

Service Interface Introduction

Service Interface
Support in NSX-T 3.2
Connectivity
Originally developed to connect vlan
backed segments.
Overlay Segments also supported.
On a Tier0, Uplink are recommended.
On a Tier1, Service Interfaces are
mandatory to provide connectivity to
vlan backed segment (no Uplink on
Tier0).
Service interfaces using the same
vlan backed segment can not be
instantiated on different logical
routers hosted on the same edge
node.
Topologies
Can be used to interconnect Tier-1
gateways together (static routing
manually configured).
Service Interfaces are supported in
Active/Standby topologies only.
Networking and Services
Dynamic Routing protocols are not
supported (OSPF, BGP). Except for
EVPN Route Server Mode
Static Routing is supported. IPsec not
supported
A Service Interface supports the
following services:
• DHCP relay.
• DHCP Server
• NAT
• Gateway Firewall
• Native Load Balancer

Service Interface
Introduction
Service Interface*: Interface
connecting Vlan backed logical
switch to provide connectivity to
vlan backed physical or virtual
workloads.
*Referred as Centralized Service Port (CSP) in NSX-T
2.3
Baremetal
Servers
Tier-1
Tier-1
Gateway
Tier-0
Gateway
Vlan Segment
SI-2
Baremetal
Servers

Service Interface
Introduction
Service Interface: Can also be
connected to Overlay Segment for
Load balancer use cases.
Tier-1
Gateway
SI-2
Overlay or Vlan
Segment
SI-1
Tier-1
Gateway
Tier-1
Gateway
Standalone
Physical
Router
Vlan Segment
Baremetal
Servers
Tier-0
Gateway

Service Interface Topology
Interconnect Tier1 gateways
Service interface can be used to connect 2
Tier1 gateway together.
A service interface is used on the Org VDC
Tier1
A Linked Segment is used on the vApp Tier1
Static routing must be configured manually on
both Tier1 gateways:
• Org VDC Tier1 should have static routes for
the vApp tier1 segments :
• Static route to 10.2.2.0/24 with a next hop of
172.16.0.2
172.16.0.2
• vApp Tier1 should have a default route
pointing to the Service Port Tier1 interfaces
hosted on the Org VDC:
172.16.0.1

Service Interface
Supported Topology
Identical Overlay backed
segment used on different Tier-1
Service Interface Gateways.
10.1.1.1/24 on Tier-1 Tenant-01
10.1.1.2/24 on Tier-1 Tenant-02
Supported Topology
Tier-1
Gateway
Tenant-01
Segment – Vlan 10
Active: Edge-01
Tier-1
Gateway
Tenant-02
Tier-0
Gateway
Same Active Edge node
OVERLAY BACKED
Active: Edge-01

Service Interface
NOT Supported Topology
Identical VLAN backed segment
used on different Tier-1 Service
Interface Gateways.
10.1.1.1/24 on Tier-1 Tenant-01
10.1.1.2/24 on Tier-1 Tenant-02
Topology not supported.
Tier-1 must be in different edge
clusters if they share interfaces
on the same segment.
Tier-1
Gateway
Tenant-01
Segment – Vlan 10
Active: Edge-01
Tier-1
Gateway
Tenant-02
Tier-0
Gateway
Same Active Edge node
VLAN BACKED
Active: Edge-01

Topology

Topology Used
Edge Node VM Design - 2 pNICs Design
NSX-T Edge 2 PNIC Design
Single N-VDS to simplify design and deployment. Multi-TEP on
Edge is supported.
Single VLAN per uplink is recommended (BGP / OSPF / Static).
Easier to troubleshoot.
Best Practice in the network industry.
Named Teaming Policy is used to map each vNIC to NSX Segment
to pNIC to ToR (Detailed in the next slides).
Single VLAN and Overlay Transport Zone.
VLAN used
VLAN 110 is the vlan used for the TEP and is shared
between TOR-01 and TOR-02
VLAN 10 is the vlan used to establish the BGP peering
between TOR-01 and T0-SR on EDGE-NODE-01
VLAN 300 is the vlan used to connect by the Service
Interface Segment on NSX-T
N-VDS-01
Mgmt IP
Uplink-1 Uplink-2
Edge-01
Mgmt IP
Trunk-TOR-LEFT
A/S Failover Order
Trunk-TOR-RIGHT
A/S Failover Order
ToR-01 ToR-02
Management-PG
VDS
Uplink 1 Uplink 2
VLAN10
TEP-IP-1 TEP-IP-2
ESXi Host
N-VDS-01
Uplink-1 Uplink-2
TEP-IP-1 TEP-IP-2
Tier-0 (A)
eth0 fp-eth2 eth0 fp-eth2
fp-eth0 fp-eth1 fp-eth1
fp-eth0
VLAN 110 -TEP
VLAN 10 and 11 - BGP
Uplink configured as a trunk
VLAN 110 - TEP
VLAN 12 and 13 - BGP
VLAN13
Edge-02
VLAN11
Tier-0 (S)
VLAN12
VLAN 300 – Service Interface VLAN 300 – Service Interface
VLAN300 VLAN300

Topology Used
Layer 3 Topology
Layer 3 Design
Separate VLANs are used for each uplink between the TORs and
the T0-SR on EN01 and EN02
VLANs / IP Addressing plan
VLAN 300 is the vlan used for the service interface on the
T0.
Virtual Machine on a standard DVPG (non NSX-T
overlay)

vSphere Configuration – vNIC - DVPG
- Edge Node Virtual Machine vNIC
- DVPG TOR-Left
- DVPG TOR-Right
- DVPG Service Interface (VM vNIC)

Edge Node Virtual Machine vNIC
Network Adapter 2
This vSphere vNIC is considered as fp-eth0 for NSX-T
The adapter type is VMXNet3
Network Adapter 3
This vSphere vNIC is considered as fp-eth1 for NSX-T
The adapter type is VMXNet3
Network Adapter 4 is not connected to anything.

Distributed Port Group Configuration – TOR LEFT (01)

Distributed Port Group Configuration – TOR RIGHT (02)

Distributed Port Group Configuration – Service Interface DPG (VM vNIC)
 Teaming and Failover policy doesn’t really matter in our example
as the VM could be hosted anywhere in the DC

NSX-T Configuration
- Uplink Profile – Teaming Policies
- Service Interface Segment – TOR01
- Service Interface on T0
- Verification

NSX-T Configuration
Uplink Profile
An Uplink profile defines the way N-VDS
operates
Transport VLAN: TEP Vlan
MTU: N-VDS MTU
Teaming Policies:
- Default Teaming Policy: Multi TEP (fp-eth0
and fp-eth1)
- TOR-1: Use uplink-1 only (fp-eth0)
- TOR-2: Use uplink-2 only (fp-eth1)

NSX-T Configuration
Teaming policies
Teaming Policies Configuration:
- Default Teaming Policy:
- Multi TEP – Load Balance Source
- Uplink-1 maps to fp-eth0
- Uplink-2 maps to fp-eth1
- Named Teaming Policy TOR-1:
- Uplink-1 maps to fp-eth0 only
- Not supported to have a standby uplink
- Name Teaming Policy TOR-2:
- Uplink-2 maps to fp-eth1 only
- Not supported to have a standby uplink

NSX-T Configuration
Segment for the Service Interface – TOR 01
Creation of a VLAN Backed Segment
This segment is not attached to a specific T0
or T1.
The Transport zone must be VLAN Based
The Subnet is not set. It will be configured on
the Service interface itself (T0 construct)
Transport VLAN must match dvPG VLAN
connecting the virtual machine.
Uplink Teaming Policy:
- Supports a single Teaming Policy
- TOR-1:
- Normal Behavior:
- Use fp-eth0 connected to dvPG TOR-LEFT
- dvPG Config (Active/Standby) : dvUPLINK1
- Failover Behavior:
- Use fp-eth0 connected to dvPG TOR-LEFT

NSX-T Configuration
Service Interface on the T0
Interface type must be “Service”
IP Address in the same range as the Virtual
Machine. 172.31.210.1 will be the VM’s default
gateway.
Connected to the Segment created earlier.

NSX-T Configuration
SRV-EDGE-01(tier0_sr)> get interfaces
Fri May 21 2021 UTC 05:14:33.012
Logical Router
UUID VRF LR-ID Name Type
a76ffe3e-8ed8-4509-a65b-5e52e43cda15 13 46 SR-Tier0-Tenant01 SERVICE_ROUTER_TIER0
Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable)
Interface : ddb58170-00eb-4651-90e8-ac3dec6fccd7
Ifuid : 399
Name : Service-Interface
Fwd-mode : IPV4_AND_IPV6
Internal name : service-399
Mode : lif
Port-type : service
IP/Mask : 172.31.210.1/24
MAC : 00:50:56:ab:ce:ac
VLAN : 300
Access-VLAN : untagged
LS port : 531af945-0f5a-43c9-9313-843e622d8027
Urpf-mode : STRICT_MODE
DAD-mode : LOOSE
RA-mode : SLAAC_DNS_TRHOUGH_RA(M=0, O=0)
Admin : up
Op_state : up
MTU : 9000
arp_proxy :
EDGE-01 Get Interfaces - ACTIVE

NSX-T Configuration
Fri May 21 2021 UTC 05:14:36.985
Logical Router
126bf0b3-1c60-466b-ae09-f89562e33634 13 43 SR-Tier0-Tenant01 SERVICE_ROUTER_TIER0
Ifuid : 397
Mode : lif
Port-type : service
IP/Mask : 172.31.210.1/24
MAC : 00:50:56:ab:84:94
VLAN : 300
LS port : 531af945-0f5a-43c9-9313-843e622d8027
DAD-mode : LOOSE
Admin : up
Op_state : down
MTU : 9000
arp_proxy :
EDGE-02 Get Interfaces - STANDBY

NSX-T Configuration
Representation within vSphere and NSX-T Edge VM
Only Service Interface traffic is represented in
this diagram for simplicity reason
Uplink Teaming Policy configured on the
segment: TOR-01
If multiple Edge Node are hosted by the same
ESXi hypervisor, all traffic will be pinned to the
same uplink. In this case, all edges have been
deployed with the same uplink profile.
Traffic for VLAN 300 (Service Interface) will be
pinned to fp-eth0 from an NSX-T standpoint
and to Uplink-1 from a vSphere perspective.
N-VDS-01
Mgmt IP
Uplink-1 Uplink-2
Edge-01
Mgmt IP
Trunk-TOR-LEFT
A/S Failover Order
Trunk-TOR-RIGHT
A/S Failover Order
ToR-01 ToR-02
Management-PG
VDS
Uplink 1 Uplink 2
VLAN10
TEP-IP-1 TEP-IP-2
ESXi Host
N-VDS-01
Uplink-1 Uplink-2
TEP-IP-1 TEP-IP-2
Tier-0 (A)
fp-eth0
Uplink configured as a trunk Uplink configured as a trunk
VLAN13
Edge-02
VLAN11
Tier-0 (S)
VLAN12
NORMAL MODE
VLAN300 VLAN300

NSX-T Configuration
packer@tenant01-service-interface-01:~$ arp -a
_gateway (172.31.210.1) at 00:50:56:ab:ce:ac [ether] on ens192
packer@tenant01-service-interface-01:~$ ping 10.1.1.10 -c 1
PING 10.1.1.10 (10.1.1.10) 56(84) bytes of data.
64 bytes from 10.1.1.10: icmp_seq=1 ttl=62 time=0.717 ms
Connectivity on the Virtual machine on the dvPG
T0-SR EDGE 01 MAC Interface
Successful ping to Northbound VM
Successful ping to Overlay VM

NSX-T Configuration
[root@srv-esxi-01:~] esxtop
PORT-ID USED-BY TEAM-PNIC DNAME PKTTX/s MbTX/s PSZTX PKTRX/s MbRX/s PSZRX %DRPTX %DRPRX
67108959 2147032:SRV-EDGE-01.eth2 vmnic1 DvsPortset-0 3.39 0.00 66.00 6.78 0.00 91.00 0.00 0.00
[root@srv-esxi-01:~] pktcap-uw --switchport 67108960 --dir 2 --vlan 300 -o /tmp/capture.pcap
Verification – ESXTOP (N)
 ICMP Traffic is being received on fp-eth0
 Packet 64 – 68 : ICMP with Overlay VM
 Packet 146 – 149: ICMP with Northbound VM

NSX-T Configuration
In this case, the T0-SR hosted on the Edge
Node 02, will send a Gratuitous ARP on VLAN
300 so that the devices can update their ARP
entry for 172.31.210.1 . (see next slide)
N-VDS-01
Mgmt IP
Uplink-1 Uplink-2
Edge-01
Mgmt IP
Trunk-TOR-LEFT
A/S Failover Order
Trunk-TOR-RIGHT
A/S Failover Order
ToR-01 ToR-02
Management-PG
VDS
Uplink 1 Uplink 2
VLAN10
TEP-IP-1 TEP-IP-2
ESXi Host
N-VDS-01
Uplink-1 Uplink-2
TEP-IP-1 TEP-IP-2
Tier-0 (DOWN)
fp-eth0
VLAN13
Edge-02
VLAN11
Tier-0 (A)
VLAN12
EDGE FAILURE
VLAN 300 – Service Interface
VLAN300 VLAN300

NSX-T Configuration
_gateway (172.31.210.1) at 00:50:56:ab:84:94 [ether] on ens192

NSX-T Configuration
Since Uplink-1 is down, Network traffic for
VLAN 300 (Service Interface) will be pinned to
Uplink-2 (dvPG configuration)
From an NSX-T standpoint, Traffic is still pinned
to fp-eth0
vSphere is handling the Failover (not NSX-T)
N-VDS-01
Mgmt IP
Uplink-1 Uplink-2
Edge-01
Mgmt IP
Trunk-TOR-LEFT
A/S Failover Order
Trunk-TOR-RIGHT
A/S Failover Order
ToR-01 ToR-02
Management-PG
VDS
Uplink 1 Uplink 2
VLAN10
TEP-IP-1 TEP-IP-2
ESXi Host
N-VDS-01
Uplink-1 Uplink-2
TEP-IP-1 TEP-IP-2
Tier-0 (A)
fp-eth0
VLAN13
Edge-02
VLAN11
Tier-0 (S)
VLAN12
TOR FAILURE
VLAN300 VLAN300

NSX-T Configuration
_gateway (172.31.210.1) at 00:50:56:ab:ce:ac [ether] on ens192

NSX-T Configuration
- Service Interface Segment using another
uplink teaming policy for testing purpose
(ToR02)
- Verification

NSX-T Configuration
Segment for the Service Interface – TOR 02
Teaming Policy has been changed to TOR-
02.
Uplink Teaming Policy:
- Supports a single Teaming Policy
- TOR-2:
- Normal Behavior:
- Use fp-eth1 connected to dvPG TOR-RIGHT
- Failover Behavior:
- Use fp-eth1 connected to dvPG TOR-RIGHT

NSX-T Configuration
Fri May 21 2021 UTC 05:14:33.012
Logical Router
a76ffe3e-8ed8-4509-a65b-5e52e43cda15 13 46 SR-Tier0-Tenant01 SERVICE_ROUTER_TIER0
Ifuid : 292
Mode : lif
Port-type : service
IP/Mask : 172.31.210.1/24
MAC : 00:50:56:ab:03:e1
VLAN : 300
LS port : 531af945-0f5a-43c9-9313-843e622d8027
DAD-mode : LOOSE
Admin : up
Op_state : up
MTU : 9000
arp_proxy :
EDGE-01 Get Interfaces - ACTIVE

NSX-T Configuration
Fri May 21 2021 UTC 07:31:32.497
Logical Router
126bf0b3-1c60-466b-ae09-f89562e33634 13 43 SR-Tier0-Tenant01 SERVICE_ROUTER_TIER0
Ifuid : 397
Mode : lif
Port-type : service
IP/Mask : 172.31.210.1/24
MAC : 00:50:56:ab:f6:7d
VLAN : 300
LS port : 531af945-0f5a-43c9-9313-843e622d8027
DAD-mode : LOOSE
Admin : up
Op_state : down
MTU : 9000
arp_proxy :
EDGE-02 Get Interfaces - STANDBY

NSX-T Configuration
segment: TOR-02
same uplink. In this case, all edge nodes have
been deployed with the same uplink profile.
N-VDS-01
Mgmt IP
Uplink-1 Uplink-2
Edge-01
Mgmt IP
Trunk-TOR-LEFT
A/S Failover Order
Trunk-TOR-RIGHT
A/S Failover Order
ToR-01 ToR-02
Management-PG
VDS
Uplink 1 Uplink 2
VLAN10
TEP-IP-1 TEP-IP-2
ESXi Host
N-VDS-01
Uplink-1 Uplink-2
TEP-IP-1 TEP-IP-2
Tier-0 (A)
fp-eth0
VLAN13
Edge-02
VLAN11
Tier-0 (S)
VLAN12
NORMAL MODE
VLAN300 VLAN300

NSX-T Configuration
_gateway (172.31.210.1) at 00:50:56:ab:03:e1 [ether] on ens192

NSX-T Configuration
[root@srv-esxi-01:~] esxtop
PORT-ID USED-BY TEAM-PNIC DNAME PKTTX/s MbTX/s PSZTX PKTRX/s MbRX/s PSZRX %DRPTX %DRPRX
[root@srv-esxi-01:~] pktcap-uw --switchport 67108959 --dir 2 --vlan 300 -o /tmp/capture.pcap
Verification – ESXTOP (N)
 ICMP Traffic is being received on fp-eth1
 Packet 64 – 68 : ICMP with Overlay VM
 Packet 146 – 149: ICMP with Northbound VM

NSX-T Configuration
segment: TOR-02
In this case, the T0-SR hosted on the Edge
Node 02, will send a Gratuitous ARP on VLAN
so that the devices can update their ARP entry
for 172.31.210.1 . (see next slide)
N-VDS-01
Mgmt IP
Uplink-1 Uplink-2
Edge-01
Mgmt IP
Trunk-TOR-LEFT
A/S Failover Order
Trunk-TOR-RIGHT
A/S Failover Order
ToR-01 ToR-02
Management-PG
VDS
Uplink 1 Uplink 2
VLAN10
TEP-IP-1 TEP-IP-2
ESXi Host
N-VDS-01
Uplink-1 Uplink-2
TEP-IP-1 TEP-IP-2
Tier-0 (DOWN)
fp-eth0
VLAN13
Edge-02
VLAN11
Tier-0 (A)
VLAN12
VLAN300 VLAN300
EDGE FAILURE

NSX-T Configuration
_gateway (172.31.210.1) at 00:50:56:ab:f6:7d [ether] on ens192

NSX-T Configuration
segment: TOR-02
N-VDS-01
Mgmt IP
Uplink-1 Uplink-2
Edge-01
Mgmt IP
Trunk-TOR-LEFT
A/S Failover Order
Trunk-TOR-RIGHT
A/S Failover Order
ToR-01 ToR-02
Management-PG
VDS
Uplink 1 Uplink 2
VLAN10
TEP-IP-1 TEP-IP-2
ESXi Host
N-VDS-01
Uplink-1 Uplink-2
TEP-IP-1 TEP-IP-2
Tier-0 (A)
fp-eth0
VLAN13
Edge-02
VLAN11
Tier-0 (S)
VLAN12
VLAN300 VLAN300
TOR FAILURE

NSX-T and Service Interfaces presentation

Recommended

Recommended

More Related Content

Similar to NSX-T and Service Interfaces presentation

Similar to NSX-T and Service Interfaces presentation (20)

Recently uploaded

Recently uploaded (20)

NSX-T and Service Interfaces presentation