NSX-T Architecture and Components.pptx

© 2019 VMware, Inc.
NSX-T Data Center
Architecture and Components
Quinton Coelho

VMware NSX-T Data Center | 1-2
Importance
As a Data Center Solutions Architect, you need a broad understanding of the VMware Virtual
Cloud Network framework and the solutions that it offers for addressing challenges in your data
center.
You must also understand the NSX-T Data Center architecture and components to properly
design, deploy, and manage a data center that meets your business requirements.

Agenda
NSX-T Data Center Architecture
NSX-T Data Center Management Cluster
NSX-T Logical Switching (Segments)
NSX-T Logical Routing

NSX Data Center Architecture for Private Cloud, Public Cloud, and Containers
NSX
Control
Plane
Data
Plane
Management
Plane
Private or Public
Cloud Infrastructure
NSX Central Controller
NSX Manager
Node
(VPN Gateway, DirectConnect, ExpressRoute)
Public Cloud
Linux VM Windows VM
NSX Cloud
Gateway
VMware Cloud on AWS
Private Cloud
NSX Edge
VM or Bare Metal
ESXi KVM
N-VDS N-VDS
Pivotal
Container Service
Container
Cloud Service
Manager
Bare Metal
NSX
Cloud Foundry Adapter
NSX Container Plug-In
K8/OS Adapter
Multi-Hypervisor
Kubernetes
Pivotal
Application Service
AWS Azure
VMware
IBM

Characterization of Critical Architecture Components
• Interact with multiple compute
managers:
‒ Multiple vCenter Server
systems for ESXi
‒ Cloud services manager
‒ Network container plug-in
• No dependencies on hypervisor
• Availability: Standalone with HA,
VIP, and external LB
• Scale-out Distributed Control plane:
‒ Central and local processing
• No dependencies on hypervisor,
overlay, or subnet
• Central place for all objects,
including DFW
• Availability: Majority-based
• Resources reservation required
• Simplified distributed routing
• DPDK 10/25/40 Gbps N-S BW:
‒ Virtual and bare metal
‒ Sub-second convergence
• Services: FW NAT, LB, DHCP,
meta-data proxy, L2 bridging
• Availability: Active-standby and
ECMP
• Resources reservation required
NSX Manager Appliance
NSX Edge
NSX Controller
NSX Manager

NSX-T Data Center Management
Cluster

NSX-T Data Center includes the management, control, and data planes.
Data
Plane
ESXi
host
N-VDS
KVM
host
N-VDS
NSX Edge
Bare Metal
Server
NSX
Linux
VM
NSX
Windows
VM
NSX
NSX
Cloud
GW
NAT
Private Cloud
Public Cloud
VMware
Cloud on
AWS
Management
and
Control
Plane
VMs Containers
NSX Manager Cluster
Configurable Through GUI, REST, or CMP
Cloud Service Manager
vCenter Server
AWS
Azure
VMware
IBM

This architectural separation enables scalability without affecting workloads.
NSX-T Data Center Components
Management and control plane:
• Converged management and control
plane cluster
• Three-node cluster for scale and high
availability
• UI/API for interacting with user,
automation, and CMP platforms
• Validates and stores desired configuration
• Maintains and propagates dynamic state
Distributed data plane:
• Hosts workloads (VMs, containers,) and
services)
• Implements distributed routing and
firewalling

About the Management and Control Planes
Each node on the management and control planes includes policy, management, and control
functions:
• The three-node NSX management cluster provides high availability and scalability.
• The UI or API interacts with users, automation, and CMPs.
• NSX policy configures networking and security functions.
• The desired configuration is validated and replicated across all nodes.
• The dynamic state is maintained and propagated across all nodes.
Management and Control
Planes
Configurable Through GUI, REST, or CMP
NSX Management Cluster
Cloud Service Manager
vCenter Server

About the Data Plane
The data plane works in the following ways:
• Includes multiple endpoints, such as ESXi and KVM hosts, NSX Edge
• Contains various workloads, such as VMs, containers, and applications running on
bare metal servers
• Forwards data plane traffic
• Uses a scale-out distributed forwarding model
• Implements logical switching, distributed and centralized routing, and firewall filtering
Data Plane
Private
Cloud
Public Cloud:
AWS
VMware Cloud on AWS
Microsoft Azure
Linux VM Windows
VM
NSX
Cloud
Gateway
NSX
Bare Metal
Server
NSX Edge
N-VDS N-VDS
VMs Containers
ESXi Host ESXi Host

About the NSX Management Cluster
• The management plane includes
the policy and manager roles.
• The central control plane includes
the controller role.
The NSX management cluster is
formed by a group of three manager
nodes.
The desired state is replicated in the
distributed persistent database,
providing the same configuration view
to all nodes in the cluster.
NSX Manager is available in four form
factors for different deployment
scenarios.
Management
Plane
Control
Plane
Manager A Manager B Manager C
Policy Role
Manager Role
Controller Role
Distributed Persistent Database
NSX Manager is an appliance with built-in roles:

NSX Manager Deployment Options (1)
• No L2 adjacent requirement.
• All three node IPs can be used for GUI and
API access. However, when that node fails,
a different IP has to be used.
API or GUI Client
IP A IP B IP C
Default

API or GUI Client
IP 10.1.1.10 IP 10.1.1.11 IP 10.1.1.12
Cluster Virtual IP
10.1.1.1
Recommended
• Low cost.
• Low complexity.
• Single IP address can be used for API and UI access.
• Single subnet only.
• No UI and API load distribution.

IP A IP B IP C
API or GUI Client
VIP
10.1.1.1
Not Recommended
• Single IP availability.
• Multisubnet: No L2 across management racks.
• More complex setup with LB configuration required.
• Complex life cycle management and compatibility
• Costly: Benefits might be overrated.

NSX Management Clusters with a Virtual IP Address
The NSX Manager cluster is highly available
and configured in the following way:
• All managers must be on the same subnet.
• One manager node is elected as the
leader.
• The cluster’s virtual IP address is attached
to the leader manager.
• Gratuitous Address Resolution Protocol
(GARP) is used if the leader manager fails.
• The cluster virtual IP address is used for
traffic destined for NSX Manager nodes.
• A single virtual IP address is used for API
and GUI client access

NSX Policy Functions
• It provides a centralized location
for configuring networking and
security across the environment.
• Users can enter the intended
configuration in the NSX
Manager simplified UI.
• The policy role allows users to
specify the final desired state of
the system without being
concerned about the current
state or underlying
implementation.
Management
Plane
Control
Plane
Policy Role
Manager Role
Controller Role
NSX Policy Manager provides the following functionalities:

NSX Manager Functions
• Receives and validates the
configuration from the NSX
policy
• Stores the configuration in the
distributed persistent database
(CorfuDB)
• Publishes the configuration to
the central control plane
• Installs and prepares the data
plane components
• Retrieves the statistical data
from data plane components
Management
Plane
Control
Plane
Policy Role
Manager Role
Controller Role
NSX Manager provides the following functions:

NSX Controller Functions
• Provides control plane
functionality, such as logical
switching, routing, and distributed
firewall
• Computes all ephemeral runtime
states, based on configuration
from the management plane
• Disseminates topology
information reported by the data
plane elements
• Pushes stateless configurations
to forwarding engines
Management
Plane
Control
Plane
Policy Role
Manager Role
Controller Role
NSX Controller maintains the realized state of the system and configures the data plane.
NSX Controller performs the following main functions:

Control Plane Components (1)
Control plane functions in an NSX-T Data
Center are divided into the central control
plane (CCP) and the local control plane (LCP):
• The CCP exists as part of NSX Manager
nodes and is offered by the NSX
Controller role.
• The LCP exists on host transport nodes or
on NSX Edge transport nodes.

Control Plane Components (2)
Management
Plane
CCP
LCP
LCP
LCP
Transport Node Transport Node Transport Node
RabbitMQ
The management
plane pushes the
user config down to
the CCP through
RabbitMQ.
The CCP receives
the config and
pushes it down to
the data plane
through an RPC.
RPC
The CCP performs the following functions:
• Computes the ephemeral runtime state,
based on the configuration from the
management plane
• Disseminates information reported by the
data plane elements using the LCP
The LCP performs the following functions:
• Monitors local link status
• Computes most ephemeral runtime states
based on updates from the data plane and
the CCP
• Pushes stateless configurations to
forwarding engines

Data Plane Functions
The data plane forwards packets based on configurations populated by the control plane and
reports topology information to the control plane.
The data plane has the following responsibilities:
• Maintains the status of and handles failover between multiple links or tunnels
• Performs stateless forwarding based on tables and rules populated by the control plane
• Maintains packet-level statistics

Data Plane Components
Data plane components, referred to as transport nodes, include the following types:
Hypervisor transport nodes:
• Act as a forwarding plane for VM traffic
• Provide support for ESXi and KVM
hypervisors
Bare metal transport nodes:
• Include Linux-based workloads running on
bare metal servers and containers running
on bare metal servers without a hypervisor
NSX Edge cluster:
• Contains edge transport nodes (VM or
bare metal)
• Provides stateful and gateway services

Logical Switching

Overlay-Backed Logical Switches
In the Policy Manager,
logical switches are
referred to as segments.
vm1 vm2 vm3 vm4 vm5 vm6 vm7 vm8
Logical Switch
Logical View
Physical View
Spine
Leaf
TEP1
TEP2
TEP3
TEP4
TEP5
TEP6
TEP7
TEP8
TEP9
HV1
HV2
HV3
HV4
HV5
HV6
HV7
HV8
HV9
vm1
vm2
vm4
vm3
vm5
vm7
vm8
vm9

Transport Nodes and N-VDS
A transport node (TN) is a device prepared for NSX Data Center and participates in traffic
forwarding (data plane). A transport node can be a hypervisor or an edge node.
An NSX virtual distributed switch (N-VDS) is the NSX Data Center software component that
performs switching functionality on a transport node:
• The N-VDS typically owns several physical NICs of the transport node.
• The N-VDS on different transport nodes are independent.
• The N-VDS has a name assigned for grouping and management.
HV1 Transport Node
N-VDS.1
Name: LAB
HV2 Transport Node
N-VDS.2
Name: Prod
HVn Transport Node
N-VDS.n
Name: Prod

A transport zone defines the span of logical networks over the physical infrastructure:
• Has an N-VDS name that is used to identify the N-VDS to bind to on transport nodes.
• Has a transport zone type: overlay or VLAN.
A logical switch is a virtual L2 broadcast domain:
• Logical switches are defined as part of a transport zone.
• Logical switches created within a TZ inherit the transport zone type (VLAN or overlay).
• Logical switch span is defined by its transport zone.
Transport Zones and Logical Switches (Segments)
Overlay Transport Zone TZ1 N-VDS Name: Prod
HV1 Transport Node
N-VDS.1
Name: LAB
HV2 Transport Node
N-VDS.2
Name: Prod
HVn Transport Node
N-VDS.n
Name: Prod
TNs Attached to the Production
Transport Zone
TN Not Attached to the
Transport Zone
Overlay LS
Overlay LS

You must identify N-VDS (and the physical uplinks from N-VDS):
• The N-VDS name field in the transport zone is used to identify the NSX virtual distributed
switch to use on the transport node.
• The following types of transport zones are available:
– Overlay transport zones
– VLAN transport zones
Transport Zone Binding and N-VDS Name
Overlay Transport Zone TZ1 N-VDS Name: Prod
HV1 Transport Node
N-VDS.1
Name: LAB
HV2 Transport Node
N-VDS.2
Name: Prod
HVn Transport Node
N-VDS.n
Name: Prod
TNs Attached to the Production
Transport Zone
TN Not Attached to the
Transport Zone
Overlay LS
Overlay LS

Logical Routing

Logical Routers
Logical router:
• Provides E-W routing
between different logical
segments
• Peers with the physical
infrastructure for N-S
routing
• Can provides network
services like Network
Address Translation(NAT),
load balancing, perimeter
firewall, VPN, and so on
Logical Switch 2
Logical Switch 1
Logical Router
Physical
Router
Downlink
Uplink
10.1.1.0/24 10.2.2.2.0/24
10.2.2.1/24
10.1.1.1/24

Services Router
Logical Routers: Component Terminology
• Runs locally in the transport nodes
participating in the NSX fabric.
• Typically runs as kernel module in the
hypervisor.
• Provides distributed E-W routing.
• Traffic between different subnets on
same hypervisor does not leave the
hypervisor.
• Responsible for providing on/off ramp
gateway services including N-S
routing.
• Provides centralized services, such as
NAT, BGP, LB, Edge Firewall,
Connectivity to the physical.
• The services router is instantiated as a
service on an appliance called the
Edge node.
Distributed Router DR SR

Each ESXi host has its own copy of each configured DLR instance.
Logical Routers: Distributed Component
Tier-0
Logical
Router
ESXi-1 ESXi-2 KVM
DR DR DR
10.1.1.0/24 10.2.2.2.0/24
10.1.1.10/24 10.2.2.10/24 10.1.1.20/24
10.2.2.20/24

Whenever a service that cannot be distributed is enabled on a logical router, a services router or
services component is instantiated.
A services router is instantiated for the following services:
A services router is instantiated on an appliance called the edge node.
Logical Routers: Services Component
Load
Balancing
P to V
Gateway
Router Perimeter
Firewall
VPN
NAT DHCP

NSX-T Data Center User
Configuration
Background Process
Topology View: Distributed Router and Services Router Interaction
Physical
Router
10.1.1.10/24 10.2.2.10/24
LS2
LS1
LS
2
LS1
Create Uplink
Interface
SR
EN1
169.254.0.1
169.254.0.2
NSX Management plane auto-plumbs
this link (internal LS) and routing
between DR and SR.
Tier-0 LR
10.1.1.20/24
Tier-0 LR
10.1.1.1 10.2.2.1
DR
10.1.1.1 10.2.2.1
10.1.1.10/24 10.2.2.10/24
10.1.1.20/24
Tier-0 SR Routing Table
10.1.1.0/24 through 169.254.0.1
10.2.2.0/24 through 169.254.0.1
Tier-0 DR Routing table
0.0.0.0/0 via 169.254.0.2
Uplink
Interface

Multitier Routing

Logical Routing: Multitier Topology (1)
The Tier-0 logical router connects to the physical infrastructure.
Tier-0 logical router:
• Manual management
Tier-1 logical router:
• Role- Per tenant first hop router
• Cloud Management Platform
(CMP) driven Management
Benefits:
• Tenant Isolation:
– Separates control for Infra
and Tenant admin
– Eliminates dependency on
physical infrastructure when
a new tenant is provisioned
Tier-0
Logical Router
Physical
Router
Tier-1
Logical Router
Tier-1
Logical Router
RouterLink
(100.64.0.0/31)
Uplink
Downlink
Tenant-1 Tenant-2

Multiple interface and route types are found in a multitier topology.
Interface types:
• Uplink: Used to connect to physical
infrastructure.
• RouterLink: Used to interconnect Tier-0 and
Tier-1 logical routers.
• DownLink: Used to connect logical switches.
Route types:
• Static: Configured by user.
• NSX route: Automatically configured by NSX
through connected state and T1 route
advertisement configuration.
Tier-0
Logical Router
Physical
Router
Tier-1
Logical Router
20.20.20.0/24 30.30.30.0/24
100.64.224.0/31
100.64.224.1/31

The diagram shows a walkthrough of route advertisement and route redistribution auto plumbing.
Tier-0
Logical Router
Physical
Router
Tier-1
Logical Router
20.20.20.0/24 30.30.30.0/24
100.64.224.0/31
100.64.224.1/31
20.20.20.0/24 & 30.30.30.0/24 will be
flagged as t1c (Tier1- Connected)
routes
Tier-0 redistributes 20.20.20.0/24 &
30.30.30.0/24
Default route with next hop IP as
100.64.224.0/31
20.20.20.0/24 & 30.30.30.0/24
are seen as eBGP routes with
next hop IP as 192.168.240.3/24
192.168.240.3/24
192.168.240.1/24

Route advertisement and route redistribution can be verified on the Routing tab.
Tier-0
Logical Router
Physical
Router
Tier-1
Logical Router
20.20.20.0/24 30.30.30.0/24

In multitier distributed routing:
• Tier-0 and Tier-1 routers are also instantiated on the hypervisors to prevent hairpinning.
• Fully distributed architecture: As much routing as possible is performed upfront at the source.
ESXi-1
Tier-0 DR
Tenant 1
Tier-1 DR
Tenant 2
Tier-1 DR
ESXi-2
Tier-0 DR
Tenant 1
Tier-1 DR
Tenant 2
Tier-1 DR
100.64.224.0/31 100.64.224.2/31
100.64.224.1/31 100.64.224.3/31
100.64.224.0/31 100.64.224.2/31
100.64.224.1/31 100.64.224.3/31

Simplified multitier distributed routing:
• Tier-0 connects to physical devices.
• Tier-1 provides tenant-specific routing.
Two-Tier Routing with Connectivity Options for Workload (1)
Tenant 1 Tenant 2
Tier-0 ECMP or
Active-Standby
Web LS App LS Db LS Web LS App LS Db LS
Tier-1 Logical Router E-
W Distributed
Tier-1 Logical Router
Active-Standby
VLAN Logical Switch
Overlay Logical
Switch
NAT
FW
LB
VPN
Distinct Routing
Peer

Stateful services runs in a centralized mode:
• FW, NAT, LB DHCP, VPN, and metadata proxy
• Bridging services
Tier-0 services:
• DPDK-based forwarding: Routing and bridging
• Services: NAT, FW, DHCP, and metadata proxy
• Centralized overlay logical switch segments
• Centralized VLAN logical switch segments
Tier-1 services:
• Tenant routing
• FW, NAT, LB, and VPN services
Two-Tier Routing with Connectivity Options for Workload (2)

Summary

NSX-T Architecture and Components.pptx

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to NSX-T Architecture and Components.pptx

Similar to NSX-T Architecture and Components.pptx (20)

Recently uploaded

Recently uploaded (20)

NSX-T Architecture and Components.pptx