Mss solution guide

Confidential. Copyright © Arista 2020. All rights reserved.Confidential. Copyright © Arista 2020. All rights reserved.
MSS Solution Guide
Asoka De Saram
03/24/2020
v1.2

Confidential. Copyright © Arista 2020. All rights reserved.2
Agenda
● MSS Overview
● Use Cases
● Key Benefits
● Supported Platforms
● Prerequisites
● Operational Details
● Deployment with Layer-2 Transparent Firewalls
● Deployment with Layer-3 Firewalls
● Troubleshooting MSS Issues

Confidential. Copyright © Arista 2020. All rights reserved.
Solution Overview
● Arista Macro Segmentation Service (MSS) insert security devices in to the
path of the traffic.
● It is specifically aimed at Physical to Physical and Physical to virtual
workloads
● MSS is a software driven dynamic and scalable network service
● MSS is a unique solution that places the control of policy enforcement
directly in the hands of the network administrator.
● Arista Cloud Vision Exchange (CVX) provides single point of integration
and orchestration.
● Arista MSS communicates with Firewalls using the available API, and
requests the security policies of interest.
● Firewalls can be located in a central location and can operate in a bump in
the wire mode or full switching/routing mode.
3

Use Cases
● The use cases illustrated in the following slides discuss two widely widely
encountered security challenges.
● The first use case deals with securing east-west traffic between physical-to-
physical (P-to-P) and physical (P-to-V) servers.
● The second use case deals with isolating lines of businesses or one tenant from
another in a shared network infrastructure deployment.
4

Securing East-West Traffic
5
Legacy Approach MSS Approach
Web Application Database
Firewall Firewall Firewall Firewall
VXLAN
FW Manager

Tenant Isolation & Security
Legacy Approach Modern Approach
Finance Sales Marketing
Firewall Firewall Firewall Firewall
VXLAN
FW Manager
Finance Sales Marketing

Key Benefits
7
Arista Macro-Segmentation Services (MSS) offers the following key benefits:
● Insert security between any physical and virtual workloads in the data center
● Automatic and seamlessly orchestrated service insertion - eliminating manual
steering of traffic per workload or tenant
● Security policies follows the host and application throughout the network
● No proprietary frame formats, tagging, or encapsulation
● One point of control - e.g. the security policy manager for physical firewalls
● No server reconfiguration or per application overhead

Supported Firewall Vendors
8
Vendor Name Minimum Software Release FW Deployment Mode
Palo Alto Networks PanOS 8.0.8 or above L2 Transparent
L3 Routed
Fortinet FortiOS 5.6.3 or above L2 Transparent
L3 Routed
CheckPoint R80.30 L3 Routed

Supported Arista Hardware Platforms
9
Deployment Mode Supported Platforms Firewall Vendor Minimum EOS version
L2 transparent 7050X, 7050X2, 7060X, 7060X2,
7060X3, 7280, 7020R
Palo Alto Networks 4.20.1
L2 transparent 7050X, 7050X2, 7060X, 7060X2,
7060X3, 7280, 7020R
Fortinet 4.21.4
L3 Routed 7060X, 7060X2, 7060X3 Palo Alto Networks 4.21.5
L3 Routed 7050X, 7050X2, 7050X3, 720XP Palo Alto Networks 4.21.6
L3 Routed 7060X, 7060X2, 7060X3, 7050X,
7050X2, 7050X3, 720XP
Fortinet 4.22.1
L3 Routed 7020R,7500R,7500R2,7280R,7280R2 Palo Alto
Fortinet
4.23.0
L3 Routed 7060X, 7060X2, 7060X3, 7050X,
7050X2, 7050X3, 720XP,
7020,7500R,7500R2,7280R,7280R2
CheckPoint 4.23.0

MSS Prerequisites
● Fully configured IP Fabric
● Direct flow supported platforms
● Firewalls policies with appropriate MSS tags
● Layer-2 adjacency between FWs and TOR switches
● CVX configured for MSS service
Additional Resources:
MSS Design and Deployment Guide
10

MSS Operations
Step 1: Arista CloudVision Exchange (CVX) as a single point of control
11
IP Fabric with VXLAN
Single point of
integration to the
physical infrastructure
Arista Leaf Spine Architecture

MSS Operations
Step 2: Firewall rules are implemented by the security team
12
● CloudVision Exchange (CVX) will send a request to the firewall manager to
provide the details of the security policies
● There is continuous polling between CVX and the firewall manager

MSS Operations
Step 3: CloudVision Exchange applies an intercept to steer interesting traffic
13
MSS
CVX implements intercept in leaf switches for
interesting traffic per FW rule
CVX continuous to receive state change
real time
● Once a firewall policy has been created with the configured tag(s) that
affect a host that CloudVision Exchange is aware of through SysDB state.
● CVX matches the hosts physical switch port against it’s database.
● CVX then pushes intercept rules to the leaf switches where the source is
located as well as the service leaf where the firewall is attached.

MSS Operations
Step 4: Data plane traffic steering with Macro Segmentation Service.
14
Leaf switches start sending
intercepted traffic to service leaf
Compute Leaf Service Leaf
Service leaf sends original
intercepted traffic to Firewall
Service Leaf

MSS Operations
Step 4: Data plane traffic steering with Macro Segmentation Service.
15
Service leaf switch sends inspected
traffic to final destination
Compute Leaf Service Leaf
Firewall applies all rules for the traffic to
allow/deny/log sends original traffic back to
Service leaf switch
Service Leaf

MSS with Layer-2 Transparent Firewalls
● The reference designed illustrated is an example of a typical MSS
deployment with two 3 tier application(s) in a multi tenant environment.
● The goal of this design is to isolate tenants as well as limit access
between hosts in the database zone(s), application zone(s), and web
zone(s).
● In this reference design Firewalls are acting in Transparent Mode.
16

Tenant B
Tenant A
untrust-dbb
trust-dbb
untrust-appb
trust-appb
untrust-wbb
trust-webb
untrust-dba
trust-dba
untrust-appa
trust-appa
untrust-weba
trust-wba
Reference Design
17
Firewall
Web
Server
Firewall
App
Server
Firewall database
Firewall
Web
Server
Firewall
App
Server
Firewall database
End
User

Different zones, vWires, interfaces and VLANs
18
Tenant A Ingress Zone Egress Zone vWire VLAN vWire Ingress
Subinterface
vWire Ingress
Subinterface
Database untrust-dba trust-dba dba 102 ae1.102 ae2.102
Application untrust-appa trust-appa appa 101 ae1.101 ae2.101
Web untrust-weba trust-weba weba 100 ae1.100 ae2.100
Tenant B Ingress Zone Egress Zone vWire VLAN vWire Ingress
Subinterface
vWire Ingress
Subinterface
Database untrust-dbb trust-dbb dbb 202 ae1.202 ae2.202
Application untrust-appb trust-appb appb 201 ae1.201 ae2.201
Web untrust-webb trust-webb webb 200 ae1.200 ae2.200

Logical Topology
● From a logical point of view each server is “on-a-stick”
● MSS steers traffic towards the firewall and requires the traffic to egress
the other side of firewall on the trusted side of the same vWire.
● Once a policy with the proper tag is created, MSS steers ingress traffic
towards the untrusted side of the vWire.
● The firewall then actions against the traffic as specified in the policy, and
the traffic egresses out of the trusted vWire
● The return traffic from the firewall is then reinserted in to the original
VLAN/VNI on the service switch and bridged to the appropriate
destination.
19

Logical Topology
20
untrust-
weba
ae1.100
trust-
weba
ae2.100
Firewall
VLAN 100
untrust-
appa
ae1.101
trust-
appa
ae2.101
Firewall
VLAN 101
untrust-
dba
ae1.102
trust-
dba
ae2.102
Firewall
VLAN 102
trust-
webb
ae1.200
untrust-
webb
ae1.200
Firewall
VLAN 200
trust-
appb
ae1.201
untrust-
appb
ae1.201
Firewall
VLAN 201
trust-
dbb
ae1.202
unrust-
dbb
ae1.202
Firewall
VLAN 202
appb
webb
dbb
appa
weba
dba
Switch
Fabric
End User
Tenant B Tennant A

Firewall Policies
● End users access the web server(s) through port TCP 443. The traffic
flows through the active firewall to the web server(s)from the untrust-
web(a/b) security zone to the trust-web(a/b) security zone.
● In cases where the intercepted host does not initiate a session, a return
rule may be required for the firewall to allow traffic through. This rule
should not be tagged if the rule allowing traffic in the other direction is
tagged.
● The web server(s) accesses the application server(s) through port TCP 80
after traversing the active firewall from the untrust-app(a/b) to the trust-
app(a/b) zone
● From there, the application server(s) accesses the database(s) through
port TCP 1433 in the untrust-db(a/b) zone to the trust-db(a/b) zone
21

Firewall Policies In Panorama
22
Tenant A
Tenant B

CVX Cluster
Firewall Cluster
MLAG PairMLAG Pair
Physical Topology
23
Spine1 Spine2
Intercept
Switch 1
Intercept
Switch 2
Intercept
Switch 3
Service
Switch 1
Service
Switch 2
weba-1
appa-1
dba-1
webb-1
appb-1
dbb-1
weba-2
appa-2
dba-2
webb-2
appb-2
dbb-2
appa-3
dba-3
webb-3
appb-3
dbb-3
weba-3
Active Passive
Layer-3 ECMP
Layer-2 VXLAN Overlay
CVX-01
CVX-02
CVX-03
Firewall
Manager
Management
Switch

Terminology
This reference design uses the following terminology:
● Intercept Switch/VTEP: Top of the rack switch and VXLAN tunnel
endpoint connected to host from which traffic is intercepted in this design,
intercept-1, intercept-2 and intercept-3.
● Service Switch/VTEP: Top of the rack switch and VXLAN tunnel endpoint
connected to firewall. In this design service-1 and service-2.
● Service VNI: VXLAN tunnel created to redirected intercepted traffic to the
firewall
● Intercept Interface: The interface at the top of the rack switch that
receives the packet from the host being intercepted.
● Egress/Near interface: The interface on the service VTEP that forwards
the intercepted to the firewall.
24

Terminology
● Ingress/Far Interface: The interface on the service VTEP that receives
the traffic back from the firewall
● VXLAN: Virtual eXtensible LAN - standards based method of of
encapsulating Layer-2 traffic across a Layer-3 fabric.
● CVX: Arista Cloud Vision Exchange (CVX) is a part of CloudVision and is a
virtual instance of the same Extensible Operating System (EOS) that runs
on physical switches. It functions as a point of integration between Palo
Alto Networks Firewalls or Panorama and the Arista network in order to
steer interesting traffic to the firewall.
25

MSS Configuration
The steps below outline how to configure Arista MSS
Step 1: Deploy Cloud Vision Exchange
The first step is to deploy CloudVision Exchange and configure the Arista TOR
switches to connect to it. A CVX cluster of 3 instances with hostnames cvx-01,
cvx-02, and cvx-03 has been configured for this design.
Please refer to the CVX configuration guide for more information.
26

MSS Configuration
Step 2: Enable VXLAN Control Service on CVX
Once the three Arista CVX instances have been deployed and the TOR
switches have been configured to be managed by them, the VXLAN Control
Service (VCS) must be enabled on every CVX instance.
The VXLAN Control service allows hardware VXLAN Tunnel Endpoints (VTEPs)
to share state with each other in order to establish VXLAN tunnels without the
need for a multicast control plane.
27

MSS Configuration
28
CVX-01#config
CVX-01(config)#cvx
CVX-01(config-cvx)#service vxlan
CVX-01(config-cvx-vxlan)#no shutdown
CVX-01(config-cvx-vxlan)#
CVX-02#config
CVX-02(config)#cvx
CVX-03#config
CVX-03(config)#cvx

MSS Configuration
Step 3: Configure intercept switch and service switch ports
This step involves configuration of the switch ports connected to the hosts,
whose traffic needs to be steered to firewalls and service switch which is
connected to the firewalls
Intercept switch configuration
The switch ports connected to the hosts whose traffic needs to be intercepted,
need to be configured as 802.1Q trunks with the VLAN that is mapped to the
VNI requiring interception. Unique VLAN IDs are configured for each tier of the
application. VXLAN to VNI mapping also needs to be configured. All switches
are configured identically:
29

Intercept Switch 1
MSS Configuration
30
intercept-1#configure
intercept-1(config)#vlan 100
intercept-1(config-vlan-100)#name weba
intercept-1(config-vlan-100)#vlan 101
intercept-1(config-vlan-101)#name appa
intercept-1(config-vlan-102)#name dba
intercept-1(config-vlan-200)#name webb
intercept-1(config-vlan-201)#name appb
intercept-1(config-vlan-202)#name dbb

MSS Configuration
Intercept Switch 1
31
intercept-1(config)#interface Ethernet50/1
intercept-1(config-if-Et50/1)# switchport mode trunk
intercept-1(config-if-Et50/1)# switchport trunk allowed vlan 100-102,200-202
intercept-1(config)#interface Ethernet3
intercept-1(config-if-Et3)# switchport mode trunk
intercept-1(config-if-Et3)# switchport trunk allowed vlan 100-102,200-202

MSS Configuration
VXLAN to VLAN Mapping Configuration - Intercept Switch-1
32
intercept-1#config
intercept-1(config)#interface Vxlan1
intercept-1(config-if-Vx1)# vxlan source-interface Loopback0
intercept-1(config-if-Vx1)# vxlan controller-client
intercept-1(config-if-Vx1)# vxlan udp-port 4789
intercept-1(config-if-Vx1)# vxlan vlan 100 vni 1000

MSS Configuration
Enable CVX Client in intercept-1
33
intercept-1# config
intercept-1(config)#management cvx
intercept-1(config-mgmt-cvx)# no shutdown
intercept-1(config-mgmt-cvx)# server host 10.92.59.103
intercept-1(config-mgmt-cvx)# source-interface Management1
CVX Cluster
Management IP
addresses

Intercept Switch 2
MSS Configuration
34

MSS Configuration
35
intercept-2(config-if-Et1)# switchport mode trunk
intercept-2(config-if-Et1)# switchport trunk allowed vlan 100-102,200-202
Intercept Switch 2

MSS Configuration
36
intercept-2#config

MSS Configuration
37
intercept-2# config
CVX Cluster
Management IP
addresses

Rack-3Rack-1
MLAG
MLAG Configuration
To provide redundancy and fault tolerance it is recommend that MLAG is
configured between the TOR switches within the same racks
38
Intercept
Switch 1
Intercept
Switch 2
weba-1
appa-1
dba-1
webb-1
appb-1
dbb-1
weba-2
appa-2
dba-2
webb-2
appb-2
dbb-2
Service
Switch 1
Service
Switch 2
Active Passive
MLAG
Firewall Cluster

MLAG Configuration Rack-1/Switch-1
intercept-1#config
intercept-1(config-vlan-4093)# name LEAF_PEER_L3
intercept-1(config-vlan-4093)# trunk group LEAF_PEER_L3
intercept-1(config-vlan-4094)# name MLAG_PEER
intercept-1(config-vlan-4094)# trunk group MLAG
intercept-1(config-vlan-4094)#interface Port-Channel1000
intercept-1(config-if-Po1000)# description MLAG-Peer
intercept-1(config-if-Po1000)# switchport trunk allowed vlan 2-4094
intercept-1(config-if-Po1000)# switchport trunk group LEAF_PEER_L3
intercept-1(config-if-Po1000)# switchport trunk group MLAG
intercept-1(config-if-Et51/1)# description intercept-2
intercept-1(config-if-Et51/1)# switchport trunk allowed vlan 2-4094
intercept-1(config-if-Et51/1)# channel-group 1000 mode active
39

MLAG Configuration Rack-1/Switch1
intercept-1#config
intercept-1(config)#interface Vlan4093
intercept-1(config-if-Vl4093)# ip address 172.16.1.1/30
intercept-1(config-if-Vl4093)#mlag configuration
intercept-1(config-mlag)# domain-id rack-1
intercept-1(config-mlag)# local-interface Vlan4093
intercept-1(config-mlag)# peer-address 172.16.1.2
intercept-1(config-mlag)# peer-link Port-Channel1000
intercept-1(config)#interface Port-Channel1001
intercept-1(config-if-Po1001)# switchport trunk allowed vlan 100-102,200-202
intercept-1(config-if-Po1001)# switchport mode trunk
intercept-1(config-if-Po1001)# mlag 1001
intercept-1(config-if-Po1001)#interface Port-Channel1002
40

intercept-1#config
intercept-1(config-if-Et3)# channel-group 1001 mode active
intercept-1(config-if-Et3)#interface Ethernet50/1
41

intercept-2#config
intercept-2(config-vlan-4093)# name LEAF_PEER_L3
intercept-2(config-vlan-4093)# trunk group LEAF_PEER_L3
intercept-2(config-vlan-4094)# name MLAG_PEER
intercept-2(config-vlan-4094)# trunk group MLAG
intercept-2(config-vlan-4094)#interface Port-Channel1000
intercept-2(config-if-Po1000)# description MLAG-Peer
intercept-2(config-if-Po1000)# switchport trunk allowed vlan 2-4094
intercept-2(config-if-Po1000)# switchport trunk group LEAF_PEER_L3
intercept-2(config-if-Po1000)# switchport trunk group MLAG
intercept-2(config-if-Et51/1)# description intercept-1
intercept-2(config-if-Et51/1)# switchport trunk allowed vlan 2-4094
42

intercept-2#config
intercept-2(config-if-Vl4093)# ip address 172.16.1.2/30
intercept-2(config-if-Vl4093)#mlag configuration
intercept-2(config-mlag)# domain-id rack-1
intercept-2(config-mlag)# local-interface Vlan4093
intercept-2(config-mlag)# peer-address 172.16.1.1
intercept-2(config-mlag)# peer-link Port-Channel1000
intercept-2(config)#interface Port-Channel1001
intercept-2(config-if-Po1001)#interface Port-Channel1002
43

intercept-2#config
intercept-2(config-if-Et3)# channel-group 1001 mode active
intercept-2(config-if-Et3)#interface Ethernet50/1
44

Intercept Switch 3
MSS Configuration
45

MSS Configuration
46
Intercept Switch 3
Note: For untagged traffic configure a native VLAN on the port using “switchport trunk
native vlan” command.

MSS Configuration
47
intercept-3#config

MSS Configuration
48
intercept-3# config
CVX Cluster
Management IP
addresses

MSS Configuration
Service Switch Port Configuration
A service switch defined as the switch connecting to the firewalls. Switch ports
connected to the firewalls are configured as trunk ports, with allowed VLANs
set to “none”. As MSS builds intercept rules based on configured firewall
policies. CVX will dynamically configure VLANs as required on these ports.
49

MSS Configuration
service-1#config
service-1(config)#interface Port-Channel1001
service-1(config-if-Po1001)# switchport trunk allowed vlan none
service-1(config-if-Po1001)# switchport mode trunk
service-1(config-if-Po1001)# spanning-tree portfast
service-1(config-if-Po1001)# spanning-tree bpdufilter enable
service-1(config-if-Po1001)#interface Port-Channel1002
50
Service Switch-1

MSS Configuration
51
Service Switch-1

MSS Configuration
service-1(config)#interface Ethernet5
service-1(config-if-Et5)# description PA-MGMT38-P
service-1(config-if-Et5)# switchport trunk allowed vlan none
service-1(config-if-Et5)# switchport mode trunk
service-1(config-if-Et5)# channel-group 1003 mode on
service-1(config-if-Et5)# spanning-tree portfast
service-1(config-if-Et5)# spanning-tree bpdufilter enable
service-1(config-if-Et5)#interface Ethernet6
service-1(config-if-Et6)# switch port mode trunk
52
Service Switch-1
Passive (Standby) Firewall

MSS Configuration
service-1(config-if-Et7)# description PA-MGMT37-A
53
Service Switch-1
Active (Hot) Firewall

MSS Configuration
service-2#config
54
Service Switch-2

MSS Configuration
55
Service Switch-2

MSS Configuration
service-2(config)#interface Ethernet5
56
Service Switch-2

MSS Configuration
57
Service Switch-2

MSS Configuration
Note: Dynamically mapped VLANs are not shown in the switchport
configuration. They can be viewed by issuing the “show vlan” command to the
switch once a policy is applied.
VXLAN to VNI mapping also needs to be configured. Both switches are
configured identically:
Spanning-tree portfast must be configured with BPDU filter enabled.
Alternatively spanning-tree must be disable on the interfaces connected to the
firewalls. Failure to do so will result in “far-end” interfaces going to
“discarding” state.
58

Rack-3Rack-1
MLAG
MLAG Configuration
To provide redundancy and fault tolerance it is recommend that MLAG is
configured between the TOR switches within the same racks
59
Intercept
Switch 1
Intercept
Switch 2
weba-1
appa-1
dba-1
webb-1
appb-1
dbb-1
weba-2
appa-2
dba-2
webb-2
appb-2
dbb-2
Service
Switch 1
Service
Switch 2
Active Passive
MLAG
Firewall Cluster

service-1# config
service-1(config)#vlan 4093
service-1(config-vlan-4093)# name LEAF_PEER_L3
service-1(config-vlan-4093)# trunk group LEAF_PEER_L3
service-1(config-vlan-4093)#vlan 4094
service-1(config-vlan-4094)# name MLAG_PEER
service-1(config-vlan-4094)# trunk group MLAG
service-1(config-vlan-4094)#interface Port-Channel1000
service-1(config-if-Po1000)# description MLAG-Peer
service-1(config-if-Po1000)# switchport trunk allowed vlan 2-4094
service-1(config-if-Po1000)# switchport trunk group LEAF_PEER_L3
service-1(config-if-Po1000)# switchport trunk group MLAG
60

service-1#config
service-1(config)#interface Ethernet50/1
service-1(config-if-Et50/1)# description service-2
service-1(config-if-Et50/1)# switchport trunk allowed vlan 2-4094
service-1(config-if-Et50/1)# switchport mode trunk
service-1(config-if-Et50/1)# switchport trunk group LEAF_PEER_L3
service-1(config-if-Et50/1)# channel-group 1000 mode active
service-1(config)#interface Vlan4093
service-1(config-if-Vl4093)# ip address 172.16.3.1/30
service-1(config)#mlag configuration
service-1(config-mlag)# domain-id rack-3
service-1(config-mlag)# local-interface Vlan4093
service-1(config-mlag)# peer-address 172.16.3.2
service-1(config-mlag)# peer-link Port-Channel1000
61

service-1# config
service-1(config-if-Po1001)# mlag 101
62

service-2# config
service-2(config)#vlan 4093
service-2(config-vlan-4093)# name LEAF_PEER_L3
service-2(config-vlan-4093)# trunk group LEAF_PEER_L3
service-2(config-vlan-4093)#vlan 4094
service-2(config-vlan-4094)# name MLAG_PEER
service-2(config-vlan-4094)# trunk group MLAG
service-2(config-vlan-4094)#interface Port-Channel1000
service-2(config-if-Po1000)# description MLAG-Peer
service-2(config-if-Po1000)# switchport trunk allowed vlan 2-4094
service-2(config-if-Po1000)# switchport trunk group LEAF_PEER_L3
service-2(config-if-Po1000)# switchport trunk group MLAG
63

service-2#config
service-2(config)#interface Ethernet50/1
service-2(config-if-Et50/1)# description service-1
service-2(config-if-Et50/1)# switchport trunk allowed vlan 2-4094
service-2(config-if-Et50/1)# switchport mode trunk
service-2(config-if-Et50/1)# switchport trunk group LEAF_PEER_L3
service-2(config-if-Et50/1)# channel-group 1000 mode active
service-2(config-if-Vl4093)# ip address 172.16.3.2/30
service-2(config)#mlag configuration
service-2(config-mlag)# domain-id rack-3
service-2(config-mlag)# local-interface Vlan4093
service-2(config-mlag)# peer-address 172.16.3.1
service-2(config-mlag)# peer-link Port-Channel1000
64

service-2# config
65

MSS Configuration
VXLAN to VLAN Mapping Configuration - Service Switch-1
66
service-1#config
service-1(config)#interface Vxlan1
service-1(config-if-Vx1)# vxlan source-interface Loopback0
service-1(config-if-Vx1)# vxlan controller-client
service-1(config-if-Vx1)# vxlan udp-port 4789
service-1(config-if-Vx1)# vxlan vlan 100 vni 1000

MSS Configuration
Enable CVX Client in Service-1
67
service-1# config
service-1(config)#management cvx
service-1(config-mgmt-cvx)# no shutdown
service-1(config-mgmt-cvx)# server host 10.92.59.103
service-1(config-mgmt-cvx)# source-interface Management1
CVX Cluster
Management IP
addresses

MSS Configuration
VXLAN to VLAN Mapping Configuration - Service Switch-2
68
service-2#config
service-2(config)#interface Vxlan1
service-2(config-if-Vx1)# vxlan source-interface Loopback0
service-2(config-if-Vx1)# vxlan controller-client
service-2(config-if-Vx1)# vxlan udp-port 4789

MSS Configuration
Enable CVX Client in Service-2
69
service-2# config
service-2(config)#management cvx
service-2(config-mgmt-cvx)# no shutdown
service-2(config-mgmt-cvx)# source-interface Management1
CVX Cluster
Management IP
addresses

Step 4: Enable Direct Flow on access switches
Arista MSS uses DirectFlow to steer interesting traffic from the intercepted
host to the firewall, and back. Diretflow must be enabled on every intercept
switch as well as both service switches.
Note: DirectFlow Configuration is platform dependent
On 7050X/7050X2/7050X3/7060X/7060X2 Platforms configure the following:
MSS Configuration
70
Switch#config
Switch(config)#directflow
Switch(config-directflow)# no shutdown

On 7280R/7280R2 copy and paste the following in the configuration mode
MSS Configuration
71
hardware tcam
profile direct-flow-mss
feature acl port ip
sequence 50
key size limit 160
key field dscp dst-ip ip-frag ip-protocol l4-dst-port l4-ops l4-src-port src-ip tcp-control ttl
action count drop
packet ipv4 forwarding bridged
packet ipv4 forwarding routed
packet ipv4 forwarding routed multicast
packet ipv4 mpls ipv4 forwarding mpls decap
packet ipv4 non-vxlan forwarding routed decap
packet ipv4 vxlan eth ipv4 forwarding routed decap
packet ipv4 vxlan forwarding bridged decap

MSS Configuration
!
feature acl port ipv6
sequence 30
key field dst-ipv6 ipv6-next-header ipv6-traffic-class l4-dst-port l4-ops-3b l4-src-port src-ipv6-high src-ipv6-low tcp-control
action count drop mirror
72

MSS Configuration
feature acl port mac
sequence 60
key size limit 160
key field dst-mac ether-type src-mac
packet ipv6 forwarding routed decap
73

MSS Configuration
packet mpls forwarding bridged decap
packet mpls ipv4 forwarding mpls
packet mpls ipv6 forwarding mpls
packet mpls non-ip forwarding mpls
packet non-ip forwarding bridged
!
feature acl subintf ip
sequence 45
key size limit 160
key field dscp dst-ip ip-frag ip-protocol l4-dst-port l4-ops-18b l4-src-port src-ip tcp-control ttl
!
74

MSS Configuration
feature acl subintf ipv6
sequence 20
key field dst-ipv6 ipv6-next-header l4-dst-port l4-src-port src-ipv6-high src-ipv6-low tcp-control
action count drop mirror redirect
!
feature acl vlan ip
sequence 40
key size limit 160
key field dscp dst-ip ip-frag ip-protocol l4-dst-port l4-ops-18b l4-src-port src-ip tcp-control ttl
75

MSS Configuration
feature acl vlan ipv6
sequence 15
key field dst-ipv6 ipv6-next-header l4-dst-port l4-src-port src-ipv6-high src-ipv6-low tcp-control
action count drop mirror redirect
!
feature acl vlan ipv6 egress
sequence 25
key field dscp dst-ipv6 ipv6-next-header l4-dst-port l4-src-port src-ipv6-high src-ipv6-low tcp-control
76

MSS Configuration
feature flow
key field in-port src-mac vlan
action redirect-to-vxlan
!
feature tunnel vxlan
sequence 55
key size limit 160
key field in-port vxlan-inner-etype vxlan-inner-ip-options vxlan-inner-ip-ttl
!
77

MSS Configuration
feature flow
key field in-port src-mac vlan
action redirect-to-vxlan
!
feature tunnel vxlan
sequence 55
key size limit 160
key field in-port vxlan-inner-etype vxlan-inner-ip-options vxlan-inner-ip-ttl
!
78

MSS Configuration
feature tunnel vxlan routing
sequence 10
system profile direct-flow-mss
!
directflow
no shutdown
!
79

MSS Configuration
Step 5: Enable routing on TOR Switches and SPINE Switches
CVX uses Address Resolution Protocol (ARP) to determine where intercept
hosts are physically located in the network.
It is recommended that VXLAN routing be configured on every TOR and
service switch that will be intercepting traffic to ensure that CVX is aware of
every host ARP entry. The configuration below shows the routing configuration
for each tier of the application and the entire VXLAN configuration.
80

MSS Configuration
Switch intercept-1 Routing Configuration
81
intercept-1# config
intercept-1(config-if-Vl100)# ip address virtual 100.64.100.1/24
intercept-1(config-if-Vl100)#interface Vlan101
intercept-1(config)#interface Loopback0
intercept-1(config-if-Lo0)# ip address 1.1.1.1/32
intercept-1(config-if-Lo0)#interface Loopback1

MSS Configuration
82
intercept-1#config
intercept-1(config)#ip prefix-list LOOP_BACK
intercept-1(config-ip-pfx)# seq 10 permit 1.1.1.1/32
intercept-1(config-ip-pfx)#!
intercept-1(config-ip-pfx)#route-map LOOP_BACK permit 10
intercept-1(config-route-map-LOOP_BACK)# match ip address prefix-list LOOP_BACK

MSS Configuration
83
intercept-1#config
intercept-1(config)#router bgp 65001
intercept-1(config-router-bgp)# router-id 1.1.1.11
intercept-1(config-router-bgp)# maximum-paths 8
intercept-1(config-router-bgp)# neighbor LEAF_PEER peer group
intercept-1(config-router-bgp)# neighbor LEAF_PEER remote-as 65001
ntercept-1(config-router-bgp)# neighbor LEAF_PEER next-hop-self
intercept-1(config-router-bgp)# neighbor LEAF_PEER maximum-routes 12000
intercept-1(config-router-bgp)# neighbor SPINE peer group
intercept-1(config-router-bgp)# neighbor SPINE remote-as 65000
intercept-1(config-router-bgp)# neighbor SPINE route-map LOOP_BACK out
intercept-1(config-router-bgp)# neighbor SPINE send-community
intercept-1(config-router-bgp)# neighbor SPINE maximum-routes 12000
intercept-1(config-router-bgp)# neighbor 100.64.1.1 peer group SPINE
intercept-1(config-router-bgp)# neighbor 172.16.1.2 peer group LEAF_PEER
intercept-1(config-router-bgp)# redistribute connected route-map LOOP_BACK

MSS Configuration
84
intercept-2# config

MSS Configuration
85
intercept-2#config

MSS Configuration
86
intercept-2#config
intercept-2(config-router-bgp)# neighbor LEAF_PEER peer group
intercept-2(config-router-bgp)# neighbor LEAF_PEER remote-as 65001
intercept-2(config-router-bgp)# neighbor LEAF_PEER next-hop-self
intercept-2(config-router-bgp)# neighbor LEAF_PEER maximum-routes 12000
intercept-2(config-router-bgp)# neighbor 172.16.1.1 peer group LEAF_PEER

MSS Configuration
87
intercept-3# config

MSS Configuration
88
intercept-3#config

MSS Configuration
89
intercept-3#config

MSS Configuration
Switch Service-1 Routing Configuration
90
service-1# config
service-1(config-if-Vl100)# ip address virtual 100.64.100.1/24
service-1(config-if-Vl100)#interface Vlan101
service-1(config)#interface Loopback0
service-1(config-if-Lo0)# ip address 3.3.3.1/32
service-1(config-if-Lo0)#interface Loopback1

MSS Configuration
91
service-1#config
service-1(config)#ip prefix-list LOOP_BACK
service-1(config-ip-pfx)# seq 10 permit 3.3.3.1/32
service-1(config-ip-pfx)#route-map LOOP_BACK permit 10
service-1(config-route-map-LOOP_BACK)# match ip address prefix-list LOOP_BACK

MSS Configuration
92
service-1#config
service-1(config)#router bgp 65003
service-1(config-router-bgp)# router-id 3.3.3.11
service-1(config-router-bgp)# maximum-paths 8
service-1(config-router-bgp)# neighbor LEAF_PEER peer group
service-1(config-router-bgp)# neighbor LEAF_PEER remote-as 65003
service-1(config-router-bgp)# neighbor LEAF_PEER next-hop-self
service-1(config-router-bgp)# neighbor LEAF_PEER maximum-routes 12000
service-1(config-router-bgp)# neighbor SPINE peer group
service-1(config-router-bgp)# neighbor SPINE remote-as 65000
service-1(config-router-bgp)# neighbor SPINE route-map LOOP_BACK out
service-1(config-router-bgp)# neighbor SPINE send-community
service-1(config-router-bgp)# neighbor SPINE maximum-routes 12000
service-1(config-router-bgp)# neighbor 100.64.3.1 peer group SPINE
service-1(config-router-bgp)# neighbor 172.16.3.2 peer group LEAF_PEER
service-1(config-router-bgp)# redistribute connected route-map LOOP_BACK

MSS Configuration
93
service-2# config
service-2(config)#interface Loopback0
service-2(config-if-Lo0)#interface Loopback1

MSS Configuration
94
service-2#config

MSS Configuration
95
service-2#config
service-2(config)#router bgp 65003
service-2(config-router-bgp)# router-id 3.3.3.12
service-2(config-router-bgp)# neighbor LEAF_PEER peer group
service-2(config-router-bgp)# neighbor LEAF_PEER remote-as 65003
service-2(config-router-bgp)# neighbor LEAF_PEER next-hop-self
service-2(config-router-bgp)# neighbor SPINE peer group
service-2(config-router-bgp)# neighbor SPINE remote-as 65000
service-2(config-router-bgp)# neighbor SPINE route-map LOOP_BACK out
service-2(config-router-bgp)# neighbor SPINE send-community
service-2(config-router-bgp)# neighbor 100.64.3.5 peer group SPINE
service-2(config-router-bgp)# neighbor 172.16.3.1 peer group LEAF_PEER
service-2(config-router-bgp)# redistribute connected route-map LOOP_BACK

MSS Configuration
96
service-2#config

MSS Configuration
spine-1#config
spine-1(config)#interface Ethernet1/1
spine-1(config-if-Et1/1)# description service-1
spine-1(config-if-Et1/1)# no switchport
spine-1(config-if-Et1/1)# ip address 100.64.3.1/30
spine-1(config-if-Et1/1)#interface Ethernet2/1
spine-1(config-if-Et2/1)# description service-2
spine-1(config-if-Et4/1)# description intercept-3
97
Switch Spine-1 Routing Configuration

MSS Configuration
spine-1(config)#interface Loopback0
spine-1(config-if-Lo0)# ip address 1.1.0.1/32
98

MSS Configuration
99
spine-1#config
spine-1(config)#router bgp 65000
spine-1(config-router-bgp)# router-id 1.1.0.1
spine-1(config-router-bgp)# maximum-paths 8
spine-1(config-router-bgp)# neighbor 100.64.1.2 remote-as 65001
spine-1(config-router-bgp)# redistribute connected

MSS Configuration
Step 6: Configure the MSS service on CVX
The next step enables Arista MSS Service on CVX. The reference design
includes three (3) CVX instances in a cluster, and the configuration must be the
same for every instance.
CVX-01 = 10.92.59.100
CVX-02 = 10.92.59.102
CVX-03 = 10.92.59.103
In this reference design Panorama (Palo Alto Firewall Manager) is configured.
Alternatively both the active and standby Palo Alto firewalls can be configured
instead of the Panorama DNS name.
In this reference design Panorama has a DNS name of “panorama-
mss.sjc.aristanetworks.com”.
100
CVX Cluster Management
IP addresses

MSS Configuration
101
Command Description
service mss Enables MSS Service on CVX
vni range 20000-30000 A dynamic range of VNI’s that will be allocated for VXLAN encapsulated
traffic to the firewall
dynamic service-set PANFW Created a set of devices, typically a pair of firewalls with the name
PANFW
tag Arista_MSS Specifies the tags that MSS looks for when reading security policy from
the firewall or firewall manager. Defaults to “Arista_MSS” - and will not
be displayed in the running configuration. More than one tag can be
configured.
state active Allows you to set the device set as active or disabled
device panorama-mss.sjc.aristanetworks.com Defines the device. Hostname or IP address that MSS will use.
type palo-alto panorama Set the firewall type
username admin password 0 admin Sets the username/password to access the device

MSS Configuration
CVX instance CVX-01
102
CVX-01#config
CVX-01(config)#cvx
CVX-01(config-cvx)# no shutdown
CVX-01(config-cvx)# peer host 10.92.59.103
CVX-01(config-cvx)# source-interface Management1
CVX-02(config-cvx)# service mss
CVX-01(config-cvx-mss)# no shutdown
CVX-01(config-cvx-mss)# vni range 20000-30000
CVX-01(config-cvx-mss)# dynamic device-set PANFW
CVX-01(config-cvx-mss-PANFW)# device panorama-mss.sjc.aristanetworks.com
CVX-01(config-cvx-mss-PANFW-panorama-mss.sjc.aristanetworks.com)# username <username> password 7 <password>
CVX-01(config-cvx-mss-PANFW-panorama-mss.sjc.aristanetworks.com)# group Arista_MSS
CVX-01(config-cvx-mss-PANFW-panorama-mss.sjc.aristanetworks.com)# state active
CVX-01(config-cvx-mss-PANFW)# type palo-alto panorama

MSS Configuration
CVX instance CVX-02
103
CVX-02#config
CVX-02(config)#cvx
CVX-02(config-cvx-mss-PANFW-panorama-mss.sjc.aristanetworks.com)#username <username> password 7 <password>
CVX-02(config-cvx-mss-PANFW-panorama-mss.sjc.aristanetworks.com)# group Arista_MSS

MSS Configuration
CVX instance CVX-03
104
CVX-03#config
CVX-03(config)#cvx
CVX-03(config-cvx-mss-PANFW-panorama-mss.sjc.aristanetworks.com)#username <username> password 7 <password>
CVX-03(config-cvx-mss-PANFW-panorama-mss.sjc.aristanetworks.com)#group Arista_MSS

Firewall Attachment to Service Switches
105
Service-1 Service-2
PA-MGMT37-A
Active
PA-MGMT38-P
Standby
MLAG
Panorama
Management Subnet
Et1/15
Et1/16
Et1/13 Et1/14 Et1/13 Et1/14Et1/15
Et1/16
Et 7 Et 8 Et 5 Et 6 Et 5 Et 6 Et 7 Et 8

MSS Configuration
Step 7: Firewall Configuration
The following firewall configuration used for the reference design.
Firewall Network Configuration
Interfaces have been configured in aggregation groups
106

Firewall Interface Configuration (Panorama)
107

Firewall Interface Configuration
Active Firewall
108
admin@PA-MGMT37-A(active)> show interface all
total configured hardware interfaces: 11
name id speed/duplex/state mac address
--------------------------------------------------------------------------------
ethernet1/13 28 1000/full/up 00:1b:17:00:25:1c
ethernet1/14 29 1000/full/up 00:1b:17:00:25:1d
ethernet1/15 30 1000/full/up 00:1b:17:00:25:1e
ethernet1/16 31 1000/full/up 00:1b:17:00:25:1f
ae1 48 [n/a]/[n/a]/up 00:1b:17:00:25:30
ae2 49 [n/a]/[n/a]/up 00:1b:17:00:25:31
aggregation groups: 2
ae1 members:
ethernet1/13 ethernet1/15
ae2 members:

Firewall Interface Configuration
Passive Firewall
109
admin@PA-MGMT38-P(passive)> show interface all
total configured hardware interfaces: 11
name id speed/duplex/state mac address
--------------------------------------------------------------------------------
ethernet1/13 28 ukn/ukn/down(power-down) 00:1b:17:00:25:1c
ethernet1/14 29 ukn/ukn/down(power-down) 00:1b:17:00:25:1d
ethernet1/15 30 ukn/ukn/down(power-down) 00:1b:17:00:25:1e
ethernet1/16 31 ukn/ukn/down(power-down) 00:1b:17:00:25:1f
ae1 48 ukn/ukn/down(unknown) 00:1b:17:00:25:30
ae2 49 ukn/ukn/down(unknown) 00:1b:17:00:25:31
aggregation groups: 2
ae1 members:
ae2 members:

Firewall vWire Configuration (Panorama)
110
Firewall Zone Configuration (Panorama)

Firewall vWire Configuration (Active)
admin@PA-MGMT37-A(active)> show virtual-wire all
total virtual-wire shown : 7
flags : m - multicast firewalling
p - link state pass-through
s - vlan sub-interface
i - ip+vlan sub-interface
t - tenant sub-interface
name interface1 interface2 flags allowed-tags
--------------------------------------------------------------------------------
weba ae1.100 ae2.100 s 100
appa ae1.101 ae2.101 s 101
dba ae1.102 ae2.102 s 102
webb ae1.200 ae2.200 s 200
appb ae1.201 ae2.201 s 201
dbb ae1.202 ae2.202 s 202
111

Firewall vWire Configuration (Standby)
admin@PA-MGMT38-P(passive)> show virtual-wire all
total virtual-wire shown : 7
flags : m - multicast firewalling
p - link state pass-through
s - vlan sub-interface
i - ip+vlan sub-interface
t - tenant sub-interface
name interface1 interface2 flags allowed-tags
--------------------------------------------------------------------------------
weba ae1.100 ae2.100 s 100
appa ae1.101 ae2.101 s 101
dba ae1.102 ae2.102 s 102
webb ae1.200 ae2.200 s 200
appb ae1.201 ae2.201 s 201
dbb ae1.202 ae2.202 s 202
112

Firewall LLDP Configuration (Panorama)
LLD (transmit and receive) must be enabled on firewall interfaces attached to
the service switches.
113

Firewall LLDP Neighbors
service-1#show lldp neighbors
Last table change time : 1 day, 18:34:09 ago
Number of table inserts : 12
Number of table deletes : 4
Number of table drops : 0
Number of table age-outs : 0
Port Neighbor Device ID Neighbor Port ID TTL
Et7 PA-MGMT37-A ethernet1/13 120
<snip>
114

Firewall LLDP Neighbors
service-2#show lldp neighbors
Last table change time : 1 day, 14:00:58 ago
Number of table inserts : 9
Number of table deletes : 1
Number of table drops : 0
Number of table age-outs : 0
Port Neighbor Device ID Neighbor Port ID TTL
<snip>
115

Firewall Tags and Address Configurations (Panorama)
116

Firewall HA Configuration
117
PA-MGMT37-A
Active
PA-MGMT38-P
Standby
ha1
ha2
ha1
ha2

Firewall Policies
For the reference design 12 policies are created in addition to the default
implicit deny policy for interzone traffic. Note that the default implicit deny will
ensure that interzone traffic will not be allowed unless a policy explicitly allows
for it.
The first policy “untrust-weba” is from the untrust web Zone A to the trust web
Zone A , that allows HTTPS (web-browsing) traffic from anywhere to web
server Zone A. A reverse policy named “trust-weba” that allows traffic in the
reverse direction must be configured for bi-directional connectivity.
118

Firewall Policies
The 3rd policy “untrust-appa” is from the untrust app zone A to the trust app
Zone A allows HTTPS(web-browsing) traffic between the web servers in Zone
A and the applications servers in Zone A.
A reverse policy named “trust-appa” that allows traffic in the reverse direction
must be configured for bi-directional connectivity.
119

Firewall Policies
The fifth policy “untrust-dba” is from the untrust database zone A to the
trusted database zone A that allows database traffic on port TCP 1433
(myssql-db) between the app server zone A and database zone A.
A reverse policy named “trust-dba” that allows traffic in the reverse direction
120

Firewall Policies
The seventh policy “untrust-webb” is from the untrust web Zone B to the trust
web Zone B , that allows HTTPS (web-browsing) traffic from anywhere to web
server Zone B.
A reverse policy named “trust-webb” that allows traffic in the reverse direction
121

Firewall Policies
The 9th policy “untrust-appb” is from the untrust app zone B to the trust app
Zone B allows HTTPS(web-browsing) traffic between the web servers in Zone
B and the applications servers in Zone B.
A reverse policy named “trust-appb” that allows traffic in the reverse direction
122

Firewall Policies
The 11th policy “untrust-dbb” is from the untrust database zone B to the
trusted database zone B that allows database traffic on port TCP 1433
(myssql-db) between the app server zone B and database zone B.
A reverse policy named “trust-dbb” that allows traffic in the reverse direction
123

MSS Configuration
124
To create a rule Arista MSS will use to intercept and redirect traffic ,
add a firewall policy with the default “Arista_MSS” tag as shown. MSS
intercept all traffic from endpoints identified in policies that match the
tag value(s) configured in CVX. The firewall will apply all rules - tagged
or untagged - to all traffic it sees.
Note that the return traffic does not require an MSS tag to function, but
still needs to be specified for the other side of the vWire
Arista_MSS tag
Policy for return traffic

Caveats
Firewall Configuration
● All policies configured on the firewall must not have any whitespace
character in their names, for example “PCI Zone” should be “PCI_Zone”.
● A maximum of 255 intercept hosts are supported per vlan.
● A firewall policy with “any” source and “any” destination zone cannot be
tagged to be used with MSS, including the default Palo Alto any/any
policy. An alternative method is to tag a single policy that defines traffic
from/to a specific host or subnet that needs to intercepted and steered to
the firewall.
125

Caveats
General
VXLAN routing with MSS is only supported with Direct (Asymmetric) Routing.
VXLAN routing with MSS is only supported with “ip address virtual”
configuration.
The current implementation divides the original layer-2 domain in to two
subsets and places the firewall(s) between the two subsets. This adds the
restriction where policies between multiple hosts can exist on either side of the
firewall. Consider the scenario where hosts A, B, and C communicate with
each other as shown below.
126
A B
C

Caveats
General
The current L2 Transparent implementation mandates logical placement of
certain hosts behind the firewall, as there is only two a virtual wire. This means,
the traffic steering, for inspection, can be achieved for traffic between host A
and host B as well as between host A and host C, but not between host B and
host C, as host B and host C would be considered to be on the same side of
the firewall as shown below.
127
C
A
B
Firewall
vWire

Macro Segmentation Service
With Layer-3 Firewalls

Macro Segmentation Service (MSS) for L3 Firewalls
● Working with a L3 firewall MSS service that runs on CVX, uses FW vendor
provided APIs to read policy and routing information from (or firewall
manager). The service then identifies a subset of policies that it needs to
act upon, based on user provided tags.
● Unlike a traditional L3 FW deployment where a gateway for endpoints in a
subnet is host on the FW and configured as the default route on the
endpoint, MSS uses DirectFlow flows to redirect traffic to the FW,
dynamically inserting it into the path for the traffic for relevant endpoints.
● In other word, the TORs intercepting traffic need to route traffic to an L2
adjacent FW, but the FW doesn't need an IP in the endpoint subnets.
129

MSS L3 Operation
Control Plane
MSS periodically polls the firewall device (or firewall manager) to obtain
policies and routing information configured at the firewall. For large policy
sets, the polling frequency can be configured using the command interval
under the service configuration mode.
It is recommended that to reduce churn in the network, the firewalls use static
routing, with routes in each zone pointing to the service TOR.
Data Plane
MSS configures DirectFlow flows on switches to influence forwarding. While
number of DirectFlow features utilize the relatively constrained TCAM
resources, MSS uses a combination of the L2/L3 forwarding tables and the
TCAM to implement traffic redirection and forwarding
130

MSS L3 Operation
Zones, Subnets, and Redirects
In order to redirect traffic, MSS installs some flows in the TCAM and upper
bound on the number of these flows can be calculated by 3 x number of
subnets that exist in different zones.
So, if zone A has 2 subnets, zone B has 3 subnets, and zone C has 4 subnets,
the total number of TCAM entries required for redirecting traffic is up to 27
entries.
In addition to the above rules, MSS also uses the existing L2 and L3
forwarding table entries, with no additional overhead, to identify endpoints
from where traffic is redirected. Even on the most constrained platform in the
network, these tables are in the order of 100K endpoints. So, MSS is capable
of redirecting line rate traffic from a large number of hosts in the network to the
firewall.
131

MSS L3 Operation
Offload rules for firewall resource optimization
To prevent overwhelming the firewall with traffic, the offload rules provide a
mechanism to filter traffic before it reaches the firewall. Users can tag policies
on the firewall for enforcement (permit and bypass firewall OR drop) and
reduce the amount of traffic redirected to the firewall.
These powerful rules accept masks and can use the full (or partial) five-tuple to
identify traffic. The offloaded rules are implemented in the form of DirectFlow
entries which consumes TCAM. Optimization of these rules and monitoring
TCAM utilization using advance telemetry features Arista provides in Cloud
Vision Portal is recommended. Hit counters per Direct Flow entry are
implemented to support the security administrator to detect malicious traffic
patterns
132

Firewall Policy Tagging
MSS L3 supports the following tags:
● Redirect: when applied to a policy, redirects all unicast IPv4 traffic from
the end-points (source/ destination IPs) in the policy to the FW.
● Redirect Verbatim: when applied to a policy, redirects just the matching
traffic (full/ partial five-tuple match) to the FW.
● Offload: when applied to a policy, enforces (permit | deny) policy at the
TOR for matching traffic based on the (full/ partial) five-tuple match. All
non-matching IP traffic from the end-points in the policy is redirected to
the FW.
● Offload Verbatim: when applied to a policy, enforces (permit | deny)
policy at the TOR for matching traffic.
133

MSS L3 with Palo Alto Network Firewalls
134

MLAG Pair
Physical Topology
LF4 LF5
SP1 SP2
Active FW Passive FW
VM Host 2 VM Host 3
webc-2
appc-2
dbc-2
webd-2
dbd-2
appd-2
webc-3
appc-3
dbc-3
webd-3
dbd-3
appd-3
HA Link
Tenant-C
Tenant-D
CVX-01 CVX-02
CVX-03
LF3LF2
VxLAN Overlay

136
LF4 LF5
PA-MGMT37-A
Active
PA-MGMT38-P
Passive
MLAG
Et1/15
Et1/16
Et1/13 Et1/14 Et1/13 Et1/14Et1/15
Et1/16
Et 7 Et 8 Et 5 Et 6 Et 5 Et 6 Et 7 Et 8
HA Peer Link(s)

Zone = Tennant-DZone = Tenant-C
IP addressing and VLAN Mappings
webc-2
100.64.104.0/24, VLAN 104
100.64.105.0/24, VLAN 105
100.64.103.0/24, VLAN 103
FireWall 100.64.204.0/24, VLAN 204
100.64.205.0/24, VLAN 205
100.64.203.0/24, VLAN 203
FireWall
Switch
Fabric
End
Users
ae1 ae2
.12
100.64.10.0/24
VLAN 10
100.64.20.0/24
VLAN 20
webc-2
.13
appc-2
.12
appc-3
.13
dbc-2
.12
dbc-3
.13
webd-2
.12
webd-2
.13
appd-2
.12
appd-3
.13
dbd-2
.12
dbd-3
.13

MSS for L3 Firewall Requirements
● VXLAN direct routing enabled on each TOR (Same as MSS L2 Fw)
● DirectFlow enabled on each TOR (Same as MSS for L2 FW)
● The firewall(s) be L2-adjacent to each of the TOR switches over a VXLAN
tunnel.
● In this reference design two additional VLANs (VLAN 10, VLAN 20) are
configured as service VLANs to provide L2 adjacencies to the firewalls
138

Active Firewall Zone Configuration
139
admin@PA-MGMT37-A(active)# show zone
zone {
Tenant-C {
network {
layer3 ae1;
}
}
Tenant-D {
network {
layer3 ae2;
}
}
}
[edit]
admin@PA-MGMT37-A(active)#

Active Firewall Interface Configuration
140

Active Firewall IP Address Object Configuration
141
Only the IP ranges assigned to specific hosts are defined.

Active Firewall Virtual Router Configuration
142
The firewall needs to have routes back to the original subnets where the
end hosts are. In this case specifically 100.64.103-105.0/24 and
100.64.203-205.0/24 subnets. Only static routes in “default” VRF are
supported in the current release. Note the firewall virtual router name must
be “default” which correspond to the default VRF that current release
supports. Tenant isolation is done through firewall zone configuration.

Active Firewall Virtual Router Definition
143

Active Firewall Static Route Configuration
144
Tenant-C
Tenant-D

Active Firewall Tag Configuration
145
● MSS redirect tag: The redirected tag, when applied to a policy, identified traffic
endpoints (using source and destination fields in the policy) from which traffic
will be redirected to the firewall for inspection.
● MSS offload tag: The offload tag, when applied to a policy, identifies a five-
tuple and the action (permit | deny) that is then enforced on the TOR switches.

Active Firewall Policy Configuration
146
● In this policy configuration Tenant-C traffic is intercepted and redirected to
the Firewall for security policy enforcement.
● Conversely Tenant-D traffic policy enforcement is offloaded from the firewall
and enforced at the TOR switches utilizing TCAM.

Active CVX Configuration
147
cvx
no shutdown
heartbeat-interval 30
heartbeat-timeout 90
peer host 10.90.164.162
peer host 10.90.164.161
source-interface Management1
!
service mss
no shutdown
vni range 20000-30000
!
dynamic device-set PANORAMA
device 10.92.59.101
username admin password 7 CF+X7x7GbctS7QTS+u8kaQ==
group Arista-MSS-Stack
state active
type palo-alto panorama
policy tag redirect MSS-redirect
policy tag offload MSS-offload
interval 10
retries 9

LF2/LF3 Configuration
148
interface Vxlan1
vxlan source-interface Loopback1
vxlan controller-client
vxlan udp-port 4789
vxlan vlan 10 vni 1010
!
Two Additional VLANs and VNIs to
provide L2 adjacency with the
firewalls

MSS L3 with Fortinet Firewalls
149

MLAG Pair
LF6 LF7 LF8 LF9
SP1
Active FW Passive FW
VM Host 1 VM Host 2
webe-2
appe-2
dbe-2
webf-2
dbf-2
appf-2
webe-3
appe-3
dbe-3
webf-3
dbf-3
appf-3
HA Link
Tenant-E
Tenant-F
CVX-04 CVX-05
CVX-06FW Mgr
Physical Topology
VxLAN Overlay
SP2

151
LF8 LF9
Fortigate
Active
Fortigate
Passive
MLAG
Et14
Et1/16
Et13 Et15 Et1/13 Et1/15Et14
Et16
Et 13 Et15 Et14 Et 16
Et14
Et16 Et13 Et 16
HA Peer Link(s)
e1 e1e2 e2
HA HA
e3
e4
e3
e4

Zone = Tennant-FZone = Tenant-E
IP addressing and VLAN Mappings
webe-2
100.64.107.0/24, VLAN 107
100.64.108.0/24, VLAN 108
100.64.106.0/24, VLAN 106
FireWall 100.64.207.0/24, VLAN 207
100.64.208.0/24, VLAN 208
100.64.206.0/24, VLAN 206
FireWall
Switch
Fabric
End
Users
ae1 ae2
.12
100.64.11.0/24
VLAN 11
100.64.21.0/24
VLAN 21
webe-2
.13
appe-2
.12
appe-3
.13
dbe-2
.12
dbe-3
.13
webf-2
.12
webf-2
.13
appf-2
.12
appf-3
.13
dbf-2
.12
dbf-3
.13

MSS for L3 Firewall Requirements
● VXLAN direct routing enabled on each TOR (Same as MSS L2 Fw)
● DirectFlow enabled on each TOR (Same as MSS for L2 FW)
● The firewall(s) be L2-adjacent to each of the TOR switches over a VXLAN
tunnel.
● In this reference design two additional VLANs (VLAN 11, VLAN 21) are
configured as service VLANs to provide L2 adjacencies to the firewalls
153

Active Firewall Interface Configuration
154

Active Firewall VDOM Definition
155

Active Firewall IP Address Object Configuration
156
Only the IP ranges assigned to specific hosts are defined.

Active Firewall Router Configuration
157
The firewall needs to have routes back to the original subnets where the
end hosts are. In this case specifically 100.64.106-108.0/24 and
100.64.206-208.0/24 subnets. Only static routes in non-root VDOM’s are
supported in the current release.

Firewall Manager Tag Configuration
158
● MSS redirect tag: The redirected tag, when applied to a policy, identified traffic
endpoints (using source and destination fields in the policy) from which traffic
will be redirected to the firewall for inspection.
● MSS offload tag: The offload tag, when applied to a policy, identifies a five-
tuple and the action (permit | deny) that is then enforced on the TOR switches.

Active Firewall Policy Configuration
159
● In this policy configuration Tenant-E traffic is intercepted and redirected to
the Firewall for security policy enforcement.
● Conversely Tenant-F traffic policy enforcement is offloaded from the firewall
and enforced at the TOR switches utilizing TCAM.

Active CVX Configuration
160
cvx
no shutdown
source-interface Management1
!
service mss
no shutdown
!
dynamic device-set fortinet
device member 10.90.164.145
!
device 10.90.164.220
username <username> password 7 <password>
group Demo
!
device member DEMO-FORT-A
!
device member MSS-Demo
state active
type fortinet fortimanager
policy tag redirect MSS-redirect
policy tag offload MSS-offload
admin domain Demo
virtual domain default

LF6/LF7 Configuration
161
interface Vxlan1
vxlan source-interface Loopback0
vxlan controller-client
vxlan udp-port 4789
!
Two Additional VLANs and VNIs to
provide L2 adjacency with the
firewalls

Thank You

Troubleshooting MSS

Troubleshooting
Ensure the MSS service is enabled
The MSS service should enabled on every CVX instance. To verify run the
following command on CVX:
164
CVX-02#show service mss status
State: Enabled
Service VNIs: 20000-30000
State: Enabled
State: Enabled

Troubleshooting
Ensure the dynamic device group is enabled
165
CVX-01#show service mss dynamic
Total policies processed: 286056
Policy Source Device Set Service Device State
------------------ ---------- ----------------------------------- ----------
palo-alto-panorama PANFW panorama-mss.sjc.aristanetworks.com active
CVX-02#show service mss dynamic
Total policies processed: 606
Policy Source Device Set Service Device State
------------------ ---------- ----------------------------------- ----------
palo-alto-panorama PANFW panorama-mss.sjc.aristanetworks.com active

Troubleshooting
The policy is not fetched from the firewall correctly
The following command will list all the policies retrieved from the firewall by
Arista MSS:
166
CVX-01#show service mss policy
-------------------------------------------------------------------
Source: PaloAltoPanorama
-------------------------------------------------------------------
Device: 001801053832
Policy: untrust_appa
Config: enabled
Status: initialized
Policy: untrust_appb
Config: enabled
Status: initialized
Policy: untrust_dba
Config: enabled
Status: initialized

Troubleshooting
If no policies are seen by Arista_MSS check if CVX is able to communicate the
firewall manager (Panorama) using the following command
167
CVX-01#show service mss dynamic status
Service Device Policy Monitoring Status:
Device: panorama-mss.sjc.aristanetworks.com
IP address: 10.92.59.101
Policy source type: PaloAltoPanorama
Aggregation Manager: True
Device group member(s):
001801053738
001801053832
Device set name: PANFW
Device set state: Active
Last seen at time: 2019 Mar 11, 14:45:03

Troubleshooting
TOR Switches cannot communicate with the CVX instances
168
CVX-01#show cvx connections
Switch 44:4c:a8:73:87:d1
Hostname: service-1
State: established
Connection timestamp: 6 days, 22:01:55 ago
Last heartbeat sent: 0:00:04 ago
Last heartbeat received: 0:00:14 ago
Out-of-band connection: Not secured
In-band connection: Not secured (SSL not supported)
Switch 44:4c:a8:73:86:a9
Hostname: intercept-1
State: established
Connection timestamp: 6 days, 22:01:55 ago
Last heartbeat sent: 0:00:04 ago
Last heartbeat received: 0:00:14 ago
Out-of-band connection: Not secured
In-band connection: Not secured (SSL not supported)

Troubleshooting
IP-MAC binding not learned by CVX
Check the status of the policy to ensure the CVX has the necessary
information to redirect traffic.
169
CVX-01# show service mss policy name untrust_weba
-------------------------------------------------------------------
Source: PaloAltoPanorama
-------------------------------------------------------------------
Device: 001801053832
Policy: untrust_weba
Config: enabled
Status: initialized
If the policy status is “pending” check the ARP table information received by CVX.

Troubleshooting
CVX-01#show service vxlan arp received
Received ARP Table
-------------------------------------------------------------------------
Switch VNI IP Address MAC Address Changes
------------------- -------- --------------- ------------------ ---------
44-4c-a8-73-87-d1 1000 100.64.100.2 44:4c:a8:c7:1c:d1 1
44-4c-a8-73-87-d1 1000 100.64.100.3 00:1c:73:90:09:a3 0
44-4c-a8-73-87-d1 1000 100.64.100.4 44:4c:a8:2f:a7:b1 0
170

Troubleshooting
If the IP address of the host is not seen in the CVX ARP table, ICMP ping a
host which is not on the same same subnet as the intercept host and verify the
ARP table information again. If ARP information for the host is learned by the
CVX after the ping, check the status of the policy and ensure it’s “initialized”.
If the situation still persists, run the following commands on the intercept
VTEP. If the host MAC address is learned on the VXLAN interface, this
indicates that there is a Layer-2 loop in the network. Resolve the loop and
verify the policy status again.
171
intercept-1#show arp
Address Age (min) Hardware Addr Interface
100.64.100.2 - 444c.a8c7.1cd1 Vlan100, Vxlan1

Troubleshooting
intercept-1# show mac address-table
Mac Address Table
------------------------------------------------------------------
Vlan Mac Address Type Ports Moves Last Move
---- ----------- ---- ----- ----- ---------
100 444c.a8c7.1cd1 DYNAMIC Vx1 6 1 day, 4:07:41 ago
172

Troubleshooting
Incorrect Service VNI and Port-VLAN membership
An incorrect service VNI and port-VLAN membership can host traffic of an
initialized policy to not be correctly intercepted.
To troubleshoot, first look at which VNI is used to tunnel traffic to the Service
VTEP. This information can be obtained by running the following command
and looking at the value of the service VNI.
173
CVX-01#show service mss internal policy advertised

Troubleshooting
On the intercept an service VTEPs, check the VLAN to VNI mapping and VTEP
floodlist for the service VNI:
174
intercept-3#show interfaces vxlan 1
Vxlan1 is up, line protocol is up (connected)
Hardware is Vxlan
Source interface is Loopback0 and is active with 2.2.2.1
Replication/Flood Mode is headend with Flood List Source: VCS
Remote MAC learning via VCS
VNI mapping to VLANs
Static VLAN to VNI mapping is
[100, 1000] [101, 1001] [102, 1002] [200, 2000]
[201, 2001] [202, 2002]
Dynamic VLAN to VNI mapping for 'mss' is
[4056, 20004] [4057, 20003] [4059, 20005] [4062, 20002]
[4063, 20000] [4087, 20001]
Note: All Dynamic VLANs used by VCS are internal VLANs.
Use 'show vxlan vni' for details.
Static VRF to VNI mapping is not configured
Headend replication flood vtep list is:
100 1.1.1.1 2.2.2.1 3.3.3.1

Troubleshooting
Next, check the port VLAN membership on intercept VTEP. On the intercept
VTEP, the intercept interface should be present in the dynamically configured
service VLAN (denoted by a*). In this example, interface Et50/1.
175
intercept-3#show vlan 4056
VLAN Name Status Ports
----- -------------------------------- --------- -------------------------------
4056* VLAN4056 active Et50/1, Vx1
* indicates a Dynamic VLAN
intercept-3#

Troubleshooting
On the service VTEP, the firewall egress interface should be a member of the
service VLAN and the intercept interface needs to be a member of the original
VLAN.
176
service-1#show vlan
VLAN Name Status Ports
----- -------------------------------- --------- -------------------------------
1 default active PEt25
100 VLAN0100 active Cpu, Po1000, Po1001, Vx1
4082* VLAN4082 active Po1000, Po1002, Vx1
4093 LEAF_PEER_L3 active Cpu, Po1000
4094 MLAG_PEER active Po1000
Po1002 is the egress
interface towards the firewall
Po1001 is the ingress
interface from the firewall

Required DirectFlow rules are missing
The VTEP needs few DirectFlow rules to facilitate the packet flow. The
command “show directflow detail” help determine if rules are missing.
177
intercept-1#show directflow detail
Flow panorama:001801053832_N:Et8_7387d1+Et6_73865f_F:Et7_7387d1+Et5_73865f_V:102-1002_InsideVtepIngress-
100.64.102.3-from-Po1001-to-any: (Flow programmed)
persistent: False
priority: 0
priorityGroupType: default
hard timeout: 0
idle timeout: 0
match:
ingress interface:
Po1001
source Ethernet address: 00:1c:73:90:09:a3/ff:ff:ff:ff:ff:ff
VLAN ID: 102
actions:
set VLAN ID to: 4086
forward normally
source: mss

Troubleshooting
● The DirectFlow rules seen at the intercept VTEP can be divided in two
broad categories, ingress rules and egress rules. Ingress rules can again
be divided in to two types, host specific ingress rules and service VLAN
specific rules.
● There at most two specific ingress rules that forward the packet to service
VLAN. One rule transfers any traffic from the host coming with a VLAN tag
from the original VLAN to the service VLAN. In addition to the original
VLAN, this rule also matches host MAC address and the switch intercept
interface. This rule is always present for intercepted host host learned on
a particular VTEP.
178

Troubleshooting
● Another class of ingress rule can be observed for all host traffic that can
be received by a VTEP without a VLAN tag. As MSS supports only trunk
intercept interfaces , this type of traffic is seen only on the original VLAN
that matches the native VLAN of the intercept interface. Note that if no
intercept host is learned in the native VLAN or native VLAN is not
configured , the intercept VTEP does not have this rule.
● The service VLAN specific ingress rule allows ARP traffic on the service
VLAN to be forwarded towards the firewall. The name of this rule ends
with ARP and mentions only the service VLAN in it’s match criteria.
179

Troubleshooting
● On the egress side, there is a set of rules on every intercept interface. This
rule matches the service VLAN and translate the packet back to the original
VLAN before sending it out on the intercept interface. Note that when multiple
hosts on the same original VLAN are learned on the same intercept interface,
only one rule is sufficient to support the necessary egress translation; the rule
does not have any match criteria on the host MAC address.
● The DirectFlow rules on the service VTEP are more generic and do not match
any host MAC address, however they can also be divided in to egress and
ingress rules
● On the ingress side there is a DirectFlow rule that matches any packet
received at the Near interface on original VLAN and translate it to service
VLAN. Traffic egressing from the service VTEP towards the firewall hits an
egress rule and the packet is translated from the service VLAN to the original
VLAN.
180

Troubleshooting
service-1#show directflow detail
Flow panorama:001801053832_N:Et8_7387d1+Et6_73865f_F:Et7_7387d1+Et5_73865f_V:102-
1002_ServiceVtepIngress-from-Po1002-to-any: (Flow programmed)
persistent: False
priority: 0
hard timeout: 0
idle timeout: 0
match:
ingress interface:
Po1002
VLAN ID: 102
actions:
set VLAN ID to: 4086
forward normally
source: mss
181

Policy Status
CVX
The following CVX commands will provide the visibility to the running states of
MSS L3
• show service policy status presents a summary against each L3 policy
fetched from the firewall. This command is useful in figuring out if
necessary DirectFlow rules are configured on all VTEPs. Moreover, the
output also shows the status for all intercepted hosts. In addition, as
implicit redirect is enabled with MSS L3, each offload policy has an
associated Redirect status.
182

Policy Status
183
CVX-02#show service mss policy
Macro-Segmentation L2 Policy Table
-------------------------------------------------------------------------------
Source Device Policy Configured State Operational State
------------ ------------ ------------ ---------------------- -----------------
Macro-Segmentation L3 Policy Table
-------------------------------------------------------------------------------
Source Device Policy Offload Redirect Unconverged
status status IPs
---------------------- ----------------------------------------------- --------------- ------------- -------------- -----------
PaloAltoFirewall PA-MGMT37-A.sjc.aristanetworks.com_HAPair appc2dbc N/A Active 6 of 12
PaloAltoFirewall PA-MGMT37-A.sjc.aristanetworks.com_HAPair appd2dbd Active Active 6 of 12
PaloAltoFirewall PA-MGMT37-A.sjc.aristanetworks.com_HAPair webc2apc N/A Active 3 of 9
PaloAltoFirewall PA-MGMT37-A.sjc.aristanetworks.com_HAPair webc2dbc N/A Active 3 of 9
PaloAltoFirewall PA-MGMT37-A.sjc.aristanetworks.com_HAPair webd2appd Active Active 3 of 9
PaloAltoFirewall PA-MGMT37-A.sjc.aristanetworks.com_HAPair webd2dbd Active Active 3 of 9

Policy Status
184
CVX-02#show service mss policy detail
-------------------------------------------------------------------
Source: PaloAltoFirewall
-------------------------------------------------------------------
Device: PA-MGMT37-A.sjc.aristanetworks.com_HAPair
Policy (L3): appc2dbc
Offload Status: N/A
Redirect Status: Active
Tags: MSS-redirect
VRF: default
IP Addresses:
Active: 100.64.104.2
Active: 100.64.104.3
Active: 100.64.104.4
Active: 100.64.105.2
Active: 100.64.105.3
The output of show service mss policy status detail can be used to
troubleshoot issues against each policy

Policy Status
Programming of Flows
show directflow detail displays if an intercepted host is correctly
programmed at the hardware.
185
intercept-4#show directflow detail
Flow bypassPeerlink: (Flow programmed)
persistent: True
priority: 65535
tableType: ifp
hard timeout: 0
idle timeout: 0
match:
ingress interface:
Et102
Ethernet type: IPv4
actions:
forward normally
source: config
matched: 0 packets, 0 bytes

Policy Hit Counter
● If the security policies are tagged as “MSS-offload” at the Firewall, the
security administrator will not see the policy hit counter getting
incremented on the Firewall UI.
● To this end Arista provides a way to view the directflow counters at the
TOR switch for security administrator to monitor security policy
enforcement.
186
intercept-6#show directflow counters | nz
Flow Name Source Matched packets Matched bytes
--------- ------ --------------- -------------
default:spm:PA-MGMT37-A.sjc.aristanetworks.com_HAPair:30000::100.64.203.4/32::100.64.204.2/32::::drop
mssl3 10 1220
default:spm:PA-MGMT37-A.sjc.aristanetworks.com_HAPair:10000::100.64.103.0/24:MssL3-intercept-host-
group:::::nh-100.64.10.254 mssl3 25 3050
Total matched packets: 35

End of Presentation

Mss solution guide

More Related Content

What's hot

Similar to Mss solution guide

Recently uploaded

Mss solution guide