Sdn aci for cisco private cloud building onprem.pdf

-
#CiscoLiveAPJC
Steve Sharman – Solutions Engineer
BRKDCN-2984
BRKDCN-2984
ACI – The Foundation of an Internal
Private Cloud
(aka “not just another network…”)

-
#CiscoLiveAPJC © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco Webex App
Questions?
Use Cisco Webex App to chat
with the speaker after the session
Find this session in the Cisco Live Mobile App
Click “Join the Discussion”
Install the Webex App or go directly to the Webex space
Enter messages/questions in the Webex space
How
Webex spaces will be moderated
by the speaker until November 15, 2024.
1
2
3
4
https://ciscolive.ciscoevents.com/
ciscolivebot/#BRKDCN-2984
5
© 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public
BRKDCN-2984

-
At little bit of background to this session…
BRKDCN-2984 6

-
Agenda
• Setting the scene
• Converting your fabric into Application Centric “mode”
• Working with ESGs
• Understanding ACI Security
• External Connectivity
• Increasing Security
• Automation Blueprints
BRKDCN-2984 8

-
Icons
App Profile
AP
Subnets
Subnets
Bridge Domain
BD
Tenant
VRF
Contract
Cont
Subject
Subj
Filter
Filt
Path
Path
L3out
L3out
EPG
P
C I
CCI
EPG
P
C I
CCI
External EPG
extEPG
P
C I
CCI
P
C I
CCI
ESG
P
C I
CCI
ESG
P
C I
CCI
ESG
ESG
EPG
EPG
Entry
Entry
*arrows indicate expected direction of connection i.e. from consumer to provider
BRKDCN-2984 10

-
You are going to see lots (and lots) of diagrams…
192.168.150.0_24
BD 192.168.151.0_24
BD 192.168.152.0_24
BD
Network-
segments
AP
192.168.150.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.151.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.152.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
The details are there for your
reference so that you can rebuild
in your own environment
192.168.150.0_24
BD
192.168.150.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.151.0_24
BD
192.168.151.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.152.0_24
BD
192.168.152.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
Network-
segments
AP
BRKDCN-2984 11

-
Setting the scene…
Designing your Tenants…
Switching to Application Centric mode…
Working with ESGs…
Understanding ACI security…
External Connectivity…
Increasing Security…
Automation Blueprints…
Wrapping up…

-
Public Cloud
infrastructure…

-
AWS reference architecture
https://docs.aws.amazon.com/vpc/latest/userguide/extend-intro.html
Customer Site
AWS Backbone
eu-west-1
eu-west-1-production
NAT gateway Route table
Public subnet Private subnet
Route table
Internet gateway eu-west-2
eu-west-2-production
NAT gateway Route table
Public subnet Private subnet
Route table
Internet gateway
Transit
Gateway
Customer Site
BRKDCN-2984 14

-
Network Connectivity and Security are mandatory in
the cloud…
BRKDCN-2984 15

-
Different clouds run different hypervisors
BRKDCN-2984 16

-
Executive
Sponsorship
New Culture
Evolution Instead
of Revolution
New Talent
Attraction
Partnerships 2.0
Scaling
Think Agile
Cross Functional
Teams
BRKDCN-2984 17

-
A cloud operating model succeeds best when there is a
new organizational culture…
18
BRKDCN-2984

-
Cloud operating models have changed the way that
security is implemented…
BRKDCN-2984 19

-
With a cloud operating model, security rules are typically
declared with the application constructs…
BRKDCN-2984 20

-
Conversely, within enterprise Data Centers security has
been implemented by network and/or security
administrators at a VRF boundary…
BRKDCN-2984 21

-
Traditional Enterprise Security Model
Traffic is routed through a firewall which
typically becomes a pinch point with
thousands of rules
Inside
Outside
ubuntu-01 ubuntu-02
permit ubuntu-01 ubuntu-02 tcp 5201
BRKDCN-2984 22

-
What are the network characteristics required to enable
us to operate in a cloud like manner…?
BRKDCN-2984 23

-
ACI is the foundation for an internal private cloud…!
Per-application
service-chaining
Hybrid cloud capability;
public cloud-like networking
constructs
Single API Model for 100s of
switches and 1000s of ports;
cloud-like consumption
model
Day0 automation out-of-the-
box; physical fabric and
underlay
Infrastructure as Code with
Ansible and Terraform
Pervasive
Security Model
Automation | Classification and Segmentation | Security
BRKDCN-2984 24

-
The ACI reference application from circa 2014…
BRKDCN-2984 26

-
The mythical three tier application…!
ACI Fabric
App DB
Web
Outside
(Tenant VRF)
QoS
Filter
QoS
Service
QoS
Filter
Application Policy
Infrastructure
Controller
APIC
BRKDCN-2984 27

-
Our reference application for this presentation…
BRKDCN-2984 28

-
Online Boutique
https://github.com/GoogleCloudPlatform/microservices-demo
BRKDCN-2984 29

-
Online Boutique
Source/Consumer Target/Provider Target/Provider
Port
cart Redis cache TCP 6379
checkout cart
currency
email
payment
product catalog
shipping
TCP 7070
TCP 7000
TCP 8080
TCP 50051
TCP 3550
TCP 50051
frontend adservice
cart
checkout
currency
product catalog
recommendation
shipping
TCP 9555
TCP 7070
TCP 5050
TCP 7000
TCP 3550
TCP 8080
TCP 50051
outside frontend TCP 80/8080
recommendation product catalog TCP 3550
frontend checkout
email
shipping currency
cart
product catalog
recommendation
adservice
Redis cache
payment
BRKDCN-2984 30

BRKDCN-2984 38

-
Design Considerations…
BRKDCN-2984 39

-
Design Patterns
common
common.vrf-01
demo
subnet(s)
BD
VLAN
(Security isolation per
Bridge Domain)
EPG
subnet(s)
BD
VLAN
Bridge Domain)
EPG
Network
Segments
AP
Apps
(Optional)
AP
Security isolation across Bridge Domains
ESG
VLAN
Bridge Domain)
EPG
VLAN
Bridge Domain)
EPG
Dedicated subnets for tenants with VRFs
that can be (optionally) shared by different
Tenants
subnet(s)
BD
common
common.vrf-01
subnet(s)
BD
demo
VLAN
Bridge Domain)
EPG
VLAN
Bridge Domain)
EPG
Network
Segments
AP
Apps
(Optional)
AP
ESG
VLAN
Bridge Domain)
EPG
VLAN
Bridge Domain)
EPG
Typically, fewer larger subnets which can
be (optionally) shared across Tenants
subnet(s)
BD
common
common.vrf-01
VLAN
Bridge Domain)
EPG
subnet(s)
BD
VLAN
Bridge Domain)
EPG
Network
Segments
AP
Apps
(Optional)
AP
ESG
VLAN
Bridge Domain)
EPG
VLAN
Bridge Domain)
EPG
Used for functions which are accessible
from any Tenant
Everything in the “common”
Tenant is not typically seen
VRFs and BDs in “common”
with EPGs and ESGs in the
“user” tenant
VRFs in “common” with
BDs, EPGs and ESGs in
the “user” tenant
Objects in the common tenant
should have unique names,
e.g. common.vrf-01
BRKDCN-2984 40

-
Design Patterns
subnet(s)
BD
demo
vrf-01
VLAN
Bridge Domain)
EPG
subnet(s)
BD
VLAN
Bridge Domain)
EPG
Network
Segments
AP
Apps
(Optional)
AP
ESG
VLAN
Bridge Domain)
EPG
VLAN
Bridge Domain)
EPG
Dedicated VRFs and subnets for each
Tenant with Dedicated L3outs
shared-services
vrf-01
subnet(s)
BD
demo
vrf-01
VLAN
Bridge Domain)
EPG
subnet(s)
BD
VLAN
Bridge Domain)
EPG
Network
Segments
AP
Apps
(Optional)
AP
ESG
VLAN
Bridge Domain)
EPG
VLAN
Bridge Domain)
EPG
Tenant with Shared L3out
demo
VLAN
Bridge Domain)
EPG
Network
Segments
AP
Apps
(Optional)
AP
Security isolation
across Bridge Domains
ESG
VLAN
Bridge Domain)
EPG
test
VLAN
Bridge Domain)
EPG
Network
Segments
AP
Apps
(Optional)
AP
Security isolation
across Bridge Domains
ESG
VLAN
Bridge Domain)
EPG
subnet(s)
BD
common
common.vrf-01
shared-services
vrf-01
EPG and ESG in the “user” Tenant with the
VRF in the “common” Tenant, and a Shared
L3out in shared-services
Each Tenant has one or more
network security groups
Network team controls
inbound/outbound routing
Each Tenant has one or more
endpoint security groups
Large subnets can be
shared across Tenants
All networking constructs
contained within a Tenant
BRKDCN-2984 41

-
Each Tenant has their own IP Range
IP range per Tenant
BRKDCN-2984 42

-
Network engineers “view” of their ACI environment…
BRKDCN-2984 43

-
Workloads identified by IP and Mac address
192.168.150.0_24
BD
192.168.150.0_24
EPG
192.168.151.0_24
BD
192.168.151.0_24
EPG
192.168.152.0_24
BD
192.168.152.0_24
EPG
192.168.153.0_24
BD
192.168.153.0_24
EPG
192.168.154.0_24
BD
192.168.154.0_24
EPG
192.168.155.0_24
BD
192.168.155.0_24
EPG
192.168.156.0_24
BD
192.168.156.0_24
EPG
AP
demo
vrf-01
network-segments
Typical “Network Centric” mode
deployment where there is a 1:1
mapping between Bridge
Domains and EPGs
The “network-segments”
Application Profile contains all
the EPGs which provide the
network backing
BRKDCN-2984 44

-
What does the application owner care about…?
BRKDCN-2984 45

-
DNS names, IP addresses, Default Gateways, and
Security Rules…
BRKDCN-2984 46

-
Online Boutique
BRKDCN-2984 47

-
Online Boutique
Source/Consumer Target/Provider Target/Provider
Port
cart Redis cache TCP 6379
checkout cart
currency
email
payment
product catalog
shipping
TCP 7070
TCP 7000
TCP 8080
TCP 50051
TCP 3550
TCP 50051
frontend adservice
cart
checkout
currency
product catalog
recommendation
shipping
TCP 9555
TCP 7070
TCP 5050
TCP 7000
TCP 3550
TCP 8080
TCP 50051
outside frontend TCP 80/8080
recommendation product catalog TCP 3550
frontend checkout
email
shipping currency
cart
product catalog
recommendation
adservice
Redis cache
payment
BRKDCN-2984 48

-
Where is our application running…?
192.168.150.0_24
BD 192.168.151.0_24
BD 192.168.152.0_24
BD 192.168.153.0_24
BD 192.168.154.0_24
BD 192.168.155.0_24
BD 192.168.156.0_24
BD
demo
vrf-01
192.168.150.0_24
EPG 192.168.151.0_24
EPG 192.168.152.0_24
EPG 192.168.153.0_24
EPG 192.168.154.0_24
EPG 192.168.155.0_24
EPG 192.168.156.0_24
EPG
AP
frontend
checkout
email
cart
product catalog
adservice
Redis cache
currency
recommendation
payment
shipping
The application endpoints require
communication across different
subnets, which is typically achieved
using “vzAny” or “Preferred Groups”
network-segments
BRKDCN-2984 49

-
Let’s convert to “Application Centric” mode…
192.168.150.0_24
BD 192.168.151.0_24
BD 192.168.152.0_24
BD 192.168.153.0_24
BD 192.168.154.0_24
BD 192.168.155.0_24
BD 192.168.156.0_24
BD
demo
vrf-01
192.168.150.0_24
EPG 192.168.151.0_24
EPG 192.168.152.0_24
EPG 192.168.153.0_24
EPG 192.168.154.0_24
EPG 192.168.155.0_24
EPG 192.168.156.0_24
EPG
AP
frontend
checkout
email
cart
product catalog
adservice
Redis cache
currency
recommendation
payment
shipping
network-segments
AP
AP
online-boutique
all-services
ESG
The application endpoints communicate
openly within the Endpoint Security Group
even though they’re connected to different
Bridge Domains
New Application Profile created
for the application Endpoint
Security Group
BRKDCN-2984 50

-
What does this mean to the network admin…?
BRKDCN-2984 51

-
Application Visibility…!
Application Endpoint
IP and MAC Information
Endpoint names
Host Information
Switch and Interface
Information
VLAN Information
EPG/Subnet Information
Tag Information
BRKDCN-2984 52

-
Correlate Endpoints to Switch Interfaces…
BRKDCN-2984 53

-
What if don’t want my classification to be this granular…?
BRKDCN-2984 54

-
Broad-brush classification…
192.168.150.0_24
BD 192.168.151.0_24
BD 192.168.152.0_24
BD 192.168.153.0_24
BD 192.168.154.0_24
BD 192.168.155.0_24
BD 192.168.156.0_24
BD
demo
vrf-01
192.168.150.0_24
EPG 192.168.151.0_24
EPG 192.168.152.0_24
EPG 192.168.153.0_24
EPG 192.168.154.0_24
EPG 192.168.155.0_24
EPG 192.168.156.0_24
EPG
network-segments
AP
AP
production
production-workloads
ESG
AP
pre-production
Production workloads
pre-production-workloads
ESG
Pre-Production
workloads
BRKDCN-2984 55

-
What if I’d like to gather data on a specific group of
endpoints…?
BRKDCN-2984 56

-
Endpoint classification for monitoring…
192.168.150.0_24
BD 192.168.151.0_24
BD 192.168.152.0_24
BD 192.168.153.0_24
BD 192.168.154.0_24
BD 192.168.155.0_24
BD 192.168.156.0_24
BD
demo
vrf-01
192.168.150.0_24
EPG 192.168.151.0_24
EPG 192.168.152.0_24
EPG 192.168.153.0_24
EPG 192.168.154.0_24
EPG 192.168.155.0_24
EPG 192.168.156.0_24
EPG
network-segments
AP
AP
online-boutique
all-services
ESG
BRKDCN-2984 57

-
Endpoint classification for monitoring…
192.168.150.0_24
BD 192.168.151.0_24
BD 192.168.152.0_24
BD 192.168.153.0_24
BD 192.168.154.0_24
BD 192.168.155.0_24
BD 192.168.156.0_24
BD
demo
vrf-01
192.168.150.0_24
EPG 192.168.151.0_24
EPG 192.168.152.0_24
EPG 192.168.153.0_24
EPG 192.168.154.0_24
EPG 192.168.155.0_24
EPG 192.168.156.0_24
EPG
network-segments
AP
AP
online-boutique
all-services
ESG
monitor
ESG
Monitoring Group with
Intra ESG Contract
BRKDCN-2984 58

-
Setting the scene…
Designing your Tenants…
Switching to Application
Centric mode…
Working with ESGs…
Understanding ACI security…
External Connectivity…
Increasing Security…
Automation Blueprints…
Wrapping up…

-
All we need are the application names and the
associated IP addresses…!
BRKDCN-2984 60

-
Application Knowledge taken from any source
vCenter
Tags/Names
Orchestration
APIC
Application name +
endpoint IP addresses
Application name +
VM Names or VM Tags
Tag Selectors
• Endpoint MAC
• Endpoint IP
• BD subnet
• Static endpoint
• VM name
• VM Tag
IP subnet selector
EPG selector
Application
Knowledge
CMDB
e.g. SNOW
DNS
Application Monitoring
e.g. AppDynamics
Application Security
e.g. Secure Workload
BRKDCN-2984 61

-
Physical or virtual workloads, with or without VMM
Integration…!
BRKDCN-2984 62

-
You can convert to Application Centric mode in two
simple steps…
BRKDCN-2984 63

-
vrf-01
epg-matched-esg
AP
Step 1: Create Application Profiles and Security
Groups
demo
192.168.150.0_24
BD 192.168.151.0_24
BD 192.168.152.0_24
BD
192.168.150.0_24
dynamic (P,S) vlans
EPG
192.168.151.0_24
dynamic (P,S) vlans
EPG
192.168.152.0_24
dynamic (P,S) vlans
EPG
all-services
P
C I
CCI
ESG
network-segments
P
C I
CCI
ESG
network-
segments
AP
online-boutique
AP
ESXi cluster with VMM integration
VDS portgroup name
192.168.150.0_24
Dynamic PVLAN (P,S)
VDS PG
VDS portgroup name
192.168.151.0_24
Dynamic PVLAN (P,S)
VDS PG
VDS portgroup name
192.168.152.0_24
Dynamic PVLAN (P,S)
VDS PG
network-segments
ESG
Domains and EPGs
Contracts applied to vzAny
implicitly applies to all EPGs,
ESGs, and extEPGs in the VRF
Application Profile for EPG
mapped Endpoint Security
Groups
New Application Profile and
Endpoint Security Group for the
“online-boutique” application
Contract allowing open or
restricted communication
Open communication between
all subnets through the
“network-segments” ESG
EPG/ESG Collection
P
C CCI
vzAny
BRKDCN-2984 64

-
demo
vrf-01
192.168.150.0_24
BD
192.168.150.0_24
dynamic (P,S) vlans
EPG
192.168.151.0_24
BD
192.168.151.0_24
dynamic (P,S) vlans
EPG
192.168.152.0_24
BD
192.168.152.0_24
dynamic (P,S) vlans
EPG online-boutique
AP
Network-
segments
AP
VDS portgroup name
192.168.150.0_24
Dynamic PVLAN (P,S)
VDS PG
VDS portgroup name
192.168.151.0_24
Dynamic PVLAN (P,S)
VDS PG
VDS portgroup name
192.168.152.0_24
Dynamic PVLAN (P,S)
VDS PG
ESG
ESG
New Application Profile and
Endpoint Security Group for the
“online-boutique” application
network-segments
online-boutique
network-segments
ESG
all subnets through the
“network-segments” ESG
ESG all-services
P
C I
CCI
ESG
Step 2: Tag Workloads to move into the new
Security Group
online-boutique endpoints
Domains and EPGs
epg-matched-esg
AP
network-segments
P
C I
CCI
ESG
EPG/ESG Collection
P
C CCI
vzAny
Contract allowing open or
restricted communication
Application Profile for EPG
mapped Endpoint Security
Groups
Contracts applied to vzAny
implicitly applies to all EPGs,
ESGs, and extEPGs in the VRF
BRKDCN-2984 65

-
Tagging Option 1: Static Tag Mapping (manual/automated)
ACI Application
Workload Tags
Match Endpoints to
Workload Tags
Define ESG Tag Selector
ApplicationName = online-boutique
Map MAC or IP address to Tag Value
00:50:56:A1:0A:90 = ApplicationName online-boutique
BRKDCN-2984 66

-
Automated conversion to “Application Centric”
BRKDCN-2984 67

-
Tagging Option 2: VMM Tag Mapping
vCenter Application
Workload Tags
ACI Application
Workload Tags
ACI Application Workload Tags match
vCenter Application Workload Tags
BRKDCN-2984 68

-
Automated conversion to “Application Centric”
BRKDCN-2984 69

-
Scaling application connectivity with vzAny…
BRKDCN-2984 70

-
Scaling connectivity to “application-01”
demo
vrf-01
192.168.150.0_24
BD 192.168.151.0_24
BD 192.168.152.0_24
BD
ESG all-services
P
C I
CCI
ESG
application-01
AP
Network-
segments
AP
epg-matched-esg
AP
network-segments
P
C I
CCI
ESG
ESG all-services
P
C I
CCI
ESG
application-02
AP
All EPGs, ESGs, extEPGs
vzAny
P
C CCI
P
C CCI
permit-to-all-applications
Cont
ESG
application-03
AP
all-services
ESG
P
C I
CCI
vzAny as a contract consumer defines
that all EPGs, ESGs, extEPGs are
consumers of the same contract
All applications initially provide the same
contract to vzAny. This maintains open
communication between applications
0.0.0.0/1
128.0.0.0/1
P
C I
CCI
extEPG
192.168.150.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.151.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.152.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
L3Out
BRKDCN-2984 71

-
demo
vrf-01
192.168.150.0_24
BD 192.168.151.0_24
BD 192.168.152.0_24
BD
ESG all-services
P
C I
CCI
ESG
application-01
AP
Network-
segments
AP
epg-matched-esg
AP
network-segments
P
C I
CCI
ESG
ESG all-services
P
C I
CCI
ESG
application-02
AP
vzAny
P
C CCI
P
C CCI
Cont
ESG
application-03
AP
all-services
ESG
P
C I
CCI
0.0.0.0/1
128.0.0.0/1
P
C I
CCI
extEPG
192.168.150.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.151.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.152.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
L3Out
BRKDCN-2984 72

-
epg-matched-esg
AP
network-segments
P
C I
CCI
ESG
demo
vrf-01
192.168.150.0_24
BD 192.168.151.0_24
BD 192.168.152.0_24
BD
ESG all-services
P
C I
CCI
ESG
application-01
AP
Network-
segments
AP
ESG all-services
P
C I
CCI
ESG
application-02
AP
vzAny
P
C CCI
P
C CCI
Cont
ESG
application-03
AP
all-services
ESG
P
C I
CCI
0.0.0.0/1
128.0.0.0/1
P
C I
CCI
extEPG
192.168.150.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.151.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.152.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
L3Out
BRKDCN-2984 73

-
Why are ESGs a better classification option…?
BRKDCN-2984 75

-
192.168.2.1/24
BD 192.168.3.1/24
192.168.4.1/24 sec
BD
demo
vrf-01
ESG
vmm domain
dynamic vlan allocation
EPG
Static Path 103/1/1 – vlan-30
Static Path 103/1/2 - vlan-40
EPG
192.168.1.1/24
BD
EPG
EPG
vmm domain
EPG
vmm domain
EPG
network-
segments
AP
security-
groups
AP
ESGs provides security
across the VRF
Bridge Domain with
multiple subnets and
multiple EPGs/vlans
ACI foundational building blocks:
• A Tenant provides an RBAC boundary typically linked to a
business function
• A VRF is mapped to a single Tenant
• A Bridge Domain is mapped to a single VRF
• A Bridge Domain provides one or more IP gateways (IP
secondary)
• An EPG is mapped to a single Bridge Domain
• An EPG provides network backing and maps to:
• VMM domains + static or dynamic VLAN(s)
• Static path(s) + static VLAN(s)
• An EPG defines a security boundary on a Bridge Domain
• An EPG allows open communication for endpoints in the EPG, or
(optionally) blocked communication for endpoints in the EPG
• Inter EPG communication requires contracts (typically not required
when using ESGs)
• An ESG forms a security boundary on a VRF
• An ESG allows open communication for endpoints in the ESG, or
(optionally) blocked communication for endpoints in the ESG
• Inter ESG communication requires contracts
• ESG contracts supersede EPG contracts
Bridge Domain with 1x
subnet and 1x EPG/vlan
Bridge Domain with 1x
subnet and multiple
EPGs/vlans
EPG provides security
across a BD
EPG Security vs ESG Security
BRKDCN-2984 76

-
What are our endpoint mapping options…?
BRKDCN-2984 77

-
We can use EPGs, Tagged endpoints, Tagged
subnets, or simply Static endpoint mapping…
BRKDCN-2984 78

-
vrf-01
Option 1: EPG mapping to a single security zone
79
BRKDCN-2984
demo
epg-matched-esg
AP
all-subnets
EPG: 192.168.150.0_24
EPG: 192.168.151.0_24
EPG: 192.168.152.0_24
ESG
192.168.152.0_24
BD
192.168.152.0_24
VMM Domain
Dynamic PVLAN
EPG
192.168.150.0_24
BD
192.168.150.0_24
VMM Domain
Dynamic PVLAN
EPG
192.168.151.0_24
BD
192.168.151.0_24
VMM Domain
Dynamic PVLAN
EPG
network-segments
AP
Settings:
- VMM Domain (read/write)
- Allow uSegmentation = True
- Dynamic PVLANs
EPG
VDS portgroup name
192.168.150.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.151.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.152.0_24
PVLAN (P, S)
VDS PG
all-subnets
ESG
Logical grouping by
EPGs
All EPGs mapped to
a single ESG

-
vrf-01
Option 2: EPG mapping for multiple security zones
80
BRKDCN-2984
demo
VDS portgroup name
192.168.150.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.151.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.152.0_24
PVLAN (P, S)
VDS PG
production
ESG pre-production
ESG
epg-matched-esg
AP
permit-to-pre-production
Cont
production
EPG: 192.168.150.0_24
EPG: 192.168.151.0_24
ESG
P
C I
CCI
pre-production
EPG: 192.168.152.0_24
ESG
P
C I
CCI
192.168.152.0_24
BD
192.168.152.0_24
VMM Domain
Dynamic PVLAN
EPG
192.168.150.0_24
BD
192.168.150.0_24
VMM Domain
Dynamic PVLAN
EPG
192.168.151.0_24
BD
192.168.151.0_24
VMM Domain
Dynamic PVLAN
EPG
network-segments
AP
Logical grouping by
EPGs
Logical grouping by
EPGs
Settings:
- Dynamic PVLANs
EPG
EPG group-01 EPG group-02

-
vrf-01
Option 3: Tag selectors with VMM integration
81
BRKDCN-2984
demo
applications
AP
application-01
ESG
application-02
ESG
P
C I
CCI
P
C I
CCI
permit-to-application-02
Cont
VMs matched with tag
Key: app
Value: application-01
Key: app
192.168.152.0_24
BD
192.168.152.0_24
VMM Domain
Dynamic PVLAN
EPG
192.168.150.0_24
BD
192.168.150.0_24
VMM Domain
Dynamic PVLAN
EPG
192.168.151.0_24
BD
192.168.151.0_24
VMM Domain
Dynamic PVLAN
EPG
network-segments
AP
Settings:
- Dynamic PVLANs
EPG
VDS portgroup name
192.168.150.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.151.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.152.0_24
PVLAN (P, S)
VDS PG
application-01
ESG
application-02
ESG
VM vCenter Tag = APIC Policy Tag
Key: app
Key: app
Logical grouping by
VM Tag

-
vrf-01
Option 4: Tag selectors with VMM integration and
Intermediary switches
82
BRKDCN-2984
demo
applications
AP
application-01
ESG
application-02
ESG
P
C I
CCI
P
C I
CCI
Cont
Key: app
Key: app
192.168.152.0_24
BD
192.168.152.0_24
VMM Domain
Static PVLAN
EPG
192.168.150.0_24
BD
192.168.150.0_24
VMM Domain
Static PVLAN
EPG
192.168.151.0_24
BD
192.168.151.0_24
VMM Domain
Static PVLAN
EPG
network-segments
AP
Static PVLAN
Static PVLAN on
intermediary switches
Settings:
- Manual/static PVLANs
EPG
VDS portgroup name
192.168.150.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.151.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.152.0_24
PVLAN (P, S)
VDS PG
application-01
ESG
application-02
ESG
Key: app
Key: app
Logical grouping by
VM Tag

-
vrf-01
Option 5: MAC selectors, no VMM integration
83
BRKDCN-2984
demo
ESXi cluster without VMM integration
VDS portgroup name
192.168.150.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.151.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.152.0_24
PVLAN (P, S)
VDS PG
applications
AP
application-01
ESG
application-02
ESG
P
C I
CCI
P
C I
CCI
Cont
Tag Selector
Key: app
Tag Selector
Key: app
192.168.152.0_24
BD
192.168.152.0_24
Phys Domain
Manual PVLAN
EPG
192.168.150.0_24
BD
192.168.150.0_24
Phys Domain
Manual PVLAN
EPG
192.168.151.0_24
BD
192.168.151.0_24
Phys Domain
Manual PVLAN
EPG
network-segments
AP
Settings:
- Physical Domain
- Static path bindings
- Intra EPG Isolation = True
- Proxy ARP = True
EPG
Assign an APIC policy tag to
each MAC statically on APIC
application-01
ESG
application-02
ESG
VMs MACs matched with tag
Key: app
VMs MACs matched with tag
Key: app
APIC Policy Tags:
• app:application1 -> MAC A, B, C, …
• app:application2 -> MAC X, Y, Z, …
Logical grouping by
MAC Tag

-
vrf-01
Option 6: IP selectors, no VMM integration
84
BRKDCN-2984
demo
ESXi cluster without VMM integration
VDS portgroup name
192.168.150.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.151.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.152.0_24
PVLAN (P, S)
VDS PG
applications
AP
application-01
ESG
application-02
ESG
P
C I
CCI
P
C I
CCI
Cont
Tag Selector
Key: app
Tag Selector
Key: app
192.168.152.0_24
BD
192.168.152.0_24
Phys Domain
Manual PVLAN
EPG
192.168.150.0_24
BD
192.168.150.0_24
Phys Domain
Manual PVLAN
EPG
192.168.151.0_24
BD
192.168.151.0_24
Phys Domain
Manual PVLAN
EPG
network-segments
AP
APIC Policy Tags:
• app:application1 -> IP A, B, C, …
• app:application2 -> IP X, Y, Z, … Intra EPG Isolation = True
Proxy ARP = True
Settings:
- Physical Domain
- Proxy ARP = True
EPG
each IP statically on APIC
application-01
ESG
application-02
ESG
VMs IPs matched with tag
Key: app
VMs IPs matched with tag
Key: app
Logical grouping by
IP Tag

-
VLAN
VLAN VLAN
vrf-01
Option 7: MAC selectors for bare metal
85
BRKDCN-2984
demo
application-01
ESG
application-02
ESG
Bare metal MACs matched to the APIC tag
Key: app
Bare metal MACs matched to the APIC tag
Key: app
applications
AP
application-01
ESG
application-02
ESG
P
C I
CCI
P
C I
CCI
Cont
Tag Selector
Key: app
Tag Selector
Key: app
192.168.152.0_24
BD
192.168.152.0_24
Phys Domain
EPG
192.168.150.0_24
BD
192.168.150.0_24
Phys Domain
EPG
192.168.151.0_24
BD
192.168.151.0_24
Phys Domain
EPG
network-segments
AP
Settings:
- Physical Domain
EPG
APIC Policy Tags:
• app:application1 -> MAC A, B, C, …
• app:application2 -> MAC X, Y, Z, …
each MAC statically on APIC
Logical grouping by
MAC Tag

-
VLAN
VLAN VLAN
vrf-01
Option 8: IP selectors for bare metal
86
BRKDCN-2984
demo
application-01
ESG
application-02
ESG
Bare metal IPs matched to the APIC tag
Key: app
Bare Metal IPs matched to the APIC tag
Key: app
applications
AP
application-01
ESG
application-02
ESG
P
C I
CCI
P
C I
CCI
Cont
Tag Selector
Key: app
Tag Selector
Key: app
192.168.152.0_24
BD
192.168.152.0_24
Phys Domain
EPG
192.168.150.0_24
BD
192.168.150.0_24
Phys Domain
EPG
192.168.151.0_24
BD
192.168.151.0_24
Phys Domain
EPG
network-segments
AP
Settings:
- Physical Domain
- Proxy ARP = True
EPG
APIC Policy Tags:
• app:application2 -> IP X, Y, Z, … Intra EPG Isolation = True
Proxy ARP = True
Logical grouping by
IP Tag

-
VLAN
vrf-01
Option 9: Subnet selectors with mixed Domains
87
BRKDCN-2984
demo
VDS portgroup name
192.168.150.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.151.0_24
PVLAN (P, S)
VDS PG
application-01
ESG
application-02
ESG
applications
AP
application-01
ESG
application-02
ESG
P
C I
CCI
P
C I
CCI
Cont
192.168.150.128/26
192.168.151.128/26
192.168.152.128/26
192.168.150.192/26
192.168.151.192/26
192.168.152.192/26
192.168.152.0_24
BD
192.168.152.0_24
Phys Domain
Manual PVLAN
EPG
192.168.150.0_24
BD
192.168.150.0_24
VMM Domain
Dynamic PVLAN
EPG
192.168.151.0_24
BD
192.168.151.0_24
VMM Domain
Manual PVLAN
EPG
Settings:
- VMM Domain
EPG
Endpoints matched to subnets
Key: app
Key: app
Settings:
- Physical Domain
- Proxy ARP = True
EPG
default-zone
ESG
Static PVLAN on
network-segments
AP
default-zone
AP
default-zone
ESG
P
C I
CCI
Cont
192.168.150.0/25
192.168.151.0/25
192.168.152.0/25
Key: default
Value: default-zone

-
VLAN
vrf-01
Option 10: Combined solution with/without VMM
88
BRKDCN-2984
demo
VDS portgroup name
192.168.150.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.151.0_24
PVLAN (P, S)
VDS PG
application-01
ESG
application-02
ESG
applications
AP
application-01
ESG
application-02
ESG
P
C I
CCI
P
C I
CCI
Cont
VMs and BM Tag
Key: app
VMs and BM Tag
Key: app
192.168.152.0_24
BD
192.168.152.0_24
Phys Domain
Manual PVLAN
EPG
192.168.150.0_24
BD
192.168.150.0_24
VMM Domain
Dynamic PVLAN
EPG
192.168.151.0_24
BD
192.168.151.0_24
VMM Domain
Manual PVLAN
EPG
Settings:
- VMM Domain
EPG
VMs and IPs matched with tag
Key: app
Key: app
Settings:
- Physical Domain
- Proxy ARP = True
EPG
default-zone
ESG
default-zone
AP
default-zone
ESG
Static PVLAN on
network-segments
AP
P
C I
CCI
Cont
EPG matched default
security zone
APIC Policy Tags:
• app:application2 -> IP X, Y, Z, …

-
VLAN
Option 11: Combined solution + Quarantine
89
BRKDCN-2984
vrf-01
demo
VDS portgroup name
192.168.150.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.151.0_24
PVLAN (P, S)
VDS PG
application-01
ESG
application-02
ESG
applications
AP
application-01
ESG
application-02
ESG
P
C I
CCI
P
C I
CCI
Cont
VMs and BM Tag
Key: app
VMs and BM Tag
Key: app
192.168.152.0_24
BD
192.168.152.0_24
Phys Domain
Manual PVLAN
EPG
192.168.150.0_24
BD
192.168.150.0_24
VMM Domain
Dynamic PVLAN
EPG
192.168.151.0_24
BD
192.168.151.0_24
VMM Domain
Manual PVLAN
EPG
Settings:
- VMM Domain
EPG
Key: app
Key: app
Settings:
- Physical Domain
- Proxy ARP = True
EPG
default-zone
ESG
default-zone
AP
default-zone
ESG
Key: endpoint
Value: quarantine
Static PVLAN on
quarantine
AP
quarantine
ESG
network-segments
AP
quarantine
ESG
P
C I
CCI
Cont
quarantine endpoints – match based
on VM Tag, VM name, MAC, IP
Isolated ESG to
prevent E/W traffic
EPG matched default
security zone

-
Why do we need to enable Proxy ARP for IP
mapping…
BRKDCN-2984 90

-
MAC addresses are not classified to ESGs when only IP-based
selectors are used. Switching traffic (i.e. within the same subnet)
will not use ESG contracts even if its payload has the IP address
classified to an ESG…
If two IPs in the same subnet from the same EPG are classified
into different ESGs, those two endpoints can still talk freely
through the MAC and its original EPG…
BRKDCN-2984 91

-
vrf-01
Proxy ARP
demo
epg-matched-esg
AP
192.168.150.0_24
BD
EPG
network-segments
AP
BRKDCN-2984 92
192.168.150.21
00:00:00:00:00:21
192.168.150.22
00:00:00:00:00:22
MAC to MAC
allowed
ESG
192.168.150.21
ESG
192.168.150.22
IP based ESG IP based ESG

-
Enabling “Allow Micro-Segmentation”
automatically enables Proxy ARP.
Option in a 100% virtual deployment, use
with or without Intra EPG isolation
Enabling Intra EPG isolation / Allow Micro-
Segmentation configures PVLANs on the
port group
Add an Intra EPG
Contract
Proxy ARP is only available
when Intra ESG isolation is
enabled
Enable Intra EPG isolation with
Proxy ARP if you have a mixed
virtual and physical environment
How do you enable Proxy ARP on the Leaf
Switches…?
BRKDCN-2984 93

-
vCenter tag/name matching requires read/write
vmm integration…
BRKDCN-2984 95

-
Dynamic Policy Tag matching from vCenter
Tag Collection runs
every 5 min
APIC creates dynamic VMM MAC Tags based
on the assigned Category / Tag in vCenter
Create ACI Tags to
match vCenter Tags
Tenant → Policies → Endpoint Tags
BRKDCN-2984 96

-
Static endpoint mapping…
BRKDCN-2984 98

-
Static Policy Tags on APIC
Static Endpoints
IP address ranges
MAC addresses
IP addresses
BRKDCN-2984 99

-
What if you have a Greenfield deployment…?
BRKDCN-2984 103

-
vrf-01
Greenfield option – 1:1 EPG to ESG
mapping
demo
network-security-groups
AP
192.168.150.0_24
ESG
192.168.152.0_24
BD
192.168.152.0_24
VMM Domain
Dynamic PVLAN
EPG
192.168.150.0_24
BD
192.168.150.0_24
VMM Domain
Dynamic PVLAN
EPG
192.168.151.0_24
BD
192.168.151.0_24
VMM Domain
Dynamic PVLAN
EPG
network-segments
AP
Settings:
- Dynamic PVLANs
EPG
VDS portgroup name
192.168.150.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.151.0_24
PVLAN (P, S)
VDS PG
VDS portgroup name
192.168.152.0_24
PVLAN (P, S)
VDS PG
BRKDCN-2984 104
192.168.151.0_24
ESG
192.168.152.0_24
ESG
Network backing
(VLANs)
Network security
groups
No contracts on
EPGs
Contracts on
ESGs

-
Consider automated static MAC tagging derived
from the endpoint IP address…
Works for Bare Metal and VMs, with or without
VMM Integration…
105
BRKDCN-2984

-
Understanding ACI
security…

-
Allowing open
communication…

-
There are four options to allow open
communication…
• vzAny
• Preferred Groups
• EPGs mapped Endpoint Security Groups
• Disable security (not covered, because why would you…?)
https://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-743951.html#Migrationexample
BRKDCN-2984 117

-
Existing applications typically require unrestricted
communication
192.168.150.0_24
BD 192.168.151.0_24
BD 192.168.152.0_24
BD 192.168.153.0_24
BD 192.168.154.0_24
BD 192.168.155.0_24
BD 192.168.156.0_24
BD
demo
vrf-01
192.168.150.0_24
EPG 192.168.151.0_24
EPG 192.168.152.0_24
EPG 192.168.153.0_24
EPG 192.168.154.0_24
EPG 192.168.155.0_24
EPG 192.168.156.0_24
EPG
AP
frontend
checkout
email
cart
product catalog
adservice
Redis cache
currency
recommendation
payment
shipping
The application endpoints require
communication across different
subnets, which is typically achieved
using “vzAny” or “Preferred Groups”
network-segments
BRKDCN-2984 118

-
vzAny
BRKDCN-2984 119

-
The great thing about vzAny provide/consume is that it
allows open communication between all endpoints…
The “bad” thing about vzAny provide/consume is that it
allows open communication between all endpoints…!
BRKDCN-2984 120

-
vzAny Operation – Consumer and Provider
https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/kb/b_KB_Use_vzAny_to_AutomaticallyApplyCommunicationRules_toEPGs.html
common
vzAny
P
C CCI
EPG
P
C CCI
common:default
Cont
default
Subj
default
Filt
All EPGs
EPG
P
C I
CCI
All ESGs
ESG
P
C I
CCI
All extEPGs
extEPG
P
C I
CCI
unspecified
Entry
vzAny as a contract Provider and
Consumer means that all EPGs (inc
extEPG) are implicitly Providers and
Consumers of the contract
Default contract in the “common”
tenant allows all traffic
demo
vrf-01
common.vrf-01
BRKDCN-2984 121

-
common
vzAny
P
C CCI
EPG
P
C CCI
common:default
Cont
default
Subj
default
Filt
All EPGs
EPG
P
C I
CCI
All ESGs
ESG
P
C I
CCI
unspecified
Entry
Default contract in the “common”
demo
vrf-02
common.vrf-01
vrf-01
L3Out
0.0.0.0/1
128.0.0.0/1
extEPG
P
C I
CCI
BRKDCN-2984 122

-
vzAny
P
C CCI
EPG
P
C CCI
common:default
Cont
default
Subj
default
Filt
All EPGs
EPG
P
C I
CCI
All ESGs
ESG
P
C I
CCI
unspecified
Entry
demo
vrf-01
shared-services
vrf-01
common
common.vrf-01 Default contract in the “common”
L3Out
0.0.0.0/1
128.0.0.0/1
extEPG
P
C I
CCI
BRKDCN-2984 123

-
vzAny cannot be a Provider for Shared Services
Requirement is to permit ssh from
“core-services” to all endpoints in any
given tenant i.e. tcp-src-any-dst-22
shared-services
vrf-01
core-services
P
C I
CCI
ESG
Name: permit-from-core-services
Subject: tcp
Filter: tcp-src-any-dst-22
Exported: Yes
Cont
Tenant-01
vrf-01
Subject: tcp
Imported: Yes
Cont
vzAny
EPG
P
C I
CCI
Tenant-02
vrf-01
Subject: tcp
Imported: Yes
Cont
vzAny
EPG
P
C I
CCI
Tenant-03
vrf-01
Subject: tcp
Imported: Yes
Cont
vzAny
EPG
P
C I
CCI
Use vzAny to allow
SSH to all EPGs/ESGs
Use vzAny to allow
Use vzAny to allow
BRKDCN-2984 124

-
vzAny cannot be a Provider for Shared Services
shared-services
vrf-01
core-services
P
C I
CCI
ESG
Subject: tcp
Exported: Yes
Cont
Tenant-01
vrf-01
Subject: tcp
Imported: Yes
Cont
vzAny
EPG
P
C I
CCI
Tenant-02
vrf-01
Subject: tcp
Imported: Yes
Cont
vzAny
EPG
P
C I
CCI
Tenant-03
vrf-01
Subject: tcp
Imported: Yes
Cont
vzAny
EPG
P
C I
CCI
Use vzAny to allow
Use vzAny to allow
Use vzAny to allow
vzAny cannot be a provider for
Shared Services
BRKDCN-2984 125

-
vzAny can absolutely be your friend, but remember
that vzAny contract relationships are applied to all
EPGs, ESGs, extEPGs in the VRF…
BRKDCN-2984 126

-
Preferred Groups
BRKDCN-2984 127

-
Preferred Group
Preferred Groups
demo
vrf-01
192.168.150.0_24
BD
192.168.150.0_24
Intra EPG = Unenforced
EPG
192.168.151.0_24
BD
192.168.151.0_24
EPG
192.168.152.0_24
BD
192.168.152.0_24
EPG
Enable Preferred Group
on VRF
Include EPG in
Preferred Group
Include EPG in
Preferred Group
Include EPG in
Preferred Group
There is only one
preferred group per VRF
pcTag 49160 pcTag 49159 pcTag 16393
Domains and EPGs
BRKDCN-2984 128

-
There can only be one “Preferred Group” per VRF...
It is not possible to add Contract relationships to a
“Preferred Group”
BRKDCN-2984 129

-
All EPGs mapped
to a single ESG
BRKDCN-2984 130

-
Initial state: Isolated groups of workloads
demo
vrf-01
192.168.150.0_24
BD
192.168.150.0_24
dynamic vlan
EPG
192.168.151.0_24
BD
192.168.151.0_24
dynamic vlan
EPG
192.168.152.0_24
BD
192.168.152.0_24
dynamic vlan
EPG
Network-
segments
AP
VDS portgroup name
192.168.150.0_24
Dynamic VLAN
VDS PG
VDS portgroup name
192.168.151.0_24
Dynamic VLAN
VDS PG
VDS portgroup name
192.168.152.0_24
Dynamic VLAN
VDS PG
Domains and EPGs
No communication between
portgroup/subnets 192.168.150.0,
192.168.151.0 and 192.168.152.0
BRKDCN-2984 132

-
Enable Endpoint Security Groups
Static Primary / Encap VLANs are required
when there is an intermediary switching layer
such as UCS FIs
Primary/Port Encap VLANs not
required for directly attached hosts
BRKDCN-2984 133

-
PVLAN and MAC Tagging
BRKDCN-2984 134

-
Each EPG has a unique security Tag (pcTag)
pcTag: 32771 pcTag: 49155 pcTag: 16390
BRKDCN-2984 135

-
Enabling ESG micro segmentation on a read/write VMM Domain enables
PVLANs in the hypervisor to control East/West traffic…
plus, micro segmentation also enables Proxy ARP and dynamic endpoint
MAC Tagging…
BRKDCN-2984 136

-
vzAny
(allowing ICMP)
EPG
P
C CCI
What’s the impact to traffic when we enable
uSegmentation…?
demo
vrf-01
192.168.150.0_24
BD 192.168.151.0_24
BD 192.168.152.0_24
BD
Network-
segments
AP
VDS portgroup name
192.168.150.0_24
Dynamic PVLAN (P,S)
VDS PG
VDS portgroup name
192.168.151.0_24
Dynamic PVLAN (P,S)
VDS PG
VDS portgroup name
192.168.152.0_24
Dynamic PVLAN (P,S)
VDS PG
192.168.150.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.151.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.152.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
pinger
permit-icmp
Cont
vzAny allows
ICMP
ubuntu-03
192.168.152.21
Increase in latency from 0.21ms to
814ms whilst the change takes
place – but zero packets are
dropped
BRKDCN-2984 137

-
Let’s map our EPGs to an ESG…
BRKDCN-2984 138

-
Create an Application Profile for Security Groups
New Application Profile
for Security Groups
epg-matched-security-groups
Do not create EPGs
BRKDCN-2984 139

-
Create a new ESG for Network Segments (EPGs)
Allow Intra ESG traffic i.e.
permit traffic between EPGs
Add EPGs
Enter ESG name
“group-01”
Create new
ESG
Finish
Select one or more
EPGs
Select the VRF for the ESG
to be applied against
BRKDCN-2984 140

-
Open communication within the ESG…
demo
vrf-01
192.168.150.0_24
BD 192.168.151.0_24
BD 192.168.152.0_24
BD
Network-
segments
AP
VDS portgroup name
192.168.150.0_24
Dynamic PVLAN (P,S)
VDS PG
VDS portgroup name
192.168.151.0_24
Dynamic PVLAN (P,S)
VDS PG
VDS portgroup name
192.168.152.0_24
Dynamic PVLAN (P,S)
VDS PG
192.168.150.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.151.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.152.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
epg-matched-esg
AP
network-segments
P
C I
CCI
ESG
Single security
zone
Static EPG to
ESG mapping
network-segments
ESG
subnets 192.168.150.0 and
192.168.151.0
No communication (other than
vzAny) to subnet 192.168.152.0
vzAny
(allowing ICMP)
EPG
P
C CCI
BRKDCN-2984 141

-
Matched EPGs now classified with a common pcTag
BRKDCN-2984 142

-
Let’s consider any impact to traffic when adding the
remaining EPG to the Security Group…
BRKDCN-2984 143

-
vzAny
(allowing ICMP)
EPG
P
C CCI
Add remaining EPG to Single Security Zone
demo
vrf-01
192.168.150.0_24
BD 192.168.151.0_24
BD 192.168.152.0_24
BD
Network-
segments
AP
VDS portgroup name
192.168.150.0_24
Dynamic PVLAN (P,S)
VDS PG
VDS portgroup name
192.168.151.0_24
Dynamic PVLAN (P,S)
VDS PG
VDS portgroup name
192.168.152.0_24
Dynamic PVLAN (P,S)
VDS PG
network-segments
ESG
network-segments
ESG
192.168.150.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.151.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.152.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
pinger
permit-icmp
Cont
vzAny allows
ICMP
ubuntu-03
192.168.152.21
epg-matched-esg
AP
network-segments
P
C I
CCI
ESG
Single security
zone
Add remaining EPG to
the “network-segments”
security zone
ubuntu-01
192.168.150.21
ICMP only permitted from ubuntu-01 to ubuntu-
03 after EPG 192.168.152.0_24 is added to the
“network-segments” security zone
BRKDCN-2984 144

-
All EPGs now classified with a common pcTag
BRKDCN-2984 145

-
Benefits of EPG to ESG Mapping
• More flexible than using vzAny as it is applied to specific EPGs to create one or more
security groups based on subnets/vlans*
• More secure than vzAny as EPG/ESG mapping does not include the extEPG – a contract is
required for external communication
• More integrated than vzAny as supports provider function for Shared Services
• More flexible than Preferred Groups as you can multiple ESG groups vs a single preferred
group
• More integrated than Preferred Groups as you can create a contract to the whole ESG
* Assumes 1:1 mapping between Bridge Domain and EPG. ESGs can mapping can also be performed on IP subnets
BRKDCN-2984 149

-
Allowing restricted
communication…

-
Let’s check our understanding on how contracts
work…
BRKDCN-2984 151

-
How do contracts work…?
Inside
Outside
ubuntu-01 ubuntu-02
permit ubuntu-01 ubuntu-02 tcp 5201
ubuntu-01
192.168.150.21
ubuntu-02
192.168.151.21
permit-to-vlan-11
Cont
vlan-10
(pcTag 32777)
P
C I
CCI
EPG
vlan-11
(pcTag 49162)
P
C I
CCI
EPG
EPG Security applied
at VLAN boundary
ubuntu-01
192.168.150.21
ubuntu-02
192.168.151.21
permit-to-cart-svc
Cont
frontend-svc
(pcTag 4168)
P
C I
CCI
ESG
cart-svc
(pcTag 1856)
P
C I
CCI
ESG
ESG Security applied
at VRF boundary
BRKDCN-2984 152

-
Consumer and Provider relationships are there to help you visualize the traffic
flow direction
i.e. (typically) from the consumer to the provider
Consumer and Provider relationships do not (by default) prevent TCP
connections being established from the Provider to the Consumer
BRKDCN-2984 153

-
permit-to-cart-svc
Cont
tcp
Subj
tcp-src-any-dst-7070
Filt
Entry
tcp-src-any-dst-443
Filt
tcp-src-any-dst-443
Entry
udp
Subj
udp-src-any-dst-53
Filt
udp-src-any-dst-53
Entry
icmp
Subj
icmp
Filt
icmp
Entry
redirect
Subj
tcp-src-any-dst-80
Filt
tcp-src-any-dst-80
Entry
ubuntu-01
192.168.150.21
ubuntu-02
192.168.151.21
frontend-svc
P
C I
CCI
ESG
cart-svc
P
C I
CCI
ESG
Contract Structure…
Contract name typically tied to
the Provider EPG/ESG
Subject name identifies
the protocol
Filter/Entry name identifies
protocol, src port, and dst port
BRKDCN-2984 154

-
Who hasn’t simply done this…
BRKDCN-2984 156

-
permit-any
Cont
permit-any
Subj
permit-any
Filt
unspecified
Entry
ubuntu-01
192.168.150.21
ubuntu-02
192.168.151.21
frontend-svc
P
C I
CCI
ESG
cart-svc
P
C I
CCI
ESG
permit any/any
# netcat –l [any]
# netcat –p [any] ubuntu-02 [any]
ubuntu-01
192.168.150.21
ubuntu-02
192.168.151.21
# netcat –l [any] # netcat –p [any] ubuntu-01 [any]
BRKDCN-2984 157

-
Contracts also trigger route leaking for EPGs…
BRKDCN-2984 158

-
vrf-01
AP
my-app-10
Contract Scope
ExG = Applies to either EPGs or ESGs
vrf-04
vrf-03
vrf-02
vrf-01
permit-any
Cont
permit-any
Cont
permit-any
Cont
ESG
application
endpoints
ExG
ESG
application
endpoints
ExG
my-app-01
AP
ESG
application
endpoints
ExG
ESG
application
endpoints
ExG
my-app-02
AP
ESG
application
endpoints
ExG
ESG
application
endpoints
ExG
my-app-03
AP
development
Scope
App
Scope
VRF
Scope
Tenant
Scope
Global
ESG
application
endpoints
ExG
ESG
application
endpoints
ExG
my-app-04
AP
ESG
application
endpoints
ExG
ESG
application
endpoints
ExG
my-app-05
AP
ESG
application
endpoints
ExG
ESG
application
endpoints
ExG
my-app-06
AP
vrf-06
vrf-05
production
permit-any
Cont
ESG
application
endpoints
ExG
ESG
application
endpoints
ExG
my-app-07
AP
ESG
application
endpoints
ExG
ESG
application
endpoints
ExG
my-app-08
AP
ESG
application
endpoints
ExG
ESG
application
endpoints
ExG
my-app-09
AP
Scope = Application allows
connectivity between EPGs/ESGs
within the same Application
Scope = VRF allows connectivity
between EPGs/ESGs within the
same VRF
Scope = Tenant allows connectivity between
EPGs/ESGs within the same Tenant – note the
contract also triggers route leaking
BRKDCN-2984 159
ESG
application
endpoints
ExG
ESG
application
endpoints
ExG
Scope = Global allows connectivity between
EPGs/ESGs within the between Tenants – note
the contract also triggers route leaking

-
Verifying Contract operation with netcat – Stateful = No
demo
vrf-01
ubuntu-02
P
C I
CCI
ESG
ubuntu-01
P
C I
CCI
ESG permit-to-ubuntu-02
Cont
tcp
Subj
Filt
(Stateful: No)
Entry
# netcat –l 7071
# netcat –p [any] ubuntu-02 7070
# netcat –l 5000 # netcat –p 7070 ubuntu-01 5000
# netcat –l 7070
ubuntu-01
192.168.150.21
ubuntu-02
192.168.151.21
ubuntu-01
192.168.150.21
Communication to and from
port “7070” is allowed
Stateful: No
Provider to Consumer connections
are allowed when the Provider side
port is specified as the source port
Incorrect Provider side port
ubuntu-02
192.168.151.21
“any” port is allowed
BRKDCN-2984 160

-
Verifying Contract Operation: EPG/ESG details
demo
vrf-01
ubuntu-02
P
C I
CCI
ESG
ubuntu-01
P
C I
CCI
Cont
ubuntu-01
192.168.150.21
ubuntu-02
192.168.151.21
pcTag: 38 pcTag: 5474
aci-dev-01-apic-01# show esg ubuntu-01 detail
Endpoint Security Group Data:
Tenant : demo
Application : endpoint-matched-security-groups
ESg : ubuntu-01
VRF : vrf-01
Intra ESG Isolation : unenforced
Policy Tag : 38
Consumed Contracts : permit-to-ubuntu-02
Provided Contracts :
Consumed Contracts Interface :
Qos Class : unspecified
Tag List :
IP Selectors:
Name Match Expression
-------------------- -----------------------------------------
ip=='192.168.150.21’
!output truncated
aci-dev-01-apic-01# show esg ubuntu-02 detail
Endpoint Security Group Data:
Tenant : demo
Application : endpoint-matched-security-groups
ESg : ubuntu-02
VRF : vrf-01
Intra ESG Isolation : unenforced
Policy Tag : 5474
Consumed Contracts :
Provided Contracts : permit-to-ubuntu-02
Consumed Contracts Interface :
Qos Class : unspecified
Tag List :
IP Selectors:
Name Match Expression
-------------------- -----------------------------------------
ip=='192.168.151.21’
!output truncated
BRKDCN-2984 161

-
Verifying Contract Operation: Contract details
demo
vrf-01
ubuntu-02
P
C I
CCI
ESG
ubuntu-01
P
C I
CCI
Cont
ubuntu-01
192.168.150.21
ubuntu-02
192.168.151.21
aci-dev-01-apic-01# show contract permit-to-ubuntu-02
Tenant Contract Type Qos Class Scope Subject Access-group Dir Description
---------- ---------- ---------- ------------ ---------- ---------- ---------- ---- ----------
demo permit-to-ubuntu-02 permit unspecified vrf icmp icmp both
demo permit-to-ubuntu-02 permit unspecified vrf tcp tcp-src-any-dst-7070 both
aci-dev-01-apic-01# show access-list tcp-src-any-dst-7070
Tenant : demo
Access-List : tcp-src-any-dst-7070
match tcp dest 7070
Subject: icmp
Subject: tcp
Access Control
Entry
Scope: VRF
BRKDCN-2984 162

-
Verifying Contract Operation: Drop details
demo
vrf-01
ubuntu-02
P
C I
CCI
ESG
ubuntu-01
P
C I
CCI
Cont
ubuntu-01
192.168.150.21
ubuntu-02
192.168.151.21
aci-dev-01-apic-01# show acllog deny l3 flow tenant demo vrf vrf-01 srcip 192.168.150.21
SrcIp DstIp Protocol SrcPort DstPort Node SrcIntf VrfEncap
-------------- -------------- -------- ----------- ----------- ---------- ------------ -------------
192.168.150.21 129.250.35.250 udp 38849 123 101 Ethernet1/31 VXLAN:2129922
SrcIP DstIP Protocol SrcPort DstPort
Show ACL deny
log
BRKDCN-2984 164

-
Verifying Contract operation with netcat – Stateful = Yes
demo
vrf-01
ubuntu-02
P
C I
CCI
ESG
ubuntu-01
P
C I
CCI
Cont
tcp
Subj
Filt
(Stateful: Yes)
Entry
# netcat –l 7071
# netcat –l 7070
ubuntu-01
192.168.150.21
ubuntu-02
192.168.151.21
ubuntu-01
192.168.150.21
Stateful: Yes
are blocked as the contract is a
“stateful” contract
Incorrect Provider side port
ubuntu-02
192.168.151.21
BRKDCN-2984 165

-
Verifying Contracts with Syslog and ELAM
BRKDCN-2984 166

-
Filter Entry source port = port opened on
the consumer EPG/ESG
Filter Entry destination port = port opened on
the provider EPG/ESG
src=any | dst=7070
Entry
Filt
tcp
Subj
permit-to-ubuntu-02
Cont
src=7070 | dst=any
Entry
tcp-src-7070-dst-any
Filt
tcp
Subj
permit-to-ubuntu-02
Cont
BRKDCN-2984 167

BRKDCN-2984
Getting into the
weeds…!
168

-
Reversing the Filter ports – Stateful = No
demo
vrf-01
ubuntu-02
P
C I
CCI
ESG
ubuntu-01
P
C I
CCI
Cont
tcp
Subj
# netcat –l 22
# netcat –l 5000 # netcat –p [any] ubuntu-01 5000
# netcat –l 5000
# netcat –p 22 ubuntu-02 5000
ubuntu-01
192.168.150.21
ubuntu-02
192.168.151.21
ubuntu-01
192.168.150.21
Stateful: No
are allowed when the Provider side
port is specified as the source port
Incorrect Consumer side port
ubuntu-02
192.168.151.21
Source port must be
“22”
BRKDCN-2984 169
tcp-src-22-dst-any
Filt
tcp-src-22-dst-any
(Stateful: No)
Entry

-
Why would you want to reverse the Consumer
and Provider Filters…?
BRKDCN-2984 170

-
vzAny as a contract Provider
shared-services
vrf-01
core-services
P
C I
CCI
ESG
Subject: tcp
Exported: Yes
Cont
Tenant-01
vrf-01
Subject: tcp
Imported: Yes
Cont
vzAny
EPG
P
C I
CCI
Tenant-02
vrf-01
Subject: tcp
Imported: Yes
Cont
vzAny
EPG
P
C I
CCI
Tenant-03
vrf-01
Subject: tcp
Imported: Yes
Cont
vzAny
EPG
P
C I
CCI
Use vzAny to allow
Use vzAny to allow
Use vzAny to allow
src_port = port open on the consumer EPG/ESG
dsr_port = port open on the provider side EPG/ESG
BRKDCN-2984 171

-
vzAny as a contract Provider
shared-services
vrf-01
core-services
P
C I
CCI
ESG
Subject: tcp
Exported: Yes
Cont
Tenant-01
vrf-01
Subject: tcp
Imported: Yes
Cont
vzAny
EPG
P
C I
CCI
Tenant-02
vrf-01
Subject: tcp
Imported: Yes
Cont
vzAny
EPG
P
C I
CCI
Tenant-03
vrf-01
Subject: tcp
Imported: Yes
Cont
vzAny
EPG
P
C I
CCI
Use vzAny to allow
Use vzAny to allow
Use vzAny to allow
vzAny cannot be a provider for
Shared Services
BRKDCN-2984 172

-
vzAny as a contract Consumer – Filters Reversed
shared-services
vrf-01
core-services
P
C I
CCI
ESG
Subject: tcp
Filter: tcp-src-22-dst-any
Exported: Yes
Cont
Tenant-01
vrf-01
Subject: tcp
Imported: Yes
Cont
vzAny
EPG
P
C I
CCI
Tenant-02
vrf-01
Subject: tcp
Imported: Yes
Cont
vzAny
EPG
P
C I
CCI
Tenant-03
vrf-01
Subject: tcp
Imported: Yes
Cont
vzAny
EPG
P
C I
CCI
Use vzAny to allow
Use vzAny to allow
Use vzAny to allow
Reverse the Filter ports in
the Contract
Provide the
Contract
Consume the
Contract
tcp-src-22-
means TCP port 22 is open on
the Consumer side
tcp- -dst-any
means any TCP port is open on
the Provider side
BRKDCN-2984 173
Consume the
exported contract(s)

-
Where should you place your L3outs…?
BRKDCN-2984 207

-
tenant “common”, “shared-services”, or in the
“workload/user” tenant…
BRKDCN-2984 208

-
External Connectivity
subnet(s)
BD
demo
vrf-01
VLAN
Bridge Domain)
EPG
subnet(s)
BD
VLAN
Bridge Domain)
EPG
Network
Segments
AP
Apps
(Optional)
AP
ESG
VLAN
Bridge Domain)
EPG
VLAN
Bridge Domain)
EPG
Tenant with Dedicated L3outs
shared-services
vrf-01
subnet(s)
BD
demo
vrf-01
VLAN
Bridge Domain)
EPG
subnet(s)
BD
VLAN
Bridge Domain)
EPG
Network
Segments
AP
Apps
(Optional)
AP
ESG
VLAN
Bridge Domain)
EPG
VLAN
Bridge Domain)
EPG
Tenant with Shared L3out
subnet(s)
BD
common
common.vrf-01
VLAN
Bridge Domain)
EPG
subnet(s)
BD
VLAN
Bridge Domain)
EPG
Network
Segments
AP
VLAN
Bridge Domain)
EPG
VLAN
Bridge Domain)
EPG
demo
Apps
(Optional)
AP
(Endpoints grouped by IP address*)
ESG
Shared networking with isolated security
BRKDCN-2984 209

-
What’s in a L3out…?
vrf-01-ospf-area—0.0.0.1
L3out
101/1/7
10.237.99.233/30
Path
vrf-01-all-ext-subnets
extEPG
P
C I
CCI
IP Address: 0.0.0.0/0
Scope: External Subnets for the extEPG
Subnets
102/1/7
10.237.99.237/30
Path
shared-services
vrf-01
permit-to-online-boutique
(scope = vrf)
Cont
External Device External Device
*arrows indicates direction of traffic flow i.e. from consumer to provider
Interfaces and Routing Protocols
Switches, Router IDs, Loopback
addresses, Static Routes
Subnet Classifier
BRKDCN-2984 211

-
Option 1 – Dedicated L3out per Tenant
BRKDCN-2984 212

-
Dedicated L3out
L3out
101/1/7
10.237.99.233/30
Path 102/1/7
10.237.99.237/30
Path
demo
vrf-01
permit-to-online-boutique
(scope = vrf)
Cont
192.168.150.0_24
BD
192.168.150.0_24
EPG
192.168.151.0_24
BD
192.168.151.0_24
EPG
192.168.152.0_24
BD
192.168.152.0_24
EPG
network-
segments
AP
Contract
Consumer
Contract
Provider
advertise=yes, shared=no advertise=yes, shared=no advertise=yes, shared=no
Bridge Domains set to
advertise subnet
Bridge Domains
mapped to L3out
IP Address Scope
0.0.0.0/0 External Subnets for the extEPG
Subnets
External Subnets for the extEPG:
Allows connections to/from the
endpoints/subnets through a contract
IP address: Classifies remote
endpoints/subnets
all-services
P
C I
CCI
ESG pcTag: 5490
online-boutique
AP
extEPG
P
C I
CCI
pcTag: 15
BRKDCN-2984 213

-
External Classification
IP Address:
Identifies remote endpoints/subnets
External Subnets for External EPG:
Required for contract purposes
• IP Address: identifies remote endpoints/subnets
• External Subnets for External EPG: allows packets to/from the L3out with a contract
BRKDCN-2984 214

-
Option 2 – Shared L3out
BRKDCN-2984 215

-
Shared L3out – Route Leaking between VRFs (ESGs)
L3out
101/1/7
10.237.99.233/30
Path 102/1/7
10.237.99.237/30
Path
shared-services
vrf-01
demo
vrf-01
Route leak
between VRFs
192.168.150.0_24
BD
192.168.150.0_24
EPG
192.168.151.0_24
BD
192.168.151.0_24
EPG
192.168.152.0_24
BD
192.168.152.0_24
EPG
network-
segments
AP
advertise=yes, shared=yes advertise=yes, shared=yes advertise=yes, shared=yes
all-services
P
C I
CCI
ESG pcTag: 5490
online-boutique
AP
permit-to-tn-demo-online-boutique
(scope = global, exported = yes)
Cont
Contract
Provider
IP Address Scope
Shared Security Import Subnet
Subnets
permit-to-tn-demo-online-boutique
Cont
Contract
Exported
Shared Security Import Subnet:
Leaks the pcTag of the extEPG between VRFs
IP address: Classifies remote
endpoints/subnets
External Subnets for the extEPG:
Allows connections to/from the
endpoints/subnets through a contract
extEPG
P
C I
CCI
pcTag: 41
Consumed
Contract Interface
BRKDCN-2984 216

-
External Classification and Route Leaking
IP Address:
Leaks the pcTag/Class ID between VRFs
• IP Address: identifies remote endpoints/subnets, must match a received route for route leaking purposes
• Shared Security Import Subnet: is always required as it leaks the extEPG pcTag to the target VRF
BRKDCN-2984 217

-
extEPG
P
C I
CCI
pcTag: 41
Shared L3out – Route Leaking between VRFs (ESGs)
L3out
101/1/7
10.237.99.233/30
Path 102/1/7
10.237.99.237/30
Path
shared-services
vrf-01
demo
vrf-01
Route leak
between VRFs
192.168.150.0_24
BD
192.168.150.0_24
EPG pcTag:10968
192.168.151.0_24
BD
192.168.151.0_24
EPG pcTag:12674
192.168.152.0_24
BD
192.168.152.0_24
EPG pcTag:5468
network-
segments
AP
Classify the external subnets and share
the extEPG pcTag between VRFs
IP Address Scope
Subnets
all-services
P
C I
CCI
ESG pcTag: 5490
online-boutique
AP
Prefix to leak
External Prefixes
Target Tenants
Target Tenants
Bridge Domain
Subnets
Subnets to leak
BRKDCN-2984 218

BRKDCN-2984
Getting into the
weeds…!
219

-
How does ACI Route Leaking work for EPGs…?
BRKDCN-2984 220

-
External Classification and Route Leaking
IP Address:
Leaks the pcTag/Class ID between VRFs
Shared Route Control Subnet:
Leaks a received route to another VRF
IP Address:
Must match a received route
for route leaking purposes
Aggregate Shared Routes:
Optional - Creates a prefix-list to
aggregate routes
• IP Address: identifies remote endpoints/subnets, must match a received route for route leaking purposes
• Shared Security Import Subnet: is always required as it leaks the extEPG pcTag to the target VRF
• Shared Route Control Subnet: not required when route leaking is configured under the VRF
• Aggregate Shared Routes: creates a prefix-list to aggregate routes
BRKDCN-2984 221

-
ext-subnet-10.237.96.16
extEPG
P
C I
CCI
pcTag: 41
EPG Route Leaking – L3out is the Provider
L3out
101/1/7
10.237.99.233/30
Path 102/1/7
10.237.99.237/30
Path
shared-services
vrf-01
demo
vrf-01
Route leak
between VRFs
192.168.150.0_24
BD
192.168.150.0_24
EPG pcTag:10968
192.168.151.0_24
BD
192.168.151.0_24
EPG pcTag:12674
192.168.152.0_24
BD
192.168.152.0_24
EPG pcTag:5468
network-segments
AP
advertise and share subnet
P
C I
CCI P
C I
CCI P
C I
CCI
permit-to-10.237.96.16
Cont
permit-to-10.237.96.16
Cont
Contract
Exported
Shared Route Control Subnet: creates a prefix-list matching the subnet IP
address (10.237.96.16/28) which is then leaked via MP-BGP.
Shared Security Import Subnet: programs the consumer VRF with the
pcTag of the external EPG (removes blacklist)
IP Address Scope
Shared Route Control Subnet
Subnets
pcTag leaked to target
VRF for zoning purposes
BRKDCN-2984 222

-
EPG Route Leaking – L3out is the Consumer
L3out
101/1/7
10.237.99.233/30
Path 102/1/7
10.237.99.237/30
Path
shared-services
vrf-01
demo
vrf-01
Route leak
between VRFs
192.168.150.0_24
BD
192.168.150.0_24
EPG pcTag:10968
192.168.151.0_24
BD
192.168.151.0_24
EPG pcTag:12674
192.168.152.0_24
BD
192.168.152.0_24
EPG pcTag:5468
network-segments
AP
P
C I
CCI P
C I
CCI P
C I
CCI
permit-to-10.237.96.16
Cont
permit-to-10.237.96.16
Cont
Contract
Exported
IP subnet list
advertise=yes, shared=yes
Sub Cont = no Def SVI GW
address (10.237.96.16/28) which is the leaked via MP-BGP.
Shared Security Import Subnet: programs the consumer VRF with the
pcTag of the external EPG (removes blacklist)
IP Address Scope
Subnets
ext-subnet-10.237.96.16
extEPG
P
C I
CCI
pcTag: 41
BRKDCN-2984 223

-
“Shared Route Control Subnet” must match a received route
L3out
101/1/7
10.237.99.233/30
Path 102/1/7
10.237.99.237/30
Path
shared-services
vrf-01
demo
vrf-01
Route leak
between VRFs
192.168.150.0_24
BD
192.168.150.0_24
EPG pcTag:10968
192.168.151.0_24
BD
192.168.151.0_24
EPG pcTag:12674
192.168.152.0_24
BD
192.168.152.0_24
EPG pcTag:5468
network-segments
AP
P
C I
CCI P
C I
CCI P
C I
CCI
permit-to-all-external-subnets
Cont
Cont
Contract
Exported
IP Address Scope
Subnets
all-external-subnets
extEPG
P
C I
CCI
pcTag: 41
addresses (0.0.0.0/1, 128.0.0.0/1) which is then leaked via MP-BGP
THESE ROUTES WILL NEVER MATCH, AND THEREFORE WILL NEVER LEAK…!
BRKDCN-2984 224

-
Leaking all received routes
L3out
101/1/7
10.237.99.233/30
Path 102/1/7
10.237.99.237/30
Path
shared-services
vrf-01
demo
vrf-01
Route leak
between VRFs
192.168.150.0_24
BD
192.168.150.0_24
EPG pcTag:10968
192.168.151.0_24
BD
192.168.151.0_24
EPG pcTag:12674
192.168.152.0_24
BD
192.168.152.0_24
EPG pcTag:5468
network-segments
AP
P
C I
CCI P
C I
CCI P
C I
CCI
Cont
Cont
Contract
Exported
IP Address Scope
Aggregate Shared
Aggregate Shared
Subnets
extEPG
P
C I
CCI
pcTag: 41
Aggregate Shared: creates prefix-lists matching the subnet IP addresses 0.0.0.0/1 le 32 and 128.0.0.0/1 le
32, the matching routes are then leaked via MP-BGP.
Note: the scope does not match 0.0.0.0/0, thus a received default route will not leak between VRFs.
BRKDCN-2984 225

-
IP Address Scope
0.0.0.0/0 Shared Route Control Subnet
Subnets
Leaking a default route
L3out
101/1/7
10.237.99.233/30
Path 102/1/7
10.237.99.237/30
Path
shared-services
vrf-01
demo
vrf-01
Route leak
between VRFs
192.168.150.0_24
BD
192.168.150.0_24
EPG pcTag:10968
192.168.151.0_24
BD
192.168.151.0_24
EPG pcTag:12674
192.168.152.0_24
BD
192.168.152.0_24
EPG pcTag:5468
network-segments
AP
P
C I
CCI P
C I
CCI P
C I
CCI
Cont
Cont
Contract
Exported
extEPG
P
C I
CCI
pcTag: 41
Shared Route Control Subnet: creates a prefix-list matching the
subnet IP addresses (0.0.0.0/0) which is then leaked via MP-BGP
Do Not configure “External Subnets for the extEPG” for 0.0.0.0/0
BRKDCN-2984 226

-
Why are we classifying with 0.0.0.0/1 and
128.0.0.0/1…?
BRKDCN-2984 227

-
Non dedicated border Leafs
hx-prod-fi-a hx-prod-fi-b
Hyperflex nodes UCS C series servers
Workloads attached
to border Leafs
Upstream
network
L3Out to
external routers
BRKDCN-2984 228

-
Shared L3out as Provider
L3out
101/1/7
10.237.99.233/30
Path
extEPG
P
C I
CCI
102/1/7
10.237.99.237/30
Path
shared-services
vrf-01 (2129920)
shared-services.vrf-01-all-ext-subnets
Cont
Subnets
demo
vrf-01 (2555904)
ssharman
vrf-01 (3047426)
vzAny
(software updates)
P
C I
CCI
EPG
aci-dev-01-apic-01# fabric 101 show ip route vrf shared-services:vrf-01
----------------------------------------------------------------
Node 101 (aci-dev-01-leaf-101)
----------------------------------------------------------------
0.0.0.0/0, ubest/mbest: 1/0
*via 10.237.99.234, eth1/7, ... ospf-default, type-2, tag 1
10.237.99.160/28, ubest/mbest: 1/0, attached, direct, pervasive
*via 10.1.176.66%overlay-1, ... static, tag 4294967292, rwVnid: vxlan-2555904
(scope = global, imported = yes)
Cont
aci-dev-01-apic-01# fabric 101 show ip route vrf ssharman:vrf-01
----------------------------------------------------------------
----------------------------------------------------------------
0.0.0.0/0, ubest/mbest: 1/0
*via 10.237.99.234%shared-services:vrf-01, eth1/7, ... rwVnid: vxlan-2129920
!
!output truncated
vzAny
(software updates)
EPG
P
C I
CCI
Cont
Route leak
between VRFs
----------------------------------------------------------------
----------------------------------------------------------------
0.0.0.0/0, ubest/mbest: 1/0
!
!output truncated
Default route via shared-
services:vrf-01
services:vrf-01
Traffic routed via the external
network allows communication
between workloads in different
Tenants despite no routes or
contracts in place
Default route to external network. Routes
to Tenant subnets via overlay-1 BRKDCN-2984 229

-
Shared L3out as Consumer
L3out
101/1/7
10.237.99.233/30
Path
extEPG
P
C I
CCI
102/1/7
10.237.99.237/30
Path
shared-services
vrf-01 (2129920)
permit-to-tn-demo
Cont
Subnets
demo
vrf-01 (2555904)
ssharman
vrf-01 (3047426)
vzAny
(software updates)
P
C I
CCI
EPG
aci-dev-01-apic-01# fabric 101 show ip route vrf shared-services:vrf-01
----------------------------------------------------------------
----------------------------------------------------------------
0.0.0.0/0, ubest/mbest: 1/0
*via 10.237.99.234, eth1/7, ... ospf-default, type-2, tag 1
permit-to-tn-ssharman
Cont
----------------------------------------------------------------
----------------------------------------------------------------
0.0.0.0/0, ubest/mbest: 1/0
!
!output truncated
vzAny
(software updates)
EPG
P
C I
CCI
permit-to-tn-demo
Cont
Route leak
between VRFs
----------------------------------------------------------------
----------------------------------------------------------------
0.0.0.0/0, ubest/mbest: 1/0
!
!output truncated
services:vrf-01
services:vrf-01
Traffic routed via the external
network allows communication
between workloads in different
Tenants despite no routes or
contracts in place
permit-to-tn-ssharman
Cont
Default route to external network. Routes
to Tenant subnets via overlay-1 BRKDCN-2984 230

-
Recommendation
Do not use 0.0.0.0/0 in route leaking design…!
BRKDCN-2984 231

-
#CiscoLiveAPJC © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 259
BRKDCN-2984
Where should you place your L4-7 devices…?
Do you want to use Layer 2 or Layer 3 redirects for
service insertion…?

-
tenant “common”, “shared-services”, or in the
“workload/user” tenant…
BRKDCN-2984 260

-
Virtual firewall deployment
shared-services
VRF aware firewalls defined in “shared-
services” and exported to “user” tenants
ftdv-04
ftdv-04-eth5-gig-0-2
vrf: ciscolive
vrf: ssharman
vrf: demo
Benefits of virtual firewall / IPS
• One or more virtual firewalls exported to “user” tenants as required
• Virtual firewalls used for targeted service insertion
• Firewall throughput matches application requirements
• Firewall ruleset reduced to application requirements
• Firewall security group members pushed/pulled from APIC (where available)
demo
Imported firewall
vrf: demo
ciscolive
Imported firewall
vrf: ciscolive
BRKDCN-2984 261

-
ACI Endpoint Update App (optional)
https://dcappcenter.cisco.com/aci-endpoint-update.html
BRKDCN-2984 262

-
Setting up PBR to a one arm attached firewall…
BRKDCN-2984 263

-
permit-to-cart-svc
Cont
redirect
Subj
tcp-src-any-dst-80
Filt
tcp-src-any-dst-80
Entry
tcp-src-any-dst-443
Filt
tcp-src-any-dst-443
Entry
ubuntu-01
192.168.150.21
ubuntu-02
192.168.151.21
frontend-svc
P
C I
CCI
ESG
cart-svc
P
C I
CCI
ESG
Redirect applied to all Filters under the Subject…
Contract name typically tied to
the Provider EPG/ESG
Subject = redirect
Filter/Entry name identifies
protocol, src port, and dst port
6.6.6.0_24
BD
BRKDCN-2984 264
Service Graph
Redirect
IP: 6.6.6.11
MAC: 00:50:56:a1:ac:90

-
Setting up PBR to a two arm attached firewall…
BRKDCN-2984 265

-
Two arm service graph – L2
No IP address
BD
Service Graph
Consumer
Bridge Domain
No IP address
BD
Service Graph
Provider
Bridge Domain
vlan-12
EPG
vlan-13
EPG
ftd-4112-cluster
Port-channel-10 Port-channel-30
Shadow EPG
Provider Bridge
Domain
Shadow EPG
Consumer
Bridge Domain
Consumer EPG Provider EPG
0008.E3D4.E5F6
0008.E3D4.BBBB
The redirect policy on the
Consumer Bridge domain points
to the MAC on the Provider side
port-channel sub-interface i.e.
0008.E3D4.E5F6
The redirect policy on the
Provider Bridge domain points to
the MAC on the Consumer side
port-channel sub-interface i.e.
0008.E3D4.BBBB
BVI pseudo-IP
frontend-svc
P
C I
CCI
ESG
cart-svc
P
C I
CCI
ESG
redirect
Subj
tcp-src-any-dst-80
Filt
tcp-src-any-dst-80
Entry
tcp-src-any-dst-443
Filt
tcp-src-any-dst-443
Entry
BRKDCN-2984 266
permit-to-cart-svc
Cont
Cons
Cons
Prov
Prov

-
Do not use Two Arm with L3 redirect…
BRKDCN-2984 267

-
192.168.152.0_24
BD 192.168.152.0_24
BD
Two arm service graph – L3 Cons to Prov
6.6.6.0_24
BD
Service Graph
Consumer
Bridge Domain
7.7.7.0_24
BD
Service Graph
Provider
Bridge Domain
vlan-12
EPG
vlan-13
EPG
ftd-4112-cluster
Shadow EPG
Shadow EPG
Consumer
Bridge Domain
frontend-svc
P
C I
CCI
ESG
cart-svc
P
C I
CCI
ESG
redirect
Subj
tcp-src-any-dst-80
Filt
tcp-src-any-dst-80
Entry
tcp-src-any-dst-443
Filt
tcp-src-any-dst-443
Entry
BRKDCN-2984 268
permit-to-cart-svc
Cont
Cons
Cons
Prov
Prov
> show route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
<output truncated>
Gateway of last resort is not set
C 6.6.6.0 255.255.255.0 is directly connected, Port-channel10-vlan-12
L 6.6.6.10 255.255.255.255
is directly connected, Port-channel10-vlan-12
C 7.7.7.0 255.255.255.0 is directly connected, Port-Cannel30-vlan-13
L 7.7.7.10 255.255.255.255 is directly connected, Port-Cannel30-vlan-13
S 192.168.151.0 255.255.255.0 [1/0] via 6.6.6.1, Port-channel10-vlan-12
Firewall static
routes
Provider Bridge
Domain

-
192.168.151.0_24
BD 192.168.152.0_24
BD
Two arm service graph – L3 Prov to Cons
6.6.6.0_24
BD
Service Graph
Consumer
Bridge Domain
7.7.7.0_24
BD
Service Graph
Provider
Bridge Domain
vlan-12
EPG
vlan-13
EPG
ftd-4112-cluster
Shadow EPG
Shadow EPG
Consumer
Bridge Domain
frontend-svc
P
C I
CCI
ESG
cart-svc
P
C I
CCI
ESG
redirect
Subj
tcp-src-any-dst-80
Filt
tcp-src-any-dst-80
Entry
tcp-src-any-dst-443
Filt
tcp-src-any-dst-443
Entry
BRKDCN-2984 269
permit-to-cart-svc
Cont
Cons
Cons
Prov
Prov
> show route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
<output truncated>
Gateway of last resort is not set
C 6.6.6.0 255.255.255.0 is directly connected, Port-channel10-vlan-12
L 6.6.6.10 255.255.255.255
is directly connected, Port-channel10-vlan-12
C 7.7.7.0 255.255.255.0 is directly connected, Port-Cannel30-vlan-13
L 7.7.7.10 255.255.255.255 is directly connected, Port-Cannel30-vlan-13
Firewall static
routes
Provider Bridge
Domain
Traffic received on the Firewall consumer interface.
The Firewall has a static route pointing back out of the
incoming interface, therefore traffic does not traverse
the consumer/provider interfaces.

-
Layer 2 – two arm attached
• Transparent IPS mode
• Slightly higher throughput
• No routing
Layer 2 vs Layer 3 firewall insertion
270
• Layer 3 - one arm attached
• Simple “default” routing
• Easier installation method
• Used by more than 95% of customers
BRKDCN-2984

-
Using Service Graphs for Security Enforcement
and/or Application Dependency Mapping…
BRKDCN-2984 274

-
online-boutique
AP
Dynamic Endpoint Updates
demo
vrf-01
shared-services
vrf-01
128.0.0.0/1
extEPG
P
C I
CCI
6.6.6.0_24
BD
Service Graph
Redirect
Route leak
between VRFs
frontend-service
ESG
P
C I
CCI
P
C I
CCI
Name: permit-to-online-boutique-backend-services
Scope: global
Exported: no
Subject: permit-any
Service Graph: yes – redirect-to-ftdv-02-eth7-gig-0-4
Stateful: no
Filter: permit-any
Cont
backend-services
ESG
P
C I
CCI
P
C I
CCI
Name: permit-to-online-boutique-frontend-services
Scope: global
Exported: yes (to shared-services)
Subject: tcp
Stateful: no
Cont
Intra ESG isolation = enforced
FMC EPU
APIC
Name: intra-esg-online-boutique-backend-services
Scope: vrf
Exported: no
Subject: permit-any
Stateful: no
Filter: permit-any
Cont
EPU retrieves dynamic
endpoint information from
APIC and updates FMC
FMC updates FTD with
dynamic endpoint information
BRKDCN-2984 275

-
online-boutique
AP
Targeted Flow Analysis
demo
vrf-01
shared-services
vrf-01
128.0.0.0/1
extEPG
P
C I
CCI
6.6.6.0_24
BD
Service Graph
Redirect
Route leak
between VRFs
frontend-service
ESG
P
C I
CCI
P
C I
CCI
Scope: global
Subject: tcp
Stateful: no
Cont
FTD configured to generate Syslog
messages for all flows
FMC EPU
APIC
Syslog
Scope: global
Exported: no
Subject: permit-any
Stateful: no
Filter: permit-any
Cont
Scope: vrf
Exported: no
Subject: permit-any
Stateful: no
Filter: permit-any
Cont
backend-services
ESG
P
C I
CCI
P
C I
CCI
FTD Syslog flow information
for manual ADM
BRKDCN-2984 276

-
online-boutique
AP
Targeted Flow Analysis and Enforcement
demo
vrf-01
shared-services
vrf-01
128.0.0.0/1
extEPG
P
C I
CCI
6.6.6.0_24
BD
Service Graph
Redirect
Route leak
between VRFs
frontend-service
ESG
P
C I
CCI
P
C I
CCI
Scope: global
Subject: tcp
Stateful: no
Cont
FTD configured to generate NetFlow records
and Syslog messages for all flows
FMC EPU
APIC
Cisco Secure Workload Syslog
Scope: global
Exported: no
Subject: permit-any
Stateful: no
Filter: permit-any
Cont
Scope: vrf
Exported: no
Subject: permit-any
Stateful: no
Filter: permit-any
Cont
backend-services
ESG
P
C I
CCI
P
C I
CCI
FTD updates CSW and Splunk
with flow information for ADM
CSW updates FMC with
dynamic firewall policies
BRKDCN-2984 277

-
Step 1: Assign Endpoints to the “correct” ESG…
BRKDCN-2984 279

-
Assign endpoints by Tagging the endpoints to the
correct group…
demo
vrf-01
192.168.150.0_24
BD 192.168.151.0_24
BD 192.168.152.0_24
BD
ESG all-services
P
C I
CCI
ESG
application-01
AP
Network-
segments
AP
epg-matched-esg
AP
network-segments
P
C I
CCI
ESG
ESG all-services
P
C I
CCI
ESG
application-02
AP
vzAny
P
C CCI
P
C CCI
Cont
ESG
application-03
AP
all-services
ESG
P
C I
CCI
0.0.0.0/1
128.0.0.0/1
P
C I
CCI
extEPG
192.168.150.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.151.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
192.168.152.0_24
dynamic (P,S) vlans
EPG
P
C I
CCI
L3Out
A single contract allows workloads to
move between ESGs without breaking
network forwarding
BRKDCN-2984 280

-
Step 2: Decide how to tighten security…
BRKDCN-2984 281

-
Application security options…
Open contracts to
vzAny
Open contracts
between applications
Restricted contracts
Stateful contracts
Service Graphs (SG)
Leverage ESG for
Application Mapping
Intra ESG with SG for
L4-7 enforcement
Discreet application
tier ESGs
Split apps into discreet
Tenants/VRFs – repeat
Cisco Secure
Workload
Application Knowledge
Security
Which applications can
communicate?
Which application ports
need exposing?
Control the direction of
session establishment?
Insert L4-7 device
between applications?
Discover E/W flows
within the application?
Control E/W flows
within the application?
Security groups for
each application tier?
BRKDCN-2984 282

-
Remember this from a little earlier…
BRKDCN-2984 283

-
permit-any
Cont
permit-any
Subj
permit-any
Filt
unspecified
Entry
ubuntu-01
192.168.150.21
ubuntu-02
192.168.151.21
frontend-svc
P
C I
CCI
ESG
cart-svc
P
C I
CCI
ESG
permit any/any
# netcat –l [any]
# netcat –p [any] ubuntu-02 [any]
ubuntu-01
192.168.150.21
ubuntu-02
192.168.151.21
# netcat –l [any] # netcat –p [any] ubuntu-01 [any]
Communication allowed to/from any
protocol/port in both directions
BRKDCN-2984 284

-
permit any/any with vzAny
demo
vzAny
P
C CCI
EPG
P
C CCI
permit-to-frontend-svc
Cont
All EPGs
EPG
P
C I
CCI
All ESGs
ESG
P
C I
CCI
vrf-01
ubuntu-01
192.168.150.21
frontend-svc
P
C I
CCI
ESG
permit-src-any-dst-any
Subj
Filt
Entry
any external
device
any ESG
attached device
Communication allowed on any
protocol/port to the frontend-svc
any EPG
attached device
All extEPGs
extEPG
P
C I
CCI
Communication allowed on any
protocol/port from the frontend-svc
BRKDCN-2984 285

-
#CiscoLiveAPJC © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 286
BRKDCN-2984
Consumer Contract Filter Provider Notes
vzAny
(all EPGs/ESGs/extEPGs)
permit-to-all-applications permit-src-any-dst-any application-01 vzAny with permit-src-any-dst-any allows
bi-directional communication on any port
between all applications
vzAny
(all EPGs/ESGs/extEPGs)
permit-to-all-applications permit-src-any-dst-any application-02 vzAny with permit-src-any-dst-any allows
bi-directional communication on any port
between all applications
application-04 permit-to-application-03 permit-src-any-dst-any application-03 Bi-directional communication on any port
between application-04 and application-03
application-03
application-05
permit-to-application-04 permit-tcp-src-any-dst-443 application-04 Communication from application-03 and
application-05
application-04 permit-to-application-05
(Service graph to FTD for inter
and intra application flows)
permit-src-any-dst-any application-05 Bi-directional communication on any port
via firewall between application-04 and
application-05
application-05
AP
ESG
all-services
P
C I
CCI
ESG
P
C I
CCI
application-04
AP
ESG
all-services
P
C I
CCI
ESG
P
C I
CCI
application-03
AP
ESG
all-services
P
C I
CCI
ESG
P
C I
CCI
application-02
AP
ESG
all-services
P
C I
CCI
ESG
P
C I
CCI
application-01
AP
ESG
all-services
P
C I
CCI
ESG
P
C I
CCI
vzAny
P
C CCI
P
C CCI
vrf-01
demo
Cont
Cont
Cont
Cont

-
Let’s tighten the contract to our online-boutique
application…
BRKDCN-2984 287

-
Tighten access to our online-boutique application…
demo
vrf-01
vzAny
P
C I
CCI
EPG
P
C I
CCI
Name: permit-to-core-services
Imported: yes (from shared-services)
Subject: udp
Stateful: no
Filter: udp-src-any-dst-53
udp-src-any-dst-123
Cont
Imported: yes (from shared-services)
Subject: tcp
Stateful: no
Cont
shared-services
vrf-01
128.0.0.0/1
Shared Route Control Subnet*
extEPG
P
C I
CCI
vzAny allows access
to/from “core-services”
Contracts exported from
“shared-services”
ESG all-services
P
C I
CCI
ESG
192.168.150.0_24
BD
192.168.150.0_24
dynamic (P,S) vlans
P
C I
CCI
EPG
P
C I
CCI
192.168.151.0_24
BD
192.168.151.0_24
dynamic (P,S) vlans
P
C I
CCI
EPG
P
C I
CCI
192.168.152.0_24
BD
192.168.152.0_24
dynamic (P,S) vlans
P
C I
CCI
EPG
P
C I
CCI
network-segments
P
C I
CCI
ESG
online-boutique
AP
epg-matched-esg
AP
Network-
segments
AP
Name: permit-to-tn-demo-online-boutique
Imported: yes (from demo)
Subject: tcp
Stateful: yes
Cont
Route leak
between VRFs
demo:online-boutique
Tighten access to our
application
Contract Exported
“outside” to application
requires ports TCP
80/8080
Name: permit-to-tn-demo-online-boutique
Subject: tcp
Stateful: yes
Cont
BRKDCN-2984 288

-
all-services
ESG
Application tiers across subnets
Application Centric Blueprint #1 – ESG “wrapper” for all services
demo
vrf-01
online-boutique
AP
frontend
checkout
email
cart
product catalog
adservice
Redis cache
currency
recommendation
payment
shipping
P
C I
CCI
Consumers
Single security zone for
all application services
BRKDCN-2984 290

-
all-services
ESG all-services
ESG
Application Centric Blueprint #2 – Intra ESG Isolation
demo
vrf-01
online-boutique
AP
frontend
checkout
email
cart
product catalog
adservice
Redis cache
currency
recommendation
payment
shipping
P
C I
CCI
Consumers
Firewall/IPS
Single isolated security zone
for all application services
Intra ESG contract with
Service Graph redirect
to Firewall/IPS
Protect against application
vulnerabilities such as Log4j
BRKDCN-2984 291

-
redis
ESG
all-services
ESG
Application Centric Blueprint #3 – Dedicated AP/ESG for backend database
demo
vrf-01
online-boutique
AP
frontend
checkout
email
cart
product catalog
adservice
Redis cache
currency
recommendation
payment
shipping
Consumers
P
C I
CCI
databases
AP
P
C I
CCI
P
C I
CCI
Dedicated Application Profile
and ESG (with contract) for
database services
BRKDCN-2984 292

-
redis
ESG
all-services
ESG
Application Centric Blueprint #4 – Inbound firewall/IPS + backend contract
demo
vrf-01
online-boutique
AP
frontend
checkout
email
cart
product catalog
adservice
Redis cache
currency
recommendation
payment
shipping
Consumers
P
C I
CCI
databases
AP
P
C I
CCI
P
C I
CCI
Inbound firewall/IPS
Inbound Firewall between
Consumers and Application
database services
BRKDCN-2984 293

-
redis
ESG
all-services
ESG
Application Centric Blueprint #5 – Inbound firewall/IPS + backend firewall/IPS
demo
vrf-01
online-boutique
AP
frontend
checkout
email
cart
product catalog
adservice
Redis cache
currency
recommendation
payment
shipping
Consumers
P
C I
CCI
databases
AP
P
C I
CCI
P
C I
CCI
Inbound firewall
Database firewall
Consumers and Application
Backend Firewall between
Application and Database
BRKDCN-2984 294

-
Application Centric Blueprint #6 – ESG per application tier
demo
vrf-01
online-boutique
AP
Consumers
redis
P
C I
CCI
ESG
P
C I
CCI
frontend
P
C I
CCI
ESG
P
C I
CCI
checkout
P
C I
CCI
ESG
P
C I
CCI
payment
P
C I
CCI
ESG
P
C I
CCI
email
P
C I
CCI
ESG
P
C I
CCI
product
catalogue
P
C I
CCI
ESG
P
C I
CCI
adservice
P
C I
CCI
ESG
P
C I
CCI
cart
P
C I
CCI
ESG
P
C I
CCI
shipping
P
C I
CCI
ESG
P
C I
CCI
currency
P
C I
CCI
ESG
P
C I
CCI
recommendation
P
C I
CCI
ESG
P
C I
CCI
each application service
Requires application
dependency map
BRKDCN-2984 295

-
Application Centric Blueprint #7 – Dedicated AP/ESG for backend database
demo
vrf-01
online-boutique
AP
Consumers
databases
AP
redis
P
C I
CCI
ESG
P
C I
CCI
frontend
P
C I
CCI
ESG
P
C I
CCI
checkout
P
C I
CCI
ESG
P
C I
CCI
payment
P
C I
CCI
ESG
P
C I
CCI
email
P
C I
CCI
ESG
P
C I
CCI
product
catalogue
P
C I
CCI
ESG
P
C I
CCI
adservice
P
C I
CCI
ESG
P
C I
CCI
cart
P
C I
CCI
ESG
P
C I
CCI
shipping
P
C I
CCI
ESG
P
C I
CCI
currency
P
C I
CCI
ESG
P
C I
CCI
recommendation
P
C I
CCI
ESG
P
C I
CCI
database services
BRKDCN-2984 296

-
Application Centric Blueprint #8 – ESG per application tier + frontend firewall/IPS
demo
vrf-01
online-boutique
AP
Consumers
databases
AP
Frontend firewall/IPS
redis
P
C I
CCI
ESG
P
C I
CCI
frontend
P
C I
CCI
ESG
P
C I
CCI
checkout
P
C I
CCI
ESG
P
C I
CCI
payment
P
C I
CCI
ESG
P
C I
CCI
email
P
C I
CCI
ESG
P
C I
CCI
product
catalogue
P
C I
CCI
ESG
P
C I
CCI
adservice
P
C I
CCI
ESG
P
C I
CCI
cart
P
C I
CCI
ESG
P
C I
CCI
shipping
P
C I
CCI
ESG
P
C I
CCI
currency
P
C I
CCI
ESG
P
C I
CCI
recommendation
P
C I
CCI
ESG
P
C I
CCI
database services
Consumers and application
“frontend”
BRKDCN-2984 297

-
Application Centric Blueprint #9 – ESG per application tier + frontend firewall/IPS + backend firewall/IPS
demo
vrf-01
online-boutique
AP
Consumers
databases
AP
Inbound firewall
Database firewall
redis
P
C I
CCI
ESG
P
C I
CCI
frontend
P
C I
CCI
ESG
P
C I
CCI
checkout
P
C I
CCI
ESG
P
C I
CCI
payment
P
C I
CCI
ESG
P
C I
CCI
email
P
C I
CCI
ESG
P
C I
CCI
product
catalogue
P
C I
CCI
ESG
P
C I
CCI
adservice
P
C I
CCI
ESG
P
C I
CCI
cart
P
C I
CCI
ESG
P
C I
CCI
shipping
P
C I
CCI
ESG
P
C I
CCI
currency
P
C I
CCI
ESG
P
C I
CCI
recommendation
P
C I
CCI
ESG
P
C I
CCI
“frontend”
BRKDCN-2984 298

-
Application Centric Blueprint #10 – ESG per application tier + frontend, backend, and payment firewall/IPS
demo
vrf-01
online-boutique
AP
Consumers
databases
AP
Inbound firewall
Database firewall
redis
P
C I
CCI
ESG
P
C I
CCI
frontend
P
C I
CCI
ESG
P
C I
CCI
checkout
P
C I
CCI
ESG
P
C I
CCI
payment
P
C I
CCI
ESG
P
C I
CCI
email
P
C I
CCI
ESG
P
C I
CCI
product
catalogue
P
C I
CCI
ESG
P
C I
CCI
adservice
P
C I
CCI
ESG
P
C I
CCI
cart
P
C I
CCI
ESG
P
C I
CCI
shipping
P
C I
CCI
ESG
P
C I
CCI
currency
P
C I
CCI
ESG
P
C I
CCI
recommendation
P
C I
CCI
ESG
P
C I
CCI
Payment firewall Firewall between
“checkout” and “payment”
“frontend”
BRKDCN-2984 299

-
The ultimate aim is to provide a fully consumable
fabric where resources are automated on
demand…
BRKDCN-2984 300

-
vrf-01
Example Internal Private Cloud Design – shared subnet(s)
demo
vzAny
P
C I
CCI
EPG
P
C I
CCI
permit-to-core-services
(exported from shared-services)
Cont
permit-from-core-services
Cont
application-1
AP
all-services
ESG
P
C I
CCI
EPG
P
C I
CCI
P
C I
CCI
network-segments
AP
application-2
AP
all-services
ESG
P
C I
CCI
application-3
AP
all-services
ESG
P
C I
CCI
Name: 10.1.0.0_16
Gateway: 10.1.0.1/16
Shared Between VRFs: Yes
Advertise Externally: Yes
BD
common
common.vrf-01
shared-services
vrf-01
Route leak
between VRFs
128.0.0.0/1
extEPG
P
C I
CCI
Application endpoints deployed to an EPG
“landing zone” in “enforced” mode to
prevent E/W traffic inside both the
hypervisor and the network
vzAny allows access
Bridge Domain in the
“common” tenant can be
shared across multiple tenants
Endpoints security policy
moved to application ESGs
based on tag policy
permit-to-tn-demo-application-1
(exported to shared-services)
Cont
Cont
Cont
Contracts exported to
vzAny cannot be a
provider for shared
services
Tenant VRF not required as
the “landing zone” EPG is
mapped to the BD in
“common”
BRKDCN-2984 301

-
vrf-01
Example Internal Private Cloud Design – Auto Cleanup
Isolation
demo
vzAny
P
C I
CCI
EPG
P
C I
CCI
permit-to-core-services
Cont
permit-from-core-services
Cont
application-1
AP
all-services
ESG
P
C I
CCI
EPG
P
C I
CCI
P
C I
CCI
network-segments
AP
application-2
AP
all-services
ESG
P
C I
CCI
application-3
AP
all-services
ESG
P
C I
CCI
Name: 10.1.0.0_16
Gateway: 10.1.0.1/16
Shared Between VRFs: Yes
Advertise Externally: Yes
BD
common
common.vrf-01
shared-services
vrf-01
Route leak
between VRFs
128.0.0.0/1
extEPG
P
C I
CCI
Application endpoints deployed to an EPG
“landing zone” in “enforced” mode to
prevent E/W traffic inside both the
hypervisor and the network
vzAny allows access
Bridge Domain in the
“common” tenant can be
shared across multiple tenants
Endpoints security policy
moved to application ESGs
based on tag policy
Cont
Cont
Cont
Contracts exported to
vzAny cannot be a
provider for shared
services
Tenant VRF not required as
the “landing zone” EPG is
mapped to the BD in
“common”
BRKDCN-2984 302

-
Select one or more Design Patterns…
Carefully consider the use of:
• Tenant “common”
• Using a “shared services” tenant
• vzAny
• Dedicated border Leafs (recommended)
• Contract scopes
• External EPG with the classifier 0.0.0.0/0
BRKDCN-2984 304

-
Benefits of Shared Service model…
• Looks and feels like a Public Cloud model of working
• Network team maintains control of North / South route peering
• Network team maintains control of Inter VRF route leaking
• Each Tenant can control their own CIDR range
• Each Tenant can control their own security rules
• Each Tenant can have private (non routable subnets)
• Security services can be easily inserted in the Tenants
• Do not use 0.0.0.0/0 as the extEPG classifier in a shared model
BRKDCN-2984 305

-
Implement ESG “wrappers”…
Wrapping applications into ESGs provides the following benefits
for both virtual and physical workloads:
• Improved application visibility
• Improved auditing capabilities
• Improved troubleshooting
• Intelligent service insertion
• Security tied applications rather than network segments
• Reduce the reliance on monolithic physical security devices
BRKDCN-2984 306

-
Automation Considerations…
• A simple consumption model is everything
• Single API for all networking functions
• Application security requirements should be declared to the infrastructure
• Add virtual application firewalls to deployments if required
• Large physical monolithic firewalls are useful at network boundaries, however they should
only provide broad security rules
• Remove unnecessary overlay networks that add layers of complexity
BRKDCN-2984 307

-
Getting started resources
• Visual Studio Code with extensions
• Yaml
• Indent Rainbow
• Hashi Terraform
• https://netascode.cisco.com/solutions/aci/terraform/overview
• https://developer.cisco.com/docs/nexus-as-code/introduction/
• https://github.com/netascode/terraform-aci-nac-aci/tree/main
• https://github.com/netascode/nac-aci-simple-example
• https://github.com/netascode/nac-aci-comprehensive-example
• https://github.com/spsharman/CiscoLive2024
• https://github.com/spsharman/aci-prod
• https://github.com/spsharman/aci-dev-01
• https://tl10k.dev/categories/terraform/nexus-as-code-architecture/
308
CISCOU-2033

-
ESG Design Guide
https://www.cisco.com/c/en/us/td/docs/dcn/whitepapers/cisco-aci-esg-design-guide.html?cachemode=refresh
BRKDCN-2984 309

-
Now available on dCloud
https://dcloud2-sjc.cisco.com/content/demo/333928?returnPathTitleKey=content-view
BRKDCN-2984 310

-
Try the Walk in Lab (LABDCN-2287) in the World of
Solutions…
BRKDCN-2984 311

-
Complete Your Session Evaluations
312
Complete a minimum of 4 session surveys and the Overall Event Survey to
claim a Cisco Live T-Shirt.
Complete your surveys in the Cisco Live mobile app.
BRKDCN-2984

-
Continue
your education • Visit the Cisco Showcase
for related demos
• Book your one-on-one
Meet the Engineer meeting
• Attend the interactive education
with DevNet, Capture the Flag,
and Walk-in Labs
• Visit the On-Demand Library
for more sessions at
www.CiscoLive.com/on-demand
Contact me at: ssharman@cisco.com
LABDCN-2287 – ACI Segmentation…
BRKDCN-2634 – Deploying EVPN G/W…
BRKDCN-2673 – Nexus-as-Code…
BRKDCN-2910 – Upgrading ACI…
BRKDCN-2949 – ACI Multi-Pod…
BRKDCN-2980 – ACI Multi-Site…
BRKDCN-3900 – ACI Forwarding…
BRKDCN-3982 – ACI PBR Deep Dive…
BRKDCN-2984 313

Sdn aci for cisco private cloud building onprem.pdf

More Related Content

Similar to Sdn aci for cisco private cloud building onprem.pdf

Sdn aci for cisco private cloud building onprem.pdf