SpringOne Platform 2017 Sai Chaitanya, VMware; Usha Ramachandran, Pivotal "In this session, you will learn how a SDN platform like VMware NSX can enable networking, security and operations for Cloud Foundry apps. We will look at the core and swappable components of the Cloud Foundry networking stack to understand how a third party CNI plugin like NSX can replace the batteries-included plugin. We will then introduce you to the Cloud Foundry Network Policy Model that enables an app developer or operator to apply Network Security Policy for a CF application and compare it to the SDN Network Policy Model. Finally, we will show how this integration works through a demonstration."

1. Cloud Foundry Networking with VMware
NSX
Usha Ramachandran, Pivotal Sai Chaitanya, VMware
@ushaRusha @sai_chaitanya
1

2. Disclaimer

3. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Safe Harbor Statement
The following is intended to outline the general direction of Pivotal's offerings. It is
intended for information purposes only and may not be incorporated into any
contract. Any information regarding pre-release of Pivotal offerings, future updates
or other planned modifications is subject to ongoing evaluation by Pivotal and is
subject to change. This information is provided without warranty or any kind,
express or implied, and is not a commitment to deliver any material, code, or
functionality, and should not be relied upon in making purchasing decisions
regarding Pivotal's offerings. These purchasing decisions should only be based on
features currently available. The development, release, and timing of any features
or functionality described for Pivotal's offerings in this presentation remain at the
sole discretion of Pivotal. Pivotal has no obligation to update forward looking
information in this presentation.
3

4. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Agenda
4
Introduction
Pivotal Cloud
Foundry
NSX-V Integration
New Features in
Cloud Foundry
Networking
NSX-T Integration
1 2 3 4 5

5. Introduction

6. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Platform Operator Needs
6
Security
• Network Security
• Authorization and Authentication
• Platform Security
Platform Stability
• Day-2 operations
• Faster patching and upgrades
Visibility
• Billing and auditing
• Triage and debugging
OPERATOR
Keep the platform
running smoothly

7. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Application Developer Needs
7
DEVELOPER
Create applications to
meet business goals
Different applications types
• Micro-services
• Clustering Apps
• Latency-sensitive or secure services
Focus on business logic
• Tools and frameworks for easy development
• Write once, run anywhere
Speed and Agility
• Self-Service – no tickets!
• Minimal impact during upgrades

8. Pivotal Cloud Foundry

9. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
PCF Technical Primer
9
Cloud
Controller
Diego
cell
cell_1
Go
Router
Deploy app
Uploads app and
invokes scheduler
App scheduled to a
container host
CF Services for
persistent storage
CF app instance
( Container) – stateless,
aka state persisted
externally
Register route:
app_a.cfapps.cloud.com →
cell_1_ip : port_num
Go
Router
App access
cf push app_a
N
A
T
Load Balancer
*.cfapps.cloud.com
→ GoRouters

10. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Traditional Network Security in Cloud Foundry
10
PCI - Space
PCF Prod
Non PCI - Space
ASGs
Collection of egress allow rules that specify {IP CIDR,
Port, Protocol} that an app can access
Applied to entire foundation or at CF space level
PCF Services -
PCI Net
Challenges
Cannot specify policy at app granularity
PCI and non PCI containers can share some container host
Source Destination Port and
Proto
Action
Any PCI Services tcp, 3306 Allow
Any Any any Deny
PCF Services –
non PCI Net
✓
✗

11. NSX-V Integration

12. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Provides
Faithful Reproduction of Network & Security Services in Software
Switchi
ng
Routin
g
Firewal
ling
Distributed
Any Network Hardware
NSX
Platform
NSX vSwitch
Logical Switch
Logical Router
Logical Firewall
Logical Load Balancer
VMware NSXSoftware Networking Platform
Centralized
Routin
g
Load
Balanci
ng
Firewal
ling
VPN

13. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
PCF Infra Networking and Load Balancing
requirements
Four Private Networks
Ops
Manager
Ops Man
Director
PCF Infra Network -
192.168.10.0/26
Go
Router
Deployment Network -
192.168.20.0/22
Go
Router
cell cell
Services Network - 192.168.28.0/22 Other External Services - 192.168.24.0/22

14. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
PCF and NSX-V Logical Networking & Load Balancing
Basic Routing Design
NSX LS Infra - 192.168.10.0/26
NSX LS Deployment - 192.168.20.0/22
Service Source Destination
Source NAT 192.168.10.0/16 External IP 1
Dest NAT External IP 2 Ops Man IP
Service VIP Pool
Load
Balancing
External IP 3 Go Router IPs
Load
Balancing
External IP 4 Diego Brain IPs
External Network – 10.114.214.0/24
NSX ESG
NSX LS Services

15. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
PCF Infrastructure Security Requirements
NSX LS Infra - 192.168.10.0/26
NSX LS Deployment - 192.168.20.0/22
External Network – 10.114.214.0/24
NSX ESG
NSX LS Services
Source Destination Service Action
Any Ops_Man SSH,
HTTP,
HTTPS
Allow
Any VIP_GoRTR HTTP,
HTTPS
Allow
… …… …… Allow
… …… …… Allow
Any Any Deny Deny
ESG Firewall to protect the PCF foundation
http://docs.pivotal.io/pivotalcf/1-11/refarch/vsphere/vsphere_nsx_cookbook.html#load_balancer

16. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Cloud Foundry Isolation Segments
16
Isolation Segments
Dedicated set of diego cells to enable compute
isolation of apps
Can be assigned to CF org or space
Apps (and instances) in org or space will only be
scheduled to their own dedicated cells
Benefits
Apps of different kinds can be deployed with compute
isolation on shared foundation – e.g. PCI and non-PCI
Save operational and cost overhead of maintaining
multiple foundations

17. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
PCF Isolation Segments and NSX-V
Ops Manager and NSX integration for CF Isolation Segments
Deploy Isolation
Segment
Ops Manager
deploys
dedicated
Diego cells for
IS
Ops Manager
adds Diego
cells to NSX-V
SG
If SG with same name as Isolation
segment, exists VMs are added to SG
If SG with name of Isolation
Segment is not found, create SG
and adds VMs
As Diego Cells are added / deleted, NSX SG membership is maintained

18. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
PCF Isolation Segments and NSX-V
Compute Isolation and Network Isolation
Source Destin Service Action
SG_PCI PCI_Services HTTP,
HTTPS
Allow
SG_non_
PCI
Non_PCI_Servi
ces
HTTP,
HTTPS
Allow
Any Any Deny Deny
DFW segmentation policy
Stateful Network Segmentation & Monitoring at
the Org / Space granularity

19. New Features in CF Networking

20. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Legacy Networking
20

21. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Desired State
21

22. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
PCF 1.11 – New Networking Features
22
Connec&vity Policy
Exis&ng
Features
Silk CNI plugin
Unique IP on overlay
3rd party plugins
App to App
Dynamic
CLI or API
Self Service
ASGs
Egress Cell
IP:SNAT
Ingress Cell
IP:DNAT

23. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Container Networking Interface (CNI) is an
industry standard API for container runtimes
to call third party networking plugins

24. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
“Batteries-included” Networking – Silk CNI
24
cell
PCF Deployment Network - 192.168.20.0/22
cell
PCF Container Network – 10.255.0.0/1610.255.10.0/24
Single Overlay network for all containers in a single foundation
Defaults to a /16 range to allow for ~250 cells with ~250 containers per cell
Access to external services and through GoRouter continue to use the PCF Deployment
Network
10.255.11.0/24

25. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Architecture Overview
Policy Server
VM
Policy
DB
- - - - - Internal API - - - - - -
- - - - External API - - - - - -
Diego Cell
Garden External Networker
Silk CNI Plugin
CNI API
- - - - Garden External Networker API - - - -
VXLAN
Policy Agent
iptables
NAT from
host
(NetIn)
ASGs
(NetOut)

26. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Application Level Policies
26
APP 1 APP 2
Overlay Network
Underlay Network
CellCell
APP 3
Cell
Allow APP1 -> APP 2
Policy
Outgoing
connections
Connections
from goRouter

27. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Policy Configuration - CF CLI or API
27
$ cf add-network-policy SOURCE_APP --desOnaOon-app DESTINATION_APP [(--protocol (tcp | udp)
--port RANGE)]
Allow two apps to talk to each other
List policies
$ cf network-policies [--source SOURCE_APP]
Revoke the policy for two apps to talk to each other
$ cf remove-network-policy SOURCE_APP --desOnaOon-app DESTINATION_APP --protocol (tcp |
udp) --port RANGE

28. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Use Cases
28
APP 1
APP 1
frontend
billing
billing
billing
Checkout
Checkout
Auth
inventory
inventory
inventory
inventory
Secure Microservices
Direct east-west communication
Private microservices do not need public
routes
Fine-grained application level policies
boot
peer
peer
Clustering Applications
Same source and destination in policy
Communicate on an TCP or UDP port

29. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
29

30. NSX-T Integration

31. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
NSX-T
31
Network and Security Platform for cloud native and traditional apps
CNI Integration with Cloud Foundry
Native “Container” Networking & Security
Common operational model for
traditional and cloud native
Integrated with data center network,
tools & processes
Leverage existing investments
On Premise and Public Cloud
workloads

32. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.
NSX-T Architecture and Components
32

33. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Microsegmentation for CF apps
33
1
2
3
Inter Microservice – same
cloud naOve plaXorm instance
Inter Microservice – mulOple insta
of CNA plaXorm/s
Microservice to VM or Database
app
Two approaches to configure Policy
CF Network Policy
NSX Manager

34. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Demo
34
PCF integraOon with NSX-T 2.1
Automated provisioning of network,
security as part of app deployment
MicrosegmentaOon for CF app
NaOve Container Networking
Common troubleshooOng tooling
for Containers and VMs with NSX
Traceflow

35. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
NSX-T Integration with PAS 2.0
35
NSX-T is integrated into Cloud Foundry through CNI
Benefits:
● Common operational model for cloud-native and
traditional apps
● C2C policies for app communication are enforced
by NSX-T
● ASGs are enforced by NSX-T
● NSX Manager can be used to configure policies
from apps to external entities (augment or
replace ASGs)
● Application identity - each Org gets an SNAT
address
● Leverage NSX-T features for logging, Traceflow
and metrics across the data center
Additional Notes
NSX-T support is only
available for greenfield
installations
This release supports
NSX-T on vSphere only
Silk will continue to be
supported as the
“batteries-included”
plugin
NSX-V will continue to be
supported

36. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Operator Workflow
36
● Configure Director
with NSX-T
credentials
● Apply Changes
● Import NSX-T tile
from VMware
● Configure tile
● Apply Changes
● Import PAS tile
● Choose “External”
CNI plugin
● Apply Changes

37. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-
NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
NSX-T CNI Integration
Policy Server
VM
Policy
DB
- - - - - Internal API - - - - - -
- - - - External API - - - - - -
Diego Cell
Garden External Networker
NSX-T CNI Plugin
CNIAPI
- - - - Garden External Networker API - - - -
NSX Node
Agent
OVS
NSX
Container
Plugin
Hyperbus

38. Learn More. Stay Connected.
Reach out to Pivotal or VMware to learn more and schedule POCs
38
#springone@s1p