This document summarizes a presentation about Romana Cloud Native SDN and multi-tenant networking in Kubernetes. It discusses how Romana embeds tenant and segment IDs in IP addresses to provide layer 3 isolation and tenancy without using overlays. This simplifies operations by eliminating the need for VLANs, VXLANs, and other complex networking constructs. The presentation includes examples of how Romana assigns addresses and isolates tenants, pods, and tiers through network policies. It concludes with an outline of a demo to showcase these capabilities by launching isolated tenants and enforcing policies between tiers.

1. Kubernetes v1.2
Multi-tenant Networking
Romana Cloud Native SDN
Chris Marino
Robert Starmer
romana.ioKubernetes Meetup 2/11/16

2. Multi-tenant Networking
• Agenda
• Cloud Native Networks
• Romana Cloud Native SDN
• How it works
• Demo
• Q & A
Kubernetes Meetup 2/11/16 romana.io Slide 1

3. Cloud Native vs. Enterprise Networks
• Amazon AWS Style v. Enterprise Apps
• Service orientation (Cattle) v. Endpoint orientation (Pets)
• Network requirements
• Reachable IP addresses v. Auto discovered MAC (ARP on VLANs)
• Service orientation further decouples apps from infrastructure
• No VM migration
• No IP Failover
• Good News: Cloud Native apps don’t need layer 2 networks
• Layer 2 networks introduce a lot of SDN complexity
• Bad News: Layer 2 networks provided a convenient way to isolate apps
romana.ioKubernetes Meetup 2/11/16 Slide 2

4. Romana Cloud Native SDN
• Layer 3 based isolation and tenancy model
• Topology-aware addressing
• Embed tenant and segment IDs in IP addresses
• Requires nothing more than standard L3 routing
• Hierarchical design simplifies scalable deployment
• No virtual network required
• Native performance and visibility
• Eliminates overlays
romana.ioKubernetes Meetup 2/11/16 Slide 3

5. Complexity melts away
• No VLANs, VXLANs, VTEP/VNID, OpenFlow, OVS/OVN/OVSDB
• Route aggregation simplifies operations
• Static routing eliminates need for route distribution (BGP, XMPP, KVS)
• Reduces the number of firewall rules (i.e. network v. endpoint)
• Simplifies Operations
• Existing tools, techniques and diagnostics all just work
• Existing security, policy and control systems all work
• Firewalls, IDS, LB, etc., etc., etc.
Kubernetes Meetup 2/11/16 romana.io Slide 4

6. How does it work?
• Assign CIDR length for host (node), tenant and segment
• Example: host 16, tenant 24, segment 28
• On every host, each tenant gets a real physical CIDR
• Tenant can further sub-net for their own private segments
• Configure IP addresses that maintain reachability
• Apply layer 3 firewall rules for network isolation
Kubernetes Meetup 2/11/16 romana.io Slide 5

7. Example
Kubernetes Meetup 2/11/16 romana.io Slide 6
Bit location 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
Field
Capacity 0 0 0 0 1 0 1 0
Example: Bits Length Purpose
10/8 Network 8 10/8 Network
Hosts 8 Up to 255 Hosts
Tenants 8 Up to 255 Tenants
Segments 4 Up to 16 Segments per Tenant
Endpoints 4 Up to 16 Endpoints per Segment
Host 1 ID CIDR or IP Host 2 ID CIDR or IP Host 3 ID CIDR or IP
Physical Addr 192.168.0.10 Physical Addr 192.168.0.11 Physical Addr 192.168.0.12
Host 1 10.1/16 Host 2 10.2/16 Host 3 10.3/16
Tenant 1 10.1.1/24 Tenant 1 10.2.1/24 Tenant 1 10.3.1/24
Segment 1 10.1.1.16/28 Segment 1 10.2.1.16/28 Segment 1 10.3.1.16/28
Pod 1 11 Pod 1 4 Pod 1 4
Pod 2 14 Pod 2 5 Pod 2 5
Tenant 2 10.1.2/24 Tenant 1 10.2.1/24 Tenant 2 10.3.2/24
Segment 1 10.1.2.16/28 Segment 2 10.2.1.32/28 Segment 1 10.3.2.32/28
Pod 1 4 Pod 1 9 Pod 1 9
Pod 2 8 Pod 2 12 Pod 2 12
Location
10/8 Net Mask Host ID Bits (8) Tenant ID Bits (8) Segment ID and IID
Up to 255 Hosts Up to 255 Tenants 255 Endpoints for each Tenant
10.1.1.27
10.3.2.28
10.3.2.25
10.3.1.21
10.3.1.20
10.2.1.44
10.2.1.41
10.2.1.21
10.2.1.20
10.1.2.24
10.1.2.20
10.1.1.40
32
28
24
16
8
29-32
25-28
17-24
9-16
1-8

8. Host 1: 192.168.0.10 on Port 1
Host 2: 192.168.0.11 on Port 2
Host 3: 192.168.0.12 on Port 3
Router,
Switch
or VPC
Physical Deployment
Kubernetes Meetup 2/11/16 romana.io
192.168.0.10 192.168.0.11 192.168.0.12
Host 1
Pod 1
1.1.27
G/W: 10.1.0.1/16
Pod 2
1.1.40
Pod 1
1.2.20
Pod 2
1.2.24
Tap
Interfaces
Host 2
Pod 1
2.1.20
G/W: 10.2.0.1/16
Pod 2
2.1.21
Pod 1
2.1.41
Pod 2
2.1.44
Tap
Interfaces
Host 3
Pod 1
3.1.20
G/W: 10.3.0.1/16
Pod 2
3.1.21
Pod 1
3.2.25
Pod 2
3.2.28
Tap
Interfaces
Slide 7

9. Romana Project
• Cloud Native SDN
• All details available at romana.io
• Open source
• Apache 2.0
• Written in Go
• www.github.com/romana
• Release v0.6.4 available now
• Integration with OpenStack
• Kubernetes integration very soon
romana.ioKubernetes Meetup 2/11/16 Slide 8

10. Node n
Node n
Node n
Node n
Node n
KubletAgent
Kube
Proxy
Docker
/rkt
Pod Pod
iptables
CNI
Romana
Romana Networks
Kubernetes Meetup 2/11/16 romana.io
K8S Master
IPAM
Routes
Tenant
DB
Topology
Controllers
Scheduler
API
etcd
ThirdParty Resource
Network Policy
Schema
Slide 9
Policy
/apis/romana.io/demo/v1
Pod/Service
Spec
Network Policy

11. Network Policy Resource
Kubernetes Meetup 2/11/16 romana.io Slide 10
name: network-policy.romana.io
apiVersion: extensions/v1beta1
kind: ThirdPartyResource
description: “Romana Network Policy Third Party Resource
Schema"
versions:
- name: demo/v1
Resulting API Endpoint
/apis/romana.io/demo/v1/networkpolicy/

12. www.romana.io
Tenant t1 Pod Specifications
• Frontend
apiVersion: v1
kind: Pod
metadata:
name: nginx-frontend
labels:
app: nginx
owner: t1
tier: frontend
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
• Backend
apiVersion: v1
kind: Pod
metadata:
name: nginx-backend
labels:
app: nginx
owner: t1
tier: backend
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
Kubernetes Meetup 2/11/16
romana.io
Slide 11

13. www.romana.io
Replication Controller
• Tenant t2
apiVersion: v1
kind: ReplicationController
metadata:
name: nginx-default
spec:
replicas: 3
template:
metadata:
labels:
app: guestbook
tier: default
owner: t2
spec:
containers:
- name: nginx-default
image: nginx
ports:
- containerPort: 80
Kubernetes Meetup 2/11/16
romana.io
Slide 12

14. www.romana.io
Pod Specifications
• Frontend
apiVersion: v1
kind: Pod
metadata:
name: nginx-frontend
labels:
app: nginx
owner: t1
tier: frontend
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
• Backend
apiVersion: v1
kind: Pod
metadata:
name: nginx-backend
labels:
app: nginx
owner: t1
tier: backend
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
Kubernetes Meetup 2/11/16
romana.io
Slide 13

15. Network Policy
• Policy1
kind: NetworkPolicy
apiVersion: romana.io/demo/v1
metadata:
name: policy1
namespace: default
labels:
- owner: t1
spec:
podSelector: // Standard label selector - selects pods.
tier: backend
allowIncoming: // (Optional) List of allow rules.
- toPorts: // (Optional) List of dest ports to open.
- port: 80 // (Optional) Numeric or named port
protocol: TCP // [ TCP | UDP]
from: // (Optional) List of sources.
- pods: // (Optional) Standard label selector.
tier: frontend // (Optional) Standard label selector.
Kubernetes Meetup 2/11/16 romana.io Slide 14

16. Router,
Switch
or VPC
Demo
Kubernetes Meetup 2/11/16 romana.io
192.168.0.10 192.168.0.11
Host 1
T1
1.1.27
G/W: 10.1.0.1/16
T1
1.1.40
FE
1.2.20
BE
1.2.44
Tap
Interfaces
Host 2
T1
2.1.20
G/W: 10.2.0.1/16
Tap
Interfaces
Slide 15

17. Demo
• Running Kubernetes on x EC2 instances
• Romana Services running on Kubernetes Master
• Demo Script
1. Apply NetworkPolicy ThirdParty Schema
2. Launch Pods as different isolated tenants
3. Within a single tenant, launch Pods on separate Tiers
4. Apply Network Policy to Tiers
5. Show Policy Enforcement
Kubernetes Meetup 2/11/16 romana.io Slide 16