Kubernetes OpenContrail Meetup

KUBERNETES OPENCONTRAIL
WORKSHOP
PEDRO MARQUES, SANJU ABRAHAM
LACHLAN EVENSON
ANIKET DAPTARI

2 Copyright © 2014 Juniper Networks, Inc. www.juniper.net
WORKSHOP AGENDA
1
3
4
5
OPENCONTRAIL OVERVIEW
2 CUSTOMER USE CASES
6
KUBERNETES + OPENCONTRAIL
KUBERNETES + OPENCONTRAIL – GCE SETUP
DEPLOY APPS
PERFORMANCE

OPENCONTRAIL
ARCHITECTURE - RECAP

OPENCONTRAIL
HETEROGENEOUS NETWORKING SYSTEM
POD
AWS
/
GCE
…
Public Clouds

VIRTUAL
NETWORK
GREEN
Host + Hypervisor
Host + Hypervisor
VIRTUAL NETWORKS: LOGICAL VERSUS PHYSICAL
VIRTUAL
NETWORK
BLUE
VIRTUAL
NETWORK
YELLOW
Contrail Security Policy
(Firewall-like e.g. allow
only HTTP traffic)
Contrail Policy
with a Firewall
Service
IP fabric
(switch underlay)
G1 G2 G3
B3
B1
B2
G1
G3
G2
Y1 Y2 Y3B1 B2 B3
Y2Y3
Y1
VM and virtualized Network
function pool
Intra-network
traffic
Inter-network traffic traversing a service
… …
LOGICAL
(Policy Definition)
PHYSICAL
(Policy Enforcement)
Non-HTTP
traffic

LITHIUM TECHNOLOGIES
https://youtu.be/pZjNFcyC6Uo - https://twitter.com/lachlanevenson

KUBERNETES + OPENCONTRAIL
Opencontrail VRouter
Opencontrail VRouter Opencontrail Controller
Kube-Network-Mgr
*Opencontrail
replaces kube-proxy

VALUE
Distributed Router
§ Logical Network across any server, any rack, any cluster and any data center
§ PODs can migrate without any reworking of security policies, load balancing, etc
§ New workloads or new networks would not require provisioning of physical networks
§ Nodes in the physical network can fail without any disruption to workloads
Multi-tenancy, Full isolation and Fault tolerance
§ MAC and IP addresses are completely private per tenant
§ Any failures or configuration errors by tenants do not affect other applications or
tenants
§ Any failures in virtual layer do not propagate to physical networks

OPENCONTRAIL CONTROLLER – KUBERNETES MASTER
• Kube network mgr reading notifications from kube api server
and creating objects in opencontrail.
• A REST API Server that provides the north-bound interface to
an Orchestration System or other application
• A Rabbitmq message bus to facilitate communications amongst
internal components
• A Cassandra database for persistent storage of configuration
• A Schema transformer that learns about changes in the high
level data model over the message bus and transforms (or
compiles) these changes in the high level data model into
corresponding changes in the low level data model
• An IF-MAP Server that provides a south bound interface to
push the computed low-level configuration down to the Control
nodes
• Zookeeper (not shown in diagram) is used for allocation unique
object identifiers and to implement transactions
Opencontrail
Kube Network
Manager
Kube-ApiServer

• All Control Plane Nodes are active active
• Each vRouter uses XMPP to connect with
multiple Control Plane nodes for
redundancy
• Each Control Plane Node connects to
multiple configuration nodes for
redundancy
• BGP is used to connect with Physical
Gateway Routers or switches
• Control Plane Nodes federate using BGP
IFMAP
Server
IFMAP
Server
OPENCONTRAIL CONTROLLER – KUBERNETES MASTER

OPENCONTRAIL VROUTER – KUBERNETES MINION
• OpenContrail Kubelet plugin reads POD
info from kubelet and create ports for
interface (veth) of the POD in vrouter.
• vRouter replaces the Linux Bridge/OVS
module in host Kernel
• vRouter performs bridging (E-VPN) and
routing (L3VPN)
• vRouter performs networking services like
Security Policies, NAT, Multicast, Mirroring,
and Load Balancing
• No need for Service Nodes or L2/L3 GWs
for Routing, Broadcast/Multicast, NAT
• Routes are automatically populated and
advertised based on Policies
• Peering with network switch and routers
based on standard protocols
• Extends to workloads running on physical
and virtual machines and also across data
centers and private/public clouds
Opencontrail
Kubelet
Plugin
Kubelet
Docker
Kubernetes - CBR0 bridge
Container Container Container
POD
OpenContrail
Kubelet
Plugin

NetLink
KUBERNETES OPENCONTRAIL GATEWAY
Linux Kernel
OpenContrail vRouter Kernel Module
OpenContrail vRouter Agent
OpenContrail
Control
OpenContrail
Control
POD (External-IP)
Kube-Minion
POD (External-IP)
Kube-Minion
Tunnel
Tunnel
Kube-Master
Static
Route
(Service IP)
Internet
BGP
BGP
XMPPXMPP
T
u
n
n
e
l
MPLSoGRE
MPLSoUDP
MPLSoUDP
MPLSoUDP
OpenContrail Gateway provides gateway function for incoming external traffic into the POD

OPENCONTRAIL KEY COMPONENTS
POD POD
Virtual Networks
Connect Virtual Machines
Gateway Devices
Connect the Virtual to the Physical
Network Policy
Connect Virtual Networks

OPENCONTRAIL NETWORK POLICY
Virtual Network Policies
At a high level of abstraction, applied at the boundaries of virtual networks.
C C C
Green
POD
C C C
Red
POD
Policy
#{Protocol:Port}

OPENCONTRAIL NETWORK FUNCTION SERVICE POLICY
Service Policies
Policy based application of virtual services with scale-out.
Firewall, Intrusion Prevention, Load balancer, Cache, WAN optimizer, proxy, ...
C C C
Green
POD
C C C
Red
POD
Virtual
Service
IDS
Virtual
Service
Cache
Physical
Service
Firewall
Policy
#{Protocol:Port}
#Service{NAT + IDS + Cache + Firewall}

OPENCONTRAIL BUILDING BLOCKS
C C C
C C C
POD Virtual Network
Tenant POD Containers
Virtual Firewall
Physical Gateway Router
Non-Virtualized (Bare Metal) Server
Physical Network (Internet, L3VPN, ...)
POD
Physical
Network
Virtual Load Balancer
Service Chain
Virtualized Server hosting Virtual
Machines

CONTROL PLANE – ROUTE DISTRIBUTION
C C
POD Virtual Network
Tenant POD Containers
Dst Next Hop
G1 VIF – veth
G2 S2 à L6
POD
G1
Minion-1
vRouter Agent
VRF GREEN VN : LABEL 2
vRouter
Forwarding
Plane
POD
G2
Minion-2
vRouter Agent
VRF GREEN VN : LABEL 6
vRouter
Forwarding
Plane
Dst Next Hop
G1 S1 à L2 / L3
G2 VIF
Orchestrator
Configuration
Control
1. On minion node S1, create POD-G1, allocate address to POD (veth)
interface, generate interface route, send route to control node by XMPP.
2. On control node, it receives route, updates routing info base, propagates
route to all other BGP peers, send route to minion 2 over XMPP.
3. On minion node S2, vrouter agent receives route, updates VRF for POD
G1.
4. The same procedure applies for minion S2 to propagate route of POD-G2
to minion node S1.
S1
S2

OPENCONTRAIL KUBERNETES LABELS
OpenContrail Kubernetes
(OpenContrail Labels)
{
Name: “Tier-XYZ”
}
{
Uses:
}
POD
Virtual Network Tier-XYZ
Virtual Network Policy
NetworkTag
NetworkAccessTag
POD
POD
POD
POD POD

OPENCONTRAIL KUBERNETES LABELS
"template":{
"metadata":{
"labels":{
"app":"guestbook",
"name": "frontend",
"uses": "redis"
}
},
Example: Snippet of the POD definition that shows the OpenContrail labels
name and uses
"template":{
"metadata":{
"labels":{
"app":"redis",
"name":"redis",
"role":"slave"
}
},
POD – redis POD – guestbook
NetworkAccessTag
aka: Policy

Steps:
1. export NETWORK_PROVIDER=opencontrail
2. kube-up.sh
More details: GETTING STARTED GUIDE
https://github.com/Juniper/kubernetes/blob/opencontrail-integration/docs/getting-
started-guides/opencontrail.md

OpenContrail supports Salt and Ansible to provision public and private clouds running
Kubernetes clusters
Provisioning of Kubernetes in GCE uses Salt and Contrail modules that have Salt
templates, pillars and grains are:
² opencontrail-kubelet-plugin
² opencontrail-networking-gateway
² opencontrail-networking-master
² opencontrail-networking-minion
² opencontrail-vrouter-kernel

KUBERNETES + OPENCONTRAIL – DEPLOY APPS
guestbook-go is an example provided by Kubernetes that shows a simple multi-tier app.
1. Guestbook controller is the front end GUI that connects to one of the Redis slave instance
2. Redis slave instance gets the IP and Port of the Redis master from SkyDNS
3. Redis slave connects to Redis master and writes the data provided by guestbook UI
Guestbook
Redis Redis
Redis Master
Sky
DNS

guestbook-go can be deployed by following opencontrail.md in the getting-started-guide section
Steps:
1. Get the patch for guestbook-controller, guestbook-redis-slave and redis-master
Patch introduces “name” and “uses” labels in the json files.
2. Apply the patch:
Ex: git apply –stat patch (* execute this from the kubernetes base directory)
git apply –check patch
git apply patch
PATCH URL:
https://github.com/Juniper/contrail-kubernetes/blob/vrouter-manifest/cluster/patch_guest_book

3. Deploy guestbook app
Example:
kubectl create -f guestbook-go/redis-master-controller.json
kubectl create -f guestbook-go/redis-master-service.json
kubectl create -f guestbook-go/redis-slave-controller.json
kubectl create -f guestbook-go/redis-slave-service.json
kubectl create -f guestbook-go/guestbook-controller.json
kubectl create -f guestbook-go/guestbook-service.json

KUBERNETES + OPENCONTRAIL – PERFORMANCE
“When you can measure what you are speaking about, and express it in numbers, you know
something about it;; but when you cannot measure it, when you cannot express it in numbers,
your knowledge is of a meager and unsatisfactory kind: it may be the beginning of knowledge,
but you have scarcely, in your thoughts, advanced to the stage of science.”
-- William Thomson, Lord Kelvin
The performance results from the current production release R2.21 are:
Drum roll please ….

Test Variant Metric Msg Size
(bytes)
OC Rel 2.21
Kernel 3.13
Netperf
TCP_STREAM
VMs on different
compute,
on different VN
Throughput 16384
(3 iter)
9.10 Gbps,
9.11 Gbps,
8.95 Gbps
VMs on different
compute,
on different VN
Throughput 2048
(3 iter)
9.08 Gbps,
8.82 Gbps,
8.89 Gbps
Netperf
TCP_RR
VMs on different
compute,
on different VN
Transaction Rate RR size = 1
(3 iter)
9126.87 tps,
8008.86 tps,
8174.70 tps
Ping Latency Single Packet Ping Latency 56 (84) 2.28 ms
ICMP Flood Ping Latency 56 (84) 0.74 ms
KUBERNETES + OPENCONTRAIL – PERFORMANCE

contrail-info@juniper.net
@opencontrail
@pedro_r_marques,
https://pedrormarques.wordpress.com
@_aniket_
@LachlanEvenson
THANK YOU!

Kubernetes OpenContrail Meetup

More Related Content

What's hot

Viewers also liked

Similar to Kubernetes OpenContrail Meetup

More from Lachlan Evenson

Recently uploaded

Kubernetes OpenContrail Meetup