Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Networking for Kubernetes
A Tale from the Trenches
Cloud Engineering, eBay
Sreekanth Pothanis
Networking is inherently hard!
Complexities of running on openstack
Scale
Multitenancy
Interoperability with Legacy
Private Network model with Openstack SDN
Dedicated kube router provisioned in neutron
Private Networks
Subnet per node
L3 Routed Model
NIPAP as IPAM
Subnet per node
Fully routable pods
Network 2.0
Abstract out network boundaries from nodes to
arbitrary network scopes
IP blocks are allocated to these networ...
Network 2.0
node
pod
pod Network
Scope
IPAM
node
pod
pod
Allocation
Pools
Network
Scope
Allocation
Pool
1 uuid1
2 uuid2
IPAM controller
Cluster admin creates network
scopes + allocation pools
Kubernetes Nodes are
associated with Scopes
IPAM C...
Networking 2.0 -- host
OVS
ARP Proxy
Service to POD
Kube’s default implementation creates LBs on Nodes
Load balance on pods directly
Neutron LBaaS
Pool
Neutron...
eBay Ingress
Application Topology
POOL
Application VIP
VIP
GTM Load Balanced Pool
POOL
VIP
POOL
VIP
Region 1 Region 2 Regi...
Ingress controller
Ingress: myIngress
Status:
VIP-1
IP
GTM name
Ingress
controller
API
Server
LBMS
DNS
GTM
Ingress: myIngr...
DNS
apiVersion: v1
kind: Service
metadata:
annotations:
network.tess.io/kube2udns:
"nginx.spothanis.svc.32.tess.io.t3600t
...
Future work
Network Policy Enforcement
Globally federated Ingress -- SLB based
Upcoming SlideShare
Loading in …5
×

Tectonic Summit 2016: Networking for Kubernetes

610 views

Published on

Sreekanth Pothanis, Cloud Engineering, eBay shares a networking Kubernetes tale from the trenches.

Networking is the hardest component in any ones infrastructure, everything depends on it. Specifically when we have web scale infrastructure with tens of thousands of servers. eBay is investing heavily in Kubernetes and networking again is one of the areas we have the most difficulty with.

During the course of this talk we will go through various approaches we tried to make container networking conform to Kubernetes networking principles, while ensuring that it adapts to the existing networking models our infrastructure supports.

We would also cover how we have automated the process of setting up networking for Kubernetes clusters and how it offers seamless integration with non-Kubernetes workloads.

12/12/16

Published in: Technology
  • Be the first to comment

Tectonic Summit 2016: Networking for Kubernetes

  1. 1. Networking for Kubernetes A Tale from the Trenches Cloud Engineering, eBay Sreekanth Pothanis
  2. 2. Networking is inherently hard! Complexities of running on openstack Scale Multitenancy Interoperability with Legacy
  3. 3. Private Network model with Openstack SDN Dedicated kube router provisioned in neutron Private Networks Subnet per node
  4. 4. L3 Routed Model NIPAP as IPAM Subnet per node Fully routable pods
  5. 5. Network 2.0 Abstract out network boundaries from nodes to arbitrary network scopes IP blocks are allocated to these network scopes Scopes can represent a host or a higher level aggregation Supports legacy and other complex network zoning
  6. 6. Network 2.0 node pod pod Network Scope IPAM node pod pod Allocation Pools Network Scope Allocation Pool 1 uuid1 2 uuid2
  7. 7. IPAM controller Cluster admin creates network scopes + allocation pools Kubernetes Nodes are associated with Scopes IPAM Controller assigns IP based on scope of the node selected by Kube scheduler Pods are annotated with IPs Tessnet plugin configures the pods with annotated IP Kube Scheduler IPAM controller Tess NetPlugin Pod: myPod Host: A Pod:myPod Node A notMyPod myPod 10.10.11.4 Tessnet Pluginkubelet Network Scope1 Allocation Pools 10.10.12.0/22 10.11.1.0/24 Node: A Node: B Node: C Network Scope2 API Server Host: A IP: 10.10.1.4 Pod: myPod Host: A IP: 10.10.1.4 Pod: myPod “network_scope”: “netscope1”
  8. 8. Networking 2.0 -- host OVS ARP Proxy
  9. 9. Service to POD Kube’s default implementation creates LBs on Nodes Load balance on pods directly Neutron LBaaS Pool Neutron LBaaS VIP POD POD POD POD
  10. 10. eBay Ingress Application Topology POOL Application VIP VIP GTM Load Balanced Pool POOL VIP POOL VIP Region 1 Region 2 Region 3 Global Name (omg.g.ebay.com) MONITOR MONITOR MONITOR Application VIPApplication VIP
  11. 11. Ingress controller Ingress: myIngress Status: VIP-1 IP GTM name Ingress controller API Server LBMS DNS GTM Ingress: myIngress Ingress: myIngress Status: VIP-1 IP Ingress: myIngress Status: VIP-1 IP GTM name
  12. 12. DNS apiVersion: v1 kind: Service metadata: annotations: network.tess.io/kube2udns: "nginx.spothanis.svc.32.tess.io.t3600t INtAt10.149.4.27" Kube2DNS controller DNS API Server
  13. 13. Future work Network Policy Enforcement Globally federated Ingress -- SLB based

×