The document compares various Container Network Interface (CNI) plugins for Kubernetes, detailing the architecture and components involved in Kubernetes networking. It outlines the requirements and functionality of CNI plugins, highlighting specific implementations such as Flannel, Calico, and AWS VPC CNI. Additionally, it provides insights into their performance, installation, configuration, and unique capabilities.

1. Comparison of existing CNI
plugins for Kubernetes
Adam Hamsik
adam.hamsik@lablabs.io

2. Labyrinth Labs
Rock-solid infrastructure and DevOps
● Building rock-solid and secure foundations for all your digital operations. Our
mission is to let you focus on your business without ever needing to worry about
technical issues again.
● Making you ready for growing traffic, safe against new security vulnerabilities and
data-loss.
2

3. Kubernetes Architecture
● Master acts as the control plane for Kubernetes. Kubernetes master is
responsible for maintaining the desired state for cluster. The “master” refers to a
collection of processes managing and is responsible at a minimum for running
scheduler, cluster controller, API Server.
● Nodes acts as the “worker” of Kubernetes cluster. The nodes in a cluster are the
machines (VMs, physical servers, etc) that run your applications and cloud
workflows. The Kubernetes master controls each node.
3

4. Kubernetes Architecture vol. 2
4

5. Kubernetes Master
● Kube-apiserver
● Etcd
● Kube-controller-manager
● Cloud-controller-manager
● Kube-scheduler
● CNI Plugin Control Plane
5

6. Kubernetes Node
● Kubelet
● Kube-proxy
● Container runtime
● CNI plugin
6

7. Kubernetes Plugins
● CRI (Container Runtime Interface) is an interface used for communication
between kubelet and container runtime
○ Docker
○ Cri-o
○ rkt
● CSI (Container Storage Interface) is an unified interface between container
orchestration systems and storage vendors
○ EBS
○ NetApp
○ Ceph
● CNI (Container Networking Interface) is an interface between network namespace
and container runtime
7

8. What is a CNI Plugin ?
● Simplest interface between container runtime and network
implementation
● Originated at CoreOS as part of Rkt Container runtime
● CNCF project
● Repository: https://github.com/containernetworking/cni
● Specification: the API between runtimes and network plugins
8

9. Kubernetes CNI Requirements
● pods on a node can communicate with all pods on all nodes without NAT
● agents on a node (e.g. system daemons, kubelet) can communicate with all pods
on that node
● pods in the host network of a node can communicate with all pods on all nodes
without NAT
● containers within a Pod share their network namespaces
○ You can access other container services through localhost
9

10. Kubernetes CNI Implementation
● Containers in a pod exists within network namespace and share same IP -
○ this setup allow for intrapod communication over localhost
● Pod are given cluster unique IP for the duration of its lifecycle
● Services are given a persistent cluster unique IP that spans Pods lifecycle
● External connectivity is generally handled by an infrastructure (default GWs)
10

11. Kubernetes CNI Workflow
● Container runtime executes CNI plugin with given config
● CNI Plugin executes IPAM module to configure IP addresses on a configured
interface
11

12. Demo Time vol.1
● Show
● CNI Plugin executes IPAM module to configure IP addresses on a configured
interface
12

13. List of existing CNI Plugins
● Overlay Network plugins
○ Flannel
○ Weave-net
● Routed Network plugins
○ AWS-VPC
○ Kube-router
○ Calico
○ Canal
● IPAM modules
○ Dhcp
○ host-local
● Multi CNI plugins
○ Damn
○ Multus
○ CNI-Genie
13

14. Flannel - Detailed description
● Developed by CoreOS
● Relatively easy to install/configure
● Setups layer 3 IPv4 overlay network over multiple backends
○ VXLAN
○ Host-gw
○ UDP
○ IPIP(Experimental)
○ IPSec(Experimental)
● Doesn’t support Network policies
14

15. Flannel - Detailed description
15https://www.slideshare.net/lorispack/using-coreos-flannel-for-docker-networking

16. Calico - Detailed description
● Best overall choice for performance, flexibility and power
● Calico requires Layer 3 network which is using BGP protocol to route packets
between hosts and pods
● Network Policies can be created to manage network access between pods
16

17. Calico - Detailed description
17https://www.docker.com/blog/networking-in-docker-enterprise-edition-2-0/

18. Kube-router - Detailed description
● Turnkey solutions for:
○ Pod Networking
■ kube-router handles Pod networking efficiently with direct routing thanks to the BGP
protocol and the GoBGP Go library.
○ IPVS/LVS Service Proxy
■ Kube-router uses battle-tested Linux LVS/IPVS to provide a service proxy and provides rich
set of scheduling options and enables advanced use-cases like DSR
○ Network Load Balancer
■ Kube-router has the ability to advertise service VIP's to L3 fabric BGP peers. So you can do
network load balancing with ECMP.
18

19. AWS VPC CNI - Detailed description
● Plugin used only in AWS deployments offers
○ High throughput
○ High availability
○ Low latency
○ Minimal network jitter
● Allocates/Attaches new AWS Elastic Networking Interfaces (ENIs) to nodes
○ There is a maximum number of interfaces attachable to running instance
○ VPC flow logs
○ VPC routing policies
○ security groups
19

20. Questions ?
adam.hamsik@lablabs.io
www.lablabs.io
20

21. More information
● http://events17.linuxfoundation.org/sites/events/files/slides/Container%20Netwo
rking%20Deep%20Dive.pdf
● https://thenewstack.io/hackers-guide-kubernetes-networking/
● https://www.cncf.io/wp-content/uploads/2017/11/Introduction-to-CNI-2.pdf
● https://itnext.io/benchmark-results-of-kubernetes-network-plugins-cni-over-
10gbit-s-network-36475925a560
● https://www.altoros.com/blog/kubernetes-networking-writing-your-own-simple-
cni-plug-in-with-bash/
21