This document compares existing CNI plugins for Kubernetes and provides descriptions of popular plugins like Flannel, Calico, Kube-router, and AWS VPC CNI. It explains that CNI plugins provide the interface between container runtimes and network implementations, and describes the CNI workflow and requirements for pod networking in Kubernetes.

1. Comparison of existing CNI
plugins for Kubernetes
Adam Hamsik
adam.hamsik@lablabs.io

2. Labyrinth Labs
Rock-solid infrastructure and DevOps
● Building rock-solid and secure foundations for all your digital operations. Our
mission is to let you focus on your business without ever needing to worry
about technical issues again.
● Making you ready for growing traffic, safe against new security vulnerabilities
and data-loss.
2

3. Kubernetes Architecture
● Master acts as the control plane for Kubernetes. Kubernetes master is
responsible for maintaining the desired state for cluster. The “master” refers to
a collection of processes managing and is responsible at a minimum for running
scheduler, cluster controller, API Server.
● Nodes acts as the “worker” of Kubernetes cluster. The nodes in a cluster are the
machines (VMs, physical servers, etc) that run your applications and cloud
workflows. The Kubernetes master controls each node.
3

4. Kubernetes Architecture vol. 2
4

5. Kubernetes Master
● Kube-apiserver
● Etcd
● Kube-controller-manager
● Cloud-controller-manager
● Kube-scheduler
● CNI Plugin Control Plane
5

6. Kubernetes Node
● Kubelet
● Kube-proxy
● Container runtime
● CNI plugin
6

7. Kubernetes Plugins
● CRI (Container Runtime Interface) is an interface used for communication
between kubelet and container runtime
○ Docker
○ Cri-o
○ rkt
● CSI (Container Storage Interface) is an unified interface between container
orchestration systems and storage vendors
○ EBS
○ NetApp
○ Ceph
● CNI (Container Networking Interface) is an interface between network
namespace and container runtime
7

8. What is a CNI Plugin ?
● Simplest interface between container runtime and network
implementation
● Originated at CoreOS as part of Rkt Container runtime
● CNCF project
● Repository: https://github.com/containernetworking/cni
● Specification: the API between runtimes and network plugins
8

9. Kubernetes CNI Requirements
● pods on a node can communicate with all pods on all nodes without NAT
● agents on a node (e.g. system daemons, kubelet) can communicate with all
pods on that node
● pods in the host network of a node can communicate with all pods on all nodes
without NAT
● containers within a Pod share their network namespaces
○ You can access other container services through localhost
9

10. Kubernetes CNI Implementation
● Containers in a pod exists within network namespace and share same IP -
○ this setup allow for intrapod communication over localhost
● Pod are given cluster unique IP for the duration of its lifecycle
● Services are given a persistent cluster unique IP that spans Pods lifecycle
● External connectivity is generally handled by an infrastructure (default GWs)
10

11. Kubernetes CNI Workflow
● Container runtime executes CNI plugin with given config
● CNI Plugin executes IPAM module to configure IP addresses on a configured
interface
11

12. Demo Time vol.1
● Show
● CNI Plugin executes IPAM module to configure IP addresses on a configured
interface
12

13. List of existing CNI Plugins
● Overlay Network plugins
○ Flannel
○ Weave-net
● Routed Network plugins
○ AWS-VPC
○ Kube-router
○ Calico
○ Canal
● IPAM modules
○ Dhcp
○ host-local
● Multi CNI plugins
○ Damn
○ Multus
○ CNI-Genie
13

14. Flannel - Detailed description
● Developed by CoreOS
● Relatively easy to install/configure
● Setups layer 3 IPv4 overlay network over multiple backends
○ VXLAN
○ Host-gw
○ UDP
○ IPIP(Experimental)
○ IPSec(Experimental)
● Doesn’t support Network policies
14

15. Flannel - Detailed description
15
https://www.slideshare.net/lorispack/using-coreos-flannel-for-docker-networking

16. Calico - Detailed description
● Best overall choice for performance, flexibility and power
● Calico requires Layer 3 network which is using BGP protocol to route packets
between hosts and pods
● Network Policies can be created to manage network access between pods
16

17. Calico - Detailed description
17https://www.docker.com/blog/networking-in-docker-enterprise-edition-2-0/

18. Kube-router - Detailed description
● Turnkey solutions for:
○ Pod Networking
■ kube-router handles Pod networking efficiently with direct routing thanks to the BGP
protocol and the GoBGP Go library.
○ IPVS/LVS Service Proxy
■ Kube-router uses battle-tested Linux LVS/IPVS to provide a service proxy and provides rich
set of scheduling options and enables advanced use-cases like DSR
○ Network Load Balancer
■ Kube-router has the ability to advertise service VIP's to L3 fabric BGP peers. So you can do
network load balancing with ECMP.
18

19. AWS VPC CNI - Detailed description
● Plugin used only in AWS deployments offers
○ High throughput
○ High availability
○ Low latency
○ Minimal network jitter
● Allocates/Attaches new AWS Elastic Networking Interfaces (ENIs) to nodes
○ There is a maximum number of interfaces attachable to running instance
○ VPC flow logs
○ VPC routing policies
○ security groups
19

20. Questions ?
adam.hamsik@lablabs.io
www.lablabs.io
20

21. More information
● http://events17.linuxfoundation.org/sites/events/files/slides/Container%20Net
working%20Deep%20Dive.pdf
● https://thenewstack.io/hackers-guide-kubernetes-networking/
● https://www.cncf.io/wp-content/uploads/2017/11/Introduction-to-CNI-2.pdf
● https://itnext.io/benchmark-results-of-kubernetes-network-plugins-cni-over-1
0gbit-s-network-36475925a560
● https://www.altoros.com/blog/kubernetes-networking-writing-your-own-simpl
e-cni-plug-in-with-bash/
21