An introduction to Container Network Interface (CNI), including what problems it want solve and how it works. Also contains a example about how to write a simple CNI plugin with golang

1. Introduction to CNI
(Container Network Interface)
Hwchiu (Hung-Wei Chiu)
Linkernetworks.com

2. Who Am I
• Hung-Wei Chiu ( )
• hwchiu@linkernetworks.com
• Blog: hwchiu.com
• Experience
• Software Engineer at Linker Networks
• Co-Founder of SDNDS-TW (Software Defined Network)
• Co-Found of CUTNG(Cloud Native Taiwan User Group)
• Open Source Experience
• SDN Related Projects (ONOS/Mininet/Floodlight)
• OVS-CNI

3. Outline
• Introduction to CNI
• How to write a CNI in golang

4. Before Taking About The CNI

5. Do You Heard Before?
• Linux network namespace
• Kernel function
• Docker
• Docker Network
• Bridge/Host..etc
• Kubernetes
• Flannel/Calico/Weave..etc

6. So, What Is Docker?
How It Works ?

8. A Simple HTTP Server
• docker run --name my-nginx -p 8080:80 nginx
• Use the localhost:8080 to communicate with nginx.
• How it works ?

9. Step By Step
1. Create a Linux Bridge
Linux Host
br0

10. Step By Step
1. Create a Linux Bridge
2. Create a Container
Linux Host
br0
Container
(Nginx)

11. Step By Step
1. Create a Linux Bridge
2. Create a Container
3. Create a veth pair
Linux Host
br0
Container
(Nginx)
veth234 veth123

12. Step By Step
1. Create a Linux Bridge
2. Create a Container
3. Create a veth pair
4. Attach veth pari to
container and bridge
(also rename)
Linux Host
br0
Container
(Nginx)
veth234
eth0

13. Step By Step
1. Create a Linux Bridge
2. Create a Container
3. Create a veth pair
4. Attach veth pari to container
and bridge (also rename)
5. Assign an IP address to
container
Linux Host
br0
Container
(Nginx)
veth234
eth0172.16.2.5/24

14. Step By Step
1. Create a Linux Bridge
2. Create a Container
3. Create a veth pair
4. Attach veth pari to container and
bridge (also rename)
5. Assign an IP address to container
6. Setup a iptablses rule for 8080:80
Linux Host
br0
Container
(Nginx)
veth234
eth0172.16.2.5/24

15. In The Previous Example
• The networking part is handled by the linux network namepsace (ns)
• veth is used to connect two different ns

16. Do We Have Any Other Options ?
• Docker run –network=…
• Bridge (bydefault)
• Host
• ContainerID
• Docker networks (CNM)
• Create your network.

17. How About Other Container System ?
• LXC
• rkt
• Mesos
• Kubernetes
• …etc

18. We Need To Make It Simple
• Develop once, run everywhere
• That’s CNI (Container Network Interface)
• https://github.com/containernetworking/cni
• Developed by go language

19. What Is CNI
• A CNCF (Cloud Native Computing Foundation) project
• For Linux Containers
• Consists of a specification and libraries for writing plugins.
• Only care about networking connectivity of containers
• Create/Remove

20. Who Use CNI
• rkt - container engine
• Kubernetes - a system to simplify container operations
• OpenShift - Kubernetes with additional enterprise features
• Cloud Foundry - a platform for cloud applications
• Apache Mesos - a distributed systems kernel
• Amazon ECS - a highly scalable, high performance container management
service

21. Network Connectivity
• Use the previous docker example, The CNI will do
• Create the Linux Bridge
• Create the veth and attach to the container (ns)
• Find a IP address and assign the IP to the Linux Bridge
• Other staffs (You can do anything you want)

22. Others CNI
• SR-IOV (Physical NIC to container)
• OVS (Use OpenvSwitch rather than Linux Bridge)
• Flannel (Support tunnel via UDP/VXLAN)
• MacVlan/IPVlan
• PTP
• Vlan
• …etc

23. So, How To Develop a CNI Plugin?

24. Let’s See A Example

25. First
• Assume we have already implemented a CNI called simple-cni
• Assume we have create a network namespace (ns) vir the following
command
• ip netns add ns1
• We have a json config contains the information we need.
• {
”name”: “simple-cni”
}

26. Second
• Execute the following command
• sudo
CNI_COMMAND=ADD
CNI_CONTAINERID=ns1
CNI_NETNS=/var/run/netns/ns1
CNI_IFNAME=eth10 CNI_PATH=`pwd`
./simple-cni < config

27. Explain
• COMMAND
• ADD/DELETE/VERSION
• CONTAINERID
• Just a ID…
• NETNS
• The location of ns
• IFNAME
• NIC name in the container
• PATH
• Where to find the binary
• Stdin
• Just a json config

28. What The Simple-CNI do
• Load the information from the config (bridge name, IP address)
• Create a Linux Bridge
• Create a veth and attach to $NETNS
• Rename the NIC to $IFNAME
• Set the IP address to the NIC (We call it IPAM )

29. It’s Go Time
https://github.com/hwchiu/CNI_Tutorial_2018

30. Skeleton
• We should implement two function (Add/Delete) for CNI_COMMAND
• We will get those data via skel.CmdArgs

33. First
• We should add a special function init

34. First(Cont’d)
• Decode the StdinData to out structure.
• You can define any data you want.
• In my example. I get the bridge name and IP address from the config.

35. First
• Decode the StdinData to out structure.

36. Create a Linux Bridge
• We have to ways to create a linux bridge
• Call the linux command (brctl addbr ….)
• Use the netlink to create a linux bridge
• We use this method our example.

37. Create a Linux Bridge
• Prepare a bridge object netlink.Bridge{}
• Create a bridge via netlink.LinkAdd
• brctl add br
• Up the Linux bridge via netlink.LinkSetUp
• ifconfig xxx up

39. Second
• Create a veth pair via netlink.Veth
• Setup the veth via netlink.LinkSetUp
• Move one side of veth to another ns via netlink.LinkSetNsFd
• Setup the NICs of the veth via netlink.LinkSetUp

40. Second(cont’d)
• We can create a veth on the host ns and move one side into container ns.
• Or, we can create a veth on the container ns and move one side into host ns.
• Choose any approach you like.

41. Second
• The better way is to use the function provide by
containernetworking/plugins/pkg/ip package.

42. The simple way.
• Get the NS Object from the ns.GetNs
• Call the SetupVeth on the continaer ns.

44. Third.
• We need to attach the one side of the veth into the Linux bridge
• First, get the Link Object via netlink.LinkByName
• Second, attach the link to bridge via netlink.LinkSetMaster

45. Now
• We have created the Linux bridge
• We have create a veth and connect the host ns and container ns.
• We also attach the veth to the Linux Bridge
Linux Host
br0
Network
Namespace
veth234
eth0

46. Next
• We need to handle the IPAM (IP address management)
• In this example, we get the IP address from the config.
• We can set the ip address via netlink.AddrAdd

48. Let’s Demo Now.

49. Other Things About CNI
• Build-in IPAM
• Host
• DHCP
• DIY

50. Complicated Config Examples

51. By The Way

53. Q&A