Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Writing the Container Network Interface(CNI) plugin in golang

1,106 views

Published on

An introduction to Container Network Interface (CNI), including what problems it want solve and how it works.
Also contains a example about how to write a simple CNI plugin with golang

Published in: Technology
  • Be the first to comment

Writing the Container Network Interface(CNI) plugin in golang

  1. 1. Introduction to CNI (Container Network Interface) Hwchiu (Hung-Wei Chiu) Linkernetworks.com
  2. 2. Who Am I • Hung-Wei Chiu ( ) • hwchiu@linkernetworks.com • Blog: hwchiu.com • Experience • Software Engineer at Linker Networks • Co-Founder of SDNDS-TW (Software Defined Network) • Co-Found of CUTNG(Cloud Native Taiwan User Group) • Open Source Experience • SDN Related Projects (ONOS/Mininet/Floodlight) • OVS-CNI
  3. 3. Outline • Introduction to CNI • How to write a CNI in golang
  4. 4. Before Taking About The CNI
  5. 5. Do You Heard Before? • Linux network namespace • Kernel function • Docker • Docker Network • Bridge/Host..etc • Kubernetes • Flannel/Calico/Weave..etc
  6. 6. So, What Is Docker? How It Works ?
  7. 7. A Simple HTTP Server • docker run --name my-nginx -p 8080:80 nginx • Use the localhost:8080 to communicate with nginx. • How it works ?
  8. 8. Step By Step 1. Create a Linux Bridge Linux Host br0
  9. 9. Step By Step 1. Create a Linux Bridge 2. Create a Container Linux Host br0 Container (Nginx)
  10. 10. Step By Step 1. Create a Linux Bridge 2. Create a Container 3. Create a veth pair Linux Host br0 Container (Nginx) veth234 veth123
  11. 11. Step By Step 1. Create a Linux Bridge 2. Create a Container 3. Create a veth pair 4. Attach veth pari to container and bridge (also rename) Linux Host br0 Container (Nginx) veth234 eth0
  12. 12. Step By Step 1. Create a Linux Bridge 2. Create a Container 3. Create a veth pair 4. Attach veth pari to container and bridge (also rename) 5. Assign an IP address to container Linux Host br0 Container (Nginx) veth234 eth0172.16.2.5/24
  13. 13. Step By Step 1. Create a Linux Bridge 2. Create a Container 3. Create a veth pair 4. Attach veth pari to container and bridge (also rename) 5. Assign an IP address to container 6. Setup a iptablses rule for 8080:80 Linux Host br0 Container (Nginx) veth234 eth0172.16.2.5/24
  14. 14. In The Previous Example • The networking part is handled by the linux network namepsace (ns) • veth is used to connect two different ns
  15. 15. Do We Have Any Other Options ? • Docker run –network=… • Bridge (bydefault) • Host • ContainerID • Docker networks (CNM) • Create your network.
  16. 16. How About Other Container System ? • LXC • rkt • Mesos • Kubernetes • …etc
  17. 17. We Need To Make It Simple • Develop once, run everywhere • That’s CNI (Container Network Interface) • https://github.com/containernetworking/cni • Developed by go language
  18. 18. What Is CNI • A CNCF (Cloud Native Computing Foundation) project • For Linux Containers • Consists of a specification and libraries for writing plugins. • Only care about networking connectivity of containers • Create/Remove
  19. 19. Who Use CNI • rkt - container engine • Kubernetes - a system to simplify container operations • OpenShift - Kubernetes with additional enterprise features • Cloud Foundry - a platform for cloud applications • Apache Mesos - a distributed systems kernel • Amazon ECS - a highly scalable, high performance container management service
  20. 20. Network Connectivity • Use the previous docker example, The CNI will do • Create the Linux Bridge • Create the veth and attach to the container (ns) • Find a IP address and assign the IP to the Linux Bridge • Other staffs (You can do anything you want)
  21. 21. Others CNI • SR-IOV (Physical NIC to container) • OVS (Use OpenvSwitch rather than Linux Bridge) • Flannel (Support tunnel via UDP/VXLAN) • MacVlan/IPVlan • PTP • Vlan • …etc
  22. 22. So, How To Develop a CNI Plugin?
  23. 23. Let’s See A Example
  24. 24. First • Assume we have already implemented a CNI called simple-cni • Assume we have create a network namespace (ns) vir the following command • ip netns add ns1 • We have a json config contains the information we need. • { ”name”: “simple-cni” }
  25. 25. Second • Execute the following command • sudo CNI_COMMAND=ADD CNI_CONTAINERID=ns1 CNI_NETNS=/var/run/netns/ns1 CNI_IFNAME=eth10 CNI_PATH=`pwd` ./simple-cni < config
  26. 26. Explain • COMMAND • ADD/DELETE/VERSION • CONTAINERID • Just a ID… • NETNS • The location of ns • IFNAME • NIC name in the container • PATH • Where to find the binary • Stdin • Just a json config
  27. 27. What The Simple-CNI do • Load the information from the config (bridge name, IP address) • Create a Linux Bridge • Create a veth and attach to $NETNS • Rename the NIC to $IFNAME • Set the IP address to the NIC (We call it IPAM )
  28. 28. It’s Go Time https://github.com/hwchiu/CNI_Tutorial_2018
  29. 29. Skeleton • We should implement two function (Add/Delete) for CNI_COMMAND • We will get those data via skel.CmdArgs
  30. 30. First • We should add a special function init
  31. 31. First(Cont’d) • Decode the StdinData to out structure. • You can define any data you want. • In my example. I get the bridge name and IP address from the config.
  32. 32. First • Decode the StdinData to out structure.
  33. 33. Create a Linux Bridge • We have to ways to create a linux bridge • Call the linux command (brctl addbr ….) • Use the netlink to create a linux bridge • We use this method our example.
  34. 34. Create a Linux Bridge • Prepare a bridge object netlink.Bridge{} • Create a bridge via netlink.LinkAdd • brctl add br • Up the Linux bridge via netlink.LinkSetUp • ifconfig xxx up
  35. 35. Second • Create a veth pair via netlink.Veth • Setup the veth via netlink.LinkSetUp • Move one side of veth to another ns via netlink.LinkSetNsFd • Setup the NICs of the veth via netlink.LinkSetUp
  36. 36. Second(cont’d) • We can create a veth on the host ns and move one side into container ns. • Or, we can create a veth on the container ns and move one side into host ns. • Choose any approach you like.
  37. 37. Second • The better way is to use the function provide by containernetworking/plugins/pkg/ip package.
  38. 38. The simple way. • Get the NS Object from the ns.GetNs • Call the SetupVeth on the continaer ns.
  39. 39. Third. • We need to attach the one side of the veth into the Linux bridge • First, get the Link Object via netlink.LinkByName • Second, attach the link to bridge via netlink.LinkSetMaster
  40. 40. Now • We have created the Linux bridge • We have create a veth and connect the host ns and container ns. • We also attach the veth to the Linux Bridge Linux Host br0 Network Namespace veth234 eth0
  41. 41. Next • We need to handle the IPAM (IP address management) • In this example, we get the IP address from the config. • We can set the ip address via netlink.AddrAdd
  42. 42. Let’s Demo Now.
  43. 43. Other Things About CNI • Build-in IPAM • Host • DHCP • DIY
  44. 44. Complicated Config Examples
  45. 45. By The Way
  46. 46. Q&A

×