Successfully reported this slideshow.
Your SlideShare is downloading. ×

Overview of kubernetes network functions

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 89 Ad

Overview of kubernetes network functions

Download to read offline

In this slides, I briefly introduce the network function in the kubernetes and explain how kubernetes implement them.
Those function includes the container network interface (CNI) and kubernetes service.
In the last, I introduce the multus CNI which is designed for multiple networks in the container and it's necessary in some use case, such as SDN/NFV/5G

In this slides, I briefly introduce the network function in the kubernetes and explain how kubernetes implement them.
Those function includes the container network interface (CNI) and kubernetes service.
In the last, I introduce the multus CNI which is designed for multiple networks in the container and it's necessary in some use case, such as SDN/NFV/5G

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to Overview of kubernetes network functions (20)

Advertisement

More from 宏瑋 邱 (20)

Recently uploaded (20)

Advertisement

Overview of kubernetes network functions

  1. 1. Kubernetes networkoverview
  2. 2. COSCUP2018 x openSUSE.Asia GNOME.Asia I am Hung-Wei Chiu Co-organizer of SDNDS-TW Co-organizer of CNTUUG I love Linux Network/Kubernetes/SDN You can find me at: blog.hwchiu.com
  3. 3. COSCUP2018 x openSUSE.Asia GNOME.Asia Outline What network functions about k8s How does above function implement What is the challenge about k8s
  4. 4. COSCUP2018 x openSUSE.Asia GNOME.Asia What network function kubernetes providers?
  5. 5. COSCUP2018 x openSUSE.Asia GNOME.Asia What Container Network ○ Connectivity ○ DNS Kubernetes services
  6. 6. COSCUP2018 x openSUSE.Asia GNOME.Asia Do You Know How Container Works?
  7. 7. COSCUP2018 x openSUSE.Asia GNOME.Asia Containersvs.VMs Containers are isolated, but share OS and where appropriate bins/libraries
  8. 8. COSCUP2018 x openSUSE.Asia GNOME.Asia HowDockerWorks We know docker is isolated, but how does it works? Linux kernel support the Namespaces mechanisms to partition kernel resources to different processes
  9. 9. COSCUP2018 x openSUSE.Asia GNOME.Asia HowDockerWorks Mount namespaces IPC namespaces PID namespaces Network namespaces User namespaces UTS namespaces ○ Unix Time System
  10. 10. COSCUP2018 x openSUSE.Asia GNOME.Asia NetworkNamespace Isolate the network functions. Including the ○ Network interfaces ○ Routing rules ○ Netfilter (iptables)
  11. 11. COSCUP2018 x openSUSE.Asia GNOME.Asia Linux Host
  12. 12. COSCUP2018 x openSUSE.Asia GNOME.Asia docker0 Linux Host Linux Host
  13. 13. COSCUP2018 x openSUSE.Asia GNOME.Asia docker0 docker0 ns1 Linux Host Linux Host Linux Host
  14. 14. COSCUP2018 x openSUSE.Asia GNOME.Asia docker0 docker0 docker0 ns1 ns1 veth1 veth0 Linux Host Linux Host Linux Host Linux Host
  15. 15. COSCUP2018 x openSUSE.Asia GNOME.Asia docker0 docker0 docker0docker0 ns1 ns1ns1 vth1 vth0 veth eth0 Linux Host Linux Host Linux Host Linux HostLinux Host
  16. 16. COSCUP2018 x openSUSE.Asia GNOME.Asia Before we talk about service, we must know why service exist.
  17. 17. COSCUP2018 x openSUSE.Asia GNOME.Asia Pods/Deployments We can deploy our applications as a containers in the kubernetes. There’re many kind of the container we can deploy ○ Pod ○ Deployment ○ Statefulset ○ DaemonSet
  18. 18. COSCUP2018 x openSUSE.Asia GNOME.Asia Deployment Node1 Nginx Node2 Nginx Node3 Nginx Kubernetes Cluster Deployment: ○ Ngnix ○ Replica: 3 10.123.234.56 10.123.234.57 10.123.234.58
  19. 19. COSCUP2018 x openSUSE.Asia GNOME.Asia Access How we application access those Nginx servers? By IP address ○ 10.123.234.56:80 ○ 10.123.234.57:80 ○ 10.123.234.58:80 What’s the problem
  20. 20. COSCUP2018 x openSUSE.Asia GNOME.Asia Deployment Node1 Nginx Node2 Nginx Node3 Nginx Kubernetes Cluster Deployment: ○ Ngnix ○ Replica: 3 10.123.234.56 10.123.234.57 10.123.234.58
  21. 21. COSCUP2018 x openSUSE.Asia GNOME.Asia Deployment Node1 Nginx Node2 Nginx Node3 Nginx Kubernetes Cluster Deployment: ○ Ngnix ○ Replica: 3 10.123.234.56 10.123.234.57 10.123.234.75
  22. 22. COSCUP2018 x openSUSE.Asia GNOME.Asia Access How we application access those Nginx servers? By IP address ○ 10.123.234.56:80 ○ 10.123.234.57:80 ○ 10.123.234.58:80 ○ 10.123.234.75:80 It’s not easy for our application to handle those ip-changed situation.
  23. 23. COSCUP2018 x openSUSE.Asia GNOME.Asia The Service is used to solve this problem.
  24. 24. COSCUP2018 x openSUSE.Asia GNOME.Asia Service Node1 Nginx Node2 Nginx Node3 Nginx Kubernetes Cluster 10.123.234.56 10.123.234.57 10.123.234.58 App Service Nginx
  25. 25. COSCUP2018 x openSUSE.Asia GNOME.Asia Service Application to Service ○ We use the DNS to access the service. ○ $(service).$(namespace).cluster.local Service to Pods ○ Service maintains all IP addresses of all Pods. ○ We call it endpoints
  26. 26. COSCUP2018 x openSUSE.Asia GNOME.Asia Service Node1 Nginx Node2 Nginx Node3 Nginx Kubernetes Cluster 10.123.234.56 10.123.234.57 10.123.234.58 App Service Nginx nginx.default endpoints
  27. 27. COSCUP2018 x openSUSE.Asia GNOME.Asia How does kubernetes implements those functions?
  28. 28. COSCUP2018 x openSUSE.Asia GNOME.Asia What Container Network Connectivity ○ Container Network Interface(CNI) Kubernetes Services ○ There’re many implementation we can choose
  29. 29. COSCUP2018 x openSUSE.Asia GNOME.Asia ContainerNetworkInterface
  30. 30. COSCUP2018 x openSUSE.Asia GNOME.Asia ContainerNetworkInterface Cloud Native Computing Foundation Project Consists of a specification and libraries. Configure network interfaces in Linux containers Concerns itself only with network connectivity of containers ○ Create/Remove
  31. 31. COSCUP2018 x openSUSE.Asia GNOME.Asia ContainerNetworkInterface Removing allocated resources when the container is deleted
  32. 32. COSCUP2018 x openSUSE.Asia GNOME.Asia Who is using CNI?
  33. 33. COSCUP2018 x openSUSE.Asia GNOME.Asia FromtheGITHUB l rkt - container engine l Kubernetes - a system to simplify container operations l OpenShift - Kubernetes with additional enterprise features l Cloud Foundry - a platform for cloud applications l Apache Mesos - a distributed systems kernel l Amazon ECS - a highly scalable, high performance container management service
  34. 34. COSCUP2018 x openSUSE.Asia GNOME.Asia So, How to use the CNI?
  35. 35. COSCUP2018 x openSUSE.Asia GNOME.Asia StepbyStep Create a kubernetes cluster Setup your CNI plugin Deploy your first Pod
  36. 36. COSCUP2018 x openSUSE.Asia GNOME.Asia Just follow the installation to install the kubernetes
  37. 37. COSCUP2018 x openSUSE.Asia GNOME.Asia How do we install the CNI?
  38. 38. COSCUP2018 x openSUSE.Asia GNOME.Asia
  39. 39. COSCUP2018 x openSUSE.Asia GNOME.Asia Handbyhand In the kubelet, we have the following parameters for CNI. --cni-bin-dir ○ /opt/cni/bin --cni-conf-dir ○ /etc/cni/net.d/ We should config the CNI for every k8s nodes.
  40. 40. COSCUP2018 x openSUSE.Asia GNOME.Asia
  41. 41. COSCUP2018 x openSUSE.Asia GNOME.Asia Let Deploy a Pod
  42. 42. COSCUP2018 x openSUSE.Asia GNOME.Asia Beforewestart Pod ○ A collection of containers
  43. 43. COSCUP2018 x openSUSE.Asia GNOME.Asia Steps Load the Pod config ○ Multiple containers Find a node to deploy the pod Create a Pause container
  44. 44. COSCUP2018 x openSUSE.Asia GNOME.Asia Linux Host Pause Container
  45. 45. COSCUP2018 x openSUSE.Asia GNOME.Asia Steps Load the Pod config ○ Multiple containers Find a node to deploy the pod Create a Pause container Load the CNI config
  46. 46. COSCUP2018 x openSUSE.Asia GNOME.Asia Linux Host Pause Container Load the CNI config from /etc/cni/net.d/…
  47. 47. COSCUP2018 x openSUSE.Asia GNOME.Asia
  48. 48. COSCUP2018 x openSUSE.Asia GNOME.Asia Steps Load the Pod config ○ Multiple containers Find a node to deploy the pod Create a Pause container Load the CNI config Execute the CNI
  49. 49. COSCUP2018 x openSUSE.Asia GNOME.Asia Callthebinary Load the binary from the config Find the binary from the /opt/cni/bin/
  50. 50. COSCUP2018 x openSUSE.Asia GNOME.Asia Linux Host Pause Container Call the /opt/cni/bin/flannel Network Connectivity
  51. 51. COSCUP2018 x openSUSE.Asia GNOME.Asia Steps Load the Pod config ○ Multiple containers Find a node to deploy the pod Create a Pause container Load the CNI config Execute the CNI Create target containers and attach to Pause container
  52. 52. COSCUP2018 x openSUSE.Asia GNOME.Asia Linux Host Pause Container Call the /opt/cni/bin/flannel Network Connectivity Busybox
  53. 53. COSCUP2018 x openSUSE.Asia GNOME.Asia Attachtoothercontainer docker run –net=… ○ bridge ○ host ○ containerID
  54. 54. COSCUP2018 x openSUSE.Asia GNOME.Asia Linux Host Pause Container Call the /opt/cni/bin/flannel Network Connectivity Busybox Pod
  55. 55. COSCUP2018 x openSUSE.Asia GNOME.Asia
  56. 56. COSCUP2018 x openSUSE.Asia GNOME.Asia Kubernetesservice There’re three implementation now. ○ User-space Kube-Proxy ○ Kernel-space iptables (default) ○ Kernel-space ipvs We use the iptables to explain how service(ClusterIP) works
  57. 57. COSCUP2018 x openSUSE.Asia GNOME.Asia Service Node1 Nginx Node2 Nginx Node3 Nginx Kubernetes Cluster 10.123.234.56 10.123.234.57 10.123.234.58 App Service Nginx nginx.default endpoints
  58. 58. COSCUP2018 x openSUSE.Asia GNOME.Asia LAB Get the Service ○ kubectl get service
  59. 59. COSCUP2018 x openSUSE.Asia GNOME.Asia LAB Get the endpoints ○ kubectl get endpoints
  60. 60. COSCUP2018 x openSUSE.Asia GNOME.Asia LAB Get the pod ip address ○ kubectl get pods –o wide
  61. 61. COSCUP2018 x openSUSE.Asia GNOME.Asia Now.Trytofetchthenginx Service name is k8s-nginx-cluster Use the nslookup to lookup the IP ○ nslookup k8s-nginx-cluster ○ nslookup k8s-nginx-cluster.default ■ default is the namespace of the service
  62. 62. COSCUP2018 x openSUSE.Asia GNOME.Asia
  63. 63. COSCUP2018 x openSUSE.Asia GNOME.Asia Trytofetchthenginx In the pod: curl k8s-nginx-cluster
  64. 64. COSCUP2018 x openSUSE.Asia GNOME.Asia Howitworks? First, we can get the VIP from the hostname. ○ It’s just a DNS request. Second, we can access the nginx from that VIP address. ○ iptables!!
  65. 65. COSCUP2018 x openSUSE.Asia GNOME.Asia Lte’swatchtheiptables First, we can use the service name to filter the iptables rules. sudo iptables-save | grep ”k8s- nginx-cluster”
  66. 66. COSCUP2018 x openSUSE.Asia GNOME.Asia Lte’swatchtheiptables Remember? There’re three endpoints for the service now.
  67. 67. COSCUP2018 x openSUSE.Asia GNOME.Asia workflowsPackets Packets Packets Match Services’ ClusterIP Find the endpoints DNAT KUBE-SERVICES KUBE-SVC-XXXX KUBE-SEP-XXXX Enter iptables PREROUTING Jump to other chain Jump to other chain Jump to other chain
  68. 68. COSCUP2018 x openSUSE.Asia GNOME.Asia Howdowechoosewhichonetouse? When we match the clusterIP:Port, goto another custom chain. ○ 10.105.100.214:80
  69. 69. COSCUP2018 x openSUSE.Asia GNOME.Asia Howdowechoosewhichonetouse? Use the random to choose what endpoint we use.
  70. 70. COSCUP2018 x openSUSE.Asia GNOME.Asia EP1 EP2 EP3 P < 0.33 P < 0.5 EP1 EP2 EP3 P= 1/3 P= 2/3 * 1/2 = 1/3 P= 2/3 * 1/2 = 1/3
  71. 71. COSCUP2018 x openSUSE.Asia GNOME.Asia EP1 EP2 EP3 P < 0.2 P < 0.25 EP1 EP2 EP3 EP4 EP5 P < 0.33 P < 0.5 EP4 EP5 P = 0.2 P = 4/5 * 1/4 = 1/5 P = 4/5 * 3/4 * 1/3 = 1/5 P = 4/5 * 3/4 * 2/3 * 1/2 = 1/5 P = 4/5 * 3/4 * 2/3 * 1/2 = 1/5
  72. 72. COSCUP2018 x openSUSE.Asia GNOME.Asia Howdowechoosewhichonetouse? K8S create a custom chain for each endpoints. First rule is a SNAT ○ The Ngnix want to access outside. Second is DNAT ○ Change the IP to one of the endpoints
  73. 73. COSCUP2018 x openSUSE.Asia GNOME.Asia Now, We Know The Basic Function Of Kubernetes Network.
  74. 74. COSCUP2018 x openSUSE.Asia GNOME.Asia What is the next step of kubernetes network
  75. 75. COSCUP2018 x openSUSE.Asia GNOME.Asia challenges For different use cases ○ 5G/NFV/IoT Network features ○ High performance ○ Low latency Network infrastructure ○ Multiple network ■ Separate the data/control network.
  76. 76. COSCUP2018 x openSUSE.Asia GNOME.Asia Networkfeaturesc We use the hardware/smart NIC for those requirements before. We also have some mechanism in the software. ○ DPDK ○ SR-IOV ○ … ○ etc
  77. 77. COSCUP2018 x openSUSE.Asia GNOME.Asia Network Interface Card Linux Kernel Network Stack Network Driver Application Network Interface Card Linux Kernel Network Stack Network Driver Application Kernel Space User Space DPDK
  78. 78. COSCUP2018 x openSUSE.Asia GNOME.Asia How integrate those with kubernetes?
  79. 79. COSCUP2018 x openSUSE.Asia GNOME.Asia CNI We use some CNI for those functions. Intel had developed the CNI for those functions. ○ Call sriov-cni ○ https://github.com/intel/sriov-cni
  80. 80. COSCUP2018 x openSUSE.Asia GNOME.Asia Node1 Node2 Node3 PodA PodA PodA PodB PodA PodB Flannel (Control Network) br0 br0 br0 Data Network (192.168.0.0/16)
  81. 81. COSCUP2018 x openSUSE.Asia GNOME.Asia Problem For those container using the DPDK/SR-IOV, it can’t use any kubernetes service now. Since the network function is handled by DPDK/SR-IOV now. How to solve this?
  82. 82. COSCUP2018 x openSUSE.Asia GNOME.Asia Multus There’s a discussion in the github about that requirement. Intel develop a CNI plugin to support multiple network for a Pod. ○ It’s called Multus CNI Multus call CNIs one by one.
  83. 83. COSCUP2018 x openSUSE.Asia GNOME.Asia Pod Container eth0 Pod Container eth0 eth1 eth2 Previous What Multus Provide
  84. 84. COSCUP2018 x openSUSE.Asia GNOME.Asia , , You need to create first
  85. 85. COSCUP2018 x openSUSE.Asia GNOME.Asia Node1 Node2 Node3 PodA PodA PodA PodB PodA PodB Flannel (Control Network) br0 br0 br0 Data Network (192.168.0.0/16) Data Network (10.56.10/24)
  86. 86. COSCUP2018 x openSUSE.Asia GNOME.Asia Node1 Node2 Node3 PodA PodA PodA PodB PodA PodB Flannel (Control Network) br0 br0 br0 Data Network (192.168.0.0/16) Data Network (10.56.10/24)
  87. 87. COSCUP2018 x openSUSE.Asia GNOME.Asia Node1 Node2 Node3 PodA PodA PodA PodB PodA PodB Flannel (Control Network) br0 br0 br0 Data Network (192.168.0.0/16) Data Network (10.56.10/24)
  88. 88. COSCUP2018 x openSUSE.Asia GNOME.Asia Node1 Node2 Node3 PodA PodA PodA PodB PodA PodB Flannel (Control Network) br0 br0 br0 Data Network (192.168.0.0/16) Data Network (10.56.10/24)
  89. 89. COSCUP2018 x openSUSE.Asia GNOME.Asia Q&A

×