Kubernetes Networking
Bryan Boreham, Director of Engineering
@bboreham
What does Weave do?
Weave lets devops
iterate faster with:
• observability &
monitoring
• continuous delivery
• container networks &
firewalls
Kubernetes is our #1
platform
What you should learn
1. How clients talk to services in Kubernetes
2. Connecting containers: overlay vs native
3. Connecting into your cluster: NodePort,
HostPort, LoadBalancer, Ingress
4. Be better equipped to troubleshoot
https://media.timeout.com/images/103755908/630/472/image.jpg
• Docker
• Kubernetes
• Weave
Who is working with...
What is Kubernetes?
https://image.shutterstock.com/z/stock-photo-vintage-photo-of-a-man-working-on-complex-machine-1392745.jpg
What is Kubernetes?
NodeNode Node
Master
NodeNode Node
Kubernetes runs Services
Let’s talk about Ports
A service listens on a Port at an IP address
– e.g. http on port 80, postgres on port 5432
– or your own custom service on 9090
192.1.6.4
9090
foo
Suppose we want to run two?
•Only one thing can be listening on a port
•We could give the second one a new port
number
192.1.6.4
9090
9091
foo1
foo2
How do we keep track?
• Fiddling with port numbers needs a bit of book-keeping
• Could have another service where we register all the
port numbers we’ve chosen
• Downside: this requires that every client uses the
registry
192.1.6.4
9090
9091
Service
Registry
“Where
is
foo2?”
foo1
foo2
Give every service its own IP address
•Container Networking means never having to
say “what port is it on?”
•Every service uses its native port number
10.20.30.42
9090
foo1
9090
foo2
10.20.30.43
Kubernetes Concepts
Pod
IP addr
Node
Container
Just one thing though
•Now, when we contact a service, we need to
know its IP address.
•There is a standard way to do that
DNS
“Where
is
foo2?”
10.20.30.42
foo1
foo2
10.20.30.43
• Run multiple instances of a service
• Clients should call one of them, don’t care
which
foo
10.20.10.1
foo
10.20.29.13
Now add Scaling and Redundancy
foo
10.20.30.42
DNS can do this. Right?
•Some clients will re-query on every call
•Some clients will cache the result too long
•Most clients will not round-robin
http://gunshowcomic.com/648
• DNS name resolves to a stable Virtual IP address
• Kube-proxy translates VIP to one Pod IP
Kubernetes Cluster IPs
kube-dns
10.20.30.42
kube-proxy
100.96.0.30
->10.20.30.42
foo
“Where is
foo?”
- 100.96.0.30 192.1.6.4
We need network packets to go from one pod to another
pod, across whatever sits in the middle
Let’s talk about Pod Networking
10.20.30.42
192.1.6.4192.1.6.3
10.20.9.1
Who controls your network?
http://philippel.deviantart.com/art/DUNE-Sandworm-Rising-
403336019
If you have the IP space, and you control the network, just
program the routers
Pod Network: Routes
10.20.30.42
192.1.6.4192.1.6.3
10.20.30.0/24:
via 192.1.6.4
10.20.9.0/24:
via 192.1.6.3
10.20.9.1
Packets are encapsulated before they leave the machine
Pod Network: Overlay
10.20.30.42
192.1.6.4192.1.6.3
10.20.9.1
192.1.6.3->192.1.6.4
[10.20.9.1->10.20.30.42]
The Three Commandments
...of Kubernetes Networking:
• All containers can communicate with all other
containers
• All nodes can communicate with all containers
(and vice-versa)
• The IP that a container sees itself as is the same
IP that others see it as
CNI: the Container Network Interface
kubele
t
Po
d
Interface
Plugin
Pod Network
ADD
• One high-numbered port, on every Node in the cluster
• Can bounce from one machine to another
Exposing services: NodePort
10.20.30.42
kube-proxy
:30021
->10.20.30.42:80
foo
192.1.6.4 :30021
• Specific port is mapped locally on the host
• “Don’t use hostPort unless it is absolutely necessary”
Exposing services: HostPort
10.20.30.42
:8080
->10.20.30.42:80
foo
192.1.6.4 :8080
• Layer 4 - works for any TCP-based protocol
• Available for specific implementations, e.g. ELB
Exposing services: LoadBalancer
LB
foo
cloud-controller
Programs
endpoints
kube-proxy / iptables
foo
• Layer 7 - defined for http only
• Available for specific implementations, e.g. nginx, ALB
Exposing services: Ingress
ingress controller
foo foo
Master
Example Ingress config
apiVersion: extensions/v1beta1
kind: Ingress
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /foo
backend:
serviceName: s1
servicePort: 80
- path: /bar
backend:
serviceName: s2
servicePort: 80
Recap: all you need to know
•Kubernetes runs Pods which implement
Services
•Pods need a Pod Network - routed or Overlay
•Pod network is driven via CNI
•Clients connect to Services via virtual Cluster
IPs
•Kubernetes has many ways to expose a
Service outside the cluster - each has pros and
Thanks! Questions?
We are hiring!
Engineers in SF & London
weave.works/hiring
What’s Next?
•Try Weave Cloud
– https://cloud.weave.works
•Join the Weave user group!
– meetup.com/pro/Weave
•Get in touch! (Slack, Email, etc.)
– weave.works/help

Kubernetes Networking 101

  • 1.
    Kubernetes Networking Bryan Boreham,Director of Engineering @bboreham
  • 2.
    What does Weavedo? Weave lets devops iterate faster with: • observability & monitoring • continuous delivery • container networks & firewalls Kubernetes is our #1 platform
  • 3.
    What you shouldlearn 1. How clients talk to services in Kubernetes 2. Connecting containers: overlay vs native 3. Connecting into your cluster: NodePort, HostPort, LoadBalancer, Ingress 4. Be better equipped to troubleshoot https://media.timeout.com/images/103755908/630/472/image.jpg
  • 4.
    • Docker • Kubernetes •Weave Who is working with...
  • 5.
  • 6.
  • 7.
  • 8.
    Let’s talk aboutPorts A service listens on a Port at an IP address – e.g. http on port 80, postgres on port 5432 – or your own custom service on 9090 192.1.6.4 9090 foo
  • 9.
    Suppose we wantto run two? •Only one thing can be listening on a port •We could give the second one a new port number 192.1.6.4 9090 9091 foo1 foo2
  • 10.
    How do wekeep track? • Fiddling with port numbers needs a bit of book-keeping • Could have another service where we register all the port numbers we’ve chosen • Downside: this requires that every client uses the registry 192.1.6.4 9090 9091 Service Registry “Where is foo2?” foo1 foo2
  • 11.
    Give every serviceits own IP address •Container Networking means never having to say “what port is it on?” •Every service uses its native port number 10.20.30.42 9090 foo1 9090 foo2 10.20.30.43
  • 12.
  • 13.
    Just one thingthough •Now, when we contact a service, we need to know its IP address. •There is a standard way to do that DNS “Where is foo2?” 10.20.30.42 foo1 foo2 10.20.30.43
  • 14.
    • Run multipleinstances of a service • Clients should call one of them, don’t care which foo 10.20.10.1 foo 10.20.29.13 Now add Scaling and Redundancy foo 10.20.30.42
  • 15.
    DNS can dothis. Right? •Some clients will re-query on every call •Some clients will cache the result too long •Most clients will not round-robin http://gunshowcomic.com/648
  • 16.
    • DNS nameresolves to a stable Virtual IP address • Kube-proxy translates VIP to one Pod IP Kubernetes Cluster IPs kube-dns 10.20.30.42 kube-proxy 100.96.0.30 ->10.20.30.42 foo “Where is foo?” - 100.96.0.30 192.1.6.4
  • 17.
    We need networkpackets to go from one pod to another pod, across whatever sits in the middle Let’s talk about Pod Networking 10.20.30.42 192.1.6.4192.1.6.3 10.20.9.1
  • 18.
    Who controls yournetwork? http://philippel.deviantart.com/art/DUNE-Sandworm-Rising- 403336019
  • 19.
    If you havethe IP space, and you control the network, just program the routers Pod Network: Routes 10.20.30.42 192.1.6.4192.1.6.3 10.20.30.0/24: via 192.1.6.4 10.20.9.0/24: via 192.1.6.3 10.20.9.1
  • 20.
    Packets are encapsulatedbefore they leave the machine Pod Network: Overlay 10.20.30.42 192.1.6.4192.1.6.3 10.20.9.1 192.1.6.3->192.1.6.4 [10.20.9.1->10.20.30.42]
  • 21.
    The Three Commandments ...ofKubernetes Networking: • All containers can communicate with all other containers • All nodes can communicate with all containers (and vice-versa) • The IP that a container sees itself as is the same IP that others see it as
  • 22.
    CNI: the ContainerNetwork Interface kubele t Po d Interface Plugin Pod Network ADD
  • 23.
    • One high-numberedport, on every Node in the cluster • Can bounce from one machine to another Exposing services: NodePort 10.20.30.42 kube-proxy :30021 ->10.20.30.42:80 foo 192.1.6.4 :30021
  • 24.
    • Specific portis mapped locally on the host • “Don’t use hostPort unless it is absolutely necessary” Exposing services: HostPort 10.20.30.42 :8080 ->10.20.30.42:80 foo 192.1.6.4 :8080
  • 25.
    • Layer 4- works for any TCP-based protocol • Available for specific implementations, e.g. ELB Exposing services: LoadBalancer LB foo cloud-controller Programs endpoints kube-proxy / iptables foo
  • 26.
    • Layer 7- defined for http only • Available for specific implementations, e.g. nginx, ALB Exposing services: Ingress ingress controller foo foo Master
  • 27.
    Example Ingress config apiVersion:extensions/v1beta1 kind: Ingress spec: rules: - host: foo.bar.com http: paths: - path: /foo backend: serviceName: s1 servicePort: 80 - path: /bar backend: serviceName: s2 servicePort: 80
  • 28.
    Recap: all youneed to know •Kubernetes runs Pods which implement Services •Pods need a Pod Network - routed or Overlay •Pod network is driven via CNI •Clients connect to Services via virtual Cluster IPs •Kubernetes has many ways to expose a Service outside the cluster - each has pros and
  • 29.
    Thanks! Questions? We arehiring! Engineers in SF & London weave.works/hiring
  • 30.
    What’s Next? •Try WeaveCloud – https://cloud.weave.works •Join the Weave user group! – meetup.com/pro/Weave •Get in touch! (Slack, Email, etc.) – weave.works/help