Istio is a service mesh for Kubernetes that offers advanced networking features. It provides intelligent routing, resiliency, and security features, so that service authors don’t have to keep reimplementing them. Istio is rapidly taking off and ther are great introductory talks everywhere. However in this session, we will dive deep to explore precisely how it does what it does, following one brave little packet in from the Internet and back out again. At each point we’ll see how to configure the features of that component to exploit Istio’s full potential. This will give a great insight into Istio’s full power, and its fascinating architecture.

1. @mt165
mt165.co.uk
QCon London
March 2019
& the architecture along the way!

2. The life of a packet through Istio @mt165
Objectives
Learn how a packet traverses an Istio/Envoy/Kubernetes system
See what control plane calls are made in that process
Build a useful mental model for reasoning about, and debugging Istio

3. The life of a packet through Istio @mt165
Prerequisites
Basic networking knowledge
Intermediate Kubernetes knowledge
An understanding of what Istio is and does

4. The life of a packet through Istio @mt165
Outline
● Context and Introduction
● Networking and Containers
● Pilot and Routing
● Mixer and Policy
● Citadel and mTLS

5. The life of a packet through Istio @mt165
Context and Introduction

6. The life of a packet through Istio @mt165
Why?

7. The life of a packet through Istio @mt165

8. The life of a packet through Istio @mt165

9. The life of a packet through Istio @mt165
Istio
“An open platform to connect, secure, control, and observe services.”

10. The life of a packet through Istio @mt165
Networking and Containers

11. The life of a packet through Istio @mt165
Service AIngress

12. The life of a packet through Istio @mt165
Service A
Envoy
Envoy
Envoy
Envoy
IngressLoad
Balancer
Node
port
Cluster
IP
*.example.com
Cluster
IP

13. The life of a packet through Istio @mt165
Service A

14. The life of a packet through Istio @mt165
Envoy
SvcA
Service A

15. The life of a packet through Istio @mt165
“Containers”
nginx
nginx
supervisord
mnt
uts
pid
user
ipc
net

16. The life of a packet through Istio @mt165
Kubernetes Pods
nginx
nginx
supervisord
mnt
uts
pid
user
ipc
net
logger
fluentd
mnt
uts

17. The life of a packet through Istio @mt165
Kubernetes Pods
nginx
nginx
supervisord
mnt
uts
pid
user
ipc
net
logger
fluentd
mnt
uts
192.168.0.42
eth0
lo
sockets
iptables
routes

18. The life of a packet through Istio @mt165
Kubernetes Pods
nginx
nginx
supervisord
mnt
uts
pid
user
ipc
net
logger
fluentd
mnt
uts
192.168.0.42
eth0
lo
sockets
iptables
routes
:8080/tcp

19. The life of a packet through Istio @mt165
Kubernetes Pods
nginx
nginx
supervisord
mnt
uts
pid
user
ipc
net
proxy
envoy
mnt
uts
192.168.0.42
eth0
lo
sockets
iptables
routes
:8080/tcp

20. The life of a packet through Istio @mt165
Sidecar Injection
pid
user
ipc
net
192.168.0.42
eth0
lo
sockets
iptables
routes

21. The life of a packet through Istio @mt165
Sidecar Injection
pid
user
ipc
net
192.168.0.42
eth0
lo
sockets
iptables
routes
alpine
sysctl -w kernel.core_pattern=...

22. The life of a packet through Istio @mt165
Sidecar Injection
pid
user
ipc
net
192.168.0.42
eth0
lo
sockets
iptables
routes
istio/proxy_init
/usr/local/bin/prepare_proxy.sh -p 15001 -u 1337

23. The life of a packet through Istio @mt165
Sidecar Injection
nginx
nginx
mnt
uts
pid
user
ipc
net
istio/proxy
envoy
mnt
uts
192.168.0.42
eth0
lo
sockets
iptables
routes
:15001/tcp

24. The life of a packet through Istio @mt165
Envoy
SvcA
Service A

25. The life of a packet through Istio @mt165
Pilot and Routing

26. The life of a packet through Istio @mt165
Envoy
SvcA
Service A
?
?
?

27. The life of a packet through Istio @mt165
Services
$ kubectl get service -o wide service-b
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service-b ClusterIP 10.98.84.169 <none> 80/TCP 90s app=service-b

28. The life of a packet through Istio @mt165
Service DNS exposure
$ dig service-b.default.svc.cluster.local.
;; ANSWER SECTION:
service-b.default.svc.cluster.local. 5 IN A 10.98.84.169

29. The life of a packet through Istio @mt165
Pods
$ kubectl get pods -o wide | grep service-b
service-b-644856485c-4rk88 1/1 Running 0 7m46s 10.32.0.4 kind-1-control-plane <none>
service-b-644856485c-dc2zv 1/1 Running 0 7m46s 10.32.0.6 kind-1-control-plane <none>
service-b-644856485c-gr75k 1/1 Running 0 7m46s 10.32.0.5 kind-1-control-plane <none>

30. The life of a packet through Istio @mt165
Endpoints
$ kubectl get endpoints service-b
NAME ENDPOINTS AGE
service-b 10.32.0.4:8080,10.32.0.5:8080,10.32.0.6:8080 8m55s

31. The life of a packet through Istio @mt165
Endpoints
$ kubectl get endpoints service-b -o yaml
...
subsets:
- addresses:
- ip: 10.32.0.4
nodeName: kind-1-control-plane
targetRef:
kind: Pod
…
ports:
- name: http
port: 8080
protocol: TCP

32. The life of a packet through Istio @mt165
Envoy
SvcA
Pilot
Control Plane API
Service A
Config to
Envoys

33. The life of a packet through Istio @mt165
Envoy
SvcA
Pilot
Control Plane API
Service A
Config to
Envoys
k8s consul zk
Data plane API

34. The life of a packet through Istio @mt165
Pilot
● Ingress Routing
● Traffic Mirroring
● Traffic Shifting
● Canary Deployments
● Circuit Breaking
● Fault Injection

35. The life of a packet through Istio @mt165
Mixer and Policy

36. The life of a packet through Istio @mt165
Envoy
SvcA
Pilot
Control Plane API
Service A Service B
Config to
Envoys

37. The life of a packet through Istio @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer
Control Plane API
Service A Service B
Config to
Envoys
Policy checks,
Telemetry

38. The life of a packet through Istio @mt165
IP 5-tuple
(src_addr, src_port, dst_addr, dst_port, proto)

39. The life of a packet through Istio @mt165
IP Router Architecture
Interrupt
Kernel module
User process
DATA PLANE
CONTROL PLANE
OSPF ARPBGP STP
Router
Information
Base
Forwarding
Information
Base

40. The life of a packet through Istio @mt165
IP Router Architecture
DATA PLANE
CONTROL PLANE
OSPF ARPBGP STP
PILOT
MIXER
ENVOYInterrupt
Kernel module
User process
Router
Information
Base
Forwarding
Information
Base

41. The life of a packet through Istio @mt165
Envoy
SvcA
Envoy
SvcB
Pilot
Mixer
Control Plane API
Service A Service B
Config to
Envoys
prom ES
REPORT
CHECK
RBAC Rate
limit
Mixer fat client Mixer fat client

42. The life of a packet through Istio @mt165
Mixer
● Check
○ ACLs / Authorization
○ Rate Limiting
● Report
○ Logs
○ Metrics
○ Tracing

43. The life of a packet through Istio @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer
Control Plane API
Service A Service B
Config to
Envoys
Policy checks,
Telemetry

44. The life of a packet through Istio @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer Citadel
Control Plane API
Service A Service B
Config to
Envoys
TLS certs
to Envoys
Policy checks,
Telemetry

45. The life of a packet through Istio @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer Citadel
Control Plane API
Service A Service B
Config to
Envoys
TLS certs
to Envoys
Policy checks,
Telemetry

46. The life of a packet through Istio @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer Citadel
Control Plane API
Service A Service B
Config to
Envoys
TLS certs
to Envoys
Policy checks,
Telemetry
Envoy
Envoy
Envoy
Envoy
Envoy
Envoy
Envoy
Envoy
Ingress Egress

47. The life of a packet through Istio @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer Citadel
Control Plane API
Service A Service B
Config to
Envoys
TLS certs
to Envoys
Policy checks,
Telemetry
API Serveretcd
kubectl

48. The life of a packet through Istio @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer Citadel
Control Plane API
Service A Service B
Config to
Envoys
TLS certs
to Envoys
Policy checks,
Telemetry
Galleyetcd
kubectl

49. The life of a packet through Istio @mt165
Outline
● Context and Introduction
● Networking and Containers
● Pilot and Routing
● Mixer and Policy
● Citadel and mTLS

50. The life of a packet through Istio @mt165
Recap
We learned:
● How a packet traverses an Istio/Envoy/Kubernetes system
● What control plane calls are made in that process
● A useful mental model for reasoning about, and debugging Istio

51. Thanks!
@mt165
QR CODE