In this session, Matt will look at Istio, a Service Mesh, but no prior knowledge of Istio or the space is needed. Istio is a “smart network” comprising traffic switches and controllers. It’s had about four different architectures over time, and we’ll look at what problem each solved, and the challenges that eventually led to it being changed. We’ll talk about how the different approaches each optimised performance, scalability, and redundancy. We’ll explore analogies between the shapes of Istio and common patterns - three-tier web apps, big network routers, etc - and how Istio’s lessons are broadly applicable.

1. Why Is Istio That Shape? @mt165
Why Is Istio That
Shape?
Matt Turner
Software Architecture Gathering, Virtual | November 2022 @mt165 | mt165.co.uk

2. Why Is Istio That Shape? @mt165
THE ENTERPRISE SERVICE MESH COMPANY

3. Why Is Istio That Shape? @mt165
Pop Quiz!

4. Why Is Istio That Shape? @mt165
Background
Service Meshes and Istio

5. Why Is Istio That Shape? @mt165
Business Value
● Top Line
● Bottom Line
● Time to Market & Speed of Iteration
● Risk Reduction

6. Why Is Istio That Shape? @mt165
Break the Monolith

7. Why Is Istio That Shape? @mt165
Pod
Pod
Pod
Break the Monolith
Namespace A
Namespace B
Namespace C
Namespace A
Namespace B
Namespace C

8. Why Is Istio That Shape? @mt165
But Don’t Just Distribute It!

9. Why Is Istio That Shape? @mt165
Technical Implications

10. Why Is Istio That Shape? @mt165

11. Why Is Istio That Shape? @mt165

12. Why Is Istio That Shape? @mt165

13. Why Is Istio That Shape? @mt165

14. Why Is Istio That Shape? @mt165

15. Why Is Istio That Shape? @mt165

16. Why Is Istio That Shape? @mt165
Service A Service B

17. Why Is Istio That Shape? @mt165
Lots of Options: In-process libraries and frameworks

18. Why Is Istio That Shape? @mt165
Polyglot environments

19. Why Is Istio That Shape? @mt165
Polyglot environments

20. Why Is Istio That Shape? @mt165
Take the Network Smarts out of the Process

21. Why Is Istio That Shape? @mt165
Add a Control Plane

22. Why Is Istio That Shape? @mt165
Istio
“An open platform to connect, secure,
control, and observe services.”

23. Why Is Istio That Shape? @mt165
What Shape is Istio?

24. Why Is Istio That Shape? @mt165
Istio 0.1 - 1.4

25. Why Is Istio That Shape? @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer Citadel
Control Plane API
Service A Service B
Envoy
Envoy
Envoy
Envoy
Ingress Egress
Envoy
Envoy
Envoy
Envoy

26. Why Is Istio That Shape? @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer Citadel
Control Plane API
Service A Service B
Envoy
Envoy
Envoy
Envoy
Ingress Egress
Envoy
Envoy
Envoy
Envoy

27. Why Is Istio That Shape? @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer Citadel
Control Plane API
Service A Service B
Envoy
Envoy
Envoy
Envoy
Ingress Egress
Envoy
Envoy
Envoy
Envoy

28. Why Is Istio That Shape? @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer Citadel
Control Plane API
Service A Service B

29. Why Is Istio That Shape? @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer Citadel
Control Plane API
Service A Service B

30. Why Is Istio That Shape? @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer Citadel
Control Plane API
Service A Service B
TLS certs
to Envoys

31. Why Is Istio That Shape? @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer Citadel
Control Plane API
Service A Service B
Config to
Envoys
Envoy
Envoy
Envoy
Envoy
Ingress Egress
Envoy
Envoy
Envoy
Envoy

32. Why Is Istio That Shape? @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer Citadel
Control Plane API
Service A Service B
Config to
Envoys
Envoy
Envoy
Envoy
Envoy
Ingress Egress
Envoy
Envoy
Envoy
Envoy
k8s consul zk
K8s API
Server

33. Why Is Istio That Shape? @mt165
Have We Seen This
Before?

34. Why Is Istio That Shape? @mt165
IP Router Architecture
DATA PLANE
CONTROL PLANE
OSPF ARP
BGP STP
Router
Information
Base
Forwarding
Information
Base
Forwarding
Engine

35. Why Is Istio That Shape? @mt165
IP Router Architecture
DATA PLANE
CONTROL PLANE
OSPF ARP
BGP STP
Router
Information
Base
Forwarding
Information
Base
Forwarding
Engine
Latency
Throughput
Concurrency
Availability

36. Why Is Istio That Shape? @mt165
IP Router Architecture
DATA PLANE
CONTROL PLANE
OSPF ARP
BGP STP
PILOT
ENVOY
Router
Information
Base
Forwarding
Information
Base
Forwarding
Engine

37. Why Is Istio That Shape? @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer Citadel
Control Plane API
Service A Service B
Policy checks,
Telemetry
Envoy
Envoy
Envoy
Envoy
Ingress Egress
Envoy
Envoy
Envoy
Envoy

38. Why Is Istio That Shape? @mt165
IP Router Architecture
Interrupt
Kernel module
User process
DATA PLANE
CONTROL PLANE
OSPF ARP
BGP STP
Router
Information
Base
Forwarding
Information
Base

39. Why Is Istio That Shape? @mt165
IP Router Architecture
DATA PLANE
CONTROL PLANE
OSPF ARP
BGP STP
PILOT
MIXER
ENVOY
Interrupt
Kernel module
User process
Router
Information
Base
Forwarding
Information
Base

40. Why Is Istio That Shape? @mt165
Envoy
SvcA
Envoy
SvcB
Pilot
Mixer
Control Plane API
Service A Service B
Config to
Envoys
prom ES
REPORT
CHECK
ACL Rate
limit
Mixer fat client Mixer fat client
Citadel

41. Why Is Istio That Shape? @mt165
Envoy
SvcA
Envoy
SvcB
Pilot Mixer Citadel
Control Plane API
Service A Service B
Config to
Envoys
TLS certs
to Envoys
Policy checks,
Telemetry
Envoy
Envoy
Envoy
Envoy
Ingress Egress
Envoy
Envoy
Envoy
Envoy

42. Why Is Istio That Shape? @mt165
Why Isn’t Istio This Shape?

43. Why Is Istio That Shape? @mt165
Mixer
Pros
● Recognisable mental mode
● Good availability
● Can horizontally scale
○ Load
○ High availability
Cons
● Performance was never great
● Very few things need coördination
between proxies

44. Why Is Istio That Shape? @mt165
µServices!
Pros
● Decoupled deployments
● Isolated security contexts
● Isolated failure domains
● Individual scaling
● Multiple languages
● Clean mapping to individual
teams/domains
Cons
● Complex to debug
● Fiddly to deploy
● Inefficient
○ Message-passing
○ Missed shared cache opportunities

45. Why Is Istio That Shape? @mt165
µServices?
Decoupled deployments
● No-one ever did

46. Why Is Istio That Shape? @mt165
µServices?
Isolated security contexts
● Citadel is obviously a security component, but…
● Pilot can re-route all your requests
● Mixer can steal all your traffic
● …a breach in any one is as bad as the others

47. Why Is Istio That Shape? @mt165
µServices?
Isolated failure domains
● Mixer is the only acutely critical service

48. Why Is Istio That Shape? @mt165
µServices?
Individual scaling
● Resource usage was dominated by one component of service: Pilot’s XDS serving

49. Why Is Istio That Shape? @mt165
µServices?
Multiple languages
● It’s all Go

50. Why Is Istio That Shape? @mt165
µServices?
Clean mapping to individual teams/domains
● We’re good at writing monoliths
● µServices were never about this anyway

51. Why Is Istio That Shape? @mt165
Istio 1.5 - Date

52. Why Is Istio That Shape? @mt165
Pilot
● That thing I said about a compiler…
● It’s a nice mental model
● It wasn’t true

53. Why Is Istio That Shape? @mt165
Pilot
Pilot
K8s API
Server
Envoy
Envoy
Envoy
Envoy
k8s
consul
zk

54. Why Is Istio That Shape? @mt165
Pilot
K8s API
Server
Envoy
Envoy
Envoy
Envoy
k8s
consul
zk

55. Why Is Istio That Shape? @mt165
Pilot
K8s API
Server
Envoy
Envoy
Envoy
Envoy
k8s
consul
zk

56. Why Is Istio That Shape? @mt165
Envoy
SvcA
Envoy
SvcB
istiod
Control Plane API
Service A Service B
Config to
Envoys
TLS certs
to Envoys
Envoy
Envoy
Envoy
Envoy
Ingress Egress
Envoy
Envoy
Envoy
Envoy
µServices

57. Why Is Istio That Shape? @mt165
Mixer
● Policy: implemented with (new) Envoy-native features

58. Why Is Istio That Shape? @mt165
Envoy
SvcA
Envoy
SvcB
istiod
Control Plane API
Service A Service B
Config to
Envoys
TLS certs
to Envoys
Envoy
Envoy
Envoy
Envoy
Ingress Egress
Envoy
Envoy
Envoy
Envoy
WASM WASM WASM WASM
Telem
etry

59. Why Is Istio That Shape? @mt165
Dataplane Topology - Host-Based Proxy
#eBPF
#sidecarless
#nomesh

60. Why Is Istio That Shape? @mt165
Envoy
Service A Service B
WASM WASM
host
istiod
Control Plane API

61. Why Is Istio That Shape? @mt165
CPU Usage
Per Request
Idle

62. Why Is Istio That Shape? @mt165
Memory Usage
Per Service Config
Overhead:
programme

63. Why Is Istio That Shape? @mt165
Dataplane Topology - “Ambient Mesh”

64. Why Is Istio That Shape? @mt165
Ambient Mesh - zTunnel
host
host
Service A
Service B
zTunnel zTunnel
Service A

65. Why Is Istio That Shape? @mt165
Ambient Mesh - “Waypoint” Proxies
host
host
Service A
Service B
zTunnel zTunnel
Service A
Envoy
K8s ServiceAccount A
Envoy
K8s ServiceAccount B

66. Why Is Istio That Shape? @mt165
Thanks!
Slides
Videos
Demo code
mt165.co.uk
Questions @mt165