Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
What’s a Service Mesh and
why do I need one?
Jeroen Reijn
#jfall
About me:
• (Java) Programmer and architect
• Big fan of the DevOps culture
• Enjoys building cloud native solutions
• Com...
Monolith?
Microservices?
Kubernetes?
Cloud?
Service mesh, ...
istio, … service mesh
Have you heard about a
service mesh before?
+
So what is a ‘Service Mesh’ and what
problem does it solve?
“A service mesh is a
dedicated infrastructure layer for
handling
service-to-service communication”
Why a dedicated layer?
Microservices
Distributed systems
Network communication
complex
Reliable communication is
Evolution networking
The evolution of networking
Computer BComputer A
Service A Service B
Networking Stack Networking Stack
Business LogicBusin...
The evolution of networking
Computer BComputer A
Service A Service B
Networking Stack Networking Stack
Business Logic
Flow...
The evolution of networking
Computer BComputer A
Networking Stack
Service A Service B
Networking Stack
Business Logic
Flow...
The 8 Fallacies of Distributed Computing
1. The network is reliable
2. Latency is zero
3. Bandwidth is infinite
4. The net...
Critical functions for microservices
Fast, reliable & safe microservices
microservice
microservice
microservice
Routing
Dy...
Routing - Service discovery
Registry client
Registry clientRegistry client
Registry client
Registry clientRegistry client
...
Resilience
Resilience - Cascading failure
Service 1 Service 2 Service 3 Service 4
The Circuit Breaker pattern
“A service client should invoke a remote service via a ‘proxy’ that
functions in a similar fas...
Circuit breaker
Half Open
Failure threshold
exceeded
Set breaker
Failure threshold
exceeded
Set breaker
Try reset after ti...
Observability of your services
Golden triangle of monitoring
Metrics
Logs
Traces
Security of microservices
• OAuth / JWT Tokens
• Mutual TLS / certificates
Computer B
The evolution of networking
Computer A
Service A Service B
Networking Stack Networking Stack
Business Logic
Flo...
Computer B
The evolution of networking
Computer A
Service A Service B
Networking Stack Networking Stack
Business Logic
Flo...
Computer B
The evolution of networking
Computer A
Service A Service B
Networking Stack Networking Stack
Business Logic
Flo...
Libraries
resilience4j
hystrix
Drawbacks of libraries
• Glue linking the libraries: expensive
• Limiting tools, runtimes, languages
• Versioning hell
• T...
Computer B
The evolution of networking
Computer A
Service A Service B
Networking Stack Networking Stack
Business Logic
Flo...
Computer B
Service B
The evolution of networking
Computer A
Service A Service B
Networking Stack
Service A
Networking Stac...
Computer B
Service B
The evolution of networking
Computer A
Networking Stack
Service A
Networking Stack
Business Logic
Flo...
OSI Model
Level 7 Application: Spring, Vertx, WFSwarm
Level 6 Presentation: Json, XML
Level 5 Session: Http 1/2, GRPC
Leve...
Computer B
Service B
The evolution of networking
Computer A
Networking Stack
Service A
Networking Stack
Business Logic
Flo...
Responsibility shift
Development team(s)
Platform team(s)
The evolution of networking
Computer B
Service D
First generation service mesh
Computer A
Service A
Proxy Proxy
Service B
Service C
Second generation service mesh - Pods and sidecars
• Container platforms
• Kubernetes
• Mesos Node
Pod Pod
Container
Proxy...
Computer B
Service B
The evolution of networking
Computer A
Networking Stack
Service A
Networking Stack
Business Logic
Flo...
Complex micro-service architectures
450 + microservices
Controlling the service mesh
Computer A
Service A
Networking Stack
Business Logic
Flow control
Sidecar proxy
Computer B
Se...
The service mesh control plane
Control plane
Proxy based Service meshes
Istio
• An open platform to connect, monitor, and
secure microservices
• Introduced by Google, Lyft, IBM and others
• Mana...
Istio - Architecture
B
Envoy Proxy
• Dynamic service discovery
• Load balancing
• TLS termination
• HTTP/2 and gRPC proxies
• Circuit breakers
• ...
Istio - Proxy configuration
YAML
Istio - Discovery and Load-balancing
Istio - Tracing
• Automatic tracing of request
• Asynchronous span reporting
• Multiple backends
• Zipkin
• Jaeger
Istio - Telemetry
Istio - Advanced routing
Istio - Security / Two way TLS
Istio Security - RBAC
• Role based access control
• Based on rules and for
instance HTTP methods
• ServiceRole (rule)
• Se...
Istio gives you:
• Telemetry
• Security
• Mutual TLS
• Role based access control
• Resilience
• Circuit-breaker
• Retry
• ...
Demo
Overhead
• Definitely not ‘free’, more parts in the system
• Proxies are used for both inbound and
outbound requests
• A l...
Debugging
• Debugging Envoy and Pilot (configuration)
• Networking Issues
• TLS issues
• Envoy bouncing requests
• …
Security
• Many new parts of the system
• Control plane components
• Proxies
• Envoys are everywhere
• Role based access c...
Istio
• Telemetry
• Security
• Circuit-breaker
• Retry
• Advanced routing
What you (want to) get What you (don’t want to) ...
But are all service meshes equal?
So we saw Istio…
Comparing Service Meshes
Source: https://kubedex.com/istio-vs-linkerd-vs-linkerd2-vs-consul/ (Sept 2018)
https://smi-spec.io
Do I really need a service mesh?
Throwing more tech at the problem…
Do you want to configure, install and
renew (mutual) TLS certificates
across an entire set of applications?
Do you want to intercept and re-route
network flows for: A/B testing, traffic
shedding or failure tolerance (circuit
break...
Do you want tracing / visibility of
application request flows within
your micro-service network?
Should I just remove
libraries from my apps?
Istio - Circuit breaking - DestinationRule
Istio - Circuit breaking - DestinationRule
Spring + Hystrix Circuit breaker fallback
Note: Hystrix is deprecated and only used as an example
Spring + Hystrix Circuit breaker fallback
Note: Hystrix is deprecated and only used as an example
Tracing
As an engineer you should still
thinkabout these concerns
Key take-aways from this talk
• A service mesh is a dedicated infra layer for service
communication
• Understand the why o...
“Please rate my talk in the
official J-Fall app”
#jfall
What's a service mesh and why do i need one?
What's a service mesh and why do i need one?
What's a service mesh and why do i need one?
What's a service mesh and why do i need one?
Upcoming SlideShare
Loading in …5
×

What's a service mesh and why do i need one?

You’ve been creating this cloud-native microservice based architecture. Continuous delivery pipelines, cloud-based deployments, and Kubernetes managed Docker containers. You are ready to scale beyond your wildest dreams. Now, while taking a step back, you notice that your services contain more than just the business logic you intended to write. Proper communication is key in a distributed system, but do you really need these extra libraries that increase the size of your microservice? Should the responsibility for reliable communication live within your application or can this be abstracted to a higher level? In this session, we will look at the concept of a Services Mesh and how it helps you put the responsibilities at the right layer. After this presentation you might have an answer to the question if you really need a service mesh.

  • Be the first to comment

  • Be the first to like this

What's a service mesh and why do i need one?

  1. 1. What’s a Service Mesh and why do I need one? Jeroen Reijn #jfall
  2. 2. About me: • (Java) Programmer and architect • Big fan of the DevOps culture • Enjoys building cloud native solutions • Community member and emeritus committer at Apache Jeroen Reijn @jreijn /jeroenreijn
  3. 3. Monolith? Microservices? Kubernetes? Cloud?
  4. 4. Service mesh, ... istio, … service mesh
  5. 5. Have you heard about a service mesh before? +
  6. 6. So what is a ‘Service Mesh’ and what problem does it solve?
  7. 7. “A service mesh is a dedicated infrastructure layer for handling service-to-service communication”
  8. 8. Why a dedicated layer?
  9. 9. Microservices Distributed systems Network communication
  10. 10. complex Reliable communication is
  11. 11. Evolution networking
  12. 12. The evolution of networking Computer BComputer A Service A Service B Networking Stack Networking Stack Business LogicBusiness Logic
  13. 13. The evolution of networking Computer BComputer A Service A Service B Networking Stack Networking Stack Business Logic Flow control Business Logic Flow control
  14. 14. The evolution of networking Computer BComputer A Networking Stack Service A Service B Networking Stack Business Logic Flow control Business Logic Flow control
  15. 15. The 8 Fallacies of Distributed Computing 1. The network is reliable 2. Latency is zero 3. Bandwidth is infinite 4. The network is secure 5. Topology doesn’t change 6. There is one administrator 7. Transport cost is zero 8. The network is homogeneous Composed by Peter Deutsch and his fellow engineers at Sun Microsystems
  16. 16. Critical functions for microservices Fast, reliable & safe microservices microservice microservice microservice Routing Dynamic discovery Load balancing Resiliency Circuit breaking Retries Rate limiting Observability Metrics Logging Tracing Security Policy Enforcement
  17. 17. Routing - Service discovery Registry client Registry clientRegistry client Registry client Registry clientRegistry client Registry client Registry clientRegistry client RegistryRegistry clientRegistry client Service A Service B Service C Service DService D Service AService A Service CService C Service BService B Service D Registry-aware HTTP client Service Registry
  18. 18. Resilience
  19. 19. Resilience - Cascading failure Service 1 Service 2 Service 3 Service 4
  20. 20. The Circuit Breaker pattern “A service client should invoke a remote service via a ‘proxy’ that functions in a similar fashion to an electrical circuit breaker” https://microservices.io/patterns/reliability/circuit-breaker.html
  21. 21. Circuit breaker Half Open Failure threshold exceeded Set breaker Failure threshold exceeded Set breaker Try reset after timeout Success Reset breaker Open Closed Success Fail (under threshold)
  22. 22. Observability of your services Golden triangle of monitoring Metrics Logs Traces
  23. 23. Security of microservices • OAuth / JWT Tokens • Mutual TLS / certificates
  24. 24. Computer B The evolution of networking Computer A Service A Service B Networking Stack Networking Stack Business Logic Flow control Flow control Circuit Breaker Service Discovery Business Logic Circuit Breaker Service Discovery Logs, metrics, traces Security Logs, metrics, traces Security
  25. 25. Computer B The evolution of networking Computer A Service A Service B Networking Stack Networking Stack Business Logic Flow control Flow control ??? ??? Circuit Breaker Service Discovery Business Logic Circuit Breaker Service Discovery Logs, metrics, traces Security Logs, metrics, traces Security
  26. 26. Computer B The evolution of networking Computer A Service A Service B Networking Stack Networking Stack Business Logic Flow control Flow control Library Library Circuit Breaker Service Discovery Business Logic Circuit Breaker Service Discovery Logs, metrics, traces Security Logs, metrics, traces Security
  27. 27. Libraries resilience4j hystrix
  28. 28. Drawbacks of libraries • Glue linking the libraries: expensive • Limiting tools, runtimes, languages • Versioning hell • Teams should not forget to add them
  29. 29. Computer B The evolution of networking Computer A Service A Service B Networking Stack Networking Stack Business Logic Flow control Flow control Library Library Circuit Breaker Service Discovery Business Logic Circuit Breaker Service Discovery Logs, metrics, traces Security Logs, metrics, traces Security
  30. 30. Computer B Service B The evolution of networking Computer A Service A Service B Networking Stack Service A Networking Stack Business Logic Flow control Library??? Flow control Library??? Circuit Breaker Service Discovery Business Logic Circuit Breaker Service Discovery Logs, metrics, traces Security Logs, metrics, traces Security
  31. 31. Computer B Service B The evolution of networking Computer A Networking Stack Service A Networking Stack Business Logic Flow control ??? Circuit Breaker Service Discovery Logs, metrics, traces Security Proxy Flow control ??? Circuit Breaker Service Discovery Logs, metrics, traces Security Proxy Business Logic
  32. 32. OSI Model Level 7 Application: Spring, Vertx, WFSwarm Level 6 Presentation: Json, XML Level 5 Session: Http 1/2, GRPC Level 4 Transport: TCP Level 1-3 Network (IP) / Data link / Physical From here To here
  33. 33. Computer B Service B The evolution of networking Computer A Networking Stack Service A Networking Stack Business Logic Flow control Proxy Proxy Circuit Breaker Service Discovery Logs, metrics, traces Security Flow control Proxy Proxy Circuit Breaker Service Discovery Logs, metrics, traces Security Business Logic
  34. 34. Responsibility shift Development team(s) Platform team(s)
  35. 35. The evolution of networking
  36. 36. Computer B Service D First generation service mesh Computer A Service A Proxy Proxy Service B Service C
  37. 37. Second generation service mesh - Pods and sidecars • Container platforms • Kubernetes • Mesos Node Pod Pod Container Proxy Container Proxy
  38. 38. Computer B Service B The evolution of networking Computer A Networking Stack Service A Networking Stack Business Logic Flow control Proxy Sidecar Proxy Circuit Breaker Service Discovery Logs, metrics, traces Security Flow control Proxy Sidecar Proxy Circuit Breaker Service Discovery Logs, metrics, traces Security Business Logic
  39. 39. Complex micro-service architectures 450 + microservices
  40. 40. Controlling the service mesh Computer A Service A Networking Stack Business Logic Flow control Sidecar proxy Computer B Service B Networking Stack Business Logic Flow control Sidecar proxy Control plane
  41. 41. The service mesh control plane Control plane
  42. 42. Proxy based Service meshes
  43. 43. Istio • An open platform to connect, monitor, and secure microservices • Introduced by Google, Lyft, IBM and others • Manages authentication, authorization, and encryption of communication between microservices • Logging, monitoring, and keeping services operational • Traffic management and policy control
  44. 44. Istio - Architecture B
  45. 45. Envoy Proxy • Dynamic service discovery • Load balancing • TLS termination • HTTP/2 and gRPC proxies • Circuit breakers • Health checks • Staged rollouts with %-based traffic split • Fault injection • Rich metrics
  46. 46. Istio - Proxy configuration YAML
  47. 47. Istio - Discovery and Load-balancing
  48. 48. Istio - Tracing • Automatic tracing of request • Asynchronous span reporting • Multiple backends • Zipkin • Jaeger
  49. 49. Istio - Telemetry
  50. 50. Istio - Advanced routing
  51. 51. Istio - Security / Two way TLS
  52. 52. Istio Security - RBAC • Role based access control • Based on rules and for instance HTTP methods • ServiceRole (rule) • ServiceRoleBinding (assign role to set of nodes)
  53. 53. Istio gives you: • Telemetry • Security • Mutual TLS • Role based access control • Resilience • Circuit-breaker • Retry • Advanced routing
  54. 54. Demo
  55. 55. Overhead • Definitely not ‘free’, more parts in the system • Proxies are used for both inbound and outbound requests • A lot of effort going on to reduce overhead
  56. 56. Debugging • Debugging Envoy and Pilot (configuration) • Networking Issues • TLS issues • Envoy bouncing requests • …
  57. 57. Security • Many new parts of the system • Control plane components • Proxies • Envoys are everywhere • Role based access control
  58. 58. Istio • Telemetry • Security • Circuit-breaker • Retry • Advanced routing What you (want to) get What you (don’t want to) get • Overhead • Debugging • Security complexity
  59. 59. But are all service meshes equal? So we saw Istio…
  60. 60. Comparing Service Meshes Source: https://kubedex.com/istio-vs-linkerd-vs-linkerd2-vs-consul/ (Sept 2018)
  61. 61. https://smi-spec.io
  62. 62. Do I really need a service mesh?
  63. 63. Throwing more tech at the problem…
  64. 64. Do you want to configure, install and renew (mutual) TLS certificates across an entire set of applications?
  65. 65. Do you want to intercept and re-route network flows for: A/B testing, traffic shedding or failure tolerance (circuit breaking)?
  66. 66. Do you want tracing / visibility of application request flows within your micro-service network?
  67. 67. Should I just remove libraries from my apps?
  68. 68. Istio - Circuit breaking - DestinationRule
  69. 69. Istio - Circuit breaking - DestinationRule
  70. 70. Spring + Hystrix Circuit breaker fallback Note: Hystrix is deprecated and only used as an example
  71. 71. Spring + Hystrix Circuit breaker fallback Note: Hystrix is deprecated and only used as an example
  72. 72. Tracing
  73. 73. As an engineer you should still thinkabout these concerns
  74. 74. Key take-aways from this talk • A service mesh is a dedicated infra layer for service communication • Understand the why of using a service mesh • Understand the operational complexity, but also the benefits e.g. transparently adds cross-cutting concerns to a microservices architecture • Think about where you want to solve specific problems
  75. 75. “Please rate my talk in the official J-Fall app” #jfall

×