Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Building a scalable microservice architecture with envoy, kubernetes and istio

366 views

Published on

Talk from O'Reilly Software Architecture Conference San Jose 2019

Microservices and containers have taken the software industry by storm. Transitioning from a monolith to microservices enables you to deploy your application more frequently, independently, and reliably. However, microservice architecture has its own challenges, and it has to deal with the same problems encountered while designing distributed systems.

Enter service mesh technology to the rescue. A service mesh reduces the complexity associated with microservices and provides functionality like load balancing, service discovery, traffic management, circuit breaking, telemetry, fault injection, and more. Istio is one of the best implementations of a service mesh at this point, while Kubernetes provides a platform for running microservices and automating deployment of containerized applications.

Join Samir Behara to go beyond the buzz and understand microservices and service mesh technologies.

Published in: Technology
  • Be the first to comment

Building a scalable microservice architecture with envoy, kubernetes and istio

  1. 1. Building a scalable Microservice Architecture With Kubernetes, Envoy and Istio
  2. 2. SAMIR BEHARA System Architect, EBSCO Samir Behara builds software solutions using cutting edge technologies. Has a Bachelor Degree in Computer Science with 13 years of IT experience. Frequent Speaker at Technical Conferences. Author of www.dotnetvibes.com @samirbehara
  3. 3. Agenda • Monolith vs Microservices • How to break a Monolith into Microservices • Complexities in a Microservice Architecture • Journey from Netflix OSS to Istio Service Mesh • The Rise of Sidecar Design Pattern • Istio Architecture and capabilities • How to make your microservices resilient & fault tolerant • Service Mesh Observability
  4. 4. Monolithic Architecture Order Management Service Monolithic Database Large Codebase Difficult to Scale Longer Development Cycle Complicated Deployments Fixed Technology stack Performance Issues Tight Coupling
  5. 5. Applying Domain Driven Design
  6. 6. Monolith Architecture – Centralized Database Order Service Title Service Currency Service Pricing Service Monolithic Shared Database API Gateway
  7. 7. Databases are private to each Microservice Order API Pricing API
  8. 8. Deploying Monolithic Applications
  9. 9. Strangler Pattern in action
  10. 10. Transform and Eliminate Pattern TRANSFORM CO-EXIST ELIMINATE
  11. 11. Microservices Deployments
  12. 12. Emergence of Microservices Shorter Development Cycle Faster Deployments Highly Scalable Right Technology Stack Business Domain Driven Resiliency & Observability High Cohesion & Loose Coupling
  13. 13. Immutable Infrastructure Declarative Configuration Horizontal Scaling Self Healing SystemsService Discovery Decoupled Architecture Load Balancing Scalable Microservices with Kubernetes
  14. 14. Microservice Architecture - Challenges
  15. 15. 8 Fallacies of Distributed Computing Fallacy Solutions The network is reliable Automatic Retries, Message Queues Latency is zero Caching Strategy, Bulk Requests, Deploy in AZs near client Bandwidth is infinite Throttling Policy, Small payloads with Microservices The network is secure Network Firewalls, Encryption, Certificates, Authentication Topology does not change No hardcoding IP, Service Discovery Tools There is one administrator DevOps Culture eliminates Bus Factor Transport cost is zero Standardized protocols like JSON, Cost Calculation The network is homogenous Circuit Breaker, Retry and Timeout Design Pattern
  16. 16. Complexities in a Microservice Architecture
  17. 17. Tracing Service Discovery Dynamic Routing Resiliency Load Balancing Netflix OSS to the rescue
  18. 18. What are the issues with Netflix OSS? ROUTING CIRCUIT BREAKER LOAD BALANCING SERVICE DISCOVERY TRACING ROUTING CIRCUIT BREAKER LOAD BALANCING SERVICE DISCOVERY TRACING INFRASTRUCTURE SERVICE A SERVICE B • Tightly coupled to the Java Platform • Not a good fit for Polyglot Architecture • Netflix Libraries needs to be embedded inside each microservice along side Business functionalities • Increases overall Application Complexity • Operational Complexity - Patching/Upgrades
  19. 19. Sidecar Design Pattern Microservice A Microservice B Microservice C Sidecar Sidecar Sidecar Service Mesh Control Plane
  20. 20. Shared Libraries vs Service Mesh Pricing Service Sidecar Order Service Sidecar Currency Service Sidecar Customer Service Sidecar Title Service Sidecar Control Plane Business Logic + Shared Libraries Business Logic + Shared Libraries Business Logic + Shared Libraries Business Logic + Shared Libraries Business Logic + Shared Libraries
  21. 21. Smart Pipes and Smart Endpoints with Service Mesh Responsibility of network is to transfer messages Responsibility of microservices is to handle Business Logic, transformations, validations and process messages. Dumb Pipes and Smart Endpoints
  22. 22. Envoy • Envoy is a high performance Open Source Proxy designed for Cloud-Native Applications • Envoy makes the network transparent to the applications • Envoy is deployed as a Sidecar Proxy to every service • All traffic in a Microservice architecture flows via the Envoy Proxy Out of Process Architecture Service Discovery Load Balancing Circuit Breakers Fault Injection Observability
  23. 23. Istio • Platform to Connect, Secure, Control and Monitor Services consistently. • Open Source Service Mesh – Governed by Google & IBM • Shifts the complexity of running a distributed microservice architecture to the infrastructure layer • Control Plane for service proxies like Envoy • Platform Independent & Language agnostic
  24. 24. Istio Features Traffic Management Policy Enforcement Observability Security Telemetry
  25. 25. Service A Service B Network Service to Service Communication over Network
  26. 26. Service A Service B Sidecar Deployment using Envoy Proxy Envoy Proxy intercepts all network traffic flowing between applications
  27. 27. Service A Service B Configuration Validation, Management and Distribution Galley
  28. 28. Service A Service B Sidecar Configuration and Traffic Management capabilities Galley Pilot Push config data to sidecar proxies
  29. 29. Service A Service B Policy Enforcement and Telemetry Collection Galley Pilot Mixer Policy Checks & Telemetry
  30. 30. Service A Service B Enable Secure Communication using mutual TLS with built-in identity and credential management Galley Pilot Mixer Citadel Push TLS certificates to sidecar proxies
  31. 31. Service A Service B Galley Pilot Mixer Citadel Istio Mesh Integrated Control Plane
  32. 32. Sidecar Container SERVICE A Pod Container JVM SERVICE B Pod Container JVM Sidecar Container Istio Data Plane with Envoy Sidecar
  33. 33. SERVICE A SERVICE B Istio Architecture PILOT CITADEL MIXER Control Plane Data Plane Service Discovery Traffic Management Resiliency Policy Enforcement Telemetry Authentication Security GALLEY Configuration Validation and Distribution HTTP, gRPC, TCP Security - mTLS Pod Pod
  34. 34. Service A Istio Gateway Service Entry Virtual Service Service B Destination Rule User Traffic Istio Traffic Management
  35. 35. Traffic Routing Envoy Service A Pod Envoy Service B Pod Envoy Service B Pod Pod Labels - version: v1 env: staging Pod Labels - version: v2 env: prodPILOT Traffic Routing Rules # Route all traffic to v1 of ServiceB kind: VirtualService metadata: name: serviceB spec: hosts: - serviceB http: - route: - destination: host: serviceB subset: v1
  36. 36. Canary Deployment Envoy Service A Pod Envoy Service B Pod Envoy Service B Pod Pod Labels - version: v1 env: staging Pod Labels - version: v2 env: prod 90% 10% PILOT Traffic Routing Rules # Percentage based Traffic Split kind: VirtualService metadata: name: serviceB spec: hosts: - serviceB http: - route: - destination: host: serviceB subset: v1 weight: 90 - destination: host: serviceB subset: v2 weight: 10
  37. 37. Dark Launches Envoy Service A Pod Envoy Service B Pod Envoy Service B Pod Pod Labels - version: v1 env: staging Pod Labels - version: v2 env: prod 100% Mirror Traffic PILOT Traffic Routing Rules # Traffic Mirroring kind: VirtualService metadata: name: serviceB spec: hosts: - serviceB http: - route: - destination: host: serviceB subset: v1 weight: 100 mirror: host: serviceB subset: v2
  38. 38. Circuit Breaker Envoy Service A Pod Envoy Service B Pod Envoy Service C Pod # Limits the number of concurrent connections and requests kind: DestinationRule metadata: name: serviceC spec: hosts: - serviceC trafficPolicy: connectionPool: http: http1MaxPendingRequests: 10 maxRequestsPerConnection: 1 tcp: maxConnections: 1
  39. 39. Outlier Detection # Detect faulty instances in the pool & remove from traffic routing kind: DestinationRule metadata: name: serviceB spec: hosts: - serviceB trafficPolicy: outlierDetection: baseEjectionTime: 20s consecutiveErrors: 3 interval: 10s maxEjectionPercent: 100 Envoy Service A Pod Envoy Service B Pod Envoy Service B Pod Pod Labels - version: v1 env: staging Pod Labels - version: v2 env: staging
  40. 40. Timeout Envoy Service A Pod Envoy Service B Pod Envoy Service C Pod # Timeout strategy for service communication over network kind: VirtualService metadata: name: serviceB spec: hosts: - serviceB http: - route: - destination: host: serviceB timeout: 10s Timeout: 10 sec Timeout: 10 sec
  41. 41. Istio Retry Policy Envoy Service A Pod Envoy Service B Pod # Retry strategy for service communication over network kind: VirtualService metadata: name: serviceB spec: hosts: - serviceB http: - route: - destination: host: serviceB retries: attempts: 3 perTryTimeout: 2s Retry: 5 5XX Error
  42. 42. Chaos Testing – Inject Delays Envoy Service A Pod Envoy Service B Pod Envoy Service B Pod Pod Labels - version: v1 env: staging Pod Labels - version: v2 env: prod # Create rule to delay traffic to ServiceB v1 kind: VirtualService metadata: name: serviceB spec: hosts: - serviceB http: - fault: delay: fixedDelay: 10s percent: 50 route: - destination: host: serviceB subset: v1 10s delay in 50% of requests
  43. 43. Chaos Testing – Inject Errors Envoy Service A Pod Envoy Service B Pod Envoy Service B Pod Pod Labels - version: v1 env: staging Pod Labels - version: v2 env: prod # Create rule to inject errors to ServiceB v1 kind: VirtualService metadata: name: serviceB spec: hosts: - serviceB http: - fault: abort: httpStatus: 500 percent: 50 route: - destination: host: serviceB subset: v2 HTTP 500 in 50% of requests
  44. 44. Monitoring your Microservices Architecture
  45. 45. The Three Pillars of Observability LOGGING METRICS TRACING
  46. 46. Prometheus Architecture
  47. 47. Visualizing the Service Mesh with Kiali • Service Mesh Observability & Configuration • Visualize Service Communication in real time • Displays Traffic Rates and Latencies • Quickly identify problem areas • Configure, Update & Validate Service Mesh
  48. 48. View the response time and request rate of each of the microservice inside the service mesh.
  49. 49. Visualization and Time Series Analytics Create your own Dashboards for Monitoring and explore the service metrics Visualization tools to help you understand your data better
  50. 50. RED Metrics for Microservices Monitoring Rate - Number of requests per second your services are serving Errors - Number of failed requests per second Duration - Amount of time each request takes to fulfil a request
  51. 51. Distributed Tracing Provides end to end visibility & insights into service requests Used to troubleshoot latency issues in a Microservice Architecture
  52. 52. Resources Istio blogs on dotnetvibes - https://dotnetvibes.com/?s=istio Katacoda Interactive Learning Platform - https://www.katacoda.com Introducing Istio Service Mesh for Microservices - By Burr Sutter and Christian Posta https://developers.redhat.com/books/introducing-istio-service-mesh-microservices/ Red Hat Developer Blogs and Tutorials - https://developers.redhat.com/topics/service-mesh/ Istio Blogs - https://istio.io/blog/ O’Reilly Live Online Training - https://www.oreilly.com/live-training
  53. 53. Thank You @samirbehara www.dotnetvibes.com

×