Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Make Java Microservices Resilient with Istio - Mangesh - IBM - CC18

1,177 views

Published on

This presentation was made by Mangesh Patankar (Developer Advocate - IBM Cloud) as part of Container Conference 2018: www.containerconf.in.

"How do we make microservices resilient and fault-tolerant? How do we enforce policy decisions, such as fine-grained access control and rate limits? How do we enable timeouts/retries, health checks, etc.?

A service-mesh architecture attempts to resolve these issues by extracting the common resiliency features needed by a microservices framework away from the applications and frameworks and into the platform itself. Istio provides an easy way to create this service mesh."

Published in: Technology
  • Be the first to comment

Make Java Microservices Resilient with Istio - Mangesh - IBM - CC18

  1. 1. Make Java Microservices Resilient with Istio IBM Code Patterns: http://developer.ibm.com/code Mangesh Patankar - Developer Advocate mapatank@in.ibm.com @MangeshPatank
  2. 2. Agenda • Evolution Of Microservice • Relation of Micro services with Container Orchestrator (K8s) • Why Service Mesh (Istio)? • Features of Istio • Demo
  3. 3. Evolution of Microservices
  4. 4. Monolithic Application to Microservices Application
  5. 5. Microservices Application Scaling
  6. 6. Microservices Application Update
  7. 7. Typically microservices are encapsulated inside containers… One:One relationship between a microservice and a container Everyone’s container journey starts with one container…. IBM Cloud - Container Service
  8. 8. At first the growth is easy to handle…. IBM Cloud - Container Service
  9. 9. Microservices Application Interactions - Simple
  10. 10. But soon it is overwhelming…we need container and microservices management IBM Cloud - Container Service
  11. 11. Enter Container Orchestrator IBM Cloud Container Service
  12. 12. Slide Title Goes Here Container Stack Physical InfrastructureLayer 1 Virtual InfrastructureLayer 2 Operating SystemLayer 3 Container EngineLayer 4 Orchestration/Scheduling Service Model Layer 5 Development Workflow Opinionated Containers Layer 6
  13. 13. Kubernetes
  14. 14. Slide Title Goes Here What is Kubernetes? • Container orchestrator • Runs and manages containers • Supports multiple cloud and bare-metal environments • Inspired and informed by Google's experiences and internal systems(Borg, Omega) • 100% Open source, written in Go • Manage applications, not machines • Rich ecosystem of plug-ins for scheduling, storage, networking
  15. 15. Intelligent Scheduling Self-healing Horizontal scaling Service discovery & load balancing Automated rollouts and rollbacks Secret and configuration management IBM Cloud- Container Service
  16. 16. Slide Title Goes Here Kubernetes Architecture API UI CLI Kubernetes Master Worker Node 1 Worker Node 2 Worker Node 3 Worker Node n Registry • Etcd • API Server • Controller Manager Server • Scheduler Server
  17. 17. Slide Title Goes Here Simplified Kubernetes Cluster
  18. 18. Kubernetes is great for Microservices… Why do we need a Service mesh and what is it?
  19. 19. Kubernetes Service Vs Service Mesh What a k8s Service component cannot do • To get more control of the traffic that goes to this API • To support many API versions • Do canary deployments • Watch and keep track of each request that comes in • L7 metrics • Traffic Control - Splitting • Rate limiting • Resiliency & Efficiency - Circuit breaking • Visibility • Security • Policy Enforcement …….
  20. 20. • Lightweight sidecars to manage traffic between services • Sidecars can do much more than just load balancing! How to build a ‘Service Mesh’ ?
  21. 21. Service Mesh • Linkerd • Conduit • Istio • ….
  22. 22. Istio
  23. 23. Istio Concepts • Pilot - Configures Istio deployments and propagate configuration to the other components of the system. Routing and resiliency rules go here • Mixer - Responsible for policy decisions and aggregating telemetry data from the other components in the system using a flexible plugin architecture • Proxy – Based on Envoy, mediates inbound and outbound traffic for all Istio-managed services. It enforces access control and usage policies, and provides rich routing, load balancing, and protocol conversion.
  24. 24. Istio Concepts
  25. 25. Istio Concepts
  26. 26. Istio Architecture
  27. 27. What is a ‘Service Mesh’ ? A network for services, not bytes " Resiliecny and efficiency ● Traffic Control ● Visibility ● Security ● Policy Enforcement
  28. 28. • Istio adds fault tolerance to your application without any changes to code • Resilience features ❖ Timeouts ❖ Retries with timeout budget ❖ Circuit breakers ❖ Health checks ❖ AZ-aware load balancing w/ automatic failover ❖ Control connection pool size and request load ❖ Systematic fault injection Resiliency
  29. 29. Code Pattern - Kubernetes, Microservices and Istio
  30. 30. Slide Title Goes Here IBM Cloud Kubernetes Container Service - IKS
  31. 31. IBM Code Pattern http://developer.ibm.com/code DEMO
  32. 32. Slide Title Goes Here Twelve-factor apps make a strong case for designing and implementing your microservices for failure. What that means is with the proliferation of microservices, failure is inevitable, and applications should be fault-tolerant. Istio, a service mesh, can help make your microservices resilient without changing application code. Developer Works Code: https://developer.ibm.com/code/patterns/make-java-microservices-resilient-with-istio/ Github: https://github.com/IBM/resilient-java-microservices-with-istio IBM Code Leverage Istio to create resilient and fault tolerant Microservices
  33. 33. MS-A Istio Ingress Envoy User Input EnvoyIstio Pilot Circuit Breaker ( X Max Conn, Y Max Pending) Administrator Set Destination Policy N requests N requests Reached maximum connec2ons – put the incoming requests in pending state Reached maximum pending requests - eject all the incoming requests. MS-B
  34. 34. MS-AEnvoy MS-B Pod 2 (Broken) Istio Pilot Circuit BreakerAdministrator Set Destination Policy 503 Load Balancing Pool for MS-B MS-B Pod 1 (Working) Eject X minutes 1 2 3 MS-BEnvoy Istio Ingress User Input N requests N requests Envoy MS-B
  35. 35. MS-B Pod Istio Pilot Timeout X seconds delay Fault Injection 504 error Administrator Set Route Rule MS-AEnvoy Envoy MS-B Istio Ingress N requests User Input N requests
  36. 36. What is a ‘Service Mesh’ ? A network for services, not bytes ● Resiliency & Efficiency ● Traffic Control ● Visibility ● Security ● Policy Enforcement
  37. 37. • // A simple traffic splitting rule • destination: serviceB.example.cluster.local • match: source: serviceA.example.cluster.local route: - tags: version: v1.5 • env: us-prod • weight: 99 • - tags: version: v2.0-alpha • env: us-staging • weight: 1 svcA Envoy Pod Service A svcB Envoy ServiceB http://serviceB.example Pod Labels: version: v1.5 env: us-prod svcB Envoy Pod Labels: version: v2.0- alpha, env:us- staging serviceB.example.cluster.loc Traffic routing rules 99% 1% Rules API Istio-Manager Traffic Splitting
  38. 38. © IBM Corporation / ConfidentialIBM Cloud I Internal Usage Only svcA Service A svcB Service B version: v1 Pod 1 Pod 2 Pod 3 Content-based traffic steering svcA Service A svcB Service B version: v1 Pod 1 Pod 2 Pod 3 svcB’ version: canary Pod 4 • // Content-based traffic steering rule • destination: serviceB.example.cluster.local match: httpHeaders: user-agent: regex: ^(.*?;)?(iPhone)(;.*)?$ precedence: 2 route: - tags: version: canary Traffic Steering
  39. 39. • Monitoring & tracing should not be an afterthought in the infrastructure • Goals • Metrics without instrumenting apps • Consistent metrics across fleet • Trace flow of requests across services • Portable across metric backend providers Istio Zipkin tracing dashboard Istio - Grafana dashboard w/ Prometheus backend Visibility
  40. 40. • Mixer collects metrics emitted by Envoys • Adapters in the Mixer normalize and forward to monitoring backends • Metrics backend can be swapped at runtime Pod Service A svcB Envoy Service B API: /svcB Latency: 10ms Status Code: 503 Src: 10.0.0.1 Dst: 10.0.0.2 …... Prometheu s InfluxDB Prometheus Adapter InfluxDB Adapter Custom Adapter Mixer Prometheu ss InfluxDB InfluxDB Custom backend Metric Flow Envoy svcA
  41. 41. • Application do not have to deal with generating spans or correlating causality • Envoys generate spans • Applications need to *forward* context headers on outbound calls • Envoys send traces to Mixer • Adapters at Mixer send traces to respective backends svcA Envoy Pod Service A svcB Envoy Service B Trace Headers X-B3-TraceId X-B3-SpanId X-B3-ParentSpanId X-B3-Sampled X-B3-Flags svcC Envoy Service C Span s Span s Prometheu s InfluxDB Zipkin Adapter Stackdriver Adapter Custom Adapter Mixer Prometheu sZipkin InfluxDB Stackdriver Custom backend Visibility : Tracing
  42. 42. Slide Title Goes Here Microservices and containers have changed application design and deployment patterns. They have also introduced new challenges, such as service discovery, routing, failure handling, and visibility to microservices. Kubernetes can handle multiple container-based workloads, including microservices, but when it comes to more sophisticated features like traffic management, failure handling, and resiliency, a microservices mesh like Istio is required. Developer Works Code: https://developer.ibm.com/code/patterns/manage-microservices-traffic-using-istio/ Github: https://github.com/IBM/microservices-traffic-management-using-istio IBM Code: Manage micro services traffic using Istio on Kubernetes
  43. 43. Slide Title Goes Here Microservices and containers have changed application design and deployment patterns. They have also introduced new challenges, such as service discovery, routing, failure handling, and visibility to microservices. Kubernetes can handle multiple container-based workloads, including microservices, but when it comes to more sophisticated features like traffic management, failure handling, and resiliency, a microservices mesh like Istio is required. Developer Works Code: https://developer.ibm.com/code/journey/manage-microservices-traffic-using-istio/ Github: https://github.com/IBM/microservices-traffic-management-using-istio IBM Code Manage micro services traffic using Istio on Kubernetes
  44. 44. Summary • Micro-service – Evolution and need • Framework supporting micro-service deployment • Scaling • Service Mesh • Frameworks for Service Mesh
  45. 45. Thank you!

×