Istio Triangle Kubernetes Meetup Aug 2019

1. Istio Connect, manage and secure microservices at scale Ram Vennam Technical Offering Manager Istio & IBM Cloud Kubernetes Service @RamVennam

2. Intelligent Scheduling Self-healing Horizontal scaling Service discovery & load balancing Automated rollouts and rollbacks Secret and configuration management Kubernetes

3. Challenges With Kubernetes 3 •The network among microservices may not be reliable. •How can my microservice handle unpredictable failures and retry? •How do I handle system degradation or topology change? •How can I monitor and trace my microservices? •As I develop multiple versions of my microservices, how can I easily dark launch and shift traffic? •How can I ensure the communication among microservices is secure? •How can I add enforce policies on my microservices?

4. Challenges with Microservices 4 Security Canary deployments A/B testing Retries and Circuit breaking Rate limiting Fault injection Policy management Telemetry

5. Solving the challenges in with not so micro microservices 5Think 2019 / DOC ID / Month XX, 2019 / © 2019 IBM Corporation https://dotnetvibes.com/2019/05/29/microservices-journey-from-netflix-oss-to-istio-service-mesh/

7. Istio Features 8Think 2019 / DOC ID / Month XX, 2019 / © 2019 IBM Corporation Intelligent Routing and Load Balancing Resiliency across Languages and Platforms Fleet Wide Policy Enforcement In-Depth Telemetry and Reporting

8. UI Order container pod How does it work? 9 Without Istio: • when service A (UI) talks to service B (Orders), it can use the local kube dns to find and talk to it directly. • If there are multiple instances of the Order, it uses standard round robin.

9. UI Order Policy container pod container check policies Envoy Request Interception 10 Istio deploys a proxy, using a sidecar pattern, that sits next to each of the services ● Service A -> Service B Client side a) Locally. envoy traps the requests, using IP Tables b) Envoy looks at that request, figures where we're going and then makes a client-side decision on where it is going to send that request c) Envoy will find the destination B host and send the request Server Side a) Checks policies in Mixer-Policy that this call is allowed, and responds to the B service request

10. UI Order container pod container Policy TelemetryPilot Citadel report Telemetry – Request and Response 11 Mixer-Telemetry is the Istio component responsible for providing policy controls and telemetry collection: • Both client and server asynchronously send data about the request for service B to Telemetry. • Data is provided for both request and response for service B

11. UI Order container pod container Policy TelemetryPilot Citadel config certs Piloting Traffic 12 The core component used for traffic management in Istio is Pilot ● Pilot lets you specify which rules you want to use to route traffic between Envoy proxies ● You configure failure recovery features such as timeouts, retries, and circuit breakers. ● Pilot also maintains a canonical model of all the services in the mesh ● Pilot uses this model to let Envoy instances know about the other Envoy instances in the mesh for service discovery

12. UI Order Orchestrate Key and certificate - Generation - Deployment - Rotation - Revocation Policy TelemetryPilot Citadel config certs Securing Traffic (Citadel) 13 Istio CA Istio:*myorg.com Istio:*myorg.com Istio:*myorg.com SAN: “Istio:foo.prod.myorg.com - Service account: foo - Namespace: prod SAN: “Istio:bar.prod.myorg.com - Service account: bar - Namespace: prod MTLS + Secure Naming Issue and Mount as Kubernetes Secrets Istio provides mutual TLS authentication between service to service communication Automatically creates certificate and key pair for each of the existing and new service accounts. Citadel stores the certificate and key pairs as Kubernetes secrets. Istio can control which microservices can talk to other

13. Istio Architecture Policy checks Policy checks Policy Telemetry Sidecar-injector Ingress- gateway Egress- gateway

14. What is Envoy ● Out of process architecture: Let’s do a lot of really hard stuff in one place and allow application developers to focus on business logic. ● Modern C++11 code base: Fast and productive. ● L3/L4 filter architecture: A TCP proxy at its core. Can be used for things other than HTTP (e.g., MongoDB, redis, stunnel replacement, TCP rate limiter, etc.). ● HTTP L7 filter architecture: Make it easy to plug in different functionality. ● HTTP/2 first! (Including gRPC and a nifty gRPC HTTP/1.1 bridge). ● Service discovery and active health checking. ● Advanced load balancing: Retry, timeouts, circuit breaking, rate limiting, shadowing, etc. ● Best in class observability: stats, logging, and tracing. ● Edge proxy: routing and TLS. https://www.youtube.com/watch?v=RVZX4CwKhGE

15. Rule Configuration Istio provides a simple configuration model to control how API calls and layer-4 traffic flow across various services in an application deployment The configuration model allows you to configure service-level properties such as circuit breakers, timeouts, and retries, as well as set up common continuous deployment tasks such as canary rollouts, A/B testing, staged rollouts with %-based traffic splits, etc. There are four traffic management configuration resources in Istio VirtualService, DestinationRule, ServiceEntry, and Gateway: • A Gateway configures a load balancer for HTTP/TCP traffic, most commonly operating at the edge of the mesh to enable ingress traffic for an application • A VirtualService defines the rules that control how requests for a service are routed within an Istio service mesh • A DestinationRule configures the set of policies to be applied to a request after VirtualService routing has occurred • A ServiceEntry is commonly used to enable requests to services outside of an Istio service mesh 16

16. 17 http://www.routetocloud.com/2019/04/introduction-to-istio-service-mesh/

17. Traffic entering the mesh Gateways (Ingress)

18. Bind to a gateway Apply routing rules VirtualService – match conditions from Gateway

19. Canary Testing: Route user:jason to reviews:v2 Others still get reviews:v1 Virtual Service - Request Routing

20. 95% -> v1 5% -> v3 Virtual Service - Traffic Shifting

21. Virtual Service - Delay & Fault Injection Inject 7 second delay

22. Destination Rules – subsets and tls Define subsets (versions) Tell clients to talk TLS with reviews

23. Rate Limits Cluster wide limits Overrides for services

25. Grafana

26. Jaeger

27. Kiali

28. Istio Performance & Latency 30 The Istio load tests mesh consists of 1000 services and 2000 sidecars with 70,000 mesh-wide requests per second. After running the tests using Istio 1.2.4, we get the following results: • The Envoy proxy uses 0.6 vCPU and 50 MB memory per 1000 requests per second going through the proxy. • The istio-telemetry service uses 0.6 vCPU per 1000 mesh-wide requests per second. • Pilot uses 1 vCPU and 1.5 GB of memory. • The Envoy proxy adds 8ms to the 90th percentile latency. https://istio.io/docs/concepts/performance-and-scalability

29. State of Istio 31 https://istio.io/about/community/customers/ Istio 1.2.4 After ~2.5 years of work ~300 developers from 100+ companies IBM, Google, VMWare, Cisco, Red Hat, Tigera, others… Many adapters Many customers

30. Manually installing Istio curl -L https://git.io/getLatestIstio | ISTIO_VERSION=1.2.4 sh - kubectl apply -f install/kubernetes/istio-demo.yaml

31. Managed Istio on IBM Cloud Kubernetes Service

32. Istio on OpenShift https://blog.openshift.com/istio-on-openshift/

33. Reasons to adopt Istio I want out-of-the-box telemetry and dashboard to monitor my services I want to perform robust testing and harden my environment. I want fine grained control over the flow the traffic in and out of my cluster I want strong identity and encryption between my services

34. Incrementally Adopting Istio

35. Start with the Ingress Gateway

36. Add services to mesh one at a time – manual sidecar injection kubectl apply -f <(istioctl kube-inject –f productpage.yaml)

37. Application Requirements and Gotchas 43 • Named service ports • Pod ports (containerPort) • Service association • Deployment labels • NET_ADMIN capability https://istio.io/docs/setup/kubernetes/additional-setup/requirements/ List of applications incompatible with Istio https:/github.com/istio/istio/issues/14743

38. Enable automatic sidecar injection (namespace) kubectl label namespace default istio-injection=enabled kubectl apply -f myapp.yaml

39. Enable mTLS https://istio.io/docs/tasks/security/authn-policy/

40. Harden https://istio.io/docs/tasks/traffic-management/ https://istio.io/docs/tasks/security/authz-http/ https://istio.io/docs/concepts/traffic-management/#sidecars https://istio.io/docs/tasks/traffic-management/egress/egress-gateway/

41. What’s new? • Installation Configuration Profiles • Improved Multicluster Integration • Limit scope using Sidecar resource • Locality-Aware Routing • Performance and Scalability Improvements • Readiness and Liveness Probes • Istio CNI plugin • Galley https://istio.io/about/notes/1.1/ https://istio.io/about/notes/1.2/ https://istio.io/about/feature-stages/

42. Multicluster and Multicloud 48 • Performance • Workload isolation • Dev/Test/Prod environments • Cost • Failover and redundancy 48

43. CLUSTER 1 CLUSTER 2 (Remote) Shared network Pilot Mixer Citadel istio-system istio-system bookinfo KUBE API KUBE API Pod: foo bookinfo Pod: bar Single network, single control plane Injector Citadel Injector • Remotes have smaller Istio • Internal CIDRs routable • Share remote cluster config • Service defined everywhere • Changing Istio service endpoints Service: bar Service: bar bar => CLUSTERIP => 10.0.221.1 Pod IP: 10.0.221.1

44. CLUSTER 1 CLUSTER 2 (Remote_ bookinfo Pod: foo Pod: bar Single control plane, separate network label: cluster1 label: cluster2 52.116.22.250 bar => CLUSTERIP => 52.116.22.250 istio-system KUBE API Citadel InjectorPilot Mixer Citadel KUBE API Injector • Remotes have smaller Istio • Split Horizon EDS • SNI routing • Service defined everywhere • Changing Istio gateway IPs • Gateway routing • Pass-through mTLS istio-system Service: bar Service: bar istio-ingressgateway +SNI

45. CLUSTER 1 CLUSTER 2 bookinfo KUBE API KUBE API bookinfo Multiple control planes istio-ingressgateway 52.116.22.250 Service Entry: bar.ns.global Pilot Mixer Citadel Injector istio-system • Simple to setup and scale • ServiceEntry for remote services • CoreDNS for resolving .global • Pass-through mTLS Pod: foo Service: bar Pod: bar Pilot Mixer Citadel Injector istio-system bar => bar.bookinfo.global => 52.116.22.250 +SNI

46. What’s coming? 52 Performance & Scalability: Control plane scalability Envoy performance UX: Istio Operator & Installer Simplified mTLS Rollout Control plane upgrades Mesh Expansion & Multi-Cluster patterns CRD Status and Debugging Tools Layering & Extensibility: Extensibility v2 (Mixer v2) Refactoring

47. Thank you! 53 Ram Vennam IBM Cloud Kubernetes Service @RamVennam rvennam@us.ibm.com ®