Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Multicluster Kubernetes: How a Service Mesh Can and Can’t Help

157 views

Published on

A look at different ways to achieve multicluster communication and when it makes sense to use a service mesh.

Published in: Technology

Multicluster Kubernetes: How a Service Mesh Can and Can’t Help

  1. 1. Multiclusterk8s: How a Service MeshCan/Can’t Help Andrew Jenkins, CTO @notthatjenkins
  2. 2. Aspen Mesh Survey at KubeCon 2019 Europe Multiple Independent Prod Dev/Test/Stage Multiple x-comm Prod Multiple (85%) (10%) Other (5%) One
  3. 3. Service Mesh Service Mesh Control Plane App A Proxy App B Proxy Service A Service B ObservabilitySecurityTraffic Management k8s apiserver Cluster
  4. 4. Service Mesh Service Mesh Control Plane App A Proxy App B Proxy Service A Service B k8s apiserver Cluster A Service Mesh Control Plane App A Proxy App B Proxy Service A Service B k8s apiserver Cluster B
  5. 5. Service Mesh App A Proxy App B Proxy Service A Service B k8s apiserver Cluster A App A Proxy App B Proxy Service A Service B k8s apiserver Cluster B Service Mesh Control Plane
  6. 6. Service Mesh App A Proxy App B Proxy Service A Service B k8s apiserver Cluster A App A Proxy App B Proxy Service A Service B k8s apiserver Cluster B Service Mesh Control Plane
  7. 7. Service Mesh Service Mesh Control Plane App A Proxy App B Proxy Service A Service B k8s apiserver Cluster A Service Mesh Control Plane App A Proxy App B Proxy Service A Service B k8s apiserver Cluster B Higher Level
  8. 8. There are many reasons to want to run multiple clusters… * Blast-radius (a problem in one cluster doesn't kill the whole system) * Environment isolation (dev, test, prod) * Reliability (a zone or region outage does not bring down the app) * Latency (run the app as close to customers as possible) * Scale (the app is too big to fit in a single cluster) * Provider diversity (for regulatory, geographic, data gravity, or other reasons) * Jurisdiction (keep user data in-country) * Upgrade scope (upgrade infra for some parts of your app but not all of it) * Avoid the need for in-place cluster upgrades * Performance isolation (teams don't want to feel each other) * Security isolation (sensitive data or untrusted code) * Organizational isolation (teams have different management domains) * Cost isolation (teams get different bills) Tim Hockin, Re: Proposing Submariner as a sig-multicluster
  9. 9. What does the Internet have to teach us?
  10. 10. NarrowPurpose Diversity …what itruns on …what runs on it ...how big it is Any system with an IP addresscan send packetsto any other system with anIP address Internet IEEE 802.3 IEEE 802.5 IEEE 802.11 RFC1577 RFC2549
  11. 11. ~100GB/wk 1983 ~100GB/day 1992 ~100GB/hour 1997 ~100GB/second2002 ~100GB/ 50ms 2007 ~100GB/ms 2019
  12. 12. Scalable Evolutionary
  13. 13. Example
  14. 14. B D C A
  15. 15. B D C A To Hops Via AJ 1 - To Hops Via Cat 1 - Routing InformationProtocol(RIP) Bellman-Ford
  16. 16. B D C A To Hops Via AJ 1 - To Hops Via Cat 1 - To Hops Via Cat 2 D
  17. 17. B D C A To Hops Via AJ 1 - To Hops Via Cat 1 - To Hops Via Cat 2 D To Hops Via Cat 3 C
  18. 18. B D C A To Hops Via AJ 1 - To Hops Via Cat 1 - To Hops Via Cat 2 D To Hops Via Cat 3 C AJ 2 A
  19. 19. B D C A To Hops Via AJ 1 - To Hops Via Cat 1 - To Hops Via Cat 2 D To Hops Via Cat 3 2 C D AJ 2 A
  20. 20. B D C A To Hops Via AJ 1 - Cat 3 B To Hops Via Cat 1 - To Hops Via Cat 2 D To Hops Via Cat 3 2 C D AJ 2 A
  21. 21. Scalable Evolutionary
  22. 22. B D C A E FG H I
  23. 23. B D C A E FG H I AS4037 AS717 AS2310 Border GatewayProtocol(BGP)
  24. 24. B D C A E FG H I AS4037 AS717 AS2310 RIP RIP RIP OSPF
  25. 25. OK, what does this have to do with k8s?
  26. 26. There are many reasons to want to run multiple clusters… * Blast-radius (a problem in one cluster doesn't kill the whole system) * Environment isolation (dev, test, prod) * Reliability (a zone or region outage does not bring down the app) * Latency (run the app as close to customers as possible) * Scale (the app is too big to fit in a single cluster) * Provider diversity (for regulatory, geographic, data gravity, or other reasons) * Jurisdiction (keep user data in-country) * Upgrade scope (upgrade infra for some parts of your app but not all of it) * Avoid the need for in-place cluster upgrades * Performance isolation (teams don't want to feel each other) * Security isolation (sensitive data or untrusted code) * Organizational isolation (teams have different management domains) * Cost isolation (teams get different bills) Tim Hockin, Re: Proposing Submariner as a sig-multicluster
  27. 27. Scalable Evolutionary
  28. 28. Unified Management – Configurethem all inoneplace Unified Trust – Crypto trusttraceable back to onecommonroot Heterogenous Network – Clusters can have overlappingor non-routableinternal IPs Independent Fault Domain – If Cluster A blows up,Cluster B is still OK Inter-Cluster Mesh Traffic –Inter-cluster traffic is still Service Mesh traffic To Multicluster, or Not to Multicluster: Inter-cluster Communication Using a Service Mesh
  29. 29. UnifiedManagement UnifiedTrust Heterogenous Network Independent Fault Domain Inter-clusterMesh Traffic Independent ✓ ✓ Common Management ✓ ✓ ✓ Flat Network ✓ ✓ ✓ Split Horizon ✓ ✓ ✓ ✓ Cluster-AwareService Routing ✓ ✓ ✓ ✓ To Multicluster, or Not to Multicluster: Inter-cluster Communication Using a Service Mesh
  30. 30. Istio: Multicluster Deployments Split Horizon ✓ Unified Mgmt ✓ Unified Trust ✓ Hetero Network ✘Fault Domain ✓ X-Cluster Mesh
  31. 31. Istio: Multicluster Deployments Cluster-Aware Service Routing ✘ Unified Mgmt ✓ Unified Trust ✓ Hetero Network ✓ Fault Domain ✓ X-Cluster Mesh
  32. 32. Recap
  33. 33. Aspen Mesh Survey at KubeCon 2019 Europe Multiple Independent Prod Dev/Test/Stage Multiple x-comm Prod Multiple (85%) (10%) Other (5%) One
  34. 34. There are many reasons to want to run multiple clusters… * Blast-radius (a problem in one cluster doesn't kill the whole system) * Environment isolation (dev, test, prod) * Reliability (a zone or region outage does not bring down the app) * Latency (run the app as close to customers as possible) * Scale (the app is too big to fit in a single cluster) * Provider diversity (for regulatory, geographic, data gravity, or other reasons) * Jurisdiction (keep user data in-country) * Upgrade scope (upgrade infra for some parts of your app but not all of it) * Avoid the need for in-place cluster upgrades * Performance isolation (teams don't want to feel each other) * Security isolation (sensitive data or untrusted code) * Organizational isolation (teams have different management domains) * Cost isolation (teams get different bills) Tim Hockin, Re: Proposing Submariner as a sig-multicluster
  35. 35. Unified Management – Configurethem all inoneplace Unified Trust – Crypto trusttraceable back to onecommonroot Heterogenous Network – Clusters can have overlappingor non-routableinternal IPs Independent Fault Domain – If Cluster A blows up,Cluster B is still OK Inter-Cluster Mesh Traffic –Inter-cluster traffic is still Service Mesh traffic To Multicluster, or Not to Multicluster: Inter-cluster Communication Using a Service Mesh
  36. 36. Thank You
  37. 37. Speaker Name Title
  38. 38. Section Title Goes Here
  39. 39. Unique, live, never-to-be repeated entertainment & experiences created by the magical interactions of many.
  40. 40. Slide Title Goes Here Observability Security Insights
  41. 41. Config data toEnvoys TLS certs toEnvoys Monitors K8s fornew pods toinject Envoys Mixer Sidecar InjectorPilot IstioControlPlane Ingress Gateway Egress GatewayEnvoy Container Flask Python Container SERVICEA SERVICEA Envoy Container Flask Python Container SERVICEA SERVICEA Envoy Container Flask Python Container SERVICEA SERVICEA Policy, quota &telemetry Citadel Managing Microservices with Istio

×