Successfully reported this slideshow.

Architectural patterns for high performance microservices in kubernetes

2

Share

1 of 90
1 of 90

More Related Content

More from Rafał Leszko

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Architectural patterns for high performance microservices in kubernetes

  1. 1. Architectural Patterns for High-Performance Microservices in Kubernetes Rafał Leszko @RafalLeszko rafalleszko.com Hazelcast
  2. 2. About me ● Cloud-Native Team Lead at Hazelcast ● Worked at Google and CERN ● Author of the book "Continuous Delivery with Docker and Jenkins" ● Trainer and conference speaker ● Live in Kraków, Poland
  3. 3. About Hazelcast ● Distributed Company ● Open Source Software ● 140+ Employees ● Products: ○ Hazelcast IMDG ○ Hazelcast Jet ○ Hazelcast Cloud @Hazelcast www.hazelcast.com
  4. 4. ● Introduction ● Distributed In-Memory Store Patterns ○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
  5. 5. Introduction
  6. 6. High-Performance Microservices ● Systems have tons of microservices ○ Netflix has over 1000! ● High Performance allows for complex use-cases, e.g. ○ AI processing ○ Mainframe optimization ○ ... ● Cloud cost optimization
  7. 7. High Performance is no longer nice-to-have
  8. 8. High Performance is a must!
  9. 9. Service 1 Service 2 v1 Service 2 v2 Service 3 Service 4 v1 Service 4 v2 Service 4 v3 Ruby Microservice World
  10. 10. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby library library library library
  11. 11. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 3 Service 4 v1 Service 4 v2 Service 4 v3 Ruby in-mem DB in-mem DB in-mem DB
  12. 12. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 3 Service 4 v1 Service 4 v2 Service 4 v3 Ruby reverse proxy reverse proxy reverse proxy
  13. 13. ● Introduction ✔ ● Distributed In-Memory Store Patterns ○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
  14. 14. 1. Embedded
  15. 15. Application Kubernetes Service library Application library Request Embedded Kubernetes Pod Kubernetes Pod
  16. 16. private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; } Embedded Cache (Pure Java implementation)
  17. 17. private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; } Embedded Cache (Pure Java implementation)
  18. 18. ● No Eviction Policies ● No Max Size Limit (OutOfMemoryError) ● No Statistics ● No built-in Cache Loaders ● No Expiration Time ● No Notification Mechanism Java Collection is not a Cache!
  19. 19. CacheBuilder.newBuilder() .initialCapacity(300) .expireAfterAccess(Duration. ofMinutes(10)) .maximumSize(1000) .build(); Embedded Cache (Java libraries)
  20. 20. Embedded Cache (Java libraries) CacheBuilder.newBuilder() .initialCapacity(300) .expireAfterAccess(Duration. ofMinutes(10)) .maximumSize(1000) .build();
  21. 21. Caching Application Layer @Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } }
  22. 22. Caching Application Layer @Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } } Be Careful, Spring uses ConcurrentHashMap by default!
  23. 23. Application Kubernetes Service library Application library Request Embedded Kubernetes Pod Kubernetes Pod
  24. 24. 1*. Embedded Distributed
  25. 25. Application Application Kubernetes Service library library Request Hazelcast Cluster Embedded Distributed Kubernetes Pod Kubernetes Pod
  26. 26. @Configuration public class HazelcastConfiguration { @Bean CacheManager cacheManager() { return new HazelcastCacheManager( Hazelcast. newHazelcastInstance()); } } Embedded Distributed Cache (Spring with Hazelcast)
  27. 27. Hazelcast Discovery Plugins
  28. 28. Hazelcast Discovery Plugins
  29. 29. Application Application Kubernetes Service library library Request Hazelcast Cluster Embedded Distributed Kubernetes Pod Kubernetes Pod
  30. 30. Embedded (Distributed) Pros Cons ● Simple configuration / deployment ● Low-latency data access ● No separate Ops Team needed ● Not flexible management (scaling, backup) ● Limited to JVM-based applications ● Data collocated with applications
  31. 31. ● Introduction ✔ ● Distributed In-Memory Store Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
  32. 32. 2. Client-Server
  33. 33. Application Kubernetes Service Application Request In-Memory Store / Computing Platform Client-Server Kubernetes Pod Kubernetes Pod
  34. 34. Client-Server Application Kubernetes Service Application Request In-Memory Store / Computing Platform Kubernetes Pod Kubernetes Pod
  35. 35. Client-Server Separate Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request In-Memory Store / Computing Platform Kubernetes Pod Kubernetes Pod
  36. 36. Client-Server Application Kubernetes Service Application Request In-Memory Store / Computing Platform Kubernetes Pod Kubernetes Pod
  37. 37. Client-Server Application Kubernetes Service Application Request In-Memory Store / Computing Platform Kubernetes Pod Kubernetes Pod
  38. 38. Client-Server
  39. 39. Client-Server Starting Hazelcast Server $ helm install hazelcast/hazelcast
  40. 40. Client-Server Hazelcast Client @Configuration public class HazelcastClientConfiguration { @Bean CacheManager cacheManager() { return new HazelcastCacheManager(HazelcastClient . newHazelcastClient()); } } Starting Hazelcast Server $ helm install hazelcast/hazelcast
  41. 41. Client-Server Hazelcast Client HazelcastInstance instance = HazelcastCacheManager( HazelcastClient. newHazelcastClient()); IMap<String, String> map = instance.getMap("my-map"); // Use in-memory storage map.put("key", "value"); // Use in-memory computing map.executeOnKeys(...); // Use stream processing, etc. Starting Hazelcast Server $ helm install hazelcast/hazelcast
  42. 42. Client-Server Separate Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request In-Memory Store / Computing Platform Kubernetes Pod Kubernetes Pod
  43. 43. 2*. Cloud
  44. 44. Application Kubernetes Service Application Request Cloud (Hazelcast as a Service) Kubernetes Pod Kubernetes Pod
  45. 45. Cloud (Hazelcast as a Service) Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request Kubernetes Pod Kubernetes Pod
  46. 46. Cloud (Hazelcast as a Service) Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request Kubernetes Pod Kubernetes Pod
  47. 47. Cloud (Hazelcast as a Service) Application Kubernetes Service Application Request Kubernetes Pod Kubernetes Pod
  48. 48. ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig().getCloudConfig() .setEnabled( true) .setDiscoveryToken( "KSXFDTi5HXPJGR0wRAjLgKe45tvEEhd"); clientConfig.setClusterName( "test-cluster"); HazelcastInstance hazelcast = HazelcastClient. newHazelcastClient(clientConfig)); Cloud (Hazelcast as a Service)
  49. 49. Client-Server (Cloud) Pros ● Data separate from applications ● Separate management (scaling, backup) ● Programming-language agnostic Cons ● Separate Ops effort ● Higher latency ● Server network requires adjustment (same region, same VPC)
  50. 50. ● Introduction ✔ ● Distributed In-Memory Store Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
  51. 51. 3. Sidecar
  52. 52. Kubernetes Service Request Hazelcast Cluster Kubernetes POD Application Container In-Mem DB Container Application Container In-Mem DB Container Kubernetes POD Sidecar
  53. 53. Sidecar Similar to Embedded: ● the same physical machine ● the same resource pool ● scales up and down together ● no discovery needed (always localhost) Similar to Client-Server: ● different programming language ● uses client to connect ● some isolation between app and server
  54. 54. ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig() .addAddress( "localhost:5701"); HazelcastInstance hazelcast = HazelcastClient .newHazelcastClient(clientConfig)); Sidecar
  55. 55. Sidecar Cache apiVersion: apps/v1 kind: Deployment ... spec: template: spec: containers: - name: application image: leszko/application - name: hazelcast image: hazelcast/hazelcast
  56. 56. Sidecar Pros Cons ● Simple configuration ● Programming-language agnostic ● Low latency ● Some isolation of data and applications ● Not flexible management (scaling, backup) ● Data collocated with application PODs
  57. 57. ● Introduction ✔ ● Distributed In-Memory Store Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ✔ ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
  58. 58. 4. Reverse Proxy
  59. 59. Application Kubernetes Service Cache Application Request Reverse Proxy Kubernetes Pod Kubernetes Pod
  60. 60. Reverse Proxy http { ... proxy_cache_path /data/nginx/cache keys_zone=one:10m; ... }
  61. 61. ● Only for HTTP ● Not distributed ● No High Availability ● Data stored on the disk NGINX Reverse Proxy Issues
  62. 62. 4*. Reverse Proxy Sidecar
  63. 63. Kubernetes Service Request Hazelcast Cluster Kubernetes POD Application Container Reverse Proxy Container Application Container Reverse Proxy Container Kubernetes POD Reverse Proxy Sidecar
  64. 64. Good
  65. 65. Reverse Proxy Sidecar Service 1 Service 2 v1 Service 2 v2 Service 3 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
  66. 66. Reverse Proxy Sidecar Service 1 Service 2 v1 Service 2 v2 Service 3 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
  67. 67. Reverse Proxy Sidecar apiVersion: apps/v1 kind: Deployment ... spec: template: spec: initContainers: - name: init-networking image: leszko/init-networking containers: - name: caching-proxy image: leszko/caching-proxy - name: application image: leszko/application
  68. 68. Reverse Proxy Sidecar Service 1 Service 2 v1 Service 2 v2 Service 3 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
  69. 69. Bad
  70. 70. Reverse Proxy @CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {} Application Cache:
  71. 71. Reverse Proxy @CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {} Application Cache: Proxy Cache: http { ... location / { add_header Cache-Control public; expires 86400; etag on; } }
  72. 72. Reverse Proxy (Sidecar) Pros Cons ● Configuration-based (no need to change applications) ● Programming-language agnostic ● Consistent with containers and microservice world ● Difficult cache invalidation ● No mature solutions yet ● Protocol-based (e.g. works only with HTTP)
  73. 73. ● Introduction ✔ ● Distributed In-Memory Store Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ✔ ○ Reverse Proxy ✔ ○ Reverse Proxy Sidecar ✔ ● Summary Agenda
  74. 74. Summary
  75. 75. application-aware?
  76. 76. application-aware? early adopter? no
  77. 77. application-aware? Reverse Proxy no no early adopter?
  78. 78. application-aware? Reverse Proxy Reverse Proxy Sidecar no yes no early adopter?
  79. 79. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? yes no yes no early adopter?
  80. 80. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? yes no yes no no early adopter?
  81. 81. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) yes no yes no no no early adopter?
  82. 82. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar yes no yes yes no no no early adopter?
  83. 83. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar cloud? yes no yes yes yes no no no early adopter?
  84. 84. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar cloud? Client-Server yes no yes yes yes no no no no early adopter?
  85. 85. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar cloud? Client-Server Cloud yes no yes yes yes yes no no no no early adopter?
  86. 86. Resources ● Code for this talk: https://github.com/leszko/caching-patterns ● Hazelcast Sidecar Container Pattern: https://hazelcast.com/blog/hazelcast-sidecar-container-pattern/ ● Hazelcast Reverse Proxy Sidecar Prototype: https://github.com/leszko/caching-injector ● NGINX HTTP Reverse Proxy Caching: https://www.nginx.com/resources/videos/best-practices-for-caching/
  87. 87. Thank You! Rafał Leszko @RafalLeszko

×