Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Where is my cache? Architectural patterns for caching microservices by example

244 views

Published on

Where is my cache? Architectural patterns for caching microservices by example

Published in: Software
  • Be the first to comment

  • Be the first to like this

Where is my cache? Architectural patterns for caching microservices by example

  1. 1. Where is my cache? Architectural patterns for caching microservices by example Rafał Leszko @RafalLeszko rafalleszko.com Hazelcast
  2. 2. About me ● Cloud Software Engineer at Hazelcast ● Worked at Google and CERN ● Author of the book "Continuous Delivery with Docker and Jenkins" ● Trainer and conference speaker ● Live in Kraków, Poland
  3. 3. About Hazelcast ● Distributed Company ● Open Source Software ● 140+ Employees ● Products: ○ Hazelcast IMDG ○ Hazelcast Jet ○ Hazelcast Cloud @Hazelcast www.hazelcast.com
  4. 4. ● Introduction ● Caching Architectural Patterns ○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
  5. 5. Why Caching? ● Performance ○ Decrease latency ○ Reduce load ● Resilience ○ High availability ○ Lower downtime
  6. 6. Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby Microservice World
  7. 7. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby cache cache cache cache
  8. 8. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby cache cache cache
  9. 9. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby cache cache cache
  10. 10. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
  11. 11. 1. Embedded
  12. 12. Application Load Balancer Cache Application Cache Request Embedded Cache
  13. 13. private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; } Embedded Cache (Pure Java implementation)
  14. 14. private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; } Embedded Cache (Pure Java implementation)
  15. 15. ● No Eviction Policies ● No Max Size Limit (OutOfMemoryError) ● No Statistics ● No built-in Cache Loaders ● No Expiration Time ● No Notification Mechanism Java Collection is not a Cache!
  16. 16. CacheBuilder.newBuilder() .initialCapacity(300) .expireAfterAccess(Duration.ofMinutes(10)) .maximumSize(1000) .build(); Embedded Cache (Java libraries)
  17. 17. Embedded Cache (Java libraries) CacheBuilder.newBuilder() .initialCapacity(300) .expireAfterAccess(Duration.ofMinutes(10)) .maximumSize(1000) .build();
  18. 18. Caching Application Layer @Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } }
  19. 19. Caching Application Layer @Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } } Be Careful, Spring uses ConcurrentHashMap by default!
  20. 20. Application Load Balancer Cache Application Cache Request Embedded Cache
  21. 21. 1*. Embedded Distributed
  22. 22. Application Application Load Balancer Cache Cache Request Hazelcast Cluster Embedded Distributed Cache
  23. 23. @Configuration public class HazelcastConfiguration { @Bean CacheManager cacheManager() { return new HazelcastCacheManager( Hazelcast.newHazelcastInstance()); } } Embedded Distributed Cache (Spring with Hazelcast)
  24. 24. DEMO
  25. 25. Hazelcast Discovery Plugins
  26. 26. Hazelcast Discovery Plugins
  27. 27. Hazelcast Discovery Plugins
  28. 28. Application Application Load Balancer Cache Cache Request Hazelcast Cluster Embedded Distributed Cache
  29. 29. Embedded Cache Pros Cons ● Simple configuration / deployment ● Low-latency data access ● No separate Ops Team needed ● Not flexible management (scaling, backup) ● Limited to JVM-based applications ● Data collocated with applications
  30. 30. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
  31. 31. 2. Client-Server
  32. 32. Application Load Balancer Application Request Cache Server Client-Server Cache
  33. 33. Application Load Balancer Application Request Cache Server Client-Server Cache
  34. 34. Application Load Balancer Application Request Cache Server Client-Server Cache Separate Management: ● backups ● (auto) scaling ● security Ops Team
  35. 35. Application Load Balancer Application Request Cache Server Client-Server Cache
  36. 36. Application Load Balancer Application Request Cache Server Client-Server Cache
  37. 37. Client-Server Cache
  38. 38. Client-Server Cache
  39. 39. Client-Server Cache $ ./start.sh Starting Hazelcast Cache Server (standalone)
  40. 40. Client-Server Cache Starting Hazelcast Cache Server (Kubernetes) $ helm install hazelcast/hazelcast
  41. 41. Client-Server Cache Hazelcast Client (Kubernetes): @Configuration public class HazelcastClientConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig().getKubernetesConfig() .setEnabled(true); return new HazelcastCacheManager(HazelcastClient .newHazelcastClient(clientConfig)); } } Starting Hazelcast Cache Server (Kubernetes) $ helm install hazelcast/hazelcast
  42. 42. Application Load Balancer Application Request Cache Server Client-Server Cache Ops Team Separate Management: ● backups ● (auto) scaling ● security
  43. 43. 2*. Cloud
  44. 44. Application Load Balancer Application Request Cloud (Cache as a Service)
  45. 45. Application Load Balancer Application Request Cloud (Cache as a Service) Management: ● backups ● (auto) scaling ● security Ops Team
  46. 46. Application Load Balancer Application Request Cloud (Cache as a Service) Management: ● backups ● (auto) scaling ● security Ops Team
  47. 47. Application Load Balancer Application Request Cloud (Cache as a Service)
  48. 48. @Configuration public class HazelcastCloudConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig().getCloudConfig() .setEnabled(true) .setDiscoveryToken("KSXFDTi5HXPJGR0wRAjLgKe45tvEEhd"); clientConfig.setGroupConfig( new GroupConfig("test-cluster", "b2f984b5dd3314")); return new HazelcastCacheManager( HazelcastClient.newHazelcastClient(clientConfig)); } } Cloud (Cache as a Service)
  49. 49. DEMO cloud.hazelcast.com
  50. 50. Client-Server (Cloud) Cache Pros ● Data separate from applications ● Separate management (scaling, backup) ● Programming-language agnostic Cons ● Separate Ops effort ● Higher latency ● Server network requires adjustment (same region, same VPC)
  51. 51. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
  52. 52. 3. Sidecar
  53. 53. Kubernetes Service (Load Balancer) Request Hazelcast Cluster Kubernetes POD Application Container Cache Container Application Container Cache Container Kubernetes POD Sidecar Cache
  54. 54. Sidecar Cache Similar to Embedded: ● the same physical machine ● the same resource pool ● scales up and down together ● no discovery needed (always localhost) Similar to Client-Server: ● different programming language ● uses cache client to connect ● clear isolation between app and cache
  55. 55. @Configuration public class HazelcastSidecarConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig() .addAddress("localhost:5701"); return new HazelcastCacheManager(HazelcastClient .newHazelcastClient(clientConfig)); } } Sidecar Cache
  56. 56. Sidecar Cache apiVersion: apps/v1 kind: Deployment ... spec: template: spec: containers: - name: application image: leszko/application - name: hazelcast image: hazelcast/hazelcast
  57. 57. Sidecar Cache Pros Cons ● Simple configuration ● Programming-language agnostic ● Low latency ● Some isolation of data and applications ● Limited to container-based environments ● Not flexible management (scaling, backup) ● Data collocated with application PODs
  58. 58. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ✔ ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
  59. 59. 4. Reverse Proxy
  60. 60. Application Load Balancer Cache Application Request Reverse Proxy Cache
  61. 61. Reverse Proxy Cache http { ... proxy_cache_path /data/nginx/cache keys_zone=one:10m; ... }
  62. 62. ● Only for HTTP ● Not distributed ● No High Availability ● Data stored on the disk NGINX Reverse Proxy Cache Issues
  63. 63. NGINX Reverse Proxy Cache Issues ● Only for HTTP ● Not distributed ● No High Availability ● Data stored on the disk
  64. 64. 4*. Reverse Proxy Sidecar
  65. 65. Kubernetes Service (Load Balancer) Request Hazelcast Cluster Kubernetes POD Application Container Reverse Proxy Cache Container Application Container Reverse Proxy Cache Container Kubernetes POD Reverse Proxy Sidecar Cache
  66. 66. Good
  67. 67. Reverse Proxy Sidecar Cache Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
  68. 68. Reverse Proxy Sidecar Cache Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
  69. 69. Reverse Proxy Sidecar Cache apiVersion: apps/v1 kind: Deployment ... spec: template: spec: initContainers: - name: init-networking image: leszko/init-networking containers: - name: caching-proxy image: leszko/caching-proxy - name: application image: leszko/application
  70. 70. Reverse Proxy Cache Container Application Container eth0 :80 Reverse Proxy Sidecar Cache :80 Pod :8000 lo :80
  71. 71. Reverse Proxy Sidecar Cache #!/bin/bash # Forward TCP traffic on port 80 to port 8000 on the eth0 interface. iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 80 -j REDIRECT --to-port 8000
  72. 72. Reverse Proxy Sidecar Cache Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
  73. 73. Reverse Proxy Sidecar Cache (Istio)
  74. 74. Reverse Proxy Sidecar Cache (Istio)
  75. 75. Bad
  76. 76. Reverse Proxy Cache @CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {} Application Cache:
  77. 77. Reverse Proxy Cache @CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {} Application Cache: Proxy Cache: http { ... location / { add_header Cache-Control public; expires 86400; etag on; } }
  78. 78. Reverse Proxy (Sidecar) Cache Pros Cons ● Configuration-based (no need to change applications) ● Programming-language agnostic ● Consistent with containers and microservice world ● Difficult cache invalidation ● No mature solutions yet ● Protocol-based (e.g. works only with HTTP)
  79. 79. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ✔ ○ Reverse Proxy ✔ ○ Reverse Proxy Sidecar ✔ ● Summary Agenda
  80. 80. Summary
  81. 81. application-aware?
  82. 82. application-aware? containers? no
  83. 83. application-aware? containers? Reverse Proxy no no
  84. 84. application-aware? containers? Reverse Proxy Reverse Proxy Sidecar no yes no
  85. 85. application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? yes no yes no
  86. 86. application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? yes no yes nono
  87. 87. application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) yes no yes no no no
  88. 88. application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) Sidecar yes no yes yes no no no
  89. 89. application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) Sidecar cloud? yes no yes yes yes no no no
  90. 90. application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) Sidecar cloud? Client-Server yes no yes yes yes no nono no
  91. 91. application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) Sidecar cloud? Client-ServerCloud yes no yes yes yes yes no nono no
  92. 92. Resources ● Hazelcast Sidecar Container Pattern: https://hazelcast.com/blog/hazelcast-sidecar-container-pattern/ ● Hazelcast Reverse Proxy Sidecar Caching Prototype: https://github.com/leszko/caching-injector ● Caching Best Practices: https://vladmihalcea.com/caching-best-practices/ ● NGINX HTTP Reverse Proxy Caching: https://www.nginx.com/resources/videos/best-practices-for-caching/
  93. 93. Thank You! Rafał Leszko @RafalLeszko rafalleszko.com

×