Architectural caching patterns for kubernetes

1. Architectural Caching Patterns for Kubernetes Rafał Leszko @RafalLeszko rafalleszko.com Hazelcast

2. About me ● Cloud-Native Team Lead at Hazelcast ● Worked at Google and CERN ● Author of the book "Continuous Delivery with Docker and Jenkins" ● Trainer and conference speaker ● Live in Kraków, Poland

3. About Hazelcast ● Distributed Company ● Open Source Software ● 140+ Employees ● Products: ○ Hazelcast IMDG ○ Hazelcast Jet ○ Hazelcast Cloud @Hazelcast www.hazelcast.com

4. ● Introduction ● Caching Architectural Patterns ○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda

5. Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby Microservice World

6. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby cache cache cache cache

7. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby cache cache cache

8. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby cache cache cache

9. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda

10. 1. Embedded

11. Application Kubernetes Service Cache Application Cache Request Embedded Cache Kubernetes Pod Kubernetes Pod

12. private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; } Embedded Cache (Pure Java implementation)

13. private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; } Embedded Cache (Pure Java implementation)

14. ● No Eviction Policies ● No Max Size Limit (OutOfMemoryError) ● No Statistics ● No built-in Cache Loaders ● No Expiration Time ● No Notification Mechanism Java Collection is not a Cache!

15. CacheBuilder.newBuilder() .initialCapacity(300) .expireAfterAccess(Duration.ofMinutes(10)) .maximumSize(1000) .build(); Embedded Cache (Java libraries)

16. Embedded Cache (Java libraries) CacheBuilder.newBuilder() .initialCapacity(300) .expireAfterAccess(Duration.ofMinutes(10)) .maximumSize(1000) .build();

17. Caching Application Layer @Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } }

18. Caching Application Layer @Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } } Be Careful, Spring uses ConcurrentHashMap by default!

19. Embedded Cache Application Kubernetes Service Cache Application Cache Request Kubernetes Pod Kubernetes Pod

20. 1*. Embedded Distributed

21. Application Application Kubernetes Service Cache Cache Request Hazelcast Cluster Embedded Distributed Cache Kubernetes Pod Kubernetes Pod

22. @Configuration public class HazelcastConfiguration { @Bean CacheManager cacheManager() { return new HazelcastCacheManager( Hazelcast.newHazelcastInstance()); } } Embedded Distributed Cache (Spring with Hazelcast)

23. Hazelcast Discovery Plugins

24. Hazelcast Discovery Plugins

25. Embedded Distributed Cache Application Application Kubernetes Service Cache Cache Request Hazelcast Cluster Kubernetes Pod Kubernetes Pod

26. Embedded Cache Pros Cons ● Simple configuration / deployment ● Low-latency data access ● No separate Ops Team needed ● Not flexible management (scaling, backup) ● Limited to JVM-based applications ● Data collocated with applications

27. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda

28. 2. Client-Server

29. Application Kubernetes Service Application Request Cache Server StatefulSet Client-Server Cache Kubernetes Pod Kubernetes Pod

30. Client-Server Cache Application Kubernetes Service Application Request Cache Server StatefulSet Kubernetes Pod Kubernetes Pod

31. Client-Server Cache Separate Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request Cache Server StatefulSet Kubernetes Pod Kubernetes Pod

34. Client-Server Cache

35. Client-Server Cache

36. Client-Server Cache Starting Hazelcast Cache Server $ helm install hazelcast/hazelcast

37. Client-Server Cache Hazelcast Client @Configuration public class HazelcastClientConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig().getKubernetesConfig() .setEnabled(true); return new HazelcastCacheManager(HazelcastClient .newHazelcastClient(clientConfig)); } } Starting Hazelcast Cache Server $ helm install hazelcast/hazelcast

38. Client-Server Cache Separate Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request Cache Server StatefulSet Kubernetes Pod Kubernetes Pod

39. 2*. Cloud

40. Application Kubernetes Service Application Request Cloud (Cache as a Service) Kubernetes Pod Kubernetes Pod

41. Cloud (Cache as a Service) Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request Kubernetes Pod Kubernetes Pod

42. Cloud (Cache as a Service) Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request Kubernetes Pod Kubernetes Pod

43. Cloud (Cache as a Service) Application Kubernetes Service Application Request Kubernetes Pod Kubernetes Pod

44. @Configuration public class HazelcastCloudConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig().getCloudConfig() .setEnabled(true) .setDiscoveryToken("KSXFDTi5HXPJGR0wRAjLgKe45tvEEhd"); clientConfig.setGroupConfig( new GroupConfig("test-cluster", "b2f984b5dd3314")); return new HazelcastCacheManager( HazelcastClient.newHazelcastClient(clientConfig)); } } Cloud (Cache as a Service)

45. Client-Server (Cloud) Cache Pros ● Data separate from applications ● Separate management (scaling, backup) ● Programming-language agnostic Cons ● Separate Ops effort ● Higher latency ● Server network requires adjustment (same region, same VPC)

46. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda

47. 3. Sidecar

48. Kubernetes Service Request Hazelcast Cluster Kubernetes POD Application Container Cache Container Application Container Cache Container Kubernetes POD Sidecar Cache

49. Sidecar Cache Similar to Embedded: ● the same physical machine ● the same resource pool ● scales up and down together ● no discovery needed (always localhost) Similar to Client-Server: ● different programming language ● uses cache client to connect ● clear isolation between app and cache

50. @Configuration public class HazelcastSidecarConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig() .addAddress("localhost:5701"); return new HazelcastCacheManager(HazelcastClient .newHazelcastClient(clientConfig)); } } Sidecar Cache

51. Sidecar Cache apiVersion: apps/v1 kind: Deployment ... spec: template: spec: containers: - name: application image: leszko/application - name: hazelcast image: hazelcast/hazelcast

52. Sidecar Cache Pros Cons ● Simple configuration ● Programming-language agnostic ● Low latency ● Some isolation of data and applications ● Not flexible management (scaling, backup) ● Data collocated with application PODs

53. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ✔ ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda

54. 4. Reverse Proxy

55. Application Kubernetes Service Cache Application Request Reverse Proxy Cache Kubernetes Pod Kubernetes Pod

56. Reverse Proxy Cache http { ... proxy_cache_path /data/nginx/cache keys_zone=one:10m; ... }

57. ● Only for HTTP ● Not distributed ● No High Availability ● Data stored on the disk NGINX Reverse Proxy Cache Issues

58. 4*. Reverse Proxy Sidecar

59. Kubernetes Service Request Hazelcast Cluster Kubernetes POD Application Container Reverse Proxy Cache Container Application Container Reverse Proxy Cache Container Kubernetes POD Reverse Proxy Sidecar Cache

60. Good

61. Reverse Proxy Sidecar Cache Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby

63. Reverse Proxy Sidecar Cache apiVersion: apps/v1 kind: Deployment ... spec: template: spec: initContainers: - name: init-networking image: leszko/init-networking containers: - name: caching-proxy image: leszko/caching-proxy - name: application image: leszko/application

65. Reverse Proxy Sidecar Cache (Istio)

66. Reverse Proxy Sidecar Cache (Istio)

67. Bad

68. Reverse Proxy Cache @CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {} Application Cache:

69. Reverse Proxy Cache @CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {} Application Cache: Proxy Cache: http { ... location / { add_header Cache-Control public; expires 86400; etag on; } }

70. Reverse Proxy (Sidecar) Cache Pros Cons ● Configuration-based (no need to change applications) ● Programming-language agnostic ● Consistent with containers and microservice world ● Difficult cache invalidation ● No mature solutions yet ● Protocol-based (e.g. works only with HTTP)

71. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ✔ ○ Reverse Proxy ✔ ○ Reverse Proxy Sidecar ✔ ● Summary Agenda

72. Summary

73. application-aware?

74. application-aware? early adopter? no

75. application-aware? Reverse Proxy no no early adopter?

76. application-aware? Reverse Proxy Reverse Proxy Sidecar no yes no early adopter?

77. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? yes no yes no early adopter?

78. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? yes no yes no no early adopter?

79. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) yes no yes no no no early adopter?

80. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar yes no yes yes no no no early adopter?

81. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar cloud? yes no yes yes yes no no no early adopter?

82. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar cloud? Client-Server yes no yes yes yes no no no no early adopter?

83. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar cloud? Client-Server Cloud yes no yes yes yes yes no no no no early adopter?

84. Resources ● Code for this talk: https://github.com/leszko/caching-patterns ● Hazelcast Sidecar Container Pattern: https://hazelcast.com/blog/hazelcast-sidecar-container-pattern/ ● Hazelcast Reverse Proxy Sidecar Caching Prototype: https://github.com/leszko/caching-injector ● NGINX HTTP Reverse Proxy Caching: https://www.nginx.com/resources/videos/best-practices-for-caching/

85. Thank You! Rafał Leszko @RafalLeszko

Architectural caching patterns for kubernetes

You are reading a preview.

Architectural caching patterns for kubernetes

More Related Content

Similar to Architectural caching patterns for kubernetes

More from Rafał Leszko

Featured

Related Books

Related Audiobooks