Recommended
More Related Content
What's hot
What's hot (20)
Similar to Architectural patterns for high performance microservices in kubernetes
Similar to Architectural patterns for high performance microservices in kubernetes (20)
More from Rafał Leszko
More from Rafał Leszko (15)
Recently uploaded
Recently uploaded (20)
Architectural patterns for high performance microservices in kubernetes
- 1. Architectural Patterns for High-Performance Microservices in Kubernetes Rafał Leszko @RafalLeszko rafalleszko.com Hazelcast
- 2. About me ● Cloud-Native Team Lead at Hazelcast ● Worked at Google and CERN ● Author of the book "Continuous Delivery with Docker and Jenkins" ● Trainer and conference speaker ● Live in Kraków, Poland
- 3. About Hazelcast ● Distributed Company ● Open Source Software ● 140+ Employees ● Products: ○ Hazelcast IMDG ○ Hazelcast Jet ○ Hazelcast Cloud @Hazelcast www.hazelcast.com
- 4. ● Introduction ● Distributed In-Memory Store Patterns ○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 5. Introduction
- 6. High-Performance Microservices ● Systems have tons of microservices ○ Netflix has over 1000! ● High Performance allows for complex use-cases, e.g. ○ AI processing ○ Mainframe optimization ○ ... ● Cloud cost optimization
- 7. High Performance is no longer nice-to-have
- 8. High Performance is a must!
- 9. Service 1 Service 2 v1 Service 2 v2 Service 3 Service 4 v1 Service 4 v2 Service 4 v3 Ruby Microservice World
- 10. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby library library library library
- 11. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 3 Service 4 v1 Service 4 v2 Service 4 v3 Ruby in-mem DB in-mem DB in-mem DB
- 12. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 3 Service 4 v1 Service 4 v2 Service 4 v3 Ruby reverse proxy reverse proxy reverse proxy
- 13. ● Introduction ✔ ● Distributed In-Memory Store Patterns ○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 14. 1. Embedded
- 16. private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; } Embedded Cache (Pure Java implementation)
- 17. private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; } Embedded Cache (Pure Java implementation)
- 18. ● No Eviction Policies ● No Max Size Limit (OutOfMemoryError) ● No Statistics ● No built-in Cache Loaders ● No Expiration Time ● No Notification Mechanism Java Collection is not a Cache!
- 20. Embedded Cache (Java libraries) CacheBuilder.newBuilder() .initialCapacity(300) .expireAfterAccess(Duration. ofMinutes(10)) .maximumSize(1000) .build();
- 21. Caching Application Layer @Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } }
- 22. Caching Application Layer @Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } } Be Careful, Spring uses ConcurrentHashMap by default!
- 26. @Configuration public class HazelcastConfiguration { @Bean CacheManager cacheManager() { return new HazelcastCacheManager( Hazelcast. newHazelcastInstance()); } } Embedded Distributed Cache (Spring with Hazelcast)
- 30. Embedded (Distributed) Pros Cons ● Simple configuration / deployment ● Low-latency data access ● No separate Ops Team needed ● Not flexible management (scaling, backup) ● Limited to JVM-based applications ● Data collocated with applications
- 31. ● Introduction ✔ ● Distributed In-Memory Store Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 32. 2. Client-Server
- 33. Application Kubernetes Service Application Request In-Memory Store / Computing Platform Client-Server Kubernetes Pod Kubernetes Pod
- 34. Client-Server Application Kubernetes Service Application Request In-Memory Store / Computing Platform Kubernetes Pod Kubernetes Pod
- 35. Client-Server Separate Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request In-Memory Store / Computing Platform Kubernetes Pod Kubernetes Pod
- 36. Client-Server Application Kubernetes Service Application Request In-Memory Store / Computing Platform Kubernetes Pod Kubernetes Pod
- 37. Client-Server Application Kubernetes Service Application Request In-Memory Store / Computing Platform Kubernetes Pod Kubernetes Pod
- 38. Client-Server
- 39. Client-Server Starting Hazelcast Server $ helm install hazelcast/hazelcast
- 40. Client-Server Hazelcast Client @Configuration public class HazelcastClientConfiguration { @Bean CacheManager cacheManager() { return new HazelcastCacheManager(HazelcastClient . newHazelcastClient()); } } Starting Hazelcast Server $ helm install hazelcast/hazelcast
- 41. Client-Server Hazelcast Client HazelcastInstance instance = HazelcastCacheManager( HazelcastClient. newHazelcastClient()); IMap<String, String> map = instance.getMap("my-map"); // Use in-memory storage map.put("key", "value"); // Use in-memory computing map.executeOnKeys(...); // Use stream processing, etc. Starting Hazelcast Server $ helm install hazelcast/hazelcast
- 42. Client-Server Separate Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request In-Memory Store / Computing Platform Kubernetes Pod Kubernetes Pod
- 43. 2*. Cloud
- 44. Application Kubernetes Service Application Request Cloud (Hazelcast as a Service) Kubernetes Pod Kubernetes Pod
- 45. Cloud (Hazelcast as a Service) Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request Kubernetes Pod Kubernetes Pod
- 46. Cloud (Hazelcast as a Service) Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request Kubernetes Pod Kubernetes Pod
- 47. Cloud (Hazelcast as a Service) Application Kubernetes Service Application Request Kubernetes Pod Kubernetes Pod
- 48. ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig().getCloudConfig() .setEnabled( true) .setDiscoveryToken( "KSXFDTi5HXPJGR0wRAjLgKe45tvEEhd"); clientConfig.setClusterName( "test-cluster"); HazelcastInstance hazelcast = HazelcastClient. newHazelcastClient(clientConfig)); Cloud (Hazelcast as a Service)
- 49. Client-Server (Cloud) Pros ● Data separate from applications ● Separate management (scaling, backup) ● Programming-language agnostic Cons ● Separate Ops effort ● Higher latency ● Server network requires adjustment (same region, same VPC)
- 50. ● Introduction ✔ ● Distributed In-Memory Store Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 51. 3. Sidecar
- 52. Kubernetes Service Request Hazelcast Cluster Kubernetes POD Application Container In-Mem DB Container Application Container In-Mem DB Container Kubernetes POD Sidecar
- 53. Sidecar Similar to Embedded: ● the same physical machine ● the same resource pool ● scales up and down together ● no discovery needed (always localhost) Similar to Client-Server: ● different programming language ● uses client to connect ● some isolation between app and server
- 54. ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig() .addAddress( "localhost:5701"); HazelcastInstance hazelcast = HazelcastClient .newHazelcastClient(clientConfig)); Sidecar
- 55. Sidecar Cache apiVersion: apps/v1 kind: Deployment ... spec: template: spec: containers: - name: application image: leszko/application - name: hazelcast image: hazelcast/hazelcast
- 56. Sidecar Pros Cons ● Simple configuration ● Programming-language agnostic ● Low latency ● Some isolation of data and applications ● Not flexible management (scaling, backup) ● Data collocated with application PODs
- 57. ● Introduction ✔ ● Distributed In-Memory Store Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ✔ ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 58. 4. Reverse Proxy
- 59. Application Kubernetes Service Cache Application Request Reverse Proxy Kubernetes Pod Kubernetes Pod
- 60. Reverse Proxy http { ... proxy_cache_path /data/nginx/cache keys_zone=one:10m; ... }
- 61. ● Only for HTTP ● Not distributed ● No High Availability ● Data stored on the disk NGINX Reverse Proxy Issues
- 62. 4*. Reverse Proxy Sidecar
- 63. Kubernetes Service Request Hazelcast Cluster Kubernetes POD Application Container Reverse Proxy Container Application Container Reverse Proxy Container Kubernetes POD Reverse Proxy Sidecar
- 65. Good
- 66. Reverse Proxy Sidecar Service 1 Service 2 v1 Service 2 v2 Service 3 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
- 67. Reverse Proxy Sidecar Service 1 Service 2 v1 Service 2 v2 Service 3 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
- 68. Reverse Proxy Sidecar apiVersion: apps/v1 kind: Deployment ... spec: template: spec: initContainers: - name: init-networking image: leszko/init-networking containers: - name: caching-proxy image: leszko/caching-proxy - name: application image: leszko/application
- 69. Reverse Proxy Sidecar Service 1 Service 2 v1 Service 2 v2 Service 3 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
- 71. Bad
- 73. Reverse Proxy @CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {} Application Cache:
- 74. Reverse Proxy @CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {} Application Cache: Proxy Cache: http { ... location / { add_header Cache-Control public; expires 86400; etag on; } }
- 75. Reverse Proxy (Sidecar) Pros Cons ● Configuration-based (no need to change applications) ● Programming-language agnostic ● Consistent with containers and microservice world ● Difficult cache invalidation ● No mature solutions yet ● Protocol-based (e.g. works only with HTTP)
- 76. ● Introduction ✔ ● Distributed In-Memory Store Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ✔ ○ Reverse Proxy ✔ ○ Reverse Proxy Sidecar ✔ ● Summary Agenda
- 77. Summary
- 81. application-aware? Reverse Proxy Reverse Proxy Sidecar no yes no early adopter?
- 82. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? yes no yes no early adopter?
- 83. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? yes no yes no no early adopter?
- 84. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) yes no yes no no no early adopter?
- 85. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar yes no yes yes no no no early adopter?
- 86. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar cloud? yes no yes yes yes no no no early adopter?
- 87. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar cloud? Client-Server yes no yes yes yes no no no no early adopter?
- 88. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar cloud? Client-Server Cloud yes no yes yes yes yes no no no no early adopter?
- 89. Resources ● Code for this talk: https://github.com/leszko/caching-patterns ● Hazelcast Sidecar Container Pattern: https://hazelcast.com/blog/hazelcast-sidecar-container-pattern/ ● Hazelcast Reverse Proxy Sidecar Prototype: https://github.com/leszko/caching-injector ● NGINX HTTP Reverse Proxy Caching: https://www.nginx.com/resources/videos/best-practices-for-caching/