Download as PDF, PPTX
More Related Content
Similar to Advanced Caching Patterns used by 2000 microservices - Code Motion
Advanced Caching Patterns used by 2000 microservices - Code Motion
- 1. Advanced Microservices Caching Patterns NatanSilnitsky - Backend Infra TL, Wix.com November 30, December 1-2, 2021
- 2. @NSilnitsky @NSilnitsky Caching Patterns Advanced Microservices CachingPatterns Natan Silnitsky Backend Infra TL, Wix.com natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil
- 3. @NSilnitsky Caching Patterns >200 Millionregistered users from 190 countries
- 4. @NSilnitsky @NSilnitsky Caching Patterns Rebecca, WixStores Stores Checkout Service * issues
- 5. @NSilnitsky @NSilnitsky Caching Patterns 1. Getapp identification 2. Start WIX APP MARKET Rebecca, Wix Stores HTTP Stores Checkout Service
- 6. @NSilnitsky @NSilnitsky Caching Patterns WIX APP MARKET Rebecca,Wix Stores 1. Get app identification 2. Start Stores Checkout Service Unreliable Requests * still available
- 7. @NSilnitsky @NSilnitsky Caching Patterns Alex, WixStores Store Catalog Stores Catalog Service retrieve
- 8. @NSilnitsky @NSilnitsky Caching Patterns Alex, WixStores Store Catalog Stores Catalog Service Request Overload DB latency and failure increase
- 9. @NSilnitsky @NSilnitsky No Cache -Increase in - loading and response - Network costs - DB failure Caching Patterns * traffic grows
- 10. @NSilnitsky @NSilnitsky With Cache - -reduce latency - reduce the needed scale - improve reliability Caching Patterns * DB is unavailable
- 11. @NSilnitsky When & howto Cache Examples from 2000 microservices of various use cases Caching Patterns #1 high risk/cost of network failure Like caching external critical configuration data. * S3 backed
- 12. @NSilnitsky When & howto Cache Examples from 2000 microservices of various use cases Caching Patterns #1 high risk/cost of network failure Like caching external critical configuration data. #2 To improve average latency for data layer access DynamoDB + CDC + LRU cache.
- 13. @NSilnitsky When & howto Cache Examples from 2000 microservices of various use cases Caching Patterns #1 high risk/cost of network failure Like caching external critical configuration data. #2 To improve average latency for data layer access DynamoDB + CDC + LRU cache. #3 Some very high external traffic cases Caches are very important - cache before the service.
- 14. @NSilnitsky When & howto Cache Examples from 2000 microservices of various use cases Caching Patterns #1 high risk/cost of network failure Like caching external critical configuration data. #2 To improve average latency for data layer access DynamoDB + CDC + LRU cache. #3 Some very high external traffic cases Caches are very important - cache before the service. #4 When NOT to - young products Don’t cache prematurely
- 15. @NSilnitsky Agenda 3 CachingPatterns: 1. S3 backed static cache 2. DynamoDB+CDC based cache 3. HTTP Reverse Proxy cache Caching Patterns
- 16. @NSilnitsky Agenda 3 CachingPatterns: 1. S3 backed static cache 2. DynamoDB+CDC based cache 3. HTTP Reverse Proxy cache Caching Patterns
- 17. @NSilnitsky Service WIX APP MARKET Caching Patterns forUnreliable Requests
- 18. @NSilnitsky Service WIX APP MARKET Caching Patterns OnInit - Fetch app defs Koboshi / Remote Data Fetcher
- 19. @NSilnitsky Service WIX APP MARKET Caching Patterns Koboshi/ Remote Data Fetcher Local cache or AWS S3
- 20. @NSilnitsky Service WIX APP MARKET Caching Patterns Koboshi/ Remote Data Fetcher Local cache AWS S3 or + Retry
- 21. @NSilnitsky @NSilnitsky Caching Patterns Cache TypeCache Size Best For Not For Read-through cache Before the 3rd party service Data has to fit into memory External source of static configuration Dynamically updating data
- 22. @NSilnitsky Next 3 CachingPatterns: 1. S3 backed static cache 2. DynamoDB+CDC based cache 3. HTTP Reverse Proxy cache Caching Patterns
- 23. @NSilnitsky Caching Patterns Store Catalog Stores Catalog Service for RequestOverload Read + Writes Browser HTTP * slower interaction
- 24. @NSilnitsky Caching Patterns Stores Catalog Service Cached KVStore 2. put Store Catalog 1. Write V1 K1 V1 K2 K3 K8 K9 V1 V2 V1 head End
- 25. @NSilnitsky Caching Patterns Stores Catalog Service Cached KVStore 1. get Store Catalog 2. Read on miss V1 K1 V1 K2 K3 K8 K9 V1 V2 V1 head End
- 26. @NSilnitsky Caching Patterns Stores Catalog Service Cached KVStore LRU Cache put Amazon DynamoDB put
- 27. @NSilnitsky Caching Patterns Stores Catalog Service Cross-DC Data replication& conflict resolution Amazon DynamoDB Cache
- 28. @NSilnitsky Caching Patterns Stores Catalog Service Amazon DynamoDB 0 12 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 Event K1- V2 is updated 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 K1 ---- V1 K2 ---- V1 K1 ----- V2 K3 ---- V1 K4 ---- V1 K5 ---- V1 CDC Topic Keeps a SEQUENCE OF EVENTS Kafka Broker Cache
- 29. @NSilnitsky Caching Patterns Stores Catalog Service Amazon DynamoDB 0 12 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 Event K1- V2 is updated 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 K1 ---- V1 K2 ---- V1 K1 ----- V2 K3 ---- V1 K4 ---- V1 K5 ---- V1 CDC Topic Keeps a SEQUENCE OF EVENTS Kafka Broker Consumer (Consumes from) Cached KV Store V1 K1 V1 K2 K3 K8 K9 V1 V2 V1 head End
- 30. @NSilnitsky Cache Caching Patterns Warm up Takemost recent events Amazon DynamoDB CDC Topic Kafka Broker Stores Catalog Service
- 31. @NSilnitsky Cache Caching Patterns Warm up(compact) Take ALL Up to date events Amazon DynamoDB CDC Topic Kafka Broker Stores Catalog Service
- 32. @NSilnitsky @NSilnitsky Caching Patterns Cache TypeCache Size Best For Not For Cache-Aside Before the DB No limitation on DB data size LRU cache size is configurable Make DB read access faster for “popular” values Highly critical application startup information
- 33. @NSilnitsky Next 3 CachingPatterns: S3 backed static cache DynamoDB+CDC based cache HTTP Reverse Proxy cache Caching Patterns
- 34.
- 35. @NSilnitsky Caching Patterns Site Renderer Service DNS resolution Load Balancers * simplifiedflow Wix DC Varnish cache * open source
- 36. @NSilnitsky Caching Patterns DNS resolution Load Balancers * simplifiedflow Varnish cache Kafka Broker Site Published Purge request (SiteId) Site Editor Service Site Renderer Service Wix DC
- 37. @NSilnitsky Caching Patterns DNS resolution Load Balancers * simplifiedflow Varnish cache Kafka Broker Site Published Purge request (SiteId) Purge ETag: SiteId Site Editor Service Varnish Purger Service Site Renderer Service Wix DC
- 38. @NSilnitsky @NSilnitsky Varnish cache Size limit dependson the configured Storage backends: 1. malloc is memory based — so the size is limited by (more) expensive memory cost 2. file is memory-backed-by-disk based — which is limited by less expensive disk cost, but can cause performance penalty due to fragmentation… Caching Patterns
- 39. @NSilnitsky @NSilnitsky Varnish cache Size limit dependson the configured Storage backends: 1. malloc is memory based — so the size is limited by (more) expensive memory cost 2. file is memory-backed-by-disk based — which is limited by less expensive disk cost, but can cause performance penalty due to fragmentation… 3. Massive Storage Engine (MSE) — also backed by file, but has a “fragmentation proof algorithm.” part of Varnish Cache Plus (paying customers only). Caching Patterns
- 40. @NSilnitsky @NSilnitsky Caching Patterns Cache TypeCache Size Best For Not For Read-through before the app Depends on storage backend Improving latency for “stable” HTTP responses Hard to invalidate aggregated data
- 41. @NSilnitsky @NSilnitsky Caching Patterns Choose Cache Pattern Isyour data crucial for startup? Is your data highly dynamic? No Yes S3 Persisted data cache
- 42. @NSilnitsky @NSilnitsky Caching Patterns Choose Cache Pattern Isyour data crucial for startup? Is your data highly dynamic? S3 Persisted data cache No Yes Is your data highly dynamic? Yes No DynamoDB backed cache / Redis Yes
- 43. @NSilnitsky @NSilnitsky Caching Patterns Choose Cache Pattern Isyour data crucial for startup? Is your data highly dynamic? S3 Persisted data cache No Yes Is your data highly dynamic? Yes No External clients & easy invalidation? No DynamoDB backed cache / Redis Varnish cache / CDN Yes No Yes
- 44.
- 45. @NSilnitsky A Java/Scala high-levelSDK for Apache Kafka. github.com/wix/greyhound Caching Patterns 0.2 is out!
- 46. @NSilnitsky @NSilnitsky Thank You! 👉 slideshare.net/NatanSilnitsky Any questions? natansil.comtwitter@NSilnitsky linkedin/natansilnitsky github.com/natansil Caching Patterns
- 47. @NSilnitsky @NSilnitsky Q&A natansil.com twitter@NSilnitsky linkedin/natansilnitskygithub.com/natansil 👉 slideshare.net/NatanSilnitsky Any questions?