7 Reasons Not to Put an External Cache in Front of Your Database.pptx
•Download as PPTX, PDF•
0 likes•248 views
This document discusses the pros and cons of placing an external cache in front of a database. It introduces Tomasz Grabiec and Tzach Livyatan from ScyllaDB and describes ScyllaDB's optimized internal caching design. External caches can increase latency and costs while ignoring the database's context and workload knowledge. ScyllaDB embeds its cache to minimize overhead and ensure data and query awareness. The document shares customer examples that improved performance and reduced costs by moving from cached databases to ScyllaDB.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to 7 Reasons Not to Put an External Cache in Front of Your Database.pptx
Similar to 7 Reasons Not to Put an External Cache in Front of Your Database.pptx (20)
More from ScyllaDB
More from ScyllaDB (20)
Recently uploaded
Recently uploaded (20)
7 Reasons Not to Put an External Cache in Front of Your Database.pptx
- 1. Tomasz Grabiec (Tomek) Distinguished Engineer at ScyllaDB Tzach Livyatan,VP Product at ScyllaDB Moderated by: Erik Costlow, InfoQ Editor 7 Reasons Not to Put an External Cache in Front of Your Database
- 2. Introductions Tomasz Grabiec, Distinguished Engineer at ScyllaDB + Years of experience with Linux and distributed systems + With ScyllaDB since the beginning of the project + An open source enthusiast Tzach Livyatan,VP Product at ScyllaDB + Long career in development, system engineering and product management. + NoSQL enthusiast
- 3. Poll Are you using cache in front of your DB?
- 4. + For data-intensive applications that require high throughput and predictable low latencies + Close-to-the-metal design takes full advantage of modern infrastructure + >5x higher throughput + >20x lower latency + >75% TCO savings + Compatible with Apache Cassandra and Amazon DynamoDB + DBaaS/Cloud, Enterprise and Open Source solutions The Database for Gamechangers 4 “ScyllaDB stands apart...It’s the rare product that exceeds my expectations.” – Martin Heller, InfoWorld contributing editor and reviewer “For 99.9% of applications, ScyllaDB delivers all the power a customer will ever need, on workloads that other databases can’t touch – and at a fraction of the cost of an in-memory solution.” – Adrian Bridgewater, Forbes senior contributor
- 5. 5 +400 Gamechangers Leverage ScyllaDB Seamless experiences across content + devices Digital experiences at massive scale Corporate fleet management Real-time analytics 2,000,000 SKU -commerce management Video recommendation management Threat intelligence service using JanusGraph Real time fraud detection across 6M transactions/day Uber scale, mission critical chat & messaging app Network security threat detection Power ~50M X1 DVRs with billions of reqs/day Precision healthcare via Edison AI Inventory hub for retail operations Property listings and updates Unified ML feature store across the business Cryptocurrency exchange app Geography-based recommendations Global operations- Avon, Body Shop + more Predictable performance for on sale surges GPS-based exercise tracking Serving dynamic live streams at scale Powering India's top social media platform Personalized advertising to players Distribution of game assets in Unreal Engine
- 6. Agenda + ScyllaDB Intro + Why latency is critical + Why placing a cache in front of a DB might be a bad idea + ScyllaDB caching design + Summary + Q&A
- 7. 7 Why Latency is critical And why ScyllaDB focuses on latency
- 8. Lower Consistent Latency -> Higher Revenue insideline.com site to reduce load times from nine seconds to 1.4 seconds, ad revenue increased three percent, and page views-per-session went up 17 percent. https://www.thinkwithgoogle.com/future-of-marketing/digital-transformation/the- google-gospel-of-speed-urs-hoelzle/ https://www.globaldots.com/resources/blog/latency-is-having-a-huge-negative-impact-on-ecommerce- companies https://www.fastcompany.com/1825005/how-one-second-could-cost-amazon-16-billion-sales
- 9. Tail Latency - why you should care Refresh User App Business Logic DB API Calls DB Calls Slowest 1% DB responses dominated UX latency
- 10. Deep dive into Low Latency Engineering https://www.p99conf.io/
- 11. 11 Why placing a cache in front of a DB might be a bad idea And when it might be useful
- 12. 12 Why users put a cache in front of a DB? Better latency for hot data
- 13. Why users put a cache in front of a DB? >50ms > 50ms Better latency for hot data
- 14. 14 Why users put a cache in front of a DB? >50ms <1ms > 50ms >50ms > 50ms Better latency for hot data
- 15. Type of caches Embedded in App / External Process
- 16. 16 Type of caches Embedded in App / External Process DAX DAX DAX External / Transparent
- 17. 17 Type of caches Embedded in App / External Process DAX DAX DAX External / Transparent DAX DAX DAX
- 18. 18 Type of caches Embedded in App / External Process External DAX DAX DAX External / Transparent
- 19. 19 Type of caches Embedded in App / External Process External DAX DAX DAX External / Transparent
- 20. 20 Type of caches Embedded in App / External Process External Embedded in DB DAX DAX DAX External / Transparent
- 21. 21 Type of caches Embedded in App / External Process External DAX DAX DAX External / Transparent Embedded in DB
- 22. 22 Why users put a cache in front of a DB? >50ms > 50ms Better latency for hot data <1ms For the same AWS region <1ms For ScyllaDB
- 23. Problems with external cache 23 + Additional latency + Additional cost + Decreases availability + Increases application complexity + Ruins the DB caching + Ignoring DB knowledge + Reduce security
- 24. Additional latency External Embedded in DB <5 ms <1ms <1ms
- 25. Additional cost External Embedded in DB <5 ms <1ms <1ms
- 28. Database hold a lot of context on the data, and workload the cache is missing: + ScyllaDB is wide-column (Key-Key-Value), while a cache might by Key-Value only. Say goodbye to partition locality and efficient partition level queries. + Structured data: Tables, User Defined Types… + Cache setting per table + Time To Live (TTL) + Materialized View and Secondary Indexes + Much more… 28 Ignoring DB knowledge - Data Modeling
- 29. Key / Value
- 30. Key / Key / Value
- 31. CREATE TABLE caching ( k int PRIMARY KEY, v1 int, v2 int, ) WITH caching = {'enabled': 'true'}; Cache setting per table
- 32. Time To Live (TTL) - Table Level Default CREATE TABLE heartrate_ttl ( pet_chip_id uuid, owner uuid, time timestamp, heart_rate int, PRIMARY KEY (pet_chip_id, time)) WITH default_time_to_live = 604800; ▪ Powerful feature to remove data that is no longer needed ▪ ScyllaDB stores the TTL for each column value
- 33. The database hold a lot of context on the data, and workload the cache is missing: + Workload prioritization + Timer per workload + Scan-resistant caching + Role-based access control + Lightweight Transactions 33 Ignoring DB knowledge - Workload Security Risk!
- 34. SELECT * FROM users BYPASS CACHE; SELECT name, occupation FROM users WHERE userid IN (199, 200, 207) BYPASS CACHE; SELECT * FROM users WHERE birth_year = 1981 AND country = 'FR' ALLOW FILTERING BYPASS CACHE; BYPASS CACHE
- 35. Workload prioritization ■ OLTP ● Small work items ● Latency sensitive ● involves narrow portion of the data ■ OLAP ● Large work items ● Throughput oriented ● Performed on large amounts of data
- 42. + Read consistency easy + Pin sstables and memtable + Thanks to collocation + ..but slow 42 Data flow RAM Disk sstable sstable sstable Read memtable
- 43. 43 Data flow with cache memtable RAM Disk Read cache sstable sstable sstable
- 45. Inefficient use of memory: + Need to cache whole buffers to cache a single row + Access locality not likely if data set >> RAM 45 Why not buffer cache? SSTable page (4K) Row (300B)
- 46. Poor negative caching: + Need to cache whole data buffer to indicate absent data 46 Why not buffer cache? SSTable page (4K) ?
- 47. Inefficient use of memory: + Redundant buffers due to LSM + Read may touch multiple SSTables + Negative caching remark pronounced 47 Why not buffer cache? sstable sstable sstable Read
- 48. High CPU overhead for reads: + Reads need to merge data from multiple sstables 48 Why not buffer cache? sstable sstable sstable Read
- 49. High CPU overhead for reads: + SSTable format optimized for compact storage, not read speed + Parsing overhead: + Need to parse index buffers sequentially + Need to parse the data file 49 Why not buffer cache?
- 50. Premature cache eviction due to SSTable compaction: + SSTable compaction removes old files => buffer invalidation + Hurts read performance by incurring misses 50 Why not buffer cache? sstable sstable sstable sstable
- 51. + Object cache + Like memtable + Optimized for low CPU overhead + Fast reads + Row-granularity caching + Reflects data in all relevant SSTables for a given object (e.g. row) 51 Cache structure
- 52. + ScyllaDB reserves and manages most of the memory on a node + Small reserve for the OS + No use of Linux page cache (only direct I/O) + Cache uses all available free memory + Shrinked on pressure from memtable and other allocations 52 Memory management memtable cache other
- 53. 53 CPU sharding CPU 0 CPU 1 CPU 2 CPU 3
- 54. 54 Thread-per-core architecture task task task task task task task + All processing in a single thread per CPU + Short tasks executed serially + Cooperative preemption
- 55. 55 Cache coherency memtable Read cache task task + Complex operations on data without dealing with concurrency + No locking or complex lock-free algorithms + Data structures and algorithms simple memtable cache
- 56. 56 Complex DQL/DML SELECT * FROM table WHERE pk = 0 and ck >= 2; DELETE FROM table WHERE pk = 0 and ck >= 2;
- 57. 57 Range queries 2 5 SELECT * FROM table WHERE ... and ck >= 2; ?
- 58. 58 Range queries 2 5 SELECT * FROM table WHERE ... and ck >= 2; Range continuity
- 59. 59 Range deletions 2 DELETE FROM table WHERE ... and ck >= 2; Range continuity + tombstone
- 60. ScyllaDB cache highlights + ScyllaDB has a fast cache + Efficient access & maintenance + Thanks to collocation with replica and design + Takes care of consistency guarantees + Handles complexities of data and query model
- 62. 62 962 C* nodes to 78 60% TCO 95% latency “By moving to ScyllaDB Enterprise software running on AWS EC2 infrastructure and on- premises, Comcast improved P99 latency by more than 95% and were able to rip out a UI cache layer “
- 63. From Redis + Elasticsearch to ScyllaDB 63 <1ms P99 Zero downtime TCO
- 64. 64 TCO Speed of Redis From Redis to ScyllaDB for Data Stores, Fraud Detection, Ad Targeting Scalability
- 65. 65 <1ms avg Latency From Redis to Cassandra to ScyllaDB Cloud 4-8msP99 Fault Tolerance
- 66. 66 Throughput P99 Read Performance Infrastructure Savings Challenges with Cassandra + Needed better throughput for reads + Too much time tuning for garbage collection + “Node sprawl” caused high infrastructure costs ScyllaDB Solution + More reliable performance for consistently better customer experience + Less administration + Frictionless transition to ScyllaDB Moving to better price-performance
- 67. Summary + Putting a cache in front of your DB might be anti-productive + A cache lacks the context the DB has for each information element. + ScyllaDB’s Internal Cache is optimized to work in ScyllaDB Context with minimal overhead + Multiple customers have switched from a Cache+DB setup to ScyllaDB, reducing the latency, increasing the throughput with less HW
- 68. Thank you for joining us today. @scylladb scylladb/ slack.scylladb.com @scylladb company/scylladb/ scylladb/
Editor's Notes
- PRESENTER - Felipe 9:59:45 AM PT – Marisa, Cynthia, Julia mute themselves. Then Marisa to START WEBINAR IN ZOOM. Felipe starts talking at 10:00AM PT Good morning everyone and welcome to our webinar. We are going to give people a few more seconds as they funnel in and we will begin shortly. Felipe to wait 30 seconds as people join the webinar. Felipe to start talking again at 10:00:30 AM PT Hi everyone and welcome! Before we get started, I’d like to quickly review a couple of housekeeping items. We welcome your questions. Please use the Q&A button, located at the bottom of your screen to ask your questions. Remember, you can enter them any time during the webinar -- you don’t have to wait till the end. We will answer as many questions as we can get to at the end of the presentation. Also, please note that today’s webinar is being recorded. We will email you a link to the recording and the slides following the event.
- PRESENTER - Felipe Before we begin we are pushing a quick poll question. Where are you in your DBaaS adoption? I am currently evaluating DBaaS solutions I already have a solution I like It's on my company strategy to shift towards a DBaaS I am looking for alternatives None of the above, I am here to learn more! Ok, thanks for those responses. Let’s get started.
- PRESENTER - Felipe For those of you who are not familiar with ScyllaDB yet, it is the database behind gamechangers - organizations whose success depends upon delivering engaging experiences with impressive speed. ScyllaDB was built with a close-to-the-metal design that squeezes every possible ounce of performance out of modern infrastructure. This translates to predictable low latency even at high throughputs. With such consistent innovation the adoption of our database technology has grown to over 400 key players worldwide
- PRESENTER - Felipe Many of you will recognize some of the companies among the selection pictured here, such as Starbucks who leverage ScyllaDB for inventory management, Zillow for real-time property listing and updates, and Comcast Xfinity who power all DVR scheduling with ScyllaDB. As it can be seen, ScyllaDB is used across many different industries and for entirely different types of use cases. More than often, your company probably has a use case that is a perfect fit for ScyllaDB and it may be that you don’t know it yet!
- Mention HWLB When a cache node fails, latency jump because the DB cache is cold - Ruins the database caching! This is not the case for ScyllaDB! Since each info element is replicated (usally 3 times) there is at least 2 nodes with hot cache. ScyllaDB has a HWLB features which allow it to gradually warm the node.
- There are only two hard things in Computer Science: cache invalidation and naming things. — Phil Karlton
- This allow to ask questions like id=17 and date between X and Y
- Expire data automatically used with TWCS If you have timestamp and TTL with no manual deletes or updates, we highly recommend you to use TWCS 2 ways: table level - create or alter table If you set set with alter table, the old data will still have no tll if any, or the data will never expire.
- Workload prioritization is all about dealing with different loads on the same cluster. Before we can define what workload prioritization is we need to define what are a workloads key characteristics. So, the main Characteristics for a workload are: Latency - this is the time it takes to the server to answer on a query measured in some time unit (usually seconds or milliseconds) - there are many variations of measuring latency but it is not that important for our discussion. What is important is that latency - although very good in Scylla DB already can’t be much reduced - and we will try to explain why later with an analogy. Throughput - is the amount of data or operations processed by the server per time unit. This can be increased by expanding the Cluster and adding HW and nodes to it. Those make up two common Workload types: OLTP - stands for Online Transaction Processing, this type of load is typically generated by individuals or by applications that needs to do a large number of unrelated queries, the most important thing about this type of traffic is that it is usually latency sensitive, good example is user operations on websites. OLAP - stands for Online Analytical Processing , this load is typically generated by a single client and involves consuming large amounts of data by running relatively small amount of queries or alternatively, large amount of queries where the individual per query latency doesn’t matter as much as the amount of queries processed per second. This is typically some kind of an analysis on the existing data ( which is often generated by OLTP operations).
- And yes, we write to the commitlog for crash recovery
- Cache is inserted like this Represents subset of data in sstables
- Cache is inserted like this Represents subset of data in sstables
- An improvement would be to manage most of memory inside Scylla. Still..
- An improvement would be to manage most of memory inside Scylla. Still..
- An improvement would be to manage most of memory inside Scylla. Still..
- … and this is enabled by the fact that cache is collocated with the replica
- An improvement would be to manage most of memory inside Scylla. Still..
- Cache is inserted like this Represents subset of data in sstables
- Cache is inserted like this Represents subset of data in sstables
- Cache is inserted like this Represents subset of data in sstables
- Repeated scans never go to disk
- Purpose: Customer case study (Recommendation/Personalization - Media Streaming; Media & Entertainment) Audience: Mixed “Comcast Cable Communications which many know as Xfinity, is a telecommunications giant headquartered in the US that provides cable TV, internet, telephone, and wireless services “The Comcast X1 platform is a cable TV and streaming video service that incorporates a cloud DVR scheduling system for 15 million households, with 2B+ RESTful calls (reads/writes) and 200+M new objects per day. “Beginning first with Oracle and later moving to Cassandra, the scheduler engineering team struggled with database latency at scale. (click) “By moving to ScyllaDB Enterprise software running on AWS EC2 infrastructure and on-premises, Comcast improved P99, P999, and P9999 latency by more than 95% and were able to rip out a UI cache layer (click) “They dramatically reduced their total database infrastructure from 962 Cassandra nodes (across multiple clusters) to 78 ScyllaDB nodes. (click) “and they reduced total costs by more than 60%, saving Comcast over $2.5M annually in infrastructure costs and staff overhead. Note Philip Zimich featured in blog and recorded Summit presentation leads the architecture, development and operations of the Comcast’s X1 Scheduler system that powers the DVR and program reminder experience for the X1 platform Blog/recorded presentation: 78 nodes is total for 6 clusters across 3 data centers using Enterprise subscriptions with AWS infrastructure and on-premises Salesforce: today 5 clusters, 4 in production (2 on EC2, 2 on premises) and totaling 100+ nodes
- Purpose: Customer case study - (Recommendation/Personalization - Media Streaming; Media & Entertainment) Audience: Mixed “Based in India, Disney+ Hotstar provides on-demand streaming services to more than 18 million paid subscribers and 300 million monthly active users. “Disney + Hotstar’s “Continue Watching” feature tracks every show for every user, capturing timestamps when last watched so users can pick up where left off on any device, to prompt users to watch next episodes, and alert users to new episodes of favorite shows. “Using Kafka for streaming data and Redis (500GB) coupled with Elasticsearch (20TB) for their 20+TB data environment, the engineering team was running into scaling, data complexity, and cost issues. They considered a number of alternatives, from Cassandra and Apache HBase to DynamoDB, ultimately selecting our database-as- a-service ScyllaDB Cloud. The gains were compelling with Disney+ Hotstar…, (click) “achieving sub-millisecond p99 latency at scale (click) “a simplified data architecture with significantly lower TCO Note Blog: calls out 20TB, sub millisecond P99.
- Purpose: Customer case study (Recommendation/Personalization - Media Streaming; Media & Entertainment) Audience: Mixed “HQ’d in Singapore, Grab is an on-demand transportation company - whether for personal rides or food or package delivery - and one of the most used mobile apps in Southeast Asia. Grab relies on Kafka to stream data for a variety of business use cases. To read the streams they needed a powerful, low-latency metadata store to aggregate the streams and initially used Redis - but it couldn’t keep up with the load. So Grab looked at Cassandra, ScyllaDB, and other NoSQL solutions, and after extensive testing, selected ScyllaDB. (click) ScyllaDB performance was on par with Redis… (click) …but without the scalability and related cost challenges. It also proved much easier than managing Cassandra. Grab now uses ScyllaDB for a variety of use cases including fraud detection, ad targeting, and data store for their front end UI.
- Purpose: Customer case study (Recommendation/Personalization - Media Streaming; Media & Entertainment) Audience: Mixed “Now part of Fox, Tubi is an ad-supported media streaming service with over 50 millions active users. “Tubi uses ML and an innovative experimentation process to personalize movie recommendations. “Tubi initially used Redis for the recommendation database, but later moved to Cassandra. As their environment grew, so did the need for better latency, throughput, fault tolerance, and maintainability. “So they moved to ScyllaDB Cloud running on AWS. In addition to eliminating JVM tuning, (click) average read latency during peak times was reduced to sub-millisecond (click) and P99 was reduced to 4-8ms.
- So why did we end up with ScyllaDB? The candidate application chosen for this POC, is our geo system that provides information about geographical entities and the relationships between them. It aggregates data from multiple systems like hotel location info, 3rd party data , etc. This rich geography dataset enables different types of data searches, using a simple REST API while guarantying single digit msec P99 read response time. To speed up API responses, we are using multi layered cache with redis as first layer and cassandra as second level. With ScyllaDB as a swap in replacement for Cassandra, I’m handing it over to Dilip for going over the infra setup, benchmark results and next steps. The use case demands a high read throughput, while tiny write throughput. As shown in the first graph whether its ScyllaDB or Cassandra the writes are almost negligible or flat line at bottom. While the real winner is on Reads where the Cassandra P99 throughput is flaky as shown by the spikes, while the ScyllaDB P99 read response times are relatively flat. This is of significant advantage to our read heavy application. In terms of throughput comparison as shown in second graph, we were able to push almost double the TPS with ScyllaDB when compared with Cassandra, especially with a flat P99 SLA. Here are some of the facts that made ScyllaDB benchmark stand out. We were able to get triple the throughput with flat single digit P99 read response times, at the same time achieve over 35% reduction in total cost of ownership. At this point it was a no brainer to switch towards ScyllaDB for this application production workload.
- Contact Us: Tomasz Grabiec: tgrabiec@scylladb.com Tzach Livyatan: tzach@scylladb.com Join our Slack Channel ScyllaDB Slack Ask your questions on our user forum ScyllaDB Community NoSQL Forum