No C-QL (Or how I learned to stop worrying, and love eventual consistency) (NoSQL Matters September 2014)
Apr. 16, 2015•0 likes•526 views
3 likes
Be the first to like this
Show More
views
Total views
0
On Slideshare
0
From embeds
0
Number of embeds
0
Report
Internet
Traditional relational databases focus on ACID, providing strong semantics that require careful synchronisation between actors that limit scalability. NoSQL Column Stores such as Cassandra, Riak and Dynamo offer another way, by eschewing strong consistency you can meet your application's needs while also increasing scalability and reliability. This talk will cover how and where to use eventual consistency.
Brian BrazilFollow
Founder at Robust PerceptionRecommended
Prometheus lightning talk (Devops Dublin March 2015)Brian Brazil
Monitoring Hadoop with Prometheus (Hadoop User Group Ireland, December 2015)Brian Brazil
Provisioning and Capacity Planning Workshop (Dogpatch Labs, September 2015)Brian Brazil
Prometheus design and philosophy Docker, Inc.
Systems Monitoring with Prometheus (Devops Ireland April 2015)Brian Brazil
Ansible at FOSDEM (Ansible Dublin, 2016)Brian Brazil
No C-QL (Or how I learned to stop worrying, and love eventual consistency) (NoSQL Matters September 2014)
- No C-QL (Or how I learned to stop worrying, and love eventual consistency) Brian Brazil Senior Software Engineer Boxever
- Overview NoSQL and eventual consistency are complementary. Why would you need it, how do you take advantage of it, and when is it useful in distributed systems?
- A Simple Example Website that counts how many times a user visits the site, and each page.
- First visit User Total Count Brian 1 User Page Count Brian index 1 Webserver DB index Read Old Total Count Insert New Total Count Read Count Insert New Count DB DB
- Second visit User Total Count Brian 2 User Page Count Brian index 1 Brian sales 1 Webserver sales Read Old Total Count Update New Total Count Read Count Insert New Count DB DB DB
- What do you think?
- Initial thoughts Looks like a simple, obvious solution
- Tabbed Browsing
- Tabbed Browsing Webserver A: Read Old Total Count B: Read Old Total Count A: Update New Total Count B: Update New Total Count . . . Webserversales index DB DB DB
- Tabbed Browsing User Total Count Brian 3 User Page Count Brian index 2 Brian sales 2 Webserver A: Read Old Total Count B: Read Old Total Count A: Update New Total Count B: Update New Total Count . . . Webserversales index DB DB DB
- What happened? Concurrent updates clashed. Need to protect against that.
- ACID to the rescue! ● Apply each set of updates atomically ● Updates don’t interfere with each other
- ACID to the rescue? ● Bottleneck for writes ● Webserver needs to retry ● Databases need to coordinate for quorum
- Aside: Cost of Reads ● Hard disk seek: 5 - 10ms
- Aside: Cost of Reads ● Hard disk seek: 5 - 10ms ● Cluster quorum read: 1 - 200ms
- Aside: Cost of Reads ● Hard disk seek: 5 - 10ms ● Cluster quorum read: 1 - 200ms ● RTT to webserver: 1 - 200ms
- Aside: Cost of Reads ● Hard disk seek: 5 - 10ms ● Cluster quorum read: 1 - 200ms ● RTT to webserver: 1 - 200ms Avoid reads on the write path.
- Eventual consistency Being slightly out of date doesn’t matter for many applications. Can we take advantage of this to avoid bottlenecks and latency?
- First visit User UniqueID Page Brian 1409778671-Server1-PID index Webserver DB index Insert Visit DB DB
- Second visit User UniqueID Page Brian 1409778671-Server1-PID index Brian 1409778689-Server1-PID sales Webserver DB sales Insert Visit DB DB
- Tabbed Browsing User UniqueID Page Brian 1409778671-Server1-PID index Brian 1409778689-Server1-PID sales Brian 1409778691-Server1-PID index Brian 1409778691-Server2-PID sales DB A: Insert Visit B: Insert Visit DB DB Webserver Webserversales index
- Analysis ● Unique IDs avoid clashes ● No bottlenecks due to consistency ● Data always internally consistent ● Fire & forget ● Only need to talk to nearest database(s) ● Resilient to network partitions
- Not all good ● Reads have to look at more data ● Need to make sure data makes it everywhere ● What if something depends on having data up to time X?
- NoSQL? Everything thus far applies equally to relational and non-relational databases.
- NoSQL Advantage ● Data locality on disk, multiple entries per row ● Replication options beyond master/slave ● Writes are cheap ● Can also have consistency where needed
- When is it good? ● Writes dominate ● Need high throughput and scalability ● Inter/intra continental databases ● 100% correct data in realtime not essential
- When is it not so good? ● Writes are relatively rare ● Scale isn’t a concern ● Data must be completely consistent
- Idempotency Each entry is unique, so can be safely replayed again and again. Gives you options for disaster recovery: Log requests to local webserver disk, replay if database goes down
- Improvements ● Take advantage of entry ordering ● Do rollups as a batch task ○ Reduces data to be processed, cutting latency ● Row per day/week/month ○ Avoids rows getting too big ● Handling updates and deletes ○ Can be done using only puts
- Advanced This is a very simple example, only doing a count. For more see “Extreme availability and self-healing data with CRDTs” at 11am with Uwe Friedrichsen in Room 2
- Summary Consistency brings bottlenecks. Eventual consistency allows for high- throughput, reliable writes. NoSQL combined with eventual consistency can scale very well.
- Questions?