This talk reviews database clusters of our time which employ synchronous replication while being ACID compliant. ACID compliance implies ability to support transactions across nodes. As part of this talk, PXC (Percona XtraDB Cluster)/Galera, Google F1 based on Spanner/CFS and MySQL Cluster will be considered. Primary objective here is to expound features of
each in order to highlight differentiating factors and commonality between them.
5. Introduction
Seed quotes..
“’Network is reliable’ - a fallacy of the distributed
system.”
“A distributed system is one in which the failure of a
computer you didn’t even know existed can render your own
computer unusable.” - Leslie Lamport
“Those who would give up essential correctness, to
purchase a little temporary scalability, deserve neither
correctness nor scalability.” - Leif Walsh
“A given row can’t be modified more than once per
RTT." - Alexey Yurchenko
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 2 / 42
6.
7. Introduction
Introduction
I A twist on CAP
Pick any Two?
ACID
SQL
Synchronous replication
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 4 / 42
8. Introduction
Introduction
I A twist on CAP
Pick any Two?
ACID
SQL
Synchronous replication
I Solution: Don’t have to pick!
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 4 / 42
9. Introduction
Introduction
I A twist on CAP
Pick any Two?
ACID
SQL
Synchronous replication
I Solution: Don’t have to pick!
I CAP and latency - eventual consistency
I Lambda architecture
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 4 / 42
12. Introduction
RAS
I Why Synchronous
Symmetry
? Easier to manage and conceive
? Build once, deploy everywhere!
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 6 / 42
13. Introduction
RAS
I Why Synchronous
Symmetry
? Easier to manage and conceive
? Build once, deploy everywhere!
Latency!
I Do we need ACID?
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 6 / 42
14. Introduction
RAS
I Why Synchronous
Symmetry
? Easier to manage and conceive
? Build once, deploy everywhere!
Latency!
I Do we need ACID?
Depends!
Concurrent workload
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 6 / 42
15. Introduction
RAS
I Why Synchronous
Symmetry
? Easier to manage and conceive
? Build once, deploy everywhere!
Latency!
I Do we need ACID?
Depends!
Concurrent workload
I Relational database
Can it be a KV/document store?
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 6 / 42
16. Introduction
RAS
I Why Synchronous
Symmetry
? Easier to manage and conceive
? Build once, deploy everywhere!
Latency!
I Do we need ACID?
Depends!
Concurrent workload
I Relational database
Can it be a KV/document store?
Everyone wants SQL though.
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 6 / 42
17. Introduction
RAS
I Why Synchronous
Symmetry
? Easier to manage and conceive
? Build once, deploy everywhere!
Latency!
I Do we need ACID?
Depends!
Concurrent workload
I Relational database
Can it be a KV/document store?
Everyone wants SQL though.
Beats every other API!
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 6 / 42
18. Review
Contemporary systems
I PXC/Galera
I NDB Cluster
I Google F1
I Others - FoundationDB, CockroachDB
I Not talking of comparisons/benchmarks.
I Apples and Pineapples!
(Since they are all clusters)
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 7 / 42
19. Review
Family of Synchronous Systems
I Transactional replication
Overhead and workarounds
One-copy equivalence
NDB: 2 PC
I Virtual Synchrony
Extended Virtual Synchrony: Galera
I Multi-phase
Paxos: Google F1
Also 2PC
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 8 / 42
20. Review Architecture
Layers
I Necessity
Strengths
Degrees of freedom
I Monolithic v/s Layered
Monolithic - Galera
Layered - F1/Spanner, NDB(?)
I Implications
Failures
Debugging
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 9 / 42
21.
22. Review Architecture
Storage
I Unbundling of translation and data
I Different strategies
Spanner
? CFS underneath
? Provides guarantees in replication
Hierarchical storage
NDB
? Separatation of data and management nodes
? Hybrid storage
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 11 / 42
23. Review Architecture
Storage
I WSREP
Plugin system for Galera Provider
? InnoDB
? More tightly bound
? In-memory and ring-buffers
Transaction awareness
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 12 / 42
25. Review Architecture
Relational Database
I NoSQL / NewSQL / OldSQL
New wine in old bottle?
Approach towards ACIDity.
? Pros and Cons
? The Fit
Does NoSQL meet RAS
? Eventual (in)Consistency
? Strict definitions
? Instrumentation
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 14 / 42
26.
27. Review Architecture
Relational Database
I Role of API
Presence of Layer
Nature of storage
API defines data model?
I NoSQL access to SQL
I ORM
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 16 / 42
28.
29. Review Architecture
Degree of Synchronicity
I Latency is a killer!
Amortization
I Synchronicity imply Causality?
I Commutative writesets
I Role of Applications/Architecture
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 18 / 42
30. Review Architecture
ACIDity
I ACIDity test?
I BASE and NoSQL
I MVCC and Synchronous Replication
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 19 / 42
31.
32. Review Architecture
ACIDity
I Does ACID require synchronous OR
I Does Synchronous demand ACIDity?
Reconciliation requires it.
? Expensive without
? Rollbacks
ACID possible without Relational semantics?
Thought: Filesystems with replication
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 21 / 42
35. Review Architecture
Locking
I Conflicts and Deadlocks
Limiting factor: Amdahl’s
Scale: number of nodes and size of transactions
= Quadratic or Quinary
= Not strictly bound
= Only a strict lower bound
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 24 / 42
36. Review Architecture
Locking
I Pessimistic Locking
Performance
Network Overhead
? More roundtrips
? Pipeline/Batching
I Hybrid approach
Switching
I Granularity of Locking
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 25 / 42
37.
38. Review Architecture
Locking
I Externalities
GPS clock - Google F1
Controlled conditions
Limitations
Lock Managers: Zookeeper, Chubby
I Lamport timestamps!
A Zeitgeber!
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 27 / 42
39. Review Architecture
Integration
I Scaling the system upwards
I Composable Transactions
I XA Support
I Extensibility
Adding other storage engines: PSA
I Geographic Replication
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 28 / 42
40. Review Architecture
Sharding
I Systems that support it
I Issues of Quorum
I Adding support externally
Spider Engine
Cluster of clusters
MySQL Fabric?
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 29 / 42
41. Review Architecture
Transition
I From single node to a cluster
I Idempotency of transactions
I Integration with Async replication
I Loss of key parts
Degree of ACIDity
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 30 / 42
42. Review Architecture
Operational usage
I Resource Manager
Moving parts
Reduces the confusion
I Backups
Impact on cluster operations
Logical and Physical backups
NoSQL interface
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 31 / 42
43. Review Architecture
Operational usage
I Integration with other HA
Load balancers and Proxies
Fencing / STONITH
Resource Managers
? Pacemaker
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 32 / 42
44.
45. Review Architecture
Eventual Consistency
“not consistent right now, maybe come back later and
things will be better”
“Full transactional consistency is one of the most
important properties of F1.”
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 34 / 42
46. Review Architecture
EC: Qualitative Analysis
I Active analysis
System-centric
Convergence time
I Passive analysis
Distributed tracing
Dangling reads
Client-centric
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 35 / 42
50. Epilogue
Limitations
I None!
I Silver bullet?
I Where you may want Async / delay
I Hot updates/inserts
I Schema changes
I Compromise on strictness/features/performance
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 38 / 42
51. Epilogue
Further Reading
I Is this a solved problem?
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 39 / 42
52. Epilogue
Further Reading
I Is this a solved problem? May be not.
I The Dangers of Replication and a Solution
I Replication Using Group Communication Over a Partitioned Network
I Transaction Processing: Concepts and Techniques
I F1: A Distributed SQL Database That Scales
I Eventually Consistent: Not What You Were Expecting?
I The Layer Concept
I Don’t Settle for Eventual Consistency
I Lambda Architecture
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 39 / 42
53. Epilogue
Image Credits
I http://upload.wikimedia.org/wikipedia/commons/e/e8/Latency_map_
world.png
I http://blog.wikimedia.org/2014/07/09/
how-ripe-atlas-helped-wikipedia-users/
I https://secure.flickr.com/photos/alexbrn/5584251627
I https://secure.flickr.com/photos/russmorris/407778776
I https://secure.flickr.com/photos/carlosluzz/561920999
I https://secure.flickr.com/photos/eiriknewth/282268782
I https://secure.flickr.com/photos/shaireproductions/5509387767
I https://secure.flickr.com/photos/grantmac/3049823203
I http://guide.couchdb.org/draft/consistency/01.png
I http://www.yeeach.com/post/583
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 40 / 42
54. Epilogue
Summary
I Questions
I Theories
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 41 / 42
55. Epilogue
About
I /me: Raghavendra Prabhu, Product Lead, Percona XtraDB
Cluster, Percona.
I Slides will be at http://www.slideshare.net/slidunder
I Twitter: randomsurfer
I LinkedIn: rdprabhu
I Github: ronin13
I Presentation under CC BY-SA 4.0
Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 42 / 42