NoSQL databases are one of the most successful technologies of Big Data era. Database community has faced many challenges even beyond Big Data era and presented several solutions. Despite all achievements, distributed transactions with external consistency remain to be one of the hardest computer science problems.

1. Dr. Parinaz Ameri
Databases through out and beyond
Big Data hype

2. Navigational
DBs
1980
Rise of
relational DBs
1990 2000 2010 2020

3. General Characteristics
Relational Schema SQL Language

4. Orders
Customers
Order Lines
Credit Cards
4
Impedance Mismatch

5. Navigational
DBs
1980
Rise of
relational DBs
1990
Object Oriented
Databases
Dominance of
relational DBs
2000
Rise of
Internet
2010 2020

7. Big Data: Volume
7
Scale up

8. Scale up vs. Scale out
8
Source: sap

9. 9
BigTable
DynamoDB

10. Graph
Columnar
Document
Key-Value
NoSQL Databases

11. Common Characteristics of NoSQL DBs
Non-Relational
Schemaless Cluster-Friendly
Weaker
Concurrency Model

12. Consistency
● Logical:
○ Bringing the DB from one consistent state to another consistent state
● Replication:
○ Propagation of write operation to various node

13. Browser Server
Database

15. Get Get
Post
Post

16. Get Get
Post
Post

17. Get Get
Post
Post

18. Get Get
Post
Post
Offline Lock
V001
V001 V001
V001
V002 V001

19. Eventual Consistency
● Update partitions independently
● Converged “Eventually”
● Probability voting on data value
● In practice: “last-write-win”
● Require conflict resolution
functionality in application layer
Source: DTBD

20. Example: Collaborative Applications
Time
Time
Hello Hello world
Hello :)Hello
Hello world :)
Hello world :)

21. Convergence Algorithms
● Operational transformation (1989)
○ C. A. Ellis and S. J. Gibbs. 1989. Concurrency control in groupware systems. In Proceedings of the 1989
ACM SIGMOD international conference on Management of data (SIGMOD '89)
○ Last Usage in the
● Conflict-free Replicated Data Types (2011-Present)
○ Sakr, Sherif and Zomaya, Albert. Conflict-free Replicated Data Types (CRDT), Encyclopedia of Big Data
Technologies, Springer International Publishing, 2018

22. Example: State of the Value
Time
Time
{a, b} {a}
{a, b, c}
{a, c}
{a, c}{a, b}

23. Example: Increment
Time
Time
X = 1 X = 2
X = 2
X = 3
X = 3X =1

24. CRTDs
● Decentralized solution
● Strong Eventual Consistency:
Once you have seen the same events, you are immediately in the same state.

25. Is it enough?
● CRTDs treat one’s transactions as debit and credit
● Deliberately inducing withdrawal races at ATMs
● Need “external consistency”

26. ● Mission-critical scalable relational DB service
○ “Spanner: Google’s Globally Distributed Database” in
ACM Transactions on Computer Systems (TOCS),
August 2013, page 22
● Lock-free distributed read transactions
● Scale out write transactions linearly
● Cross-node ACID transactions guaranteed
Google Spanner

27. Enabling Technologies
● Multi-shard transactions by a two-phase algorithm
● Updates to each shard in real-time order (serializability)
● Interval-based global time ensure external consistency
● Physical atomic clock synchronizes time within a small error bound

28. Coordinator
Participants
Time
Vote collection
Phase
Phase 1: Prepare Phase 2:
Commit
Vote
request
Yes
Prepare to
Commit
Prepared
Commit
Decision
Ack

29. Solved the problem?

30. Alternative Solutions
● Querying a single physical clock:
○ “Large-scale incremental processing using distributed transactions and notifications”,
In Proceedings of the 9th USENIX conference on Operating systems design and implementation (OSDI'10).
● Avoiding physical clock by Calvin:
○ “Calvin: fast distributed transactions for partitioned database systems”,
In Proceedings of the 2012 ACM SIGMOD

31. Summary
● NoSQL database succeed as Big Data technologies
● Need for ACID transactions not vanished
● Distributed transactions with external consistency remained to be solved

32. References
[sap] https://bit.ly/1zSUgZh
[DTBD] Big Data Principles and Paradigms/Chapter 6: Database Techniques for Big Data/ Parinaz Ameri