This document discusses NewSQL databases, which aim to provide the horizontal scalability of NoSQL databases with the ACID transactions of relational databases. It describes how NewSQL databases use techniques like consistent hashing, replication, and distributed transactions to achieve both scalability and transactions. Several examples of NewSQL databases are provided, including CockroachDB, Calvin, and NuoDB, as well as how they compare to traditional RDBMS and NoSQL databases.

1. Everything You Need to Know About NewSQL in 2020
Nikolay Stoitsev, Engineering Manager @ Halo DX

2. Journey

3. Relational DB
NoSQL

4. database size
fetched per second
updated per second

5. ACID
transactions

6. Atomicity

7. Consistency

8. Isolation

9. Durability

10. Scaling
How to scale it???

11. Horizontal
vs
Vertical

15. “Bigger servers
don’t exist”
The SRE team

16. Horizontal Scaling
10TB

17. Horizontal Scaling
2.5TB 2.5TB 2.5TB 2.5TB

18. Partitioning
& Routing

19. Replications
ID 1 - 100k ID 1 - 100kID 100k+ ID 100k+

20. Failure handling
ID 1 - 100k ID 1 - 100kID 100k+ ID 100k+
copy

21. Rebalancing
2.5TB 2.5TB 2.5TB 2.5TB
500GB
500GB
500GB
500GB

22. Horizontal scaling out of the box
Meet NoSQL

23. F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T.
Chandra, A. Fikes, and R. E. Gruber. Bigtable: A distributed storage system for
structured data. ACM Trans. Comput. Syst., 26:4:1–4:26, June 2008.

24. G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin,
S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: amazon’s highly
available key-value store. SIGOPS Oper. Syst. Rev., 41:205–220, October 2007.

25. Our choice
Apache
Cassandra

26. Why Apache Cassandra?
Sharding,
Replication,
Fault Tolerance
Decentralized
Multiple Data
Centers
Performance
Proven in
Production
Gives Control

28. But no
transactions :(

30. Consistent Hashing

31. Consistent Hashing
Two Phase Commit

32. Consistent Hashing
Two Phase Commit
Three Phase
Commit

33. Consistent Hashing
Two Phase Commit
Three Phase
Commit
Quorum

34. Consistent Hashing
Two Phase Commit
Three Phase
Commit
Quorum
Eventual
Consistency

35. Transactions + Scalability
Meet NewSQL

36. https://ai.google/research/pubs/pub27898

37. Partitioning tables in tablets
Tablet #1
Tablet #2
Tablet #3

38. Tablets are replicated
Node Node
Node
Node
Tablet #1
Tablet #1
Tablet #1

39. One leader for each tablet
Leader Follower
Follower
Leader for
tablet #1

40. Paxos to pick the leader
Leader Follower
Follower

41. Paxos to pick the leader
Leader Follower
Follower
New
Leader

42. Read queries

43. Read queries
Leader Follower
Follower

44. Read queries
Leader Follower
Follower
Can I read?
Here is the data
I have. Is it
latest?

45. Read queries
Leader Follower
Follower
Can I read?
Yes

46. Read queries
Leader Follower
Follower
Can I read?
Yes

47. Write queries

48. Write query
Leader Follower
Follower

49. Write query
Leader Follower
Follower

50. Write query
Leader Follower
Follower
Write
Write

51. Write query
Leader Follower
Follower
Write
Write
Done
Done

52. Write query
Leader Follower
Follower
Write
Write
Done
Done
Done

53. Distributed transactions

54. Distributed transaction
Leader
Follower
Follower
Leader
Follower
Follower

55. Distributed transaction
Leader
Follower
Follower
Leader
Follower
Follower
Paxos Magic

56. Distributed transaction
Leader
Follower
Follower
Leader
Follower
Follower
Transaction
Leader

57. Distributed transaction
Leader
Follower
Follower
Leader
Follower
Follower
Transaction
Leader

58. Distributed transaction
Leader
Follower
Follower
Leader
Follower
Follower
Transaction
Leader
Write

59. Distributed transaction
Leader
Follower
Follower
Leader
Follower
Follower
Transaction
Leader
Write
Done

60. Are transactions ACID?

61. Atomicity

62. Durability

63. Consistency

64. Isolation :(

65. True Time API
Atomic Clock GPS Clock

66. TrueTime API
time

67. Types of NewSQL databases
●
●
●

68. Databases with novel architecture
CockroachDB
● Open Source - https://github.com/cockroachdb/cockroach
● Easy to setup
● Horizontal scalability and high availability
● Geo-partitioning and distribution of data
● ACID transactions

69. How it works?

70. Calvin and FaunaDB
● Scalable
● ACID Transactions
● One global consensus protocol
● GraphQL
● User defined functions
● Commercial

71. Databases with novel architecture
NuoDB
● Horizontal scalability
● High availability
● ACID transactions
● Separate Transaction
Management and Storage
Management
● Used in banking
● https://nuodb.com
VoltDB
● Horizontal scalability with Geo
Replication
● High availability
● ACID transactions (Serialized)
● In-memory database
● Low latency
● Used in many domains
● https://www.voltdb.com/

72. Middlewares
Apache Ignite
● Peer-to-peer mesh network
● Distributed queries
● Distributed caching
● Storage and processing
framework
● Horizontally scalable
● Atomic types
● Replicated data structures
● https://ignite.apache.org
Apache Trafodion
● SQL query language on Apache
HBase
● Big data workloads
● ACID transactions
● https://trafodion.apache.org

73. Managed Cloud Databases
Cloud Spanner
● Horizontal scalability
● High availability
● ACID transactions
● Planet scale
● Widely used
● https://cloud.google.com/s
panner
Azure Cosmos DB
● Horizontal scalability
● High availability
● ACID transactions
● Planet scale
● Widely used
● Multi-Model
● https://azure.microsoft.com
/en-us/services/cosmos-db

74. When to use NewSQL
●
●
●

75. Summary
RDBMS
Easy to run
Has
transactions
Can’t scale
NoSQL
Can scale
Provides
availability
No transactions
NewSQL
Can Scale
Has
transactions
Lack maturity

76. Nikolay Stoitsev
Engineering Manager at Halo DX