Spanner is Google's globally distributed database that uses synchronized clocks to provide strong consistency across data centers. It addresses the CAP theorem by providing both availability and consistency for write transactions using two phase commit and read transactions using synchronized clocks. However, perfectly synchronized clocks are impossible due to network latency and the theoretical limitations of Minkowski spacetime. While Spanner works well for Google's infrastructure, its consistency guarantees rely on assumptions about clock drift that may not hold in general.

1. Spanner, True Time & the CAP Theorem
Eric Brewer, 2017
Papers We Love San Diego
May 4th, 2017
Presented by Daniel Norman
@DreamingInCode

2. Overview
Interesting citations
The original Spanner Paper
Corbett et al, Google Inc, OSDI 2012
Practical uses of synchronized clocks in
distributed systems Liskov, PODC 1991
Standing on Distributed Shoulders of Giants
Helland, ACM Queue June 2016
CAP Twelve Years Later: How the "Rules"
Have Changed Brewer, IEEE Computer 2012
The Network is Reliable
Bailis, Kingsbury, ACM Queue July 2014

3. Synopsis
● How Spanner works
● The Spanner Paper
● Networks are unreliable, except for Google’s
● Practical use of synchronized clocks
● Synchronized clocks are impossible
● Let’s talk about Minkowski
● ??
● Profit!

4. ● Geo-replicated sharding
● Two Phase Commit / Paxos for R/W transactions
● TrueTime for read-isolation
● “CA” for write transactions
● “CP” for read transactions
What is Spanner?

5. What does it span?
This bit
{

6. What does it span?
Bigtable
● Global Database
● High availability
● High performance
● eventually-consistent CDR
● Non-relational
● No SQL-like query language
Megastore
● Global Database
● Somewhat lower performance
● Somewhat lesser availability
● Synchronous CDR
● Semi-relational
● SQL-like Query language
~ This bit ~

7. What does it span?
Eventual Consistency
● Dynamo
● Cassandra
● CouchBase
● MongoDB
Serializable Consistency
● Mysql
● PostgreSQL
● SQL Server
● Aurora
~ This bit ~

8. The Spanner Paper
Basics
Limited discussion of:
● How zones work & interact
● Overarching sharding scheme
● Zonemaster and location proxy

9. Universe
Universe Master
Placement Driver
Zone
Participants
Leader
Lock Table
Tablet
Colossus
Directories
Directory Fragments
DirMove
The Spanner Paper
Basics

11. Raid 10 / LVM
pv0 pv1
lv0
lv1
lv2
lv3
lv4
pv2
Raid 1
Raid 1
Raid 1
LVM physical extent
assignment
volumes

12. The Spanner Paper
Basics
Rather like a document database

13. The Spanner Paper
Basics

14. The Spanner Paper
Basics
Google network POPs

15. The Spanner Paper
TrueTime
https://www.wired.com/2012/11/google-spanner-time/
“A person with a watch knows what time
it is. A person with two watches is never
sure” -Segal’s law

16. The Spanner Paper
TrueTime
Oh
dear...

17. The Spanner Paper
TrueTime

18. The Spanner Paper
TrueTime

19. CAP Twelve Years later, how the “Rules” have changed
Brewer, IEEE Computer, 2012
“CAP theorem ignores latency, although in practice, latency and partitions are deeply related.”
“Aspects of the CAP theorem are often misunderstood, particularly the scope of availability and consistency, which can lead to
undesirable results.”
● Consistency < Apology
● Availability is continuous, not discrete
● Partition = Latency
● CAP “2 of 3” is kind of misleading
https://www.infoq.com/articles/cap-twelve-years-later-how-the-rules-have-changed

20. The Network is Reliable
Bailis, Kingsbury, ACM Queue 2014
● Network congestion
● Configuration errors
● Router hardware failures
● Spanning tree failures
● FLP Impossibility
http://queue.acm.org/detail.cfm?id=2655736

21. Practical uses of synchronized clocks in distributed systems
Barbara Liskov, PODC 1991
http://www.dainf.cefetpr.br/~tacla/SDII/PracticalUseOfClocks.pdf
Synchronizing clocks perfectly is not possible
But synchronizing might still be useful

22. Standing on Distributed Shoulders of Giants
Pat Helland, ACM Queue June 2016
http://queue.acm.org/detail.cfm?id=2953944
Time is notional
Physicists have had their shit
together for a while

23. Spanner, TrueTime & The CAP Theorem
Partitions are not common
Latency is not a big deal
CP is ok if you spend enough to mitigate A

24. Spanner, TrueTime & The CAP Theorem
“Miniscule” is a value judgement

25. Spanner, TrueTime & The CAP Theorem
“Always” is a bit of a bold claim
Read-only transaction consistency assumes:
❌ Minkowski Spacetime
❌ Worst-case drift sufficiently pessimistic

26. Minkowski Spacetime Curved Spacetime (General Relativity)
https://simple.wikipedia.org/wiki/Minkowski_spacetime
http://sci.esa.int/lisa-pathfinder/56471-measuring-spacetime-curvature/
GET BENT

27. Simulated Simultaneity
● I think read correctness proof is impossible, or requires novel physics
● Generous error estimation -> clock drift + unpredictable spacetime curvature
● Artificially limited performance due to upper error bound

28. Conclusion
● Spanner is the DB you’ve been waiting for, sort of
● It basically works (notwithstanding significant questions about correctness)
● But only for Google, not for you
● Pay Google

29. Thank You!
Papers We Love San Diego
May 4th, 2017
Presented by Daniel Norman
@DreamingInCode
Spanner, True Time & the CAP
Theorem
Eric Brewer, 2017