The document discusses the implementation of cluster-wide causal consistency in distributed systems, outlining its definition and academic perspectives. It covers the architectural components necessary for building secure, fast, and reliable causal consistency, as well as how it can be utilized by end-users through interaction with databases. The document includes technical details about timestamp management and operational mechanisms for maintaining causal relationships among events.

1. Implementation of
Cluster-wide Causal
Consistency in

2. - What is causal consistency
- Academics view on Causal Consistency
- MongoDB architecture
- Causal Consistency building blocks
- Making Causal Consistency secure
- Making Causal Consistency fast
- Making Causal Consistency reliable
- Causal consistency for end-users
Outline

4. Client-side properties of causal consistency
- Read your writes
- Writes follow reads
- Monotonic reads
- Monotonic writes

5. Implementing with a non causally consistent
system
- Single server systems are causally consistent
- Read and write from the same node
- Add an application logic to handle the scenarios that have to be causally
consistent

7. Ordering of Events in Distributed System
Process P Process Q Process R
q2: <C2> = 11
p1
r1: <C3> = 11
q1
<C3> = 0<C1> = 10 <C2> = 10
r2: <C3> = 12
q3

8. Server-side causal consistency
Causal consistency is a partial order of events in a distributed
system. If an event A causes another event B, then causal
consistency provides an assurance that each other process of the
system observes event A before observing event B.
If an event A is not causally related to an event B then they are
concurrent.

9. System’s Architecture

10. Gossiping clusterTime
ClusterTime: {uint64}
Timestamp(1495470881, 6)

11. Ticking clusterTime
{ts: 6} ...
{ts: 5} ...
{ts: 11} ...
insert({x:1}, clusterTime: 4)
<clusterTime>: 6
<Wall clock>: 11

12. Reporting operationTime
insert({x:1})
{ts: 11} ...
{ts: 5} ...
{ok:1}, { operationTime: 12}
{ts: 12} ...
<clusterTime>: 11
<Wall clock>: 11

13. Waiting for afterClusterTime
find({x:1}, afterClusterTime: {10},
clusterTime: {15})
{ts: 6} ...
{ts: 5} ...
{x:1}, { operationTime: {11}}
{ts: 11} ...

14. Breaking clusterTime
{ts: Timestamp(1495470881, 6), term: 1},
...
{ts: Timestamp(1495470881, 5), term: 1},
...
{Timestamp(0xFFFFFFFF, 0xFFFFFFFF}
...
insert({x:1}, clusterTime:
{0xFFFFFFFF, 0xFFFFFFFE})
LogicalClock:clusterTime =
Timestamp(0xFFFFFFFF, 0xFFFFFFFE)

15. Protecting clusterTime
"$clusterTime" : {
"clusterTime" : Timestamp(1495470881, 5),
"signature" : {
"hash" : BinData(0,"7olYjQCLtnfORsI9IAhdsftESR4="),
"keyId" : NumberLong("6422998367101517844")
}
}

16. Protecting against operator errors
{ts: 6} ...
{ts: 5} ...
insert({x:1}, clusterTime: 100,000)
<clusterTime>: 6
<Wall clock>: 11
{Error}, { operationTime: 6}

17. Signing a range of clusterTime
find({x:1}, clusterTime: <val>,
signature:<hash>)
<timeRange> =
<val> | 0x0000’0000’0000’FFFF
cache:{ <timeRange>:<hash> }

18. Use dummy signatures
- When the auth is off
- When a user has advanceClusterTime privilege

19. How end users see it
let session=db.getMongo().startSession({causalConsistency: true})
db = session.getDatabase(db.getName());

20. {checking:100}
find({name:”misha”})
afterClusterTime: 15
update({name:”misha”
checking:100})
{ok:1} operationTime: 15
startSession()

21. Misha Tyulenev
misha@mongodb.com