This document summarizes a presentation about Apache Jackrabbit Oak and its use of MongoDB for content storage. It discusses how Jackrabbit Oak uses multi-version concurrency control and copy-on-write to allow concurrent access to different document versions. Transactions are implemented by assigning commit roots and checking for collisions during commit. Properties are stored with revision IDs to allow retrieval of prior versions.

1. Apache Jackrabbit Oak on MongoDB
Marcel Reutegger | Senior Software Engineer
© 2014 Adobe Systems Incorporated. All Rights Reserved.

2. Apache Jackrabbit Oak
© 2014 Adobe Systems Incorporated. All Rights Reserved.
2
About me
Software Engineer
At Day/Adobe since 2002
JCR API Specification
Apache member
Apache Jackrabbit / Oak

3. Contents
© 2014 Adobe Systems Incorporated. All Rights Reserved.
Adobe Experience Manager
Java Content Repository
Apache Jackrabbit Oak
Multiversion Concurrency Control
Transactions
Content Addressable Storage
Q & A
3

4. Adobe Experience Manager
© 2014 Adobe Systems Incorporated. All Rights Reserved.
4

5. Adobe Experience Manager – Technology Stack
© 2014 Adobe Systems Incorporated. All Rights Reserved.
5
OSGi Container
Web Framework
Java Content Repository

6. Adobe Experience Manager – Technology Stack
© 2014 Adobe Systems Incorporated. All Rights Reserved.
6
OSGi Container
Web Framework
Java Content Repository

7. Adobe Experience Manager – Technology Stack
Java Content Repository / Apache Jackrabbit Oak
Tar MongoDB RDBMS
© 2014 Adobe Systems Incorporated. All Rights Reserved.
7
.tar

8. Contents
© 2014 Adobe Systems Incorporated. All Rights Reserved.
Adobe Experience Manager
Java Content Repository
Apache Jackrabbit Oak
Multiversion Concurrency Control
Transactions
Content Addressable Storage
Q & A
8

9. Java Content Repository – Why?
“Build me a web content management system!”
© 2014 Adobe Systems Incorporated. All Rights Reserved.
9
Apache
PHP
Developer happiness
Easy: LAMP stack
Linux
MySQL
Done in 2 weeks

10. Java Content Repository – Why?
Build me a web content management system!
“Nice, but I want to organize my pages in a hierarchy.”
Apply a well known hierarchical database model
and update the application.
© 2014 Adobe Systems Incorporated. All Rights Reserved.
10
Developer happiness
Done in 4 weeks

11. Java Content Repository – Why?
Nice, but I want to organize my pages in a hierarchy.
“Can you please add structured and fulltext searches?”
Integrate with Apache Solr or Elasticsearch
© 2014 Adobe Systems Incorporated. All Rights Reserved.
Done in 4 weeks
11
Developer happiness

12. Java Content Repository – Why?
Can you please add structured and fulltext searches?
“I accidentally deleted the product page.
We need to version our content.”
Introduce new tables and rewrite the application.
© 2014 Adobe Systems Incorporated. All Rights Reserved.
Done in 8 weeks
12
Developer happiness

13. Java Content Repository – Why?
“I accidentally deleted the product page.
We need to version our content.”
“We cannot publish financial results, unless the system
has fine grained access control.”
Introduce more tables and
© 2014 Adobe Systems Incorporated. All Rights Reserved.
integrate with a directory server.
I’ll get back to you
13
next year!
Developer happiness

14. Java Content Repository – Features
JSR-283 – JCR 2.0 released 2009
Hierarchical - Structured and binary data
Query – SQL, XPath and Java language binding
Access Control on Node and Property level
Versioning – Modeled after WebDAV DeltaV (RFC 3253)
Locking – Shallow or deep
Asynchronous Observation
© 2014 Adobe Systems Incorporated. All Rights Reserved.
14

15. Contents
© 2014 Adobe Systems Incorporated. All Rights Reserved.
Adobe Experience Manager
Java Content Repository
Apache Jackrabbit Oak
Multiversion Concurrency Control
Transactions
Content Addressable Storage
Q & A
15

16. Apache Jackrabbit Oak – Design
© 2014 Adobe Systems Incorporated. All Rights Reserved.
16
Cacheable
Customizable
Scalable Support NoSQL Storage
Support Sharding

17. Apache Jackrabbit Oak – Design
© 2014 Adobe Systems Incorporated. All Rights Reserved.
17
Cacheable
Customizable
Scalable
Pluggable Storage
Custom Index Definitions

18. Apache Jackrabbit Oak – Design
© 2014 Adobe Systems Incorporated. All Rights Reserved.
18
Cacheable
Customizable
Scalable
Copy-On-Write
Multiversion Concurrency
Content Addressable Storage

19. Contents
© 2014 Adobe Systems Incorporated. All Rights Reserved.
Adobe Experience Manager
Java Content Repository
Apache Jackrabbit Oak
Multiversion Concurrency Control
Transactions
Content Addressable Storage
Q & A
19

20. Apache Jackrabbit Oak – MVCC & Copy-On-Write
© 2014 Adobe Systems Incorporated. All Rights Reserved.
20
Rev 1
/
/a /b
/a/1 /a/2 /b/1

21. Apache Jackrabbit Oak – MVCC & Copy-On-Write
/a /b
/a/1 /a/2 /b/1 /b/2
© 2014 Adobe Systems Incorporated. All Rights Reserved.
21
Rev 1
/
new

22. Apache Jackrabbit Oak – MVCC & Copy-On-Write
Rev 2
/a /b
/a/1 /a/2 /b/1 /b/2
© 2014 Adobe Systems Incorporated. All Rights Reserved.
22
Rev 1
/b'
/ /'
copy parents

23. Apache Jackrabbit Oak – MVCC & Copy-On-Write
Rev 2
/a /b
/a/1 /a/2 /b/1 /b/2
© 2014 Adobe Systems Incorporated. All Rights Reserved.
23
Rev 1
/b'
/ /'
concurrent access
to Rev 1 and Rev 2

24. Apache Jackrabbit Oak – MVCC & Copy-On-Write
collect garbage Rev 2
© 2014 Adobe Systems Incorporated. All Rights Reserved.
24
/a
/b'
/'
/a/1 /a/2 /b/1 /b/2

25. Apache Jackrabbit Oak – MVCC & Copy-On-Write
/a /b'
/a/1 /a/2 /b/1 /b/2
© 2014 Adobe Systems Incorporated. All Rights Reserved.
25
Rev 2
/'
compact

26. Apache Jackrabbit Oak – MVCC & Copy-On-Write
© 2014 Adobe Systems Incorporated. All Rights Reserved.
26
⊕
⊖
Stable snapshot view of data
Writes do not block reads
Higher storage cost
Garbage collection

27. Apache Jackrabbit Oak – The Data Model
{!
_id : “/home/john”,!
name : “john”,!
email : “john@example.com”!
}!
{!
_id : “2:/home/john”,!
name : { “r14979e4b424-0-1” : “john” },!
email : { “r14979e4b424-0-1” : “john@example.com” },!
_deleted : { “r14979e4b424-0-1” : “false” },!
_revisions : { “r14979e4b424-0-1” : “c” }!
}!
© 2014 Adobe Systems Incorporated. All Rights Reserved.
27

28. Apache Jackrabbit Oak – The Data Model
{!
_id : “2:/home/john”,!
name : { “r14979e4b424-0-1” : “john” },!
email : { “r14979e4b424-0-1” : “john@example.com” },!
_deleted : { “r14979e4b424-0-1” : “false” },!
_revisions : { “r14979e4b424-0-1” : “c” }!
}!
© 2014 Adobe Systems Incorporated. All Rights Reserved.
28

29. Apache Jackrabbit Oak – The Data Model
{!
_id : “2:/home/john”,!
name : { “r14979e4b424-0-1” : “john” },!
email : { “r14979e4b424-0-1” : “john@example.com” },!
_deleted : { “r14979e4b424-0-1” : “false” },!
_revisions : { “r14979e4b424-0-1” : “c” }!
}!
Timestamp Counter Cluster ID
© 2014 Adobe Systems Incorporated. All Rights Reserved.
29

30. Apache Jackrabbit Oak – The Data Model
{!
_id : “2:/home/john”,!
name : { “r14979e4b424-0-1” : “john” },!
email : {!
“r14979e4b424-0-1” : “john@example.com”,!
“r14979e6a941-0-1” : “john.doe@example.com”!
},!
_deleted : { “r14979e4b424-0-1” : “false” },!
_revisions : {!
“r14979e4b424-0-1” : “c”,!
“r14979e6a941-0-1” : “c”!
© 2014 Adobe Systems Incorporated. All Rights Reserved.
30
}!
}!
Change email to “john.doe@example.com”

31. Contents
© 2014 Adobe Systems Incorporated. All Rights Reserved.
Adobe Experience Manager
Java Content Repository
Apache Jackrabbit Oak
Multiversion Concurrency Control
Transactions
Content Addressable Storage
Q & A
31

32. Apache Jackrabbit Oak – Transactions
{!
_id : “2:/home/john”,!
name : { “r14979e4b424-0-1” : “john” },!
_deleted : { “r14979e4b424-0-1” : “false” },!
_commitRoot : { “r14979e4b424-0-1” : “1” }!
}!
{!
_id : “3:/home/john/profile”,!
avatar : { “r14979e4b424-0-1” : <bin> },!
_deleted : { “r14979e4b424-0-1” : “false” },!
_commitRoot : { “r14979e4b424-0-1” : “1” }!
}!
© 2014 Adobe Systems Incorporated. All Rights Reserved.
32

33. Apache Jackrabbit Oak – Transactions
© 2014 Adobe Systems Incorporated. All Rights Reserved.
33
},!
false” },!
1” }!
},!
false” },!
1” }!
{!
_id : “1:/home”,!
_deleted : { “r14979e1b312-0-1” : “false” },!
_revisions : {!
“r14979e1b312-0-1” : “c”,!
“r14979e4b424-0-1” : “c”!
}!
}!
Conditional update for commit:
{!
_id : “1:/home”, !
“_collisions.r14979e4b424-0-1” : { $exists : false }!
}!

34. Apache Jackrabbit Oak – Transactions
{!
_id : “3:/home/john/profile”,!
avatar : {!
“r14979e4b424-0-1” : <bin>!
},!
_deleted : {!
“r14979e4b424-0-1” : “false”!
},!
_commitRoot : {!
“r14979e4b424-0-1” : “1” !
}!
}!
© 2014 Adobe Systems Incorporated. All Rights Reserved.
34

35. Apache Jackrabbit Oak – Transactions
{!
_id : “3:/home/john/profile”,!
avatar : {!
“r14979e4b424-0-1” : <bin>,!
“r14979e6c7a2-0-1” : <bin>!
},!
_deleted : {!
“r14979e4b424-0-1” : “false”!
},!
_commitRoot : {!
“r14979e4b424-0-1” : “1”,!
“r14979e6c7a2-0-1” : “1” !
}!
}!
© 2014 Adobe Systems Incorporated. All Rights Reserved.
35
T-1

36. Apache Jackrabbit Oak – Transactions
{!
_id : “3:/home/john/profile”,!
avatar : {!
“r14979e4b424-0-1” : <bin>,!
“r14979e6c7a2-0-1” : <bin>,!
“r14979e6c7a3-0-1” : <bin>!
},!
_deleted : {!
“r14979e4b424-0-1” : “false”!
},!
_commitRoot : {!
“r14979e4b424-0-1” : “1”,!
“r14979e6c7a2-0-1” : “1”,!
“r14979e6c7a3-0-1” : “1”!
}!
}!
© 2014 Adobe Systems Incorporated. All Rights Reserved.
36
T-1 T-2

37. Apache Jackrabbit Oak – Transactions
© 2014 Adobe Systems Incorporated. All Rights Reserved.
37
{!
_id : “1:/home”,!
_deleted : { !
“r14979e1b312-0-1” : “false”!
},!
_revisions : {!
“r14979e1b312-0-1” : “c”!
},!
_collisions : {!
“r14979e6c7a2-0-1” : “true”!
}!
}!
T-2
Conditional update for collision marker:
{!
_id : “1:/home”, !
“_revisions.r14979e6c7a2-0-1” : { $exists : false }!
}!

38. Apache Jackrabbit Oak – Transactions
© 2014 Adobe Systems Incorporated. All Rights Reserved.
38
{!
_id : “1:/home”,!
_deleted : { !
“r14979e1b312-0-1” : “false”!
},!
_revisions : {!
“r14979e1b312-0-1” : “c”!
},!
_collisions : {!
“r14979e6c7a2-0-1” : “true”!
}!
}!
T-2
T-1
✗
Conditional update for commit:
{!
_id : “1:/home”, !
“_collisions.r14979e6c7a2-0-1” : { $exists : false }!
}!

39. Apache Jackrabbit Oak – Transactions
© 2014 Adobe Systems Incorporated. All Rights Reserved.
39
{!
_id : “1:/home”,!
_deleted : { !
“r14979e1b312-0-1” : “false”!
},!
_revisions : {!
“r14979e1b312-0-1” : “c”,!
“r14979e6c7a3-0-1” : “c”!
},!
_collisions : {!
“r14979e6c7a2-0-1” : “true”!
}!
}!
T-1 T-2
✗
{!
_id : “1:/home”, !
“_collisions.r14979e6c7a3-0-1” : { $exists : false }!
}!

40. Apache Jackrabbit Oak – Transactions
{!
_id : “3:/home/john/profile”,!
avatar : {!
“r14979e4b424-0-1” : <bin>,!
“r14979e6c7a2-0-1” : <bin>,!
“r14979e6c7a3-0-1” : <bin>!
},!
_deleted : {!
“r14979e4b424-0-1” : “false”!
},!
_commitRoot : {!
“r14979e4b424-0-1” : “1”,!
“r14979e6c7a2-0-1” : “1”,!
“r14979e6c7a3-0-1” : “1”!
}!
}!
© 2014 Adobe Systems Incorporated. All Rights Reserved.
40
T-1

41. Contents
© 2014 Adobe Systems Incorporated. All Rights Reserved.
Adobe Experience Manager
Java Content Repository
Apache Jackrabbit Oak
Multiversion Concurrency Control
Transactions
Content Addressable Storage
Q & A
41

42. Apache Jackrabbit Oak – Content Addressable Storage
© 2014 Adobe Systems Incorporated. All Rights Reserved.
42
GridFS
file chunk 0
chunk 1
…
chunk N

43. Apache Jackrabbit Oak – Content Addressable Storage
© 2014 Adobe Systems Incorporated. All Rights Reserved.
43
Oak
Node Storage Binary Storage
Binary
Chunk
0x38a7
Chunk
0xc92a
Chunk
0x8f91

44. Apache Jackrabbit Oak – Content Addressable Storage
Binary 0x38a7
© 2014 Adobe Systems Incorporated. All Rights Reserved.
44
Oak
0x38a7
Chunk
0x38a7
Chunk
0xc92a
Binary Storage
Chunk
0x8f91
Hash:
Node Storage

45. Apache Jackrabbit Oak – Content Addressable Storage
Binary
© 2014 Adobe Systems Incorporated. All Rights Reserved.
45
Oak
0x8f91
Chunk
0x38a7
Chunk
0xc92a
Binary Storage
Chunk
0x8f91
Hash:
0x38a7 0x8f91
Node Storage

46. Apache Jackrabbit Oak – Content Addressable Storage
Binary
© 2014 Adobe Systems Incorporated. All Rights Reserved.
46
Oak
0x52f1
Chunk
0x38a7
Chunk
0xc92a
Binary Storage
Chunk
0x8f91
Hash:
0x38a7 0x8f91
0x52f1
Chunk
0x52f1
Node Storage

47. Apache Jackrabbit Oak – Content Addressable Storage
© 2014 Adobe Systems Incorporated. All Rights Reserved.
47
Oak
Chunk
0x38a7
Chunk
0xc92a
Binary Storage
Chunk
0x8f91
Chunk
0x52f1
Node Storage
0x38a7 0x8f91 0x52f1

48. Apache Jackrabbit Oak – Content Addressable Storage
© 2014 Adobe Systems Incorporated. All Rights Reserved.
48
⊕
⊖
De-duplication on chunk level
Chunks are immutable
Shared Storage
Garbage collection

49. Summary
© 2014 Adobe Systems Incorporated. All Rights Reserved.
Adobe Experience Manager
Java Content Repository
Apache Jackrabbit Oak
Multiversion Concurrency Control
Transactions
Content Addressable Storage
Q & A
49

50. Q & A
© 2014 Adobe Systems Incorporated. All Rights Reserved. 50

51. © 2014 Adobe Systems Incorporated. All Rights Reserved.