Eventual consistency
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Eventual consistency

on

  • 1,459 views

Slides explaining the consistency modes for Terracotta clustered ehcache.

Slides explaining the consistency modes for Terracotta clustered ehcache.

Statistics

Views

Total Views
1,459
Views on SlideShare
1,459
Embed Views
0

Actions

Likes
1
Downloads
29
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Eventual consistency Presentation Transcript

  • 1. Terracotta Ehcache ConsistencyStrong and EventualAbhishek SanoujamMarch 29, 2011
  • 2. Ehcache-2.4 + Terracotta 3.5 Search – http://ehcache.org/documentation/search.html New Consistency modes – http://ehcache.org/documentation/terracotta/ consistency_options.html Nonstop feature – http://ehcache.org/documentation/terracotta/non_stop_cache.html Rejoin Explicit locking api merged to Ehcache interface Various other perf improvements and bug fixes – http://ehcache.org Terracotta Ehcache Consistency 2
  • 3. Terracotta Ehcache Consistency Terracotta Ehcache Consistency 3
  • 4. Terracotta Ehcache Consistency New attribute “consistency” in “terracotta” sub-element of “cache” in ehcache.xml Terracotta Ehcache Consistency 3
  • 5. Terracotta Ehcache Consistency New attribute “consistency” in “terracotta” sub-element of “cache” in ehcache.xml <cache name=”sampleCache”> <terracotta consistency=”strong|eventual” /> </cache> Terracotta Ehcache Consistency 3
  • 6. Terracotta Ehcache Consistency New attribute “consistency” in “terracotta” sub-element of “cache” in ehcache.xml <cache name=”sampleCache”> <terracotta consistency=”strong|eventual” /> </cache> Consistency can be “strong” or “eventual” Terracotta Ehcache Consistency 3
  • 7. Terracotta Ehcache Consistency New attribute “consistency” in “terracotta” sub-element of “cache” in ehcache.xml <cache name=”sampleCache”> <terracotta consistency=”strong|eventual” /> </cache> Consistency can be “strong” or “eventual” Old “coherent” attribute deprecated Terracotta Ehcache Consistency 3
  • 8. Terracotta Ehcache Consistency New attribute “consistency” in “terracotta” sub-element of “cache” in ehcache.xml <cache name=”sampleCache”> <terracotta consistency=”strong|eventual” /> </cache> Consistency can be “strong” or “eventual” Old “coherent” attribute deprecated With “strong” consistency, changes made by any node in the cluster are visible instantly Terracotta Ehcache Consistency 3
  • 9. Terracotta Ehcache Consistency New attribute “consistency” in “terracotta” sub-element of “cache” in ehcache.xml <cache name=”sampleCache”> <terracotta consistency=”strong|eventual” /> </cache> Consistency can be “strong” or “eventual” Old “coherent” attribute deprecated With “strong” consistency, changes made by any node in the cluster are visible instantly With “eventual” consistency, changes made by any node are visible to other nodes eventually Terracotta Ehcache Consistency 3
  • 10. Terracotta Ehcache Consistency New attribute “consistency” in “terracotta” sub-element of “cache” in ehcache.xml <cache name=”sampleCache”> <terracotta consistency=”strong|eventual” /> </cache> Consistency can be “strong” or “eventual” Old “coherent” attribute deprecated With “strong” consistency, changes made by any node in the cluster are visible instantly With “eventual” consistency, changes made by any node are visible to other nodes eventually http://ehcache.org/documentation/terracotta/consistency_options.html Terracotta Ehcache Consistency 3
  • 11. Strong Consistency Terracotta Ehcache Consistency 4
  • 12. Strong Consistency Changes are visible cluster-wide instantly Terracotta Ehcache Consistency 4
  • 13. Strong Consistency Changes are visible cluster-wide instantly Reads/Writes uses clustered locks – Clustered locks same as normal read-write locks, but valid through the cluster – Multiple reads possible – Writes are mutually exclusive to other writes and reads Terracotta Ehcache Consistency 4
  • 14. Strong Consistency Changes are visible cluster-wide instantly Reads/Writes uses clustered locks – Clustered locks same as normal read-write locks, but valid through the cluster – Multiple reads possible – Writes are mutually exclusive to other writes and reads Honors happens-before cluster wide (using clustered locks) Terracotta Ehcache Consistency 4
  • 15. Strong Consistency Changes are visible cluster-wide instantly Reads/Writes uses clustered locks – Clustered locks same as normal read-write locks, but valid through the cluster – Multiple reads possible – Writes are mutually exclusive to other writes and reads Honors happens-before cluster wide (using clustered locks) Supports other concurrent apis – putIfAbsent(Element element) – removeElement(Element element) – replace(Element element) – replace(Element old, Element element) Terracotta Ehcache Consistency 4
  • 16. Strong Consistency Changes are visible cluster-wide instantly Reads/Writes uses clustered locks – Clustered locks same as normal read-write locks, but valid through the cluster – Multiple reads possible – Writes are mutually exclusive to other writes and reads Honors happens-before cluster wide (using clustered locks) Supports other concurrent apis – putIfAbsent(Element element) – removeElement(Element element) – replace(Element element) – replace(Element old, Element element) Works with explicit locking apis Terracotta Ehcache Consistency 4
  • 17. Eventual Consistency Terracotta Ehcache Consistency 5
  • 18. Eventual Consistency Changes are visible to other nodes eventually (in order of ms) Terracotta Ehcache Consistency 5
  • 19. Eventual Consistency Changes are visible to other nodes eventually (in order of ms) L2 notifies clients about which keys were updated Terracotta Ehcache Consistency 5
  • 20. Eventual Consistency Changes are visible to other nodes eventually (in order of ms) L2 notifies clients about which keys were updated On receiving update from L2, those mappings are removed from clients local cache Terracotta Ehcache Consistency 5
  • 21. Eventual Consistency Changes are visible to other nodes eventually (in order of ms) L2 notifies clients about which keys were updated On receiving update from L2, those mappings are removed from clients local cache Can read stale data for some time (in order of msecs) Terracotta Ehcache Consistency 5
  • 22. Eventual Consistency Changes are visible to other nodes eventually (in order of ms) L2 notifies clients about which keys were updated On receiving update from L2, those mappings are removed from clients local cache Can read stale data for some time (in order of msecs) Suitable for apps that can tolerate reading stale data Terracotta Ehcache Consistency 5
  • 23. Eventual Consistency Changes are visible to other nodes eventually (in order of ms) L2 notifies clients about which keys were updated On receiving update from L2, those mappings are removed from clients local cache Can read stale data for some time (in order of msecs) Suitable for apps that can tolerate reading stale data No need of locks for reads/writes – Better performance compared to strong consistency Terracotta Ehcache Consistency 5
  • 24. Eventual Consistency Changes are visible to other nodes eventually (in order of ms) L2 notifies clients about which keys were updated On receiving update from L2, those mappings are removed from clients local cache Can read stale data for some time (in order of msecs) Suitable for apps that can tolerate reading stale data No need of locks for reads/writes – Better performance compared to strong consistency Supports other concurrent apis – putIfAbsent(Element element) – removeElement(Element element) – replace(Element element) – replace(Element old, Element element) Terracotta Ehcache Consistency 5
  • 25. Eventual Consistency Changes are visible to other nodes eventually (in order of ms) L2 notifies clients about which keys were updated On receiving update from L2, those mappings are removed from clients local cache Can read stale data for some time (in order of msecs) Suitable for apps that can tolerate reading stale data No need of locks for reads/writes – Better performance compared to strong consistency Supports other concurrent apis – putIfAbsent(Element element) – removeElement(Element element) – replace(Element element) – replace(Element old, Element element) Works with explicit locking apis – provides more control over locking Terracotta Ehcache Consistency 5
  • 26. Old “coherent” attribute deprecated Terracotta Ehcache Consistency 6
  • 27. Old “coherent” attribute deprecated Old “coherent” attribute deprecated Terracotta Ehcache Consistency 6
  • 28. Old “coherent” attribute deprecated Old “coherent” attribute deprecated coherent=”true” translates to consistency=”strong” Terracotta Ehcache Consistency 6
  • 29. Old “coherent” attribute deprecated Old “coherent” attribute deprecated coherent=”true” translates to consistency=”strong” coherent=”false” translates to consistency=”eventual” Terracotta Ehcache Consistency 6
  • 30. Old “coherent” attribute deprecated Old “coherent” attribute deprecated coherent=”true” translates to consistency=”strong” coherent=”false” translates to consistency=”eventual” Old coherent=”false” meant bulk-load mode (incoherent mode) – Cannot be set in config (use “eventual” consistency instead) – possible through api only now • setNodeBulkLoadEnabled(true) • setNodeBulkLoadEnabled(false) Terracotta Ehcache Consistency 6
  • 31. Bulk-load mode Terracotta Ehcache Consistency 7
  • 32. Bulk-load mode coherent=”false” - bulk load mode or incoherent mode Terracotta Ehcache Consistency 7
  • 33. Bulk-load mode coherent=”false” - bulk load mode or incoherent mode No locks are required for reads/writes Terracotta Ehcache Consistency 7
  • 34. Bulk-load mode coherent=”false” - bulk load mode or incoherent mode No locks are required for reads/writes Writes are batched heavily in the client (L1) Terracotta Ehcache Consistency 7
  • 35. Bulk-load mode coherent=”false” - bulk load mode or incoherent mode No locks are required for reads/writes Writes are batched heavily in the client (L1) Warmup of large caches happens pretty fast Terracotta Ehcache Consistency 7
  • 36. Bulk-load mode coherent=”false” - bulk load mode or incoherent mode No locks are required for reads/writes Writes are batched heavily in the client (L1) Warmup of large caches happens pretty fast Once values are fetched, changes/updates by other nodes visible after specific time (5 mins by default) Terracotta Ehcache Consistency 7
  • 37. Bulk-load mode coherent=”false” - bulk load mode or incoherent mode No locks are required for reads/writes Writes are batched heavily in the client (L1) Warmup of large caches happens pretty fast Once values are fetched, changes/updates by other nodes visible after specific time (5 mins by default) Possible through api only Terracotta Ehcache Consistency 7
  • 38. Bulk-load mode coherent=”false” - bulk load mode or incoherent mode No locks are required for reads/writes Writes are batched heavily in the client (L1) Warmup of large caches happens pretty fast Once values are fetched, changes/updates by other nodes visible after specific time (5 mins by default) Possible through api only Old api’s are deprecated – setNodeCoherent(boolean coherent) – isNodeCoherent() – isClusterCoherent() – waitUntilClusterCoherent() Terracotta Ehcache Consistency 7
  • 39. Bulk-load mode coherent=”false” - bulk load mode or incoherent mode No locks are required for reads/writes Writes are batched heavily in the client (L1) Warmup of large caches happens pretty fast Once values are fetched, changes/updates by other nodes visible after specific time (5 mins by default) Possible through api only Old api’s are deprecated – setNodeCoherent(boolean coherent) – isNodeCoherent() – isClusterCoherent() – waitUntilClusterCoherent() New api’s added in Ehcache interface: – setNodeBulkLoadEnabled(boolean enabledBulkLoad) – isNodeBulkLoadEnabled() – isClusterBulkLoadEnabled() – waitUntilClusterBulkLoadComplete() Terracotta Ehcache Consistency 7
  • 40. How it works - Strong Consistency Terracotta Ehcache Consistency 8
  • 41. How it works - Strong Consistency Terracotta Server Array (L2) Node A Node B Node C (L1) (L1) (L1) Terracotta Ehcache Consistency 8
  • 42. How it works - Strong Consistency Terracotta Server Array (L2) Node A Node B Node C (L1) (L1) (L1) put(k1, v1) Terracotta Ehcache Consistency 8
  • 43. How it works - Strong Consistency Terracotta Server Array (L2) acquireLock (WRITE) Node A Node B Node C (L1) (L1) (L1) put(k1, v1) Terracotta Ehcache Consistency 8
  • 44. How it works - Strong Consistency Terracotta Server Array (L2) acquireLock grant lock (WRITE) Node A Node B Node C (L1) (L1) (L1) put(k1, v1) Terracotta Ehcache Consistency 8
  • 45. How it works - Strong Consistency Terracotta Server Array (L2) acquireLock grant lock (WRITE) put(k1, v1) Node A Node B Node C (L1) (L1) (L1) put(k1, v1) Terracotta Ehcache Consistency 8
  • 46. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant lock (WRITE) put(k1, v1) Node A Node B Node C (L1) (L1) (L1) put(k1, v1) Terracotta Ehcache Consistency 8
  • 47. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant lock (WRITE) put(k1, v1) Node A Node B Node C (L1) (L1) (L1) (k1, v1) put(k1, v1) Terracotta Ehcache Consistency 8
  • 48. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant lock (WRITE) put(k1, v1) Node A Node B Node C (L1) (L1) (L1) (k1, v1) put(k1, v1) get(k1) Terracotta Ehcache Consistency 8
  • 49. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant lock (WRITE) put(k1, v1) Node A Node B Node C (L1) (L1) (L1) (k1, v1) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 50. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant lock (WRITE) put(k1, v1) acquireLock (READ) acquireLock (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v1) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 51. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant lock (WRITE) recall put(k1, v1) acquireLock (READ) acquireLock (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v1) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 52. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant lock (WRITE) recall put(k1, v1) acquireLock (READ) acquireLock (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v1) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 53. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant read grant lock (WRITE) lock recall put(k1, v1) acquireLock (READ) acquireLock (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v1) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 54. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall put(k1, v1) acquireLock (READ) acquireLock (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v1) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 55. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) get(k1) acquireLock (READ) acquireLock (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v1) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 56. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 acquireLock (READ) acquireLock (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v1) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 57. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 acquireLock (READ) acquireLock (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v1) (k1, v1) (k1, v1) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 58. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 acquireLock (READ) acquireLock (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v1) (k1, v1) (k1, v1)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 59. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 acquireLock (READ) acquireLock (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v1) (k1, v1) (k1, v1)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 60. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 acquireLock (READ) acquireLock recall recall (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v1) (k1, v1) (k1, v1)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 61. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 acquireLock (READ) acquireLock recall recall (READ) Node A Node B Node C (L1) (L1) (L1)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 62. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 acquireLock (READ) acquireLock recall recall (READ) Node A Node B Node C (L1) (L1) (L1)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 63. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v1 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) acquireLock (READ) acquireLock recall recall (READ) Node A Node B Node C (L1) (L1) (L1)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 64. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v2 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) acquireLock (READ) acquireLock recall recall (READ) Node A Node B Node C (L1) (L1) (L1)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 65. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v2 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) acquireLock (READ) acquireLock recall recall (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v2)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 66. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v2 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) acquireLock (READ) acquireLock recall recall (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v2)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 67. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v2 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) acquireLock (READ) acquireLock recall recall (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v2)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 68. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v2 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) acquireLock (READ) acquireLock recall recall (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v2)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 69. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v2 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) acquireLock (READ) acquireLock recall recall (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v2)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 70. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v2 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) acquireLock (READ) acquireLock recall recall (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v2)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 71. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v2 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) acquireLock (READ) acquireLock recall recall (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v2)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 72. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v2 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) acquireLock (READ) acquireLock recall recall (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v2)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 73. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v2 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 v1 put(k1, v2) acquireLock v2 get(k1) (READ) acquireLock v2recall recall (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v2)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 74. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v2 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 v1 put(k1, v2) acquireLock v2 get(k1) (READ) acquireLock v2recall recall (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v2) (k1, v2)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 75. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v2 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 v1 put(k1, v2) acquireLock v2 get(k1) (READ) acquireLock v2recall recall (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v2) (k1, v2)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 76. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v2 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 v1 put(k1, v2) acquireLock v2 get(k1) (READ) acquireLock v2recall recall (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v2) (k1, v2)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 8
  • 77. How it works - Strong Consistency Terracotta Server Array (L2) (k1, v2 ) acquireLock grant read grant read grant lock lock (WRITE) lock recall get(k1) put(k1, v1) v1 v1 put(k1, v2) acquireLock v2 get(k1) (READ) acquireLock v2recall recall (READ) Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v2) (k1, v2)put(k1, v2) put(k1, v1) get(k1) v2 get(k1) v2 Terracotta Ehcache Consistency 8
  • 78. How it works - Eventual Consistency Terracotta Ehcache Consistency 9
  • 79. How it works - Eventual Consistency Terracotta Server Array (L2) Node A Node B Node C (L1) (L1) (L1) Terracotta Ehcache Consistency 9
  • 80. How it works - Eventual Consistency Terracotta Server Array (L2) Node A Node B Node C (L1) (L1) (L1) put(k1, v1) Terracotta Ehcache Consistency 9
  • 81. How it works - Eventual Consistency Terracotta Server Array (L2) put(k1, v1) Node A Node B Node C (L1) (L1) (L1) put(k1, v1) Terracotta Ehcache Consistency 9
  • 82. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v1 ) put(k1, v1) Node A Node B Node C (L1) (L1) (L1) put(k1, v1) Terracotta Ehcache Consistency 9
  • 83. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v1 ) put(k1, v1) Node A Node B Node C (L1) (L1) (L1) (k1, v1) put(k1, v1) Terracotta Ehcache Consistency 9
  • 84. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v1 ) put(k1, v1) Node A Node B Node C (L1) (L1) (L1) (k1, v1) put(k1, v1) get(k1) Terracotta Ehcache Consistency 9
  • 85. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v1 ) put(k1, v1) Node A Node B Node C (L1) (L1) (L1) (k1, v1) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 9
  • 86. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v1 ) get(k1) put(k1, v1) get(k1) Node A Node B Node C (L1) (L1) (L1) (k1, v1) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 9
  • 87. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v1 ) get(k1) put(k1, v1) v1 get(k1) v1 Node A Node B Node C (L1) (L1) (L1) (k1, v1) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 9
  • 88. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v1 ) get(k1) put(k1, v1) v1 get(k1) v1 Node A Node B Node C (L1) (L1) (L1) (k1, v1) (k1, v1) (k1, v1) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 9
  • 89. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v1 ) get(k1) put(k1, v1) v1 get(k1) v1 Node A Node B Node C (L1) (L1) (L1) (k1, v1) (k1, v1) (k1, v1)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 9
  • 90. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v1 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) Node A Node B Node C (L1) (L1) (L1) (k1, v1) (k1, v1) (k1, v1)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 9
  • 91. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v2 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) Node A Node B Node C (L1) (L1) (L1) (k1, v1) (k1, v1) (k1, v1)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 9
  • 92. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v2 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v1) (k1, v1) (k1, v1)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 9
  • 93. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v2 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v1) (k1, v1) (k1, v1)put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 9
  • 94. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v2 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v1) (k1, v1) (k1, v1)put(k1, v2) put(k1, v1) get(k1) v1 get(k1) Terracotta Ehcache Consistency 9
  • 95. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v2 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v1) (k1, v1) (k1, v1)put(k1, v2) put(k1, v1) get(k1) v1 get(k1) Terracotta Ehcache Consistency 9
  • 96. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v2 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v1) (k1, v1) (k1, v1) v1 v1put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 9
  • 97. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v2 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) k1 k1 Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v1) (k1, v1) (k1, v1) v1 v1put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 9
  • 98. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v2 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) k1 k1 Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v1) v1 v1put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 9
  • 99. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v2 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) k1 k1 Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v1) v1 v1put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 9
  • 100. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v2 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) k1 k1 Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v1) v1 v1put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 9
  • 101. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v2 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) k1 k1 v2 v2 Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v1) v1 v1put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 9
  • 102. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v2 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) k1 k1 v2 v2 Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v1) (k1, v2) (k1, v2) v1 v1put(k1, v2) put(k1, v1) get(k1) get(k1) Terracotta Ehcache Consistency 9
  • 103. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v2 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) k1 k1 v2 v2 Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v1) (k1, v2) (k1, v2) v1 v1put(k1, v2) put(k1, v1) get(k1) get(k1) v2 v2 Terracotta Ehcache Consistency 9
  • 104. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v2 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) k1 k1 v2 v2 Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v1) (k1, v2) (k1, v2) v1 v1put(k1, v2) put(k1, v1) get(k1) get(k1) v2 v2 Terracotta Ehcache Consistency 9
  • 105. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v2 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) k1 k1 v2 v2 Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v1) (k1, v2) (k1, v2) v1 v1put(k1, v2) put(k1, v1) get(k1) get(k1) v2 v2 Terracotta Ehcache Consistency 9
  • 106. How it works - Eventual Consistency Terracotta Server Array (L2) (k1, v2 ) get(k1) put(k1, v1) v1 get(k1) v1 put(k1, v2) k1 k1 v2 v2 Node A Node B Node C (L1) (L1) (L1) (k1, v2) (k1, v2) (k1, v2) v1 v1put(k1, v2) put(k1, v1) get(k1) get(k1) v2 v2 Terracotta Ehcache Consistency 9
  • 107. How it works - Bulk load mode Terracotta Ehcache Consistency 10
  • 108. How it works - Bulk load mode Similar to eventual consistency – No locks required Terracotta Ehcache Consistency 10
  • 109. How it works - Bulk load mode Similar to eventual consistency – No locks required Puts are batched heavily in the L1 Terracotta Ehcache Consistency 10
  • 110. How it works - Bulk load mode Similar to eventual consistency – No locks required Puts are batched heavily in the L1 The L2 don’t send out invalidation notifications Terracotta Ehcache Consistency 10
  • 111. How it works - Bulk load mode Similar to eventual consistency – No locks required Puts are batched heavily in the L1 The L2 don’t send out invalidation notifications Invalidations in the local cache happen based on time (5 mins by default) Terracotta Ehcache Consistency 10
  • 112. How it works - Bulk load mode Similar to eventual consistency – No locks required Puts are batched heavily in the L1 The L2 don’t send out invalidation notifications Invalidations in the local cache happen based on time (5 mins by default) Apps read stale data for the specified time interval – configurable via tc property “ehcache.storageStrategy.dcv2.localcache.incoherentReadTimeout” Terracotta Ehcache Consistency 10
  • 113. Comparison Strong consistency – Reads/Writes happen under locks – Invalidations of local cache happens when a node acquires WRITE locks Eventual Consistency – Reads/Writes do not require locks – L2 notifies about changes to interested nodes – L1 invalidates local cache upon receiving notification from L2 – Notifications are batched and sent to only interested nodes Bulk-load mode – No locks required – Invalidations happen based on time (5 mins by default) – Puts are batched heavily in the L1 Terracotta Ehcache Consistency 11
  • 114. Summary Strong consistency – Pros • Provides coherent data across the cluster • Changes visible to all nodes instantly • Honors happens-before cluster wide – Cons • If coherent cache not desired, suffers from performance • Needs locks for both reads and writes • May result in lock hopping when data not partitioned leading to perf degradation Terracotta Ehcache Consistency 12
  • 115. Summary Terracotta Ehcache Consistency 13
  • 116. Summary Eventual Consistency – Pros • No need of locks for reads/writes • Better performance • Changes are visible across cluster in order of ms – Cons • Application can read stale data for some time Terracotta Ehcache Consistency 13
  • 117. Summary Eventual Consistency – Pros • No need of locks for reads/writes • Better performance • Changes are visible across cluster in order of ms – Cons • Application can read stale data for some time Bulk Load mode – Pros • No need of locks for reads/writes • Better performance for puts, useful for cache warmup – Cons • Application can read stale data for longer amount of time • Values once fetched, will see changes after specific amount of time Terracotta Ehcache Consistency 13
  • 118. Thank you!Abhishek Sanoujamasanoujam@terracottatech.comhttp://terracotta.orghttp://ehcache.orghttp://ehcache.org/documentation/terracotta/consistency_options.html