Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cassandra : to be or not to be @ TechTalk

153 views

Published on

This presentation is about Apache Cassandra cluster, data model, read and write operations on node and on cluster and the same about update & delete. Why Delete is overhead and why all engineers have to know business needs to build right architecture and select the best tools.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Cassandra : to be or not to be @ TechTalk

  1. 1. CASSANDRA To be or not to be?
  2. 2. ABOUT ME Andriy Rymar Senior Software Engineer @ Lohika Morning@Lohika Program Committee Member JUG Lviv Active Member 2
  3. 3. AGENDA 3
  4. 4. AGENDA Cluster Ring 3 Node 4 Node 1 Node 2 Node 3
  5. 5. AGENDA Cluster Ring Data Model 3 Row1 Row2 Row3
  6. 6. AGENDA Cluster Ring Data Model Read & Write 3 Read Write
  7. 7. AGENDA Cluster Ring Data Model Read & Write Update & Delete 3 Update Delete
  8. 8. AGENDA Cluster Ring Data Model Read & Write Update & Delete 3 To be or not to be?
  9. 9. PREFACE 4
  10. 10. TRENDS 5
  11. 11. TRENDS 5
  12. 12. TRENDS 6
  13. 13. TRENDS 6
  14. 14. 6
  15. 15. 7http://stanfordflipside.com/images/236richKid.jpg
  16. 16. 8 BUSINESS NEEDS
  17. 17. CASSANDRA To be or not to be?
  18. 18. 10 CLUSTER
  19. 19. DISCOVERY Node1 Node3 Node4 Node2 11
  20. 20. DISCOVERY Node3 Node4 Node2 Node1 11
  21. 21. DISCOVERY Node3 Node4 Node2 Hi, I’m node 2 Node1 11
  22. 22. DISCOVERY Node3 Node4 Node2 OK, here are you and me Hi, I’m node 2 Node1 11
  23. 23. DISCOVERY Node3 Node4 Node2 Hi, I’m node 3 OK, here are you and me Hi, I’m node 2 Node1 11
  24. 24. DISCOVERY Node3 Node4 Node2 Hi, I’m node 3 OK, here are you and me Hi, I’m node 2 Well, here are you, me and node2 Node1 11
  25. 25. DISCOVERY Node3 Node4 Node2 Hi, I’m node 4 Hi, I’m node 3 OK, here are you and me Hi, I’m node 2 Well, here are you, me and node2 Node1 11
  26. 26. DISCOVERY Node3 Node4 Node2 Cool, here are you, me, node2 and node3 Hi, I’m node 4 Hi, I’m node 3 OK, here are you and me Hi, I’m node 2 Well, here are you, me and node2 Node1 11
  27. 27. DISCOVERY Node3 Node4 Node2 Cool, here are you, me, node2 and node3 Hi, I’m node 4 Hi, I’m node 3 OK, here are you and me Hi, I’m node 2 Well, here are you, me and node2 Node1 11
  28. 28. DISCOVERY Node3 Node4 Node2 Cool, here are you, me, node2 and node3 Hi, I’m node 4 Hi, I’m node 3 OK, here are you and me Hi, I’m node 2 Well, here are you, me and node2 Node1 11
  29. 29. DISCOVERY Node3 Node4 Node2 Cool, here are you, me, node2 and node3 Hi, I’m node 4 Hi, I’m node 3 OK, here are you and me Hi, I’m node 2 Well, here are you, me and node2 Node1 11
  30. 30. RING Node1 Node3 Node4 Node2 12
  31. 31. DATA RANGE Long.min Long.max 13
  32. 32. DATA RANGE Long.min Long.max 13
  33. 33. RING CLUSTER Node1 Node3 Node4 Node2 Long.minLong.max 14
  34. 34. TOKEN 15
  35. 35. TOKEN 12343214 15
  36. 36. TOKEN 12343214 Long.min Long.max 15
  37. 37. INITIALTOKENS Node1 Node2 12343214 21144543 16
  38. 38. 17 TOKENS
  39. 39. Issues 17 TOKENS
  40. 40. REPLICATION 18
  41. 41. REPLICATION RF 18
  42. 42. = 2 REPLICATION RF 18
  43. 43. = 2 REPLICATION RF 18 «Hello,TechTalk» Node 1: «Hello, TechTalk» Node 2: «Hello, TechTalk»
  44. 44. REPLICATION &TOKENS Node1 Node 2Node 3 19
  45. 45. REPLICATION &TOKENS Node1 RF = 2 Node 2Node 3 19
  46. 46. REPLICATION &TOKENS RF = 2 Node 2Node 3 Node1 19
  47. 47. REPLICATION &TOKENS RF = 2 Node 2Node 3 Node1 19
  48. 48. REPLICATION &TOKENS RF = 2 Node 3 Node1 Node 2 node1 Node1 19
  49. 49. REPLICATION &TOKENS RF = 2 Node 3 Node1 Node 2 node1 Node1 19
  50. 50. REPLICATION &TOKENS RF = 2 Node 3 Node1 Node 2 node1 Node1 19
  51. 51. REPLICATION &TOKENS Node1 RF = 2 Node 3 Node 2 node1 19
  52. 52. REPLICATION &TOKENS Node1 RF = 2 Node 3 Node 2 node1 19
  53. 53. REPLICATION &TOKENS Node1 RF = 2 Node 3 Node 2 node1 Node 2 node1 19
  54. 54. VNODE 20
  55. 55. VNODE 20
  56. 56. VNODE 255 20
  57. 57. VNODE 255 From To 20
  58. 58. REPLICATION &V-NODE Node1 Node 2Node 3 21
  59. 59. REPLICATION &V-NODE Node1 RF = 2 Node 2Node 3 21
  60. 60. REPLICATION &V-NODE Node1 RF = 2 Node 2Node 3 v-node = 2 21
  61. 61. Node1 REPLICATION &V-NODE RF = 2 Node 2Node 3 v-node = 2 21
  62. 62. Node1 REPLICATION &V-NODE RF = 2 Node 2Node 3 v-node = 2 21
  63. 63. Node1 REPLICATION &V-NODE RF = 2 Node 2Node 3 v-node = 2 21
  64. 64. Node1 REPLICATION &V-NODE RF = 2 Node 2Node 3 v-node = 2 21
  65. 65. Node1 REPLICATION &V-NODE RF = 2 Node 2Node 3 v-node = 2 21
  66. 66. REPLICATION &V-NODE Node1 RF = 2 Node 2Node 3 v-node = 2 21
  67. 67. REPLICATION &V-NODE Node1 RF = 2 Node 2Node 3 v-node = 2 21
  68. 68. NODE RANGE Node1 Node3 Node4 Node2 22
  69. 69. NODE RANGE 23 Node1 Node3 Node4 Node2
  70. 70. CASSANDRA To be or not to be?
  71. 71. 25 DATA MODEL
  72. 72. KEYSPACE 26
  73. 73. KEYSPACE 26
  74. 74. KEYSPACE Table1 Table2 26
  75. 75. KEYSPACE 26 Column Family 1 Column Family 2
  76. 76. KEYSPACE Row 1 Row 2 Row 1 Row 2 26 Column Family 1 Column Family 2
  77. 77. KEYSPACE value value value value value value key key key key key key value value value key key key Row 1 Row 2 Row 1 Row 2 26 Column Family 1 Column Family 2
  78. 78. DATA DISTRIBUTION 27
  79. 79. DATA DISTRIBUTION murmur3(partition_key) 27
  80. 80. 28 KEYS
  81. 81. 28 KEYS Primary
  82. 82. 28 KEYS Primary http://www.pngall.com/man-png
  83. 83. 28 KEYS Primary Partition http://www.pngall.com/man-png
  84. 84. 28 KEYS Primary Partition http://www.pngall.com/man-png http://www.pngmart.com/image/31069
  85. 85. 28 KEYS Primary Partition Composite http://www.pngall.com/man-png http://www.pngmart.com/image/31069
  86. 86. 28 KEYS Primary Partition Composite http://www.pngall.com/man-png http://www.pngmart.com/image/31069 http://www.pngmart.com/image/21342
  87. 87. 28 KEYS Primary Partition Composite Clustering http://www.pngall.com/man-png http://www.pngmart.com/image/31069 http://www.pngmart.com/image/21342
  88. 88. 28 KEYS Primary Partition Composite Clustering http://www.pngall.com/man-png http://www.pngmart.com/image/31069 http://www.pngmart.com/image/21342 http://ujceastside.org/home-health-aides/
  89. 89. 29 PRIMARY CREATE TABLE album ( id uuid, name text, PRIMARY KEY (id) )
  90. 90. 29 PRIMARY CREATE TABLE album ( id uuid, name text, PRIMARY KEY (id) ) Primary & Partition & Clustering
  91. 91. 30 PARTITION CREATE TABLE author_book ( author text, book text, population int, PRIMARY KEY (author, book) )
  92. 92. 30 PARTITION CREATE TABLE author_book ( author text, book text, population int, PRIMARY KEY (author, book) ) Partition Clustering
  93. 93. 31 COMPOSITE & CLUSTERING CREATE TABLE teacher_lesson ( teacher text, lesson text, topic text, duration int, PRIMARY KEY ((teacher, lesson), topic, duration) )
  94. 94. 31 COMPOSITE & CLUSTERING CREATE TABLE teacher_lesson ( teacher text, lesson text, topic text, duration int, PRIMARY KEY ((teacher, lesson), topic, duration) ) Composite Clustering
  95. 95. 32 IMPORTANT
  96. 96. 32 IMPORTANT Natural ordering
  97. 97. 32 IMPORTANT Natural ordering No relations
  98. 98. 32 IMPORTANT Natural ordering No relations No ACID transactions
  99. 99. CASSANDRA to be or not to be?
  100. 100. 34 READ & WRITE
  101. 101. ARCHITECTURE Node 35
  102. 102. ARCHITECTURE Memory Permanent storage Node 35
  103. 103. ARCHITECTURE Memory MemTable Permanent storage Node 35
  104. 104. ARCHITECTURE Memory MemTable CommitLog Permanent storage Node 35
  105. 105. ARCHITECTURE Memory MemTable CommitLog Permanent storage SSTable 1 SSTable 2 SSTable 3 Node 35
  106. 106. ARCHITECTURE Memory MemTable CommitLog Permanent storage SSTable 1 SSTable 2 SSTable 3 Newer Older Node 35
  107. 107. Memory MemTable CommitLog Permanent storage SSTable 1 SSTable 2 SSTable 3 Newer Older Node 35 WRITE
  108. 108. Request Memory MemTable CommitLog Permanent storage SSTable 1 SSTable 2 SSTable 3 Newer Older Node 35 WRITE
  109. 109. Request Memory MemTable CommitLog Permanent storage SSTable 1 SSTable 2 SSTable 3 Newer Older Node 1 35 WRITE
  110. 110. Request Memory MemTable CommitLog Permanent storage SSTable 1 SSTable 2 SSTable 3 Newer Older Node 1 2 35 WRITE
  111. 111. Request Memory MemTable CommitLog Permanent storage SSTable 1 SSTable 2 SSTable 3 Newer Older Node 1 3 4 2 35 WRITE
  112. 112. Request Memory MemTable CommitLog Permanent storage SSTable 5 SSTable 4 SSTable 1 SSTable 2 SSTable 3 Newer Older Node 1 3 4 5 2 35 WRITE
  113. 113. WRITE Node 1 Node 2 Node 3 36
  114. 114. WRITE Node 1 Node 2 Node 3 RF = 3 36
  115. 115. WRITE Node 1 Node 2 Node 3 RF = 3 36 Client
  116. 116. WRITE Node 1 Node 2 Node 3 RF = 3 36 Client
  117. 117. CONSISTENCY LEVEL 37
  118. 118. CONSISTENCY LEVEL 37 CL
  119. 119. WRITE Node 1 Node 2 Node 3 RF = 3 38 Client
  120. 120. WRITE Node 1 Node 2 Node 3 RF = 3 38 Client CL = 2
  121. 121. WRITE Node 3 RF = 3 Node 2Node 1 38 Client successsuccess CL = 2
  122. 122. WRITE Node 3 RF = 3 Node 2Node 1 38 Client successsuccess CL = 2
  123. 123. READ Memory MemTable Permanent storage Node 39
  124. 124. Request READ Memory MemTable Permanent storage Node 39
  125. 125. Request READ Memory MemTable Permanent storage Node 1 39
  126. 126. Request READ Memory MemTable Permanent storage Node SSTable 1 SSTable 2 1 39
  127. 127. Request READ Memory MemTable Permanent storage Node SSTable 1BF IF SSTable 2BF IF 1 39
  128. 128. Request READ Memory MemTable Permanent storage Node SSTable 1BF IF SSTable 2BF IF 1 2 39
  129. 129. Request READ Memory MemTable Permanent storage Node SSTable 1BF IF SSTable 2BF IF 1 2 3 39
  130. 130. Request READ Memory MemTable Permanent storage Node SSTable 1BF IF SSTable 2BF IF 1 2 3 4 39
  131. 131. Request READ Memory MemTable Permanent storage Node SSTable 1BF IF SSTable 2BF IF 1 2 3 4 5 39
  132. 132. Request READ Memory MemTable Permanent storage Node SSTable 1BF IF SSTable 2BF IF 1 2 3 4 5 6 39
  133. 133. READ Node 1 Node 2 Node 3 40
  134. 134. READ Node 1 Node 2 Node 3 RF = 3 40
  135. 135. READ Node 1 Node 2 Node 3 CL = 2 RF = 3 40
  136. 136. READ Node 1 Node 2 Node 3 CL = 2 RF = 3 40 Client
  137. 137. READ Node 1 Node 2 Node 3 CL = 2 RF = 3 get digest 40 Client
  138. 138. READ Node 1 Node 2 Node 3 CL = 2 RF = 3 get digest 123453 40 Client
  139. 139. READ Node 1 Node 2 Node 3 CL = 2 RF = 3 get digest 123453 40 Client
  140. 140. CASSANDRA To be or not to be?
  141. 141. 42 UPDATE & DELETE
  142. 142. UPDATE 43
  143. 143. UPDATE Upsert 43
  144. 144. UPDATE Memory MemTable Permanent storage SSTable 1 SSTable 2 SSTable 3 Newer Older Node CommitLog 44
  145. 145. UPDATE Request Memory MemTable Permanent storage SSTable 1 SSTable 2 SSTable 3 Newer Older Node CommitLog 44
  146. 146. UPDATE Request Memory MemTable Permanent storage SSTable 1 SSTable 2 SSTable 3 Newer Older Node CommitLog 1 44
  147. 147. UPDATE Request Memory MemTable Permanent storage SSTable 1 SSTable 2 SSTable 3 Newer Older Node CommitLog 1 2 44
  148. 148. UPDATE Request Memory MemTable Permanent storage SSTable 1 SSTable 2 SSTable 3 Newer Older Node CommitLog 1 2 3 44
  149. 149. UPDATE Request Memory MemTable Permanent storage SSTable 1 SSTable 2 SSTable 3 Newer Older Node CommitLog 1 2 3 4 44
  150. 150. UPDATE Request Memory MemTable Permanent storage SSTable 1 SSTable 2 SSTable 3 Newer Older Node CommitLog 1 2 3 4 5 44
  151. 151. DELETE 45
  152. 152. DELETE https://docs.datastax.com/en/cql/3.3/cql/cql_reference/cqlDelete.html 45
  153. 153. DELETE 46
  154. 154. DELETE 46 Update flow *
  155. 155. TOMBSTONES 47 http://shipoffools.wikia.com/wiki/File:Tombstone.png
  156. 156. TOMBSTONES Overhead 47 http://shipoffools.wikia.com/wiki/File:Tombstone.png
  157. 157. TOMBSTONES 48
  158. 158. TOMBSTONES Remove 48
  159. 159. TOMBSTONES Remove Null 48
  160. 160. TOMBSTONES Remove Null TTL 48
  161. 161. CASSANDRA To be or not to be?
  162. 162. 50 SUMMARY
  163. 163. RECOMMENDATIONS https://youtu.be/pAGESQo5IjI 51 https://youtu.be/h14iML3aaK0
  164. 164. THANKYOU My contacts : Email : andriyrav@gmail.com Q / A 52 Twitter : @AndriyRymar

×