Downloaded 29 times
More Related Content
Similar to Cassandra under the hood
Cassandra under the hood
- 1.
- 2. Who I am JavaSoftware Engineer @ Lohika More than 7 years of experience Andriy Rymar 2
- 3. What we won’t •Learn how to use Cassandra • Learn about performance tuning • Learn how to manage cluster • Learn how to interact with Cassandra 3
- 4. What we will 4 Wewill learn what is Cassandra
- 5. Content • General overview •Data model • Architecture • Read & Write operations 5
- 6.
- 7. • RDBMS -is not bad • RDBMS - has been successful in the last 40 years RDBMS 7
- 8. • Slow queriesdue to complex joins, long time to reindexing data • Expensive vertical scaling and problems with horizontal scaling • When you try to replicate database you hurt the availability of the system RDBMS Issues 8
- 9.
- 10. CA, CP, AP •Consistency & Availability • Consistency & Partition-tolerance • Availability & Partition-tolerance 10
- 11. Eventual consistent Eventual consistentsystem without any failures Eventual consistent system with failures V0 V0 V0 V0 V0 V1 V0 V0 V1 V1 V1 V1 V1 V1 V1 V0 V1 V1 V1 V1V1 V1 11 V1
- 12. Solution Google BigTable 2004 Cassandra 2008 (2010, 2013) Amazon Dynamo DB 2012 12
- 13.
- 14. Cassandra cluster N1 N2 N3 A G R Tokens &Seed node & Ring representation Tokens - determine position of node in ring cluster and portion of data N1 14
- 15. Cassandra cluster N1 N2 N3 A-F G-Q R-Z pk: «Taras»,message: «Hello» Replication Factor (RF) = 2 G-Q R-Z A-F 15
- 16. Tokens Issues • Manually managetoken initial value for all nodes • Big overhead when restoring node data for(int i=0; i < CLUSTER_SIZE; i++) { System.out.println((((2**64 / CLUSTER_SIZE) * i) - 2**63)) } N1 N2N3 Replication Factor (RF) = 2 Ne w N2 16
- 17. Virtual Nodes 1 2 3 4 5 6 7 8 910 11 12 Server1Server2 Server3Server4 17
- 18. Virtual Nodes Data restoring vnode= 3 S1 S3 S2S4 RF = 2 18
- 19. V-nodes Summary • Rebalancing acluster is no longer necessary when adding or removing nodes • More powerful machines can have more v-nodes. This approach give ability to build heterogeneous Cassandra ring 19
- 20.
- 21. Introduction into datamodel KEYSPACE Table (column family) partition key column1 column2 column3 model123 value value age email test@test.com name demo14 value 21
- 22. Column family • RDBMS user nameemail title age … Taras Andriy tm@gm.com ar@gm.com Staff Engineer 27 • Column Family user … key: Taras key: Andriy value: value: email : tm@gm.com title: Staff Engineer email : ar@gm.com age: 27 22
- 23. Column family “user” :{ “Taras” : { “email” : “tm@gm.com”, “title” : “Staff Engineer” }, “Andriy” : { “email” : “ar@gm.com”, “age” : “27” } } user … key: Taras key: Andriy value: value: email : tm@gm.com title: Staff Engineer email : ar@gm.com age: 27 23
- 24. Other differences • Norelations (No Joins) • Tuples (key-value pairs) are natural sorted • May want to denormalize data model in database • No transactions 24
- 25. Type of keys •Primary key • Composite key • Partition key • Clustering key • Composite partition key 25
- 26. Example 1 CREATE TABLEalbum ( id uuid, name name, PRIMARY KEY (id) ) Primary key and also the partition key id - partition & primary key at the same time 26
- 27. Composite key Example 2 CREATETABLE author_book ( author text, book text, population int, PRIMARY KEY (author, book) ) partition key primary key 27
- 28. Example 3 Key withcomposite partition & clustering keys CREATE TABLE teacher_lesson ( teacher text, lesson text, topic text, duration int, PRIMARY KEY ((teacher, lesson), topic, duration) ) clustering keyscomposite partition key 28
- 29. Row vs Partition Rows Partitions Node1 Node 2 1234 5678 9101112 1234:user 5678:user 1234:address 5678:address 1234:details 5678:details 29
- 30. Coffee break • Generaloverview • Data model • Architecture • Read & Write operations 30
- 31.
- 32. Cassandra components API Tools Storagelayer Partitioner Replicator Failure detector Compaction Manager Messaging layer 32
- 33. Cassandra components API Tools Storagelayer Partitioner Replicator Failure detector Compaction Manager Messaging layer 33
- 34. Messaging service In clusterof 5 nodes , each node has 8 opened socket connections 34 Has 2 opened socket connections with every other node
- 35.
- 36. Gossip How Cassandra initiatessessions? • One session for any random live node • One session for any random unreachable node • If the node in point 1 is not a seed node, then create session with random seed node 36
- 37. Gossip Session 1 : GossipSyncMessage N1N2 2 : GossipAckMessage 3 : GossipAck2Message 37
- 38. Cassandra components API Tools Storagelayer Partitioner Replicator Failure detector Compaction Manager Messaging layer 38
- 39. Failure detection ϕ accrualfailure detector • Doesn’t use TRUE / FALSE • Provides continuos value • This value is called «ϕ» 39
- 40. Failure detection ϕ accrualfailure detector 40 time session 1 2 3 4 5 1s2s
- 41. Failure detection Proposed byXavier Défago in 2004 41 https://goo.gl/xS0kB0
- 42. Cassandra components API Tools Storagelayer Partitioner Replicator Failure detector Compaction Manager Messaging layer 42
- 43.
- 44. Partitioner • Murmur 3Partitioner • Random Partitioner • Byte Order Partitioner 44
- 45. Cassandra components API Tools Storagelayer Partitioner Replicator Failure detector Compaction Manager Messaging layer 45
- 46. Replicator • Replication factor= 3 Write data request N1 N2 N3 N4N1 N2 N3 • Consistency Level = 2 N1 N2 46
- 47. Consistency level • ZERO(write only) • ANY (write only) • ONE • QUORUM • ALL Push and forget Success even hinted of write First replica returned successfully N/2 +1 replica success All replica success 47 Replicator
- 48. Inconsistency • 5 nodecluster • Replication factor 3 • Consistency level 1 N1 N2 N3 N4 N5 Write Read N2 N3 N4 48 Replicator
- 49. Tuning • Use consistencylevel with at least 1 node overlap (Quorum) Write CL = 2 Read CL = 2 Replication factor = 3 N1 N2 N3 N4 N5 Write Read N2 N3 N4 49 Replicator
- 50. Tuning • Tune readand write CL separately to reach high performance Fast write Fast Read Write CL = 1 Read CL = ALL Write CL = ALL Read CL = 1 50 Replicator
- 51. Cassandra components API Tools Storagelayer Partitioner Replicator Failure detector Compaction Manager Messaging layer 51
- 52. Storage layer 52 Client Mutation Request Commitlog MemTable SSTable mem hdd add / update append Flush cleanup
- 53. Storage layer 53 Client Mutation Request Commitlog MemTable SSTable mem hdd add / update append Flush cleanup
- 54. SSTable • Representation ofMemTable • Immutable • Eventually get merged into larger SSTable files (compaction) • Has next components • Bloom filter • Index file • Data file 54
- 55. SSTable Bloom filter • Bloomfilter is used to determine correct SSTable • Bloom filter may result as FALSE positive • Stored on heap memory 55
- 56. SSTable Bloom filter 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 1 1 1 0 0 1 0 0 1 0 0 0 5 0 0 2 0 1 1 3 0 0 1 0 0 2 0 56 murmur3(`key`) = 15
- 57. SSTable Index file • Containsall row keys and their offset in data file • Each 128th key from the file will be stored into memory • Use binary search to determine right index in memory 57
- 58. SSTable Index file 58 memory hdd 126,… 127,… 128,… … 199,… 200,… 201,… 202,… 203,… … Indexfile 1, 128, 256, 384 Sampled IndexBF 201
- 59. Cassandra components API Tools Storagelayer Partitioner Replicator Failure detector Compaction Manager Messaging layer 59
- 60. Compaction • Merges SSTables •There are two compaction strategies • size-tiered • leveled 60
- 61. Compaction A B AB C C D E C C F Size-tiered (Minor compaction) D E C F 61 … Compaction Level = 2
- 62. Compaction B C D E Size-tiered (Majorcompaction) F 62 A
- 63. Data repair • Hintedhandoff • Read repair • Anti-entropy 63
- 64. Coffee break • Generaloverview • Data model • Architecture • Read & Write operations 64
- 65. Cassandra Read & Writeoperations 65
- 66.
- 67. Write Replication & Consistency N1 N4 ReplicationFactor = 3 N2 N3 Consistency Level = 2 67 Anti-entropy / Read repair Hinted handoff
- 68.
- 69. Read Snitch Function • SimpleSnitch •DynamicSnitch • PropertyFileSnitch • GossipingPropertyFileSnitch • RackInferringSnitch … 69
- 70.
- 71.
- 72. Read Snitch functions GossipingPropertyFileSnitch $CASSANDRA_HOME/conf/cassandra-rackdc.properties # indicatethe rack and dc for this node dc=DC1 rack=RAC1 72
- 73. Read In action • 7node • RF = 4 • CL = 3 Read request Node1 StorageProxy Read data Get digest Get digest Node 3 Node 4 Node 5 Node 6 Node 7 Node 2 73 Node 3 Node 4 Node 5
- 74. Read on node B SSTable I BB I 74 B
- 75.
- 76. Nishant Neeraj :«Mastering Apache Cassandra - Second Edition» Throughput comparison 76
- 77. Publication A Real ComparisonOf NoSQL Databases HBase, Cassandra & MongoDB 77 https://goo.gl/z5abRu
- 78.
- 79.
- 80.
- 81. Resources Nishant Neeraj Mastering ApacheCassandra - 2015 http://docs.datastax.com/en/cassand ra/3.x/cassandra/cassandraAbout.ht ml Thank you 81