Downloaded 56 times
More Related Content
Similar to Cassandra - Deep Dive ...
Cassandra - Deep Dive ...
- 1. Cassandra A Decentralized StructuredStorage System By Sameera Nelson
- 2. Outline … Introduction Data Model System Architecture Failure Detection & Recovery Local Persistence Performance Statistics
- 3. What is Cassandra? Distributed Storage System Manages Structured Data Highly available , No SPoF Not a Relational Data Model Handle high write throughput ◦ No impact on read efficiency
- 4. Motivation Operational Requirementsin Facebook ◦ Performance ◦ Reliability/ Dealing with Failures ◦ Efficiency ◦ Continues Growth Application ◦ Inbox Search Problem, Facebook
- 5. Similar Work GoogleFile System ◦ Distributed FS, Single master/Slave Ficus/ Coda ◦ Distributed FS Farsite ◦ Distributed FS, No centralized server Bayou ◦ Distributed Relational DB System Dynamo ◦ Distributed Storage system
- 6.
- 7. Data Model Figure fromEben Hewitt’s slides.
- 8. Supported Operations insert(table;key; rowMutation) get(table; key; columnName) delete(table; key; columnName)
- 9. Query Language CREATE TABLEusers ( user_id int PRIMARY KEY, fname text, lname text ); INSERT INTO users (user_id, fname, lname) VALUES (1745, 'john', 'smith'); SELECT * FROM users;
- 10. Data Structure Log-StructuredMerge Tree
- 11.
- 12.
- 13. Fully Distributed … No Single Point of Failure
- 14. Cassandra Architecture Partitioning Data distribution across nodes Replication Data duplication across nodes Cluster Membership Node management in cluster adding/ deleting
- 15.
- 16. Partitioning Partitions usingConsistent hashing
- 17. Partitioning Assignment into the relevant partition
- 18.
- 19. Replication Based onconfigured replication factor
- 20. Replication Different ReplicationPolicies ◦ Rack Unaware ◦ Rack Aware ◦ Data center Aware
- 21. Cluster Membership Basedon scuttlebutt Efficient Gossip based mechanism Inspired for real life rumor spreading. Anti Entropy protocol ◦ Repair replicated data by comparing & reconciling differences
- 22.
- 23.
- 24. Failure Detection Trackstate ◦ Directly, Indirectly Accrual Detection mechanism Permanent Node change ◦ Admin should explicitly add or remove Hints ◦ Data to be replayed in replication ◦ Saved in system.hints table
- 25. Accrual Failure Detector •Node is faulty, suspicion level monotonically increases. • Φ(t) k • k - threshold variable • Node is correct • Φ(t) = 0
- 26.
- 27.
- 28.
- 29. Write Operation Loggingdata in commit log/ memtable Flushing data from the memtable ◦ Flushing data on threshold Storing data on disk in SSTables Mark with tombstone Compaction Remove deletes, Sorts, Merges data, consolidation
- 30.
- 31. Read Request Direct/Background (Read repair)
- 32.
- 33. Delete Operation Datanot removed immediately Only Tombstone is written Deleted in Compacting Process
- 34. Additional Features Addingcompression Snappy Compression Secondary index support SSL support ◦ Client/ Node ◦ Node/ Node Rolling commit logs SSTable data file merging
- 35.
- 36. Performance High Throughput& Low Latency ◦ Eliminating on-disk data modification ◦ Eliminate erase-block cycles ◦ No Locking for concurrency control ◦ Maintaining integrity not required High Availability Linear Scalability Fault Tolerant
- 37.
- 38. Stats from Netflix Liner scalability
- 39.
- 40.
- 41.
- 42.