Dun ddd

291 views
234 views

Published on

Cassandra 2.0 presentation at DunDDD

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
291
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Dun ddd

  1. 1. Cassandra 2.0 Lyuben Todorov Cassandra Developer, DataStax
  2. 2. New Core Value • • • • Massive Scalability Performance Reliability Ease of Use
  3. 3. Binary Protocol
  4. 4. Binary Protocol • Paging • Batching prepared statements • Improved parameterized statements
  5. 5. Cursors Before - manual paging • Have to keep track of position • Complex for compound primary keys CREATE TABLE timeline ( ... PRIMARY KEY (uid, event_id) ); SELECT * FROM timeline WHERE (uid = :last_key AND event_id > :last_event) OR token(uid) > token(:last_key) LIMIT 100;
  6. 6. Cursors Now Statement s = new SimpleStatement(‚SELECT * FROM timeline‛); s.setFetchSize(100); // transparently fetches next pages ResultSet result = session.execute(s); for (Row row : result) { dooStuff(row); }
  7. 7. Cursors Now Statement s = new SimpleStatement(‚SELECT * FROM timeline‛); s.setFetchSize(100); // transparently fetches next pages ResultSet result = session.execute(s); for (Row row : result) { dooStuff(row); // x2 }
  8. 8. Batching PreparedStatement PreparedStatement ps = session.prepare("INSERT INTO timeline (id, event) VALUES(?, ?)"); BatchStatement batch = new BatchStatement(); batch.add(ps.bind(UUID.randomUUID(), event_1)); batch.add(ps.bind(UUID.randomUUID(), event_2)); session.execute(batch);
  9. 9. Simplified Parameterized Statements session.execute(String cql, Object… values); session.executeAsynch(String cql, Object… values); session.execute("INSERT INTO timeline (uid, event) VALUES (?, ?)", UUID.randomUUID(), event_x);
  10. 10. CQL3
  11. 11. CQL3 • SELECT DISTINCT partition_key;
  12. 12. CQL3 • SELECT DISTINCT partition_key; • CREATE TABLE IF NOT EXISTS tbl_name;
  13. 13. CQL3 • SELECT DISTINCT partition_key; • CREATE TABLE IF NOT EXISTS tbl_name; • Aliases SELECT event_id, dateOf(created_at) AS creation_date;
  14. 14. CQL3 • SELECT DISTINCT partition_key; • CREATE TABLE IF NOT EXISTS tbl_name; • Aliases SELECT event_id, dateOf(created_at) AS creation_date • ALTER TABLE tbl DROP column_name;
  15. 15. Lightweight Transactions SESSION 1 SESSION 2 SELECT * FROM users WHERE USERNAME = ‘lyuben’; SELECT * FROM users WHERE USERNAME = ‘lyuben’; [empty resultset] [empty resultset] INSERT INTO users (...) VALUES (‘lyubent’, ...) INSERT INTO users (...) VALUES (‘lyubent’, ...) Last write wins. User exists? If not, create user
  16. 16. Why not locking? request Client (locks) Coordinator internal request Replica
  17. 17. Why not locking? request Client (locks) Coordinator internal request X Replica
  18. 18. Why not locking? request Coordinator Client (locks) internal request store for replay X Replica
  19. 19. Why not locking? request Coordinator Client (locks) internal request timeout response store for replay X Replica
  20. 20. Paxos A family of protocols for solving consensus in a network of unreliable processors. Wikipedia
  21. 21. Paxos • Immediate Consistency • QUARUM based operations • Unfinished operations’ states sent to leader during prepare phase • Paxos state is durable
  22. 22. LWT – Use when Appropriate • Expensive – x4 round trips • Eventual consistency is your friend
  23. 23. Lightweight Transactions – CQL INSERT INTO USERS (username, email ...) VALUES (‘lyuben’, ‘ltodorov@datastax.com’, ... ) IF NOT EXISTS;
  24. 24. Triggers CREATE TRIGGER <name> ON <table> USING <classname>;
  25. 25. Triggers class MyTrigger implements ITrigger { public Collection<RowMutation> augment(ByteBuffer key, ColumnFamily update) { ... } }
  26. 26. Tracing • Detailed view of what’s going on • Great for debugging queries cqlsh:test> TRACING ON; Now tracing requests.
  27. 27. Tracing insert cqlsh:test> INSERT INTO example (i, j) VALUES ('key', 7); Tracing session: 69fc9cb0-4fb3-11e3-84ae-612d9c5d36d9 activity | timestamp | source | source_elapsed ------------------------------------+--------------+-----------+---------------Determining replicas for mutation | 18:09:34,722 | 127.0.0.1 | 3507 Sending message to /127.0.0.2 | 18:09:34,724 | 127.0.0.1 | 5720 Acquiring switchLock read lock | 18:09:34,732 | 127.0.0.2 | 6043 Appending to commitlog | 18:09:34,732 | 127.0.0.2 | 6305 Adding to example memtable | 18:09:34,732 | 127.0.0.2 | 6373 Enqueuing response to /127.0.0.1 | 18:09:34,733 | 127.0.0.2 | 6978 Message received from /127.0.0.2 | 18:09:34,737 | 127.0.0.1 | 19055 Processing response from /127.0.0.2 | 18:09:34,738 | 127.0.0.1 | 19993 Request complete | 18:09:34,739 | 127.0.0.1 | 20322
  28. 28. Tracing tombstone cqlsh:test> SELECT * FROM example; Tracing session: 79d55380-4fb7-11e3-9ac8-612d9c5d36d9 activity | timestamp | source | source_elapsed -----------------------------------------+--------------+-----------+--------------... Sending message to /127.0.0.2 | 18:38:39,291 | 127.0.0.1 | 601 Read 10 live and 100000 tombstoned cells | 18:38:39,291 | 127.0.0.2 | 31655 Scanned 1 rows and matched 1 | 18:38:39,292 | 127.0.0.2 | 31693 Message received from /127.0.0.2 | 18:38:39,292 | 127.0.0.1 | 33150 Enqueuing response to /127.0.0.1 | 18:38:39,292 | 127.0.0.2 | 33724 Processing response from /127.0.0.2 | 18:38:39,292 | 127.0.0.1 | 34704 Sending message to /127.0.0.1 | 18:38:39,292 | 127.0.0.2 | 35220 ...
  29. 29. Row Marker CREATE TABLE tbl ( key int, a text, b text, PRIMARY KEY (key) ); UPDATE tbl SET a=null, b=null WHERE key=1; DELETE FROM tbl where key=2;
  30. 30. Row Marker (JSON format) [ {"key": "00000001","columns": [["","",1384716960173000], ["a","52891aa0",1384716960173000,"d"], ["b","52891aa0",1384716960173000,"d"]] }, {"key": "00000002","columns": []} ] Retreied via sstable2json
  31. 31. Rapid Read Protection • Configurable per-table. • Reduces occurrences of read timeouts in overloaded / crashed replicas. • Enabled by default in 2.0.2.
  32. 32. Rapid Read Protection
  33. 33. Rapid Read Protection Configuring rapid read protection # retry if request takes longer than 10ms ALTER TABLE example WITH speculative_retry = '10ms'; # retry if request takes longer then 99% of requests ALTER TABLE example WITH speculative_retry = '99percentile';
  34. 34. Going Off Heap Managed by GC Stack Heap Not Managed by GC Native
  35. 35. Going Off Heap • Bloom Filters 1 - 2GB per billion entries
  36. 36. Going Off Heap • Bloom Filters 1 - 2GB per billion entries • Compression Offsets 1 - 3GB per TB of compressed data
  37. 37. Going Off Heap • Bloom Filters 1 - 2GB per billion entries • Compression Offsets 1 - 3GB per TB of compressed data • Partition Summary Depends on # of rows per partition
  38. 38. Compaction Leveled Row fragments Size Tiered
  39. 39. Leveled Compaction Strategy
  40. 40. Happy LCS
  41. 41. Sad LCS
  42. 42. STCS in Level 0
  43. 43. Coming up • Secondary indexes for collections (4511)
  44. 44. Coming up • Secondary indexes for collections (4511) CREATE TABLE image ( id UUID, tags set<text>, PRIMARY KEY(id) ); SELECT * FROM image WHERE tags CONTAINS ’sunny’; image_id | tags --------------------------------------+------------------------------1a3ab520-5177-11e3-91ae-612d9c5d36d9 | {'beach', 'holiday', 'sunny'} 2617d120-5177-11e3-91ae-612d9c5d36d9 | {'mountains', 'sunny'}
  45. 45. Coming up • Secondary indexes for collections (4511) • More efficient repairs (5351)
  46. 46. Coming up • Secondary indexes for collections (4511) • More efficient repairs (5351) STCS Repair Compaction
  47. 47. Coming up • Secondary indexes for collections (4511) • More efficient repairs (5351) LCS L0 L0 L1 L2
  48. 48. Coming up • Secondary indexes for collections (4511) • More efficient repairs (5351) • Custom types (5590)
  49. 49. Coming up • Secondary indexes for collections (4511) • More efficient repairs (5351) • Custom types (5590) CREATE TYPE address ( street text, city text, zip_code int, phones set<text> ) Key UTF8Type UTF8Type CREATE TABLE users ( id uuid PRIMARY KEY, name text, addresses map<int, address> ) INT32Type MapType ( Int32Type, Address)
  50. 50. Coming up • • • • Secondary indexes for collections (4511) More efficient repairs (5351) Custom types (5590) CQL aggregate functions (4914)
  51. 51. Coming up • • • • Secondary indexes for collections (4511) More efficient repairs (5351) Custom types (5590) CQL aggregate functions (4914) – AVG, MIN, MAX, MEAN, SUM, etc. SELECT sum(salary) FROM employee where country='UK';
  52. 52. Coming up • • • • • Secondary indexes for collections (4511) More efficient repairs (5351) Custom types (5590) CQL aggregate functions (4914) Many more!
  53. 53. DataStax Ac*ademy Free online cassandra training! https://datastaxacademy.elogiclearning.com/

×