On Cassandras evolutionBerlin Buzzwords(June4th 2013)Sylvain Lebresne
Apache Cassandra#bbuzzFully Distributed DatabaseMassively ScalableHigh performanceHighly reliable/available····3/22
Cassandra: the past#bbuzzCassandra 0.7 (Jan 2011):Cassandra 0.8 (Jun 2011):·Dynamic schema creationExpiring columns (TTL)S...
Cassandra: the presentCassandra 1.2 (Jan 2013):#bbuzzVirtual nodesCQL3Native protocolTracing...·····5/22
Data distribution without virtual nodes#bbuzz 6/22
Virtual nodes#bbuzz 7/22
Repairing without virtual nodes#bbuzz 8/22
Virtual nodes: repairing#bbuzz 9/22
Virtual nodes#bbuzzNot really "virtual nodes", more "multiple tokens per nodes" (but we still call themvnodes).Faster rebu...
The Cassandra Query language#bbuzzInitial version introduced in Cassandra 0.8. Version 3 (described here) is a major, more...
Storing songsid title artist album tags tracka3e64f8f... La Grange ZZTop Tres Hombres { blues, 1973 } 38a172618... Moving ...
Playlistsuser_id playlist_name title artist album song_idpcmanus My list La Grange ZZTop Tres Hombres a3e64f8f...pcmanus M...
Querying a Playlist#bbuzz--SongsinMylistwithatitlestartingbyborcSELECT*FROMplaylistsWHEREuser_id=pcmanusANDplaylist_name=M...
Native protocolBinary transport protocol for CQL3 (replace Thrift transport):See the Datastax Java Driver for a mature dri...
Request tracing#bbuzzcqlsh:foo>TRACINGON;cqlsh:foo>INSERTINTObar(i,j)VALUES(6,2);CQLactivity |timestamp |source |elapsed--...
Tracing an anti-patternid created_at valuemy_queue 1399121331 0x9b0450d30de9my_queue 1439051021 0xfc7aee5f6a66my_queue 144...
Tracing an anti-pattern#bbuzzcqlsh:foo>TRACINGON;cqlsh:foo>SELECTFROMqueuesWHEREid=myqueueORDERBYcreated_atLIMIT1;CQLactiv...
But also ...#bbuzzConcurrent schema creationImproved JBOD supportOff-heap bloom filters and compression metadataFaster (mu...
Whats next?Cassandra 2.0 is scheduled for July:#bbuzzImprovements to CQL3 and the native protocol (automatic query paging)...
<Thank You!>Questions?www http://cassandra.apache.org/twitter @pcmanusgithub github.com/pcmanus
On cassandra's evolution @ Berlin buzzwords
On cassandra's evolution @ Berlin buzzwords
Upcoming SlideShare
Loading in …5
×

On cassandra's evolution @ Berlin buzzwords

1,103 views

Published on

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,103
On SlideShare
0
From Embeds
0
Number of Embeds
45
Actions
Shares
0
Downloads
5
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

On cassandra's evolution @ Berlin buzzwords

  1. 1. On Cassandras evolutionBerlin Buzzwords(June4th 2013)Sylvain Lebresne
  2. 2. Apache Cassandra#bbuzzFully Distributed DatabaseMassively ScalableHigh performanceHighly reliable/available····3/22
  3. 3. Cassandra: the past#bbuzzCassandra 0.7 (Jan 2011):Cassandra 0.8 (Jun 2011):·Dynamic schema creationExpiring columns (TTL)Secondary indexes---·CountersFirst version of CQLAutomatic memtable tuning---Cassandra 1.0 (Oct 2011):Cassandra 1.1 (Apr 2012):·CompressionLeveled compaction--·Row level isolationConcurrent schema changesSupport for mixed SDD+HDDnodesSelf-tuning key/row caches----4/22
  4. 4. Cassandra: the presentCassandra 1.2 (Jan 2013):#bbuzzVirtual nodesCQL3Native protocolTracing...·····5/22
  5. 5. Data distribution without virtual nodes#bbuzz 6/22
  6. 6. Virtual nodes#bbuzz 7/22
  7. 7. Repairing without virtual nodes#bbuzz 8/22
  8. 8. Virtual nodes: repairing#bbuzz 9/22
  9. 9. Virtual nodes#bbuzzNot really "virtual nodes", more "multiple tokens per nodes" (but we still call themvnodes).Faster rebuilds.Allows heterogeneous nodes.Simpler load balancing when adding nodes.····10/22
  10. 10. The Cassandra Query language#bbuzzInitial version introduced in Cassandra 0.8. Version 3 (described here) is a major, moreambitious, revision.Goal: provide a much simpler, more abstracted user interface than the legacy thrift one.Kind of a "denormalized SQL".Strictly real-time oriented:····No joinsNo sub-queriesNo aggregation/GROUP BYLimited ORDER BY----11/22
  11. 11. Storing songsid title artist album tags tracka3e64f8f... La Grange ZZTop Tres Hombres { blues, 1973 } 38a172618... Moving in Stereo Fu Manchu We Must Obey { covers, 2003 } 92b09185b... Outside Woman Blues Back Door Slame Roll Away 3#bbuzzCREATETABLEsongs(iduuidPRIMARYKEY,titletext,artisttext,albumtext,trackint,tagsset<text>);--AtomicandisolatedINSERTINTOsongs(id,title,artist,album,tags,track)VALUES(a3e64f8f...,LaGrange,ZTopTresHombres,{blues,1973},3);UPDATEsongsSETartist=ZZTopWHEREid=a3e64f8f...;CQL12/22
  12. 12. Playlistsuser_id playlist_name title artist album song_idpcmanus My list La Grange ZZTop Tres Hombres a3e64f8f...pcmanus My list Moving in Stereo Fu Manchu We Must Obey 8a172618...pcmanus Other list La Grange ZZTop Tres Hombres a3e64f8f...pcmanus Other list Outside Woman Blues Back Door Slame Roll Away 2b09185b...Partition key: determines the physical node holding the rowClustering key: induces physical ordering of the rows on disk#bbuzzCREATETABLEplaylists(user_idtext,playlist_nametext,titletext,artisttext,albumtext,song_iduuid,PRIMARYKEY((user_id,playlist_name), title,album,artist));CQL13/22
  13. 13. Querying a Playlist#bbuzz--SongsinMylistwithatitlestartingbyborcSELECT*FROMplaylistsWHEREuser_id=pcmanusANDplaylist_name=MylistANDtitle>=bANDtitle<d;--50lastsongsinMylistSELECT*FROMplaylistsWHEREuser_id=pcmanusANDplaylist_name=MylistORDERBYtitleDESCLIMIT50;CQL14/22
  14. 14. Native protocolBinary transport protocol for CQL3 (replace Thrift transport):See the Datastax Java Driver for a mature driver using this new protocol(https://github.com/datastax/java-driver).#bbuzzAsynchronous (less connections)Server notifications for new nodes, schema changes, etc..Optimized for CQL3···15/22
  15. 15. Request tracing#bbuzzcqlsh:foo>TRACINGON;cqlsh:foo>INSERTINTObar(i,j)VALUES(6,2);CQLactivity |timestamp |source |elapsed-------------------------------------+--------------+-----------+---------Determiningreplicasformutation |00:02:37,015|127.0.0.1| 540Sendingmessageto/127.0.0.2 |00:02:37,015|127.0.0.1| 779Messagereceivedfrom/127.0.0.1 |00:02:37,016|127.0.0.2| 63Applyingmutation |00:02:37,016|127.0.0.2| 220AcquiringswitchLock |00:02:37,016|127.0.0.2| 250Appendingtocommitlog |00:02:37,016|127.0.0.2| 277Addingtomemtable |00:02:37,016|127.0.0.2| 378Enqueuingresponseto/127.0.0.1 |00:02:37,016|127.0.0.2| 710Sendingmessageto/127.0.0.1 |00:02:37,016|127.0.0.2| 888Messagereceivedfrom/127.0.0.2 |00:02:37,017|127.0.0.1| 2334Processingresponsefrom/127.0.0.2 |00:02:37,017|127.0.0.1| 255016/22
  16. 16. Tracing an anti-patternid created_at valuemy_queue 1399121331 0x9b0450d30de9my_queue 1439051021 0xfc7aee5f6a66my_queue 1440134565 0x668fdb3a2196my_queue 1445219887 0xdaf420a01c09my_queue 1479138491 0x3241ad893ff0#bbuzzCREATETABLEqueues(idtext,created_attimestamp,valueblob,PRIMARYKEY(id,created_at));CQL17/22
  17. 17. Tracing an anti-pattern#bbuzzcqlsh:foo>TRACINGON;cqlsh:foo>SELECTFROMqueuesWHEREid=myqueueORDERBYcreated_atLIMIT1;CQLactivity |timestamp |source |elapsed------------------------------------------+--------------+-----------+---------execute_cql3_query |19:31:05,650|127.0.0.1| 0Sendingmessageto/127.0.0.3 |19:31:05,651|127.0.0.1| 541Messagereceivedfrom/127.0.0.1 |19:31:05,651|127.0.0.3| 39Executingsingle-partitionquery |19:31:05,652|127.0.0.3| 943Acquiringsstablereferences |19:31:05,652|127.0.0.3| 973Mergingmemtablecontents |19:31:05,652|127.0.0.3| 1020Mergingdatafrommemtablesandsstables |19:31:05,652|127.0.0.3| 1081Read1livecellsand100000tombstoned |19:31:05,686|127.0.0.3| 35072Enqueuingresponseto/127.0.0.1 |19:31:05,687|127.0.0.3| 35220Sendingmessageto/127.0.0.1 |19:31:05,687|127.0.0.3| 35314Messagereceivedfrom/127.0.0.3 |19:31:05,687|127.0.0.1| 36908Processingresponsefrom/127.0.0.3 |19:31:05,688|127.0.0.1| 37650Requestcomplete |19:31:05,688|127.0.0.1| 3804718/22
  18. 18. But also ...#bbuzzConcurrent schema creationImproved JBOD supportOff-heap bloom filters and compression metadataFaster (murmur3 based) partitioner...·····19/22
  19. 19. Whats next?Cassandra 2.0 is scheduled for July:#bbuzzImprovements to CQL3 and the native protocol (automatic query paging)Compare-and-swapTriggers (experimental)Eager retriesPerformance improvements (single-pass compaction, more efficient tombstone removal,...)...··UPDATEusersSETlogin=pcmanus,name=SylvainLebresneIFNOTEXISTS;UPDATEusersSETemail=sylvain@datastax.comIFemail=slebresne@datastax.com;CQL····20/22
  20. 20. <Thank You!>Questions?www http://cassandra.apache.org/twitter @pcmanusgithub github.com/pcmanus

×