C* Summit 2013: The State of CQL by Sylvain Lebresne


Published on

Abstract Since its inception, the Cassandra Query Language (CQL) has grown and matured, resulting in the 3rd version of the language (CQL3) being finalized in Cassandra 1.2. Compared to the legacy Thrift API, CQL3 aims at providing an API that is higher level and more user friendly but still fully assumes the distributed nature of Cassandra and it's storage engine. This presentation will present CQL3, describing the reasoning and goals behind the language as well as the language itself. CQL's relationship with Thrift will be touched on, along with the CQL binary protocol that has been introduced in Cassandra 1.2. This presentation will wrap up by discussing the future of CQL.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

C* Summit 2013: The State of CQL by Sylvain Lebresne

  1. 1. The State of CQLSylvain Lebresne (@pcmanus)June 12, 2013
  2. 2. Why CQL?(Rational and goals behind CQL)What is CQL?(How do you model application with CQL)The native protocol(Transporting CQL queries)Whats next?(Cassandra 2.0 and beyond)2/26
  3. 3. DisclaimerThis presentation focuses exclusively on CQL version 3. Many things do not apply to CQL version 1 and 2.Unless explicitly state otherwise, the terms rows and columns means CQL3 rows and CQL3 columns, which doesnot map directly to the notion of rows and columns in thrift (or the internal C* implementation).··3/26
  4. 4. Why?Rational and goals behind CQL
  5. 5. The thrift API is:Cassandra has often been regarded as hard to develop against.It doesnt have to be that way!Not user friendly, hard to use.Low level.Very little abstraction.Hard to evolve (in a backward compatible way).····5/26
  6. 6. Why the hell a SQL look-alike query language?!So why not?Very easy to read.Programming Language independent.Ubiquitous, widely known.Copy/paste friendly.Easy to evolve.Does not imply slow.Doesnt force you to work with string.·······6/26
  7. 7. Hence, CQL"Denormalized SQL"Strictly real-time oriented··No joinsNo sub-queriesNo aggregationLimited ORDER BY----7/26
  8. 8. CQL: the C stands for CassandraGoals:Not goals:Provide a user friendly, productive API for C*.Make it easy to do the right thing, hard to do the wrong one.Provide higher level constructs for useful modeling patterns.Be a complete alternative to the Thrift API.····Be SQL.Abstract C* (useful) specificities away (distribution awareness, C* storage engine, ...).Be slow.···8/26
  9. 9. What is CQL?How do you model application with CQL
  10. 10. Cassandra modeling 101Efficient queries in Cassandra boils down to:And denormalization is the technique that allows to achieve this in practice.But this imply the API should:The Thrift API allows that. So does CQL.1. Data Locality at the cluster level: a query should only hit one node.2. Data Locality at the node level: C* storage engine allows data collocation on disk.expose how to collocate data in the same replica set.expose how to collocate data on disk (for a given replica).to query data that is collocated.···10/26
  11. 11. A naive e-mailing applicationWe want to model:UsersEmailsUsers inboxes (all emails received by a user in chronological order)···11/26
  12. 12. Storing user profilesCREATETABLEusers(user_iduuid,nametext,passwordtext,emailtext,picture_profileblob,PRIMARYKEY(user_id))--ThisisreallyanUPSERTINSERTINTOusers(user_id,name,password,email,picture_profile)VALUES(51b-23-ab8,SylvainLebresne,Hd3!ba,lebresne@gmail.com,0xf8ac...);--ThistooisanUPSERTUPDATEusersSETemail=sylvain@datastax.com,password=B9a1^WHEREuser_id=51b-23-ab8;CQLThe first component of the PRIMARY KEY is called the partition key.All the data sharing the same partition key is stored on the same replica set.··12/26
  13. 13. Allowing user defined propertiesSay we want the user to be able to add to this own profile a set of custom properties:user_id email name password picture_profile user_props51b-23-ab8 lebresne@gmail.com Sylvain Lebresne B9a1^ 0xf8ac... { myProperty : Whatever I want }ALTERTABLEusersADDuser_propsmap<text,text>;UPDATEusersSETuser_props[myProperty]=WhateverIwantWHEREuser_id=51b-23-ab8;SELECT*FROMusers;CQL13/26
  14. 14. Storing emailsOnly “indexed” queried are allowed. You cannot do:That is, unless you explicitely index from using:CREATETABLEemails(email_idtimeuuidPRIMARYKEY, --Embedstheemailcreationdatesubjecttext,senderuuid,recipientsset<uuid>,bodytext)--Insertsemails...CQLSELECT*FROMemailsWHEREsender=51b-23-ab8; CQLCREATEINDEXONemails(sender); CQL14/26
  15. 15. InboxesFor each user, its inbox is the list of its emails chronologically sorted.To display the inbox, we need for each email the subject, the sender and recipients names and emails.In a traditional RDBMS, we could join the users and emails table.In Cassandra, we denormalize. That is, we store the pre-computed result of queries we care about (always up todate materialized view).···Good luck to scale that!-·Collocate all the data for an inbox on the same node.Collocate all inbox emails on disk, in the order queried.This is typically the time-series kind of model for which Cassandra shines.---15/26
  16. 16. Storing inboxesCQL distinguishes 2 sub-parts in the PRIMARY KEY:In practice, we are interested by having emails stored in reverse chronological order.CREATETABLEinboxes(user_iduuid,email_idtimeuuid,sender_emailtext,recipients_emailsset<text>,subjecttext,is_readboolean,PRIMARYKEY(user_id, email_id))WITHCLUSTERINGORDERBY(email_idDESC)CQLpartition key: decides the node on which the data is storedclustering columns: within the same partition key, (CQL3) rows are physically ordered following the clusteringcolumns··16/26
  17. 17. Storing inboxes contdIn this example, this allows efficient queries of time range of emails for a given inbox.email_id dateOf(email_id) sender_email recipients_emails subjectd20-32-012 2013-06-24 00:42+0000 Yuki Morishita <yuki@datastax.com> { Sylvain Lebresne } あなたに幸せな誕生日 false17a-bf-65f 2013-03-01 17:03+0000 Aleksey Yeschenko <aleksey@datastax.com> { Sylvain Lebresne } RE: What do you think? truea9c-13-9da 2013-02-10 04:12+0000 Brandon Williams <brandon@datastax.com> { Jonathan Ellis, Sylvain Lebresne } dtests are broken!?@# true241-b4-ca0 2013-01-04 12:45+0000 Jonathan Ellis <jbellis@datastax.com> { Sylvain Lebresne } Whatzz up? true--Getallemailsforuser51b-23-ab8sinceJan01,2013inreversechronologicalorder.SELECTemail_id,dateOf(email_id),sender_email,recipients_emails,subject,is_readFROMinboxesWHEREuser_id=51b-23-ab8ANDemail_id>minTimeuuid(2013-01-0100:00+0000)ORDERBYemail_idDESC;CQL17/26
  18. 18. Handling huge inboxesWhat if inboxes can become too big? The traditional solution consists in sharding inboxes in adapted time shards(say a year), to avoid storing it all on one node.This can be easily done using a composite partition key:CREATETABLEinboxes(user_iduuid,yearint,email_idtimeuuid,sender_emailtext,recipients_namestext,subjecttext,PRIMARYKEY((user_id,year),email_id))WITHCLUSTERINGORDERBY(email_idDESC)CQL18/26
  19. 19. Upgrading from thriftFor more details on the relationship between thrift and CQL:CQL uses the same internal storage engine than ThriftCQL can read your existing Thrift column families (no data migration needed):You can read CQL3 tables from thrift, but this is not easy in practice because some CQL3 metadata are notexposed through thrift for compatibility reasons.CQL is meant to be an alternative to Thrift, not a complement to it.··cqlsh>USE"<keyspace_name>";cqlsh>DESCRIBE"<column_family_name>";cqlsh>SELECT*FROM"<column_family_name>"LIMIT20;CQL··http://www.datastax.com/dev/blog/thrift-to-cql3http://www.datastax.com/dev/blog/does-cql-support-dynamic-columns-wide-rows··19/26
  20. 20. The native protocolTransporting CQL queries
  21. 21. The native protocolA binary transport for CQL3:Want to know more about drivers using this native protocol? Stay in the room for Michaël and Patricks talk.Asynchronous (allows multiple concurrent queries per connection)Server notifications (Only for generic cluster events currently)Made for CQL3···21/26
  22. 22. Whats next?Cassandra 2.0 and beyond
  23. 23. Cassandra 2.0: CQL3Compare-and-swap supportTriggersAllow preparation of TIMESTAMP, TTL and LIMIT.Primary key columns 2ndary indexingALTER ... DROP·UPDATEloginSETpassword=fs3!cWHEREusername=pcmanusIFNOTEXISTS;UPDATEusersSETemail=sylvain@datastax.comWHEREuser_id=51b-23-ab8IFemail=slebresne@apache.org;CQL····23/26
  24. 24. Cassandra 2.0: Native protocolOne-short prepare-and-execute messageBatching of prepared statementSASL authenticationAutomatic query paging····24/26
  25. 25. After C* 2.0Continue to improve the user experience by facilitating good data modeling, while respecting Cassandra inherentspecificities.Storage engine optimizationsCollections 2ndary indexingAggregations within a partitionUser defined struct types...·····25/26
  26. 26. Thank You!(Questions?)