CQL3 and Data Modeling 101 with Apache Cassandra

865 views

Published on

Introduction to CQL3 and Cassandra Data Modeling.

Presented at SD Cassandra Meetup 2014.02.27

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
865
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
22
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

CQL3 and Data Modeling 101 with Apache Cassandra

  1. 1. CQL3 & Data Modeling 101 with Apache Cassandra San Diego Cassandra Meetup Feb 27, 2014
  2. 2. Not how you’ve done it before…
  3. 3. In the beginning… There was the Row, and the Column And the Row was fast to find and scale, And the Column was fast to order.
  4. 4. Cassandra Properties
  5. 5. C* • Column Oriented • Log Structured • Distributed Database
  6. 6. Column Oriented • Columns actually hold the data • Key/Value pair • Name can be used to store meaning as well
  7. 7. Distributed Database • Rows are used to distribute • C* pulls the entire row into memory • Can pull out individual parts or write to individual parts, but it’s still considered together
  8. 8. Log Structured Updates • Commitlog and sstables are log structured • Oriented around appending (streaming at a know location) • ==> Writes quickly • And you want to avoid rewrites
  9. 9. Random Reads • Data is scattered around the store (have to get location and random read to look it up) • Some indexing, and hopefully it’s in the vfs page cache, but still. • ==> Reads “slower”
  10. 10. General Rules of Thumb • De-normalize Everything • Duplicate your data • Organize it for reading
  11. 11. Containers
  12. 12. Keyspace • For modeling, not much • All data lives inside of Keyspaces
  13. 13. Columnfamily • aka Table • Grouping of similar data • Has unique key/row space • Where some structure is applied
  14. 14. Row • Unique inside of a column family • Key/Value where the Value is all of the columns in the row • Can handle some additional meaning to the row name • Typically “bucketing”
  15. 15. Column • Holds data values • Key/Value • Can have meaning in the column name as well
  16. 16. Thrift Interface • Operates on the raw rows and columns • Many different language drivers • Can use cassandra-cli to do this on the command line
  17. 17. Data Patterns
  18. 18. Users CF mac@mac .com NAME TWITTER TAGS mac @macmceniry admin,super,cool jsmith@mac .com NAME ICQ Employer Hobby John 89403270 Smithco Miniature Horses bb@example .com NAME IRC Bobtholomew bb@DAL NAME TWITTER Food TAGS Elizabeth @liz Cheesecake admin NAME IRC Steven steve@DARK liz@example .com steve@my .net
  19. 19. Lookup by chat handle ICQ CF IRC CF EMAIL 89403270 EMAIL bb@DAL bb@example .com jsmith@mac.com EMAIL steve@ DARK steve@my .net
  20. 20. Lookup by chat handle ICQ CF EMAIL IRC CF Name 89403270 EMAIL Name bb@example .com Bobtholomew EMAIL Name steve@my .net Steve bb@DAL jsmith@mac.com John steve@ DARK
  21. 21. HandleCF EMAIL NAME jsmith@mac.com John EMAIL NAME bb@example.com Bobtholomew EMAIL NAME steve@my.net Steven 89403270 bb@DAL steve@DARK
  22. 22. HandleCF TYPE EMAIL NAME ICQ jsmith@mac.com John TYPE EMAIL NAME IRC bb@example.com Bobtholomew TYPE EMAIL NAME IRC steve@my.net Steven 89403270 bb@DAL steve@DARK
  23. 23. How do I create these? [default@userdb] create column family usersCF; 5ecec19a-3a43-3490-8c9a-3eb2901e2e97 Waiting for schema agreement... ... schemas agree across the cluster [default@userdb] create column family handleCF; df82135c-eb1f-3abf-b9df-02c605d571d5 Waiting for schema agreement... ... schemas agree across the cluster
  24. 24. How do I insert data? [default@userdb] set handleCF[utf8(‘bb@DAL’)] … [utf8(‘NAME’)] = utf8('Bobtholomew'); Value inserted. Elapsed time: 22 msec(s). [default@userdb] set handleCF[utf8(‘bb@DAL')] … [utf8(‘EMAIL’)] = utf8(‘bb@example.com’); Value inserted. Elapsed time: 3.43 msec(s).
  25. 25. Users CF - TAGS mac@mac .com NAME TWITTER TAGS mac @macmceniry admin,super,cool jsmith@mac .com NAME ICQ Employer Hobby John 89403270 Smithco Miniature Horses bb@example .com NAME IRC Bobtholomew bb@DAL NAME TWITTER Food TAGS Elizabeth @liz Cheesecake admin NAME IRC Steven steve@DARK liz@example .com steve@my .net
  26. 26. Users CF - TAGS mac@mac .com jsmith@mac .com bb@example .com liz@example .com steve@my .net NAME TWITTER TAGS:admin TAGS:cool mac @macmceniry NAME ICQ Employer Hobby John 89403270 Smithco Miniature Horses NAME IRC Bobtholomew bb@DAL NAME TWITTER Food TAGS:admin Elizabeth @liz Cheesecake NAME IRC Steven steve@DARK TAGS:super
  27. 27. “Types” [default@userdb] set handleCF[utf8(‘bb@DAL’)] ! • Key Validator • (Column) Comparator • (Column Value) Default Validator, Metadata • BytesType, AsciiType, UTF8Type, IntegerType, Int32Type, LongType, UUIDType, TimeUUIDType, DateType, BooleanType, FloatType, DoubleType, DecimalType, CounterColumnType (, CompositeTypes)
  28. 28. What’s in a name? • Can use row names and column names to add meaning • Row name meaning creates a new distribution bin • Column name meaning can create a data hierarchy • No real change to the column family creation in the thrift interface (well, types depending on what you’re doing)
  29. 29. EventCF mac@mac .com: 20140203 08:10:15 08:15:15 09:10:15 join update logout mac@mac .com: 20140204 08:11:23 08:14:57 18:45:12 18:50:52 19:01:29 login logout login logout logout mac@mac .com: 20140205 09:23:23 09:57:44 login logout liz@example .com: 20140203 11:22:33 11:44:55 22:10:05 22:52:02 login logout login logout liz@example .com: 20140205 08:11:23 08:14:57 login logout
  30. 30. That was then, this is…
  31. 31. Now • Same underlying structure none of that has changed • • • Rows - reference quickly use for searching Columns - scan quickly user for ordering But now have usage patterns • Some have been codified into CQL
  32. 32. CQL • Thrift alternative • Simpler API • Hides the structure of the internal storage • 3 generations • Only looking at CQL3 here • cqlsh [-3]
  33. 33. How does handle look here? cqlsh:userdb> CREATE TABLE handles ( … handlename VARCHAR, … email VARCHAR, … name VARCHAR, … PRIMARY KEY (handlename) … ); cqlsh:userdb> INSERT INTO handles … (handlename, email, name) VALUES … (‘bb@DAL’, ‘bb@example.com’, ‘Bobtholomew’);
  34. 34. handles Table email name bb@example.com Bobtholomew bb@DAL
  35. 35. How does handle look here? cqlsh:userdb> SELECT * FROM handles; handlename | email | name ————————————|————————————————|————————————— bb@DAL | bb@example.com | Bobtholomew cqlsh:userdb> SELECT * FROM handles WHERE … handlename = ‘bb@DAL’; handlename | email | name ————————————|————————————————|————————————— bb@DAL | bb@example.com | Bobtholomew
  36. 36. How do I change it? cqlsh:userdb> UPDATE handles SET email=‘none’ … WHERE handlename = ‘bb@DAL’; cqlsh:userdb> SELECT * FROM handles; handlename | email | name ————————————|————————————————|————————————— bb@DAL | none | Bobtholomew
  37. 37. upsert • Update instead of Insert • • Does the same thing (as long as it’s not a key) Insert instead of Update • Overwrites data if it’s already there
  38. 38. What about our event buckets from earlier? • Can do the same thing • Creating a composite key • • USERNAME:DATE Creating a composite column • hh:mm:ss
  39. 39. EventCF mac@mac .com: 20140203 08:10:15 08:15:15 09:10:15 join update logout mac@mac .com: 20140204 08:11:23 08:14:57 18:45:12 18:50:52 19:01:29 login logout login logout logout mac@mac .com: 20140205 09:23:23 09:57:44 login logout liz@example .com: 20140203 11:22:33 11:44:55 22:10:05 22:52:02 login logout login logout liz@example .com: 20140205 08:11:23 08:14:57 login logout
  40. 40. cqlsh:userdb> CREATE TABLE events ( … username VARCHAR, … d VARCHAR, … hr INT, … min INT, … sec INT, … event VARCHAR, … PRIMARY KEY ( (username,d), hr, min, sec ) );
  41. 41. … PRIMARY KEY ( (username,d), hr, min, sec ) ); ROW NAME (C* 1.2) COLUMN NAME
  42. 42. Tags • CQL has collections • map, list, set • Collections are build similar to small/special composite columns • Can add to our existing handle table
  43. 43. cqlsh:userdb> ALTER TABLE handles ADD tags SET; cqlsh:userdb> UPDATE TABLE handles … SET tags = (‘admin’, ‘foo’); email name bb@example.com Bobtholomew bb@DAL tags:admin tags:foo
  44. 44. Design the data model so that it’s idempotent (eBay) • Counter versus Collection (what question is being asked?) Count 100 Count 200 Count 300 Likes A Likes B Likes C Likes A Likes B Likes C +user11 1393287359 +user12 1393287359 +user11 1393287359 +user12 1393280912 -user11 1393281942 +user13 1393212345 1393287100 +user12 1393287100 1393287100 +user13 1393287100 1393287100 +user14 1393287100
  45. 45. Go Forth and Model! Thank You! PS… Sony Network is hiring!

×