Intro to Cassandra and CassandraObject

5,766 views
5,325 views

Published on

Slides from my cassandra talk at Railsconf 2010, not sure how useful they'll be without context, but people were asking.

Published in: Technology, Business
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,766
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
73
Comments
0
Likes
7
Embeds 0
No embeds

No notes for slide

Intro to Cassandra and CassandraObject

  1. Cassandra & CassandraObject Michael Koziarski michael@koziarski.com
  2. Intro to Me
  3. I Don’t Have To Scale
  4. Intro to Cassandra
  5. Distributed
  6. Fault Tolerant
  7. Elastic
  8. The Ring A F B E C D
  9. The Ring A F B E C D
  10. The Ring A F B E C D
  11. The Ring A F B E C D
  12. The Ring A F B E C D
  13. The Ring A F B E C D
  14. The Ring A F B E C D
  15. The Ring someKey A F B E C D RandomPartitioner OrderPreservingPartitioner MD5(key) key
  16. Replication Factor someKey A F B E C D RF = 2
  17. Replication Factor someKey A F B E C D RF = 3
  18. Consistency Level GET 'someKey' A F B E C D ConsistencyLevel.ONE
  19. Consistency Level GET 'someKey' A F B E C D ConsistencyLevel.ONE
  20. Consistency Level GET 'someKey' A F B E C D ConsistencyLevel.ONE
  21. Consistency Level GET 'someKey' A F B E C D ConsistencyLevel.ONE
  22. Consistency Level GET 'someKey' A F B E C D ConsistencyLevel.QUORUM
  23. Consistency Level GET 'someKey' A F B E C D ConsistencyLevel.QUORUM
  24. Consistency Level GET 'someKey' A F B E C D ConsistencyLevel.QUORUM
  25. Consistency Level GET 'someKey' A F B E C D ConsistencyLevel.ALL
  26. Consistency Level GET 'someKey' A F B E C D ConsistencyLevel.ALL
  27. Consistency Level GET 'someKey' A F B E C D ConsistencyLevel.ALL
  28. Consistency Level GET 'someKey' A F B E C D ConsistencyLevel.ALL
  29. Fault Tolerance GET 'someKey' A F DEAD E C D ConsistencyLevel.QUORUM
  30. Fault Tolerance GET 'someKey' A F DEAD E C D ConsistencyLevel.QUORUM
  31. Fault Tolerance GET 'someKey' A F E C D
  32. Elastic NEW A F B E C D
  33. Elastic NEW A F B E C D
  34. Data Model
  35. Key Value Store Some someKey Value
  36. Column Store
  37. Column
  38. Column firstName Michael
  39. Row (Column Family) firstName Michael someKey lastName Koziarski
  40. Row (Super Column Family) firstName Michael someSubColumn lastName Koziarski someKey firstName Kate otherSubColumn lastName Koziarski
  41. JSON
  42. Column {    'name':  'first_name',    'value':  'Michael',    'timestamp':  1276040575 }
  43. Column {    'first_name':  "Michael" }
  44. Column Family Users["koz"]  =  {    'first_name':  'Michael',    'last_name':  'Koziarski' }
  45. Column Family Users["koz"]  =  {    'first_name':  'Michael',    'last_name':  'Koziarski' }
  46. Column Family Users["koz"]  =  {    'first_name':  'Michael',    'last_name':  'Koziarski' }
  47. Column Family Users["koz"]  =  {    'first_name':  'Michael',    'last_name':  'Koziarski' }
  48. Super Column Family UserAddresses["koz"]  =  {    "home"  :  {        "suburb":  "Berhampore"        "city":  "Wellington",        "country":  "New  Zealand"    },    "office"  :  {        "suburb":  "CBD"        "city":  "Wellington",        "country":  "New  Zealand"    } }
  49. Don’t let the name fool you
  50. One to Many Timeline["nzkoz"]  =  {    uuid_one:      "http://twitter.com/chadfowler/status/15740739666",      uuid_two:      "http://twitter.com/dhh/status/15740689762",    uuid_three:  "http://twitter.com/glv/status/15740546908" } <ColumnFamily  CompareWith="TimeUUIDType"  Name="Timeline"/>
  51. Modeling
  52. Schema Driven Modeling
  53. Start with your Data
  54. Model User firstName lastName dateOfBirth
  55. Figure out the Queries
  56. Query SELECT  firstName,  lastName   FROM      `users` WHERE  dateOfBirth  <  '1992-­‐06-­‐09'
  57. Query SELECT  dateOfBirth FROM      `users` WHERE  firstName  =  'Michael'                  AND            lastName  =  'Koziarski'
  58. Query SELECT  firstName,  lastName   FROM      `users` WHERE  YEAR(dateOfBirth)  =  1980
  59. Query SELECT  COUNT(DISTINCT  firstName) FROM  `users` WHERE  dateOfBirth  <  '1992-­‐06-­‐09'
  60. Cassandra Limitations
  61. No WHERE
  62. No WHERE Kinda
  63. No ORDER
  64. No ORDER Kinda
  65. No COUNT
  66. No COUNT Kinda
  67. No SUM
  68. Query Driven Modeling
  69. Start with the Queries
  70. Populate a data model which enables them
  71. Modeling Example
  72. Users Users["koz"]  =  {    'first_name':  "Michael",    'last_name':  "Koziarski" }
  73. Users Users["koz"]  =  {    'first_name':  "Michael",    'last_name':  "Koziarski" } connection.get(:Users,  "koz")
  74. Koziarski Family UsersByLastName["koziarski"]  =  {    uuid_one:  "koz"    uuid_two:  "kate" }
  75. Koziarski Family UsersByLastName["koziarski"]  =  {    uuid_one:  "koz"    uuid_two:  "kate" } connection.get(:UsersByLastName,  "koziarski").values.map  do  |key|    connection.get(:Users,  key) end
  76. Share my Birthday UsersByDOB["1980-­‐08-­‐15"]  =  {    uuid_one:  "koz" }
  77. Share my Birthday UsersByDOB["1980-­‐08-­‐15"]  =  {    uuid_one:  "koz" } connection.get(:UsersByDOB,  "1980-­‐08-­‐15")
  78. Users Born in 1980 UsersByDOB["1980-­‐08-­‐15"]  =  {    uuid_one:  "koz" }
  79. Users Born in 1980 UsersByDOB["1980-­‐08-­‐15"]  =  {    uuid_one:  "koz" } connection.get_range(:UsersByDOB,  :start=>"1980",  :finish=>"1981")
  80. Users Born in 1980 UsersByDOB["1980-­‐08-­‐15"]  =  {    uuid_one:  "koz" } connection.get_range(:UsersByDOB,  :start=>"1980",  :finish=>"1981") Only with OrderPreservingPartitioner
  81. A Column Family per Query
  82. Do you really need Cassandra?
  83. CassandraObject
  84. class  Customer  <  CassandraObject::Base    attribute  :first_name,        :type  =>  :string    attribute  :last_name,          :type  =>  :string    attribute  :date_of_birth,  :type  =>  :date    attribute  :signed_up_at,    :type  =>  :time_with_zone    validate  :should_be_cool    key  :uuid    index  :date_of_birth    association  :invoices,  :unique=>false,  :inverse_of=>:customer    private    def  should_be_cool        unless  ["Michael",  "Anika",  "Evan",  "James"].include?(first_name)            errors.add(:first_name,  "must  be  that  of  a  cool  person")        end    end end
  85. Motivations
  86. Prove ActiveModel
  87. Learn Cassandra
  88. Have a fun Side Project
  89. Solve my Scaling Problems
  90. Solve my Scaling Problems
  91. Mostly AR Compatible
  92. Not Compatible def  index    @people  =  @customer.people.order(params[:order]).                                                          limit(params[:limit]).                                                          where(...) end
  93. Compatible def  create    @person  =  Customer.new  params[:customer]    if  @person.save        redirect_to  @person    else        render  :action=>'new'    end end
  94. Compatible <%=  form_for(@customer)  do  |customer|  %>    <%=  customer.error_messages  %>    <%=  customer.text_field  :first_name  %>    <%=  customer.submit  "Save"  %> <%  end  %>
  95. Walkthrough
  96. class  Customer  <  CassandraObject::Base    attribute  :first_name,        :type  =>  :string    attribute  :last_name,          :type  =>  :string    attribute  :date_of_birth,  :type  =>  :date    attribute  :signed_up_at,    :type  =>  :time_with_zone    validate  :should_be_cool    key  :uuid    index  :date_of_birth    association  :invoices,  :unique=>false,  :inverse_of=>:customer    private    def  should_be_cool        unless  ["Michael",  "Anika",  "Evan",  "James"].include?(first_name)            errors.add(:first_name,  "must  be  that  of  a  cool  person")        end    end end
  97. class  Invoice  <  CassandraObject::Base    attribute  :number,  :type=>:integer    attribute  :total,  :type=>:float    attribute  :gst_number,  :type=>:string    #  indexes  can  have  a  single  entry  also.    index  :number,  :unique=>true    #  bi-­‐directional  associations  with  read-­‐repair  support.    association  :customer,  :unique=>true,  :inverse_of=>:invoices    #  Read  migration  support    migrate  1  do  |attrs|        attrs["total"]  ||=  rand(2000)  /  100.0    end    migrate  2  do  |attrs|        attrs["gst_number"]  =  "66-­‐666-­‐666"    end    key  :natural,  :attributes  =>  :number end
  98. Attributes attribute  :first_name,        :type  =>  :string attribute  :last_name,          :type  =>  :string attribute  :date_of_birth,  :type  =>  :date attribute  :signed_up_at,    :type  =>  :time_with_zone attribute  :number,                :type  =>  :integer attribute  :total,                  :type  =>  :float attribute  :gst_number,        :type  =>  :string
  99. Attributes attribute  :first_name,  :type  =>  :string @customer.first_name  =  "Michael" @customer.attributes=  {:first_name=>"Michael"}
  100. Validations validate  :should_be_cool def  should_be_cool    unless  ["Michael",  "Anika",  "Evan",  "James"].include?(first_name)        errors.add(:first_name,  "must  be  that  of  a  cool  person")    end end
  101. Validations validate  :should_be_cool def  should_be_cool    unless  ["Michael",  "Anika",  "Evan",  "James"].include?(first_name)        errors.add(:first_name,  "must  be  that  of  a  cool  person")    end end @customer.first_name  =  "Marcel" @customer.valid?  #  =>  false
  102. Validations validates_confirmation_of  :tos validates_format_of  :gst_number,  :with=>  /.../ validates_length_of  :first_name,  :max=>123
  103. Keys
  104. Key Selection Matters
  105. Key Selection Matters UsersByDOB["1980-­‐08-­‐15"]  =  {    uuid_one:  "koz" } connection.get_range(:UsersByDOB,  :start=>"1980",  :finish=>"1981")
  106. Keys key  :uuid
  107. Keys key  :uuid "bf1ba5da-­‐735a-­‐11df-­‐8b47-­‐377649cf993b"
  108. Keys key  :natural,  :attributes  =>  :number
  109. Custom Key Factories key  RedisKeyFactory.new(REDIS_CONNECTION,  "customer_key")
  110. Custom Key Factories class  RedisKeyFactory    def  initialize(connection,  key)        @connection,  @key  =  connection,  key    end        def  next_key(object)        @connection.incr(@key)    end        #  Parse  should  create  a  new  key  object  from  the  'to_param'  format    def  parse(string)        string.to_i    end        #  create  should  create  a  new  key  object  from  the  cassandra  format.    def  create(string)        string.to_i    end     end
  111. Migrations
  112. Migrations class  AddLicenseNameToArticle  <  ActiveRecord::Migration    def  self.up        add_column  :articles,  :license_name,  :string,  :default=>"Exclusive"        execute  "UPDATE  articles  SET  license_name  =  'Exclusive'                                WHERE  price_first  IS  NOT  NULL"        execute  "UPDATE  articles  SET  license_name  =  'Syndicated'                            WHERE  price_first  IS  NULL"    end    def  self.down        remove_column  :articles,  :license_name    end end
  113. Migrations {    'price_first':  45,    'schema_version':  0 }
  114. Migrations class  Article  <  CassandraObject::Base    attribute  :price_first,  :type=>:float    attribute  :license_name,  :type=>:string    migrate  1  do  |attrs|        if  attrs[:price_first]            attrs[:license_name]  =  "Exclusive"        else            attrs[:license_name]  =  "Syndicated"        end    end end
  115. Migrations class  Article  <  CassandraObject::Base    attribute  :price_first,  :type=>:float    attribute  :license_name,  :type=>:string    migrate  1  do  |attrs|        if  attrs[:price_first]            attrs[:license_name]  =  "Exclusive"        else            attrs[:license_name]  =  "Syndicated"        end    end end @article  =  Article.get("some-­‐old-­‐story") @article.license_name  #  =>  "Exclusive"
  116. Migrations {    'price_first':  45,    'schema_version':  1,    'license_name':  'Exclusive' }
  117. Indexes
  118. Indexes class  Article  <  CassandraObject::Base    attribute  :slug,  :type=>:string    key  :uuid    index  :slug,  :unique=>true end connection.insert(:ArticlesBySlug,  @article.slug,                                      {UUID.new  =>  @article.key})
  119. Indexes class  Article  <  CassandraObject::Base    attribute  :slug,  :type=>:string    key  :uuid    index  :slug,  :unique=>true end connection.insert(:ArticlesBySlug,  @article.slug,                                      {UUID.new  =>  @article.key}) @article  =  Article.find_by_slug("some-­‐slug")
  120. Indexes class  Article  <  CassandraObject::Base    attribute  :publication_date,  :type=>:date    key  :uuid    index  :publication_date,  :unique=>false end connection.insert(:ArticlesByPublicationDate,                                      @article.publication_date,      {UUID.new  =>  @article.key})
  121. Indexes class  Article  <  CassandraObject::Base    attribute  :publication_date,  :type=>:date    key  :uuid    index  :publication_date,  :unique=>false end connection.insert(:ArticlesByPublicationDate,                                      @article.publication_date,      {UUID.new  =>  @article.key}) @article  =  Article.find_all_by_publication_date(Date.today  -­‐  1)
  122. Indexes class  Article  <  CassandraObject::Base    attribute  :publication_date,  :type=>:date    key  :uuid    index  :publication_date,  :unique=>false end connection.insert(:ArticlesByPublicationDate,                                      @article.publication_date,      {UUID.new  =>  @article.key}) @article  =  Article.find_all_by_publication_date(Date.today  -­‐  1)
  123. Read-Repair results  =  [] connection.get(:ArticlesByPublicationDate,  date).each  do  |(uuid,  key)|    article  =  Article.get(uuid)    if  article.publication_date  !=  date        connection.delete(:ArticlesByPublicationDate,  date,  uuid)    else        results  <<  article    end end results
  124. Associations
  125. Associations class  Invoice  <  CassandraObject::Base    association  :customer,  :unique=>true,  :inverse_of=>:invoices end class  Customer  <  CassandraObject::Base    association  :invoices,  :unique=>false,  :inverse_of=>:customer end @customer.invoices.create!  params[:invoice] @invoice.customer
  126. Project Status
  127. Very Beta
  128. In Flux
  129. Taking Patches
  130. Thanks! Michael Koziarski michael@koziarski.com http://github.com/NZKoz/cassandra_object http://cassandra.apache.org/

×