Intro to Cassandra and CassandraObject

Cassandra & CassandraObject Michael Koziarski michael@koziarski.com

Intro to Me

I Don’t Have To Scale

Intro to Cassandra

Distributed

Fault Tolerant

Elastic

The Ring A F B E C D

The Ring someKey A F B E C D RandomPartitioner OrderPreservingPartitioner MD5(key) key

Replication Factor someKey A F B E C D RF = 2

Replication Factor someKey A F B E C D RF = 3

Consistency Level GET 'someKey' A F B E C D ConsistencyLevel.ONE

Consistency Level GET 'someKey' A F B E C D ConsistencyLevel.QUORUM

Consistency Level GET 'someKey' A F B E C D ConsistencyLevel.ALL

Fault Tolerance GET 'someKey' A F DEAD E C D ConsistencyLevel.QUORUM

Fault Tolerance GET 'someKey' A F E C D

Elastic NEW A F B E C D

Data Model

Key Value Store Some someKey Value

Column Store

Column

Column ﬁrstName Michael

Row (Column Family) ﬁrstName Michael someKey lastName Koziarski

Row (Super Column Family) ﬁrstName Michael someSubColumn lastName Koziarski someKey ﬁrstName Kate otherSubColumn lastName Koziarski

Column { 'name': 'first_name', 'value': 'Michael', 'timestamp': 1276040575 }

Column { 'first_name': "Michael" }

Column Family Users["koz"] = { 'first_name': 'Michael', 'last_name': 'Koziarski' }

Super Column Family UserAddresses["koz"] = { "home" : { "suburb": "Berhampore" "city": "Wellington", "country": "New Zealand" }, "office" : { "suburb": "CBD" "city": "Wellington", "country": "New Zealand" } }

Don’t let the name fool you

One to Many Timeline["nzkoz"] = { uuid_one: "http://twitter.com/chadfowler/status/15740739666", uuid_two: "http://twitter.com/dhh/status/15740689762", uuid_three: "http://twitter.com/glv/status/15740546908" } <ColumnFamily CompareWith="TimeUUIDType" Name="Timeline"/>

Modeling

Schema Driven Modeling

Start with your Data

Model User ﬁrstName lastName dateOfBirth

Figure out the Queries

Query SELECT firstName, lastName FROM `users` WHERE dateOfBirth < '1992-‐06-‐09'

Query SELECT dateOfBirth FROM `users` WHERE firstName = 'Michael' AND lastName = 'Koziarski'

Query SELECT firstName, lastName FROM `users` WHERE YEAR(dateOfBirth) = 1980

Query SELECT COUNT(DISTINCT firstName) FROM `users` WHERE dateOfBirth < '1992-‐06-‐09'

Cassandra Limitations

No WHERE

No WHERE Kinda

No ORDER

No ORDER Kinda

No COUNT

No COUNT Kinda

No SUM

Query Driven Modeling

Start with the Queries

Populate a data model which enables them

Modeling Example

Users Users["koz"] = { 'first_name': "Michael", 'last_name': "Koziarski" }

Users Users["koz"] = { 'first_name': "Michael", 'last_name': "Koziarski" } connection.get(:Users, "koz")

Koziarski Family UsersByLastName["koziarski"] = { uuid_one: "koz" uuid_two: "kate" }

Koziarski Family UsersByLastName["koziarski"] = { uuid_one: "koz" uuid_two: "kate" } connection.get(:UsersByLastName, "koziarski").values.map do |key| connection.get(:Users, key) end

Share my Birthday UsersByDOB["1980-‐08-‐15"] = { uuid_one: "koz" }

Share my Birthday UsersByDOB["1980-‐08-‐15"] = { uuid_one: "koz" } connection.get(:UsersByDOB, "1980-‐08-‐15")

Users Born in 1980 UsersByDOB["1980-‐08-‐15"] = { uuid_one: "koz" }

Users Born in 1980 UsersByDOB["1980-‐08-‐15"] = { uuid_one: "koz" } connection.get_range(:UsersByDOB, :start=>"1980", :finish=>"1981")

Users Born in 1980 UsersByDOB["1980-‐08-‐15"] = { uuid_one: "koz" } connection.get_range(:UsersByDOB, :start=>"1980", :finish=>"1981") Only with OrderPreservingPartitioner

A Column Family per Query

Do you really need Cassandra?

CassandraObject

class Customer < CassandraObject::Base attribute :first_name, :type => :string attribute :last_name, :type => :string attribute :date_of_birth, :type => :date attribute :signed_up_at, :type => :time_with_zone validate :should_be_cool key :uuid index :date_of_birth association :invoices, :unique=>false, :inverse_of=>:customer private def should_be_cool unless ["Michael", "Anika", "Evan", "James"].include?(first_name) errors.add(:first_name, "must be that of a cool person") end end end

Motivations

Prove ActiveModel

Learn Cassandra

Have a fun Side Project

Solve my Scaling Problems

Mostly AR Compatible

Not Compatible def index @people = @customer.people.order(params[:order]). limit(params[:limit]). where(...) end

Compatible def create @person = Customer.new params[:customer] if @person.save redirect_to @person else render :action=>'new' end end

Compatible <%= form_for(@customer) do |customer| %> <%= customer.error_messages %> <%= customer.text_field :first_name %> <%= customer.submit "Save" %> <% end %>

Walkthrough

class Customer < CassandraObject::Base attribute :first_name, :type => :string attribute :last_name, :type => :string attribute :date_of_birth, :type => :date attribute :signed_up_at, :type => :time_with_zone validate :should_be_cool key :uuid index :date_of_birth association :invoices, :unique=>false, :inverse_of=>:customer private def should_be_cool unless ["Michael", "Anika", "Evan", "James"].include?(first_name) errors.add(:first_name, "must be that of a cool person") end end end

class Invoice < CassandraObject::Base attribute :number, :type=>:integer attribute :total, :type=>:float attribute :gst_number, :type=>:string # indexes can have a single entry also. index :number, :unique=>true # bi-‐directional associations with read-‐repair support. association :customer, :unique=>true, :inverse_of=>:invoices # Read migration support migrate 1 do |attrs| attrs["total"] ||= rand(2000) / 100.0 end migrate 2 do |attrs| attrs["gst_number"] = "66-‐666-‐666" end key :natural, :attributes => :number end

Attributes attribute :first_name, :type => :string attribute :last_name, :type => :string attribute :date_of_birth, :type => :date attribute :signed_up_at, :type => :time_with_zone attribute :number, :type => :integer attribute :total, :type => :float attribute :gst_number, :type => :string

Attributes attribute :first_name, :type => :string @customer.first_name = "Michael" @customer.attributes= {:first_name=>"Michael"}

Validations validate :should_be_cool def should_be_cool unless ["Michael", "Anika", "Evan", "James"].include?(first_name) errors.add(:first_name, "must be that of a cool person") end end

Validations validate :should_be_cool def should_be_cool unless ["Michael", "Anika", "Evan", "James"].include?(first_name) errors.add(:first_name, "must be that of a cool person") end end @customer.first_name = "Marcel" @customer.valid? # => false

Validations validates_confirmation_of :tos validates_format_of :gst_number, :with=> /.../ validates_length_of :first_name, :max=>123

Key Selection Matters

Key Selection Matters UsersByDOB["1980-‐08-‐15"] = { uuid_one: "koz" } connection.get_range(:UsersByDOB, :start=>"1980", :finish=>"1981")

Keys key :uuid

Keys key :uuid "bf1ba5da-‐735a-‐11df-‐8b47-‐377649cf993b"

Keys key :natural, :attributes => :number

Custom Key Factories key RedisKeyFactory.new(REDIS_CONNECTION, "customer_key")

Custom Key Factories class RedisKeyFactory def initialize(connection, key) @connection, @key = connection, key end def next_key(object) @connection.incr(@key) end # Parse should create a new key object from the 'to_param' format def parse(string) string.to_i end # create should create a new key object from the cassandra format. def create(string) string.to_i end end

Migrations

Migrations class AddLicenseNameToArticle < ActiveRecord::Migration def self.up add_column :articles, :license_name, :string, :default=>"Exclusive" execute "UPDATE articles SET license_name = 'Exclusive' WHERE price_first IS NOT NULL" execute "UPDATE articles SET license_name = 'Syndicated' WHERE price_first IS NULL" end def self.down remove_column :articles, :license_name end end

Migrations { 'price_first': 45, 'schema_version': 0 }

Migrations class Article < CassandraObject::Base attribute :price_first, :type=>:float attribute :license_name, :type=>:string migrate 1 do |attrs| if attrs[:price_first] attrs[:license_name] = "Exclusive" else attrs[:license_name] = "Syndicated" end end end

Migrations class Article < CassandraObject::Base attribute :price_first, :type=>:float attribute :license_name, :type=>:string migrate 1 do |attrs| if attrs[:price_first] attrs[:license_name] = "Exclusive" else attrs[:license_name] = "Syndicated" end end end @article = Article.get("some-‐old-‐story") @article.license_name # => "Exclusive"

Migrations { 'price_first': 45, 'schema_version': 1, 'license_name': 'Exclusive' }

Indexes

Indexes class Article < CassandraObject::Base attribute :slug, :type=>:string key :uuid index :slug, :unique=>true end connection.insert(:ArticlesBySlug, @article.slug, {UUID.new => @article.key})

Indexes class Article < CassandraObject::Base attribute :slug, :type=>:string key :uuid index :slug, :unique=>true end connection.insert(:ArticlesBySlug, @article.slug, {UUID.new => @article.key}) @article = Article.find_by_slug("some-‐slug")

Indexes class Article < CassandraObject::Base attribute :publication_date, :type=>:date key :uuid index :publication_date, :unique=>false end connection.insert(:ArticlesByPublicationDate, @article.publication_date, {UUID.new => @article.key})

Indexes class Article < CassandraObject::Base attribute :publication_date, :type=>:date key :uuid index :publication_date, :unique=>false end connection.insert(:ArticlesByPublicationDate, @article.publication_date, {UUID.new => @article.key}) @article = Article.find_all_by_publication_date(Date.today -‐ 1)

Read-Repair results = [] connection.get(:ArticlesByPublicationDate, date).each do |(uuid, key)| article = Article.get(uuid) if article.publication_date != date connection.delete(:ArticlesByPublicationDate, date, uuid) else results << article end end results

Associations

Associations class Invoice < CassandraObject::Base association :customer, :unique=>true, :inverse_of=>:invoices end class Customer < CassandraObject::Base association :invoices, :unique=>false, :inverse_of=>:customer end @customer.invoices.create! params[:invoice] @invoice.customer

Project Status

Very Beta

In Flux

Taking Patches

Thanks! Michael Koziarski michael@koziarski.com http://github.com/NZKoz/cassandra_object http://cassandra.apache.org/

http://www.slideshare.net	70
http://en.oreilly.com	69

Intro to Cassandra and CassandraObject

by nzkoz

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 139

Statistics

Intro to Cassandra and CassandraObject Presentation Transcript