• Save
Couch Foo: CouchDB on rails
Upcoming SlideShare
Loading in...5
×
 

Couch Foo: CouchDB on rails

on

  • 5,440 views

 

Statistics

Views

Total Views
5,440
Views on SlideShare
5,423
Embed Views
17

Actions

Likes
4
Downloads
0
Comments
0

2 Embeds 17

http://www.slideshare.net 16
http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Couch Foo: CouchDB on rails Couch Foo: CouchDB on rails Presentation Transcript

  • CouchDB and Ruby George Palmer
  • “Spending more time on the couch: Using CouchDB to avoid migrations, create offline applications and scale with greater ease”
  • What is CouchDB?
  • Database Landscape
  • Database Landscape MySQL
  • Database Landscape MySQL Postgres
  • Database Landscape MySQL Postgres SQLite
  • Database Landscape MySQL Postgres SQLite
  • Database Landscape MySQL Postgres SQLite
  • Database Landscape Relational Database MySQL Postgres SQLite
  • Database Landscape Relational Database MySQL Postgres ToykoCabinet SQLite
  • Database Landscape Relational Database MySQL Postgres ToykoCabinet SQLite Redis
  • Database Landscape Relational Database MySQL Postgres ToykoCabinet SQLite Redis Memcachedb
  • Database Landscape Relational Database MySQL Postgres ToykoCabinet SQLite Redis Memcachedb
  • Database Landscape Relational Database MySQL Postgres ToykoCabinet SQLite Redis Memcachedb
  • Database Landscape Relational Database Key-Value Databases MySQL Postgres ToykoCabinet SQLite Redis Memcachedb
  • Database Landscape Amazon SimpleDB Relational Database Key-Value Databases MySQL Postgres ToykoCabinet SQLite Redis Memcachedb
  • Database Landscape Amazon SimpleDB MongoDB Relational Database Key-Value Databases MySQL Postgres ToykoCabinet SQLite Redis Memcachedb
  • Database Landscape Amazon SimpleDB MongoDB CouchDB Relational Database Key-Value Databases MySQL Postgres ToykoCabinet SQLite Redis Memcachedb
  • Database Landscape Amazon SimpleDB MongoDB CouchDB Relational Database Key-Value Databases MySQL Postgres ToykoCabinet SQLite Redis Memcachedb
  • Database Landscape Amazon SimpleDB MongoDB CouchDB Relational Database Key-Value Databases MySQL Postgres ToykoCabinet SQLite Redis Memcachedb
  • Database Landscape Amazon SimpleDB MongoDB CouchDB Document Orientated Databases Relational Database Key-Value Databases MySQL Postgres ToykoCabinet SQLite Redis Memcachedb
  • Document Orientated Databases • Data is stored in documents • Like real life there’s no limit on how the information is stored • ie schema free • Documents stored and accessed by an identifier
  • Stored as JSON { “_id”: “A312C72B”, “_rev”: “AB746C”, “name”: “Rails Underground”, “start-date”: “July 24th”, “tags”: [“ruby”, “rails”, “conference”] }
  • REST interface • Create HTTP POST or PUT /db/docid • Read HTTP GET /db/docid • Destroy HTTP DELETE /db/docid • Update HTTP PUT /db/docid
  • HTTP REST Benefits • Load Balancing • Caching • ...
  • Introducing Views { { “type”: “car” “type”: “van” “colour”... “colour”... } } { { “type”: “lorry” “type”: “van” “colour”... “colour”... } } CouchDB
  • Introducing Views { { “type”: “car” “type”: “van” “colour”... “colour”... } } { { “type”: “lorry” “type”: “van” “colour”... “colour”... } } CouchDB
  • Introducing Views { { We need to look at a “type”: “car” “colour”... “type”: “van” “colour”... subset of the } } database documents - { { or a ‘view’ of the “type”: “lorry” “colour”... “type”: “van” “colour”... database } } CouchDB
  • Views • These views are saved in the database itself under a special identifier • they must start: _design/
  • Simple View function(doc) { if (doc.Type == "van") { emit(null, doc); } }
  • Executing the query http://localhost:5984/rails-underground/_view/vans/all {"total_rows":2, "offset":0, "rows":[ {"id":"1", "key":null, "value": {"_id":"1","_rev":"4062949995","type":"van","name":"Tran sit"}}, {"id":"3", "key":null, "value": {"_id":"3","_rev":"1728947327","type":"van","name":"Tran sporter"}} ] }
  • Executing the query http://localhost:5984/rails-underground/_view/vans/all? limit=1 {"total_rows":1, "offset":0, "rows":[ {"id":"1", "key":null, "value": {"_id":"1","_rev":"4062949995","type":"van","name":"Tran sit"}} ] }
  • Simple View function(doc) { if (doc.Type == "van") { emit(doc.Name, doc); } }
  • Executing the query http://localhost:5984/rails-underground/_view/vans/all? startkey=Trans {"total_rows":1, "offset":0, "rows":[ {"id":"3", "key":”Transporter”, "value": {"_id":"3","_rev":"1728947327","type":"van","name":"Tran sporter"}} ] }
  • Reduce Query • map: function(doc) {if (doc.Type == "van") {emit(null, doc);}} // previous example • reduce: function(keys, values) { return keys.length } • This will return a result of 2 • You can optionally rereduce the result set by having a third argument on the reduce function
  • Inline Associatons { “user” : “George”, “roles” : [“supervisor”, “teamleader”], ... }
  • Associations in 1 query function(doc) { if (doc.type == "post") { map([doc._id, 0], doc); } else if (doc.type == "comment") { map([doc.post, 1], doc); } }
  • Associations in 1 query http://localhost:5984/rails-underground/_view/ posts_comments/all “key”:[“1”,0], “value”:{“_id”:”1”, “type”:”post”, “text”:”My Blog Post”} “key”:[“2”,0], “value”:{“_id”:”2”, “type”:”post”, “text”:”My 2nd Blog Post”} “key”:[“3”,0], “value”:{“_id”:”3”, “type”:”post”, “text”:”My 3rd Blog Post”} “key”:[“1”,1], “value”:{“_id”:”3”, “type”:”comment”, “text”:”You rock dude”, “post”:”1”} “key”:[“2”,1], “value”:{“_id”:”3”, “type”:”comment”, “text”:”Man you suck”, “post”:”2”}
  • Associations in 1 query http://localhost:5984/rails-underground/_view/ posts_comments/all?startkey=[“1”]&endkey=[“1”,2] “key”:[“1”,0], “value”:{“_id”:”1”, “type”:”post”, “text”:”My Blog Post”} “key”:[“2”,0], “value”:{“_id”:”2”, “type”:”post”, “text”:”My 2nd Blog Post”} “key”:[“3”,0], “value”:{“_id”:”3”, “type”:”post”, “text”:”My 3rd Blog Post”} “key”:[“1”,1], “value”:{“_id”:”3”, “type”:”comment”, “text”:”You rock dude”, “post”:”1”} “key”:[“2”,1], “value”:{“_id”:”3”, “type”:”comment”, “text”:”Man you suck”, “post”:”2”}
  • Indexes • Relational databases typically pay cost at point of insertion • CouchDB pay cost at time of checking view • Expensive if you write lots and read infrequently • Can create an update script CouchDB calls when database updated
  • Replication
  • Replication
  • Replication
  • Replication
  • Replication Nodes that go down catch up
  • Replication Partial Replicas
  • Replication Partial Replicas Offline copies
  • Conflict Management • As well as an _id field each document has a _rev • Each time a document is updated so is the _rev • Used to help determine ‘winning’ document when conflicts occur • Old revisions are available until the database is compressed
  • When to use CouchDB (and also when not)
  • Scenario I: Schema-less databases • FriendFeed moved to using MySQL in schema-less fashion because: • Adding new features became so difficult. In particular adding new indexes became too expensive for tables with 10-20 million rows
  • Rules of Database degradation • Fields become optional • Relationships become Many-to-Many [ Taken from http://push.cx/2009/rules-of-database- app-aging ]
  • Scenario II: Real world models “The real world doesn’t map to a relational database very well”
  • 5ft Shelf - Envisaged Database Design Shelf M M Book 1 1 M M M M Recommendat Price Category ions
  • 5ft Shelf - Actual Database Design Book 1 M Recommendat ions 1 M Shelf M M Book Edition 1 1 M M M M ISBN Price Category
  • 5ft Shelf - CouchDB Database Design Shelf M M Book 1 M Recommendat ions Prices ISBNs Categories Inline Associations on Book
  • Scenario III: Using replication/sharding • Offline Capability: • Satellite Office • Desktop Apps • ‘Edge’ Databases • Large amount of databases needed • Even 1 database per user
  • When not to use CouchDB • Estate Agent application • Each house has x bedrooms, y bathrooms... • Financial Applications • (Generally stuff that’s very fixed)
  • Introducing couch_foo (An exercise in learning how ActiveRecord really works)
  • Why? • Used all the existing ruby libraries at the time and found frustrations coming from ActiveRecord API • Opinion split on whether providing a full API for CouchDB is a good thing or whether more of the logic should be in application
  • CouchFoo Models class Address < CouchFoo::Base property :number, Integer property :street, String property :postcode end # - Generic types are fine if to_json # from_json are defined on the # object
  • CouchFoo Models (2) • The _id field is automatically assigned by CouchFoo and is guaranteed to be a UUID • View generation is handled automatically so you can just call the finders • Nearly all AR finders are available • Address.all • User.find_by_login, ... • (Obviously #find_by_sql isn’t)
  • CouchFoo Models (3) • Associations • Validations • Callbacks • Inheritance • Calculations • ...
  • How finders work • Each model stores its name in a ruby_class attribute in the document. • This makes it easy to filter out in a view • And then regenerate the model on retrieval • Views are generated automatically with one for each attribute lookup on a given model • Users can define their own custom view in the model as well
  • Finders • User.find(:first, :conditions => {:login => “george”}) • User.find(:first, :conditions => {:login => “george”}, :update => false) • User.find(:all, :limit => 10, :offset => 3) • User.find(:all, :use_key => [:login], :startkey => “fred”, :endkey => “george”)
  • Ordering Confusion • Relational databases order records by id which normally keeps incrementing • Thus newer records have a higher id • In CouchDB inserting a record may have a key starting ‘a126’ whereas the next insertion may have a key starting ‘3a82’ • Thus find(:all) operations aren’t always in the order you expect • This can confuse the user too - “Why is my newest photo in the middle of the list?”
  • Ordering Confusion (2) class Address < CouchFoo::Base property :number, Integer property :street, String property :postcode property :created_at, DateTime default_sort :created_at end
  • Gotchas • To keep views(indexes) at a minimum ordering is done by CouchDB if the attribute isn’t exposed in the key • This means using :order and :limit may not provide the results expected • Can get round by adding extra attributes to the query eg User.find(:all, :use_key => [:name, :admin], :conditions => {:admin => true}, :order => :name, :limit => 10)
  • Bulk Saving CouchFoo::Base.bulk_save_default = true User.all.size # => 27 User.create(:name => "george") User.all.size # => 27 User.database.commit User.all.size # => 28 # It’s the developers responsibility to commit # In Rails the after_filter can be used to avoid forgetting
  • Performance • For updates will perform worse when not in memory - eg update_all • With bulk_save can perform much better than relational databases • CouchDB 0.9 offers a few performance enhancements over 0.8 • Performance implications fully documented in CouchFoo#Base
  • Bonus
  • ActiveRecord tips • model.column_names # => {“id”, “login”,...} • also columns and columns_hash • instance.attributes # => {“name” => “George”, ...} • model.with_scope User.with_scope(:find => { :conditions => "state = 3" }) do find(1) # => SELECT * from articles WHERE state = 3 AND id = 1 end
  • Resources • Rails Freelancer • @Georgio_1999 • http://rowtheboat.com • http://github.com/georgepalmer