Persistence Smoothie: Blending SQL and NoSQL (RubyNation Edition)

Michael Bleigh
Persistence Intridea, Inc.

Smoothie
Blending SQL and NoSQL

photo by Nikki L. via Flickr

Saturday, April 10, 2010

present.ly


The Buzz


You’ve (probably)
heard a lot about
NoSQL


NoSQL is a new way
to think about
persistence


Atomicity
Consistency
Isolation
Durability


Denormalization
Eventual Consistency
Schema-Free
Horizontal Scale
Map Reduce


Map Reduce
• Massively parallel way to
process large datasets
• First you scour data and “map” a
new set of data
• Then you “reduce” the data
down to a salient result


map = function() {
this.tags.forEach(function(tag) {
emit(tag, {count: 1});
});
}

reduce = function(key, values) {
var total = 0;
for (var i = 0; i < values.length; i++) {
total += values[i].count;
return {count: total};
}


NoSQL tries to scale
(more) simply


NoSQL is going
mainstream


New York Times
Business Insider
BBC ShopWiki
GitHub Meebo
Disqus SourceForge
Sony Digg


...but not THAT
mainstream.


A word of caution...


sn’t
d oe s
QL wait
oS , it
N
s le ep
NoSQL can
divide by zero
NoSQL
to infin counte
ity, twi d
ce

NoSQL is a (growing)
collection of tools, not
a new way of life


The Ecosystem


Key-Value Stores

• Voldemort
• Redis
• Tokyo Cabinet
• Riak • MemcachedDB


Document Stores

• MongoDB • Riak
• CouchDB • FleetDB


Column(ish) Stores

• Cassandra
• HBase


Graph Databases

• Neo4j
• HypergraphDB
• InfoGrid


When should I use
this stuff?


Complex, slow joins
for “activity stream”


Complex, slow joins
for “activity stream”

Denormalize,
use Key-Value Store

Variable schema,
vertical interaction


Variable schema,
vertical interaction

Document Database
or Column Store

Modeling deep
relationships


Modeling deep
relationships

Graph Database


NoSQL solves real
scalability and data
design issues


Ben Scofield
bit.ly/state-of-nosql


Ready to go?


Just one problem...


Your data is already
in a SQL database


So now we need to
ask the question...


Yeah, it blends.


The “Hard” Way:
Do it by hand.


class Post
include MongoMapper::Document

key :title, String
key :body, String
key :tags, Array
key :user_id, Integer

def user
User.find_by_id(self.user_id)
end

def user=(some_user)
self.user_id = some_user.id
end
end

class User < ActiveRecord::Base
def posts(options = {})
Post.all({:conditions => {:user_id => self.id}}.merge(options))
end
end


Pros & Cons
• Simple, maps to your domain

• Works for small, simple ORM intersections

• MUCH simpler in Rails 3

• Complex relationships are a mess

• Makes your models fat

• As DRY as the ocean


The “Easy” Way:
DataMapper


DataMapper

• Generic, relational ORM
• Speaks pretty much everything
you’ve ever heard of
• Implements Identity Map
• Module-based inclusion

DataMapper.setup(:default, "mysql://localhost")
DataMapper.setup(:mongodb, "mongo://localhost/posts")

class Post
include DataMapper::Resource
def self.default_repository_name; :mongodb; end

property :title, String
property :body, String
property :tags, Array

belongs_to :user
end

class User
include DataMapper::Resource

property :email, String
property :name, String

has n, :posts
end


Pros & Cons
• The ultimate Polyglot ORM

• Simple relationships between persistence
engines are easy

• Jack of all trades, master of none

• Perpetuates (sometimes) false assumptions

• Legacy stuﬀ is in ActiveRecord anyway


Show and Tell:
Social Storefront


The Application
• Dummy version of a store that lets
others “follow” your purchases (like a
less creepy version of Blippy)
• Four requirements:
• users

• purchasing

• listings

• social graph


Users

• I already have an authentication
system
• I’m happy with it
• It’s Devise and ActiveRecord
• Stick with SQL

Purchasing

• Users need to be able to purchase
items from my storefront
• I can’t lose their transactions
• I need full ACID
• SQL Again

Social Graph

• I want activity streams and one
and two way relationships
• I need speed
• I don’t need consistency
• I’ll use Redis

Product Listings
• I am selling both books about
Ruby and movies about zombies
• They have very diﬀerent
properties
• Products are relatively non-
relational
• I’ll use MongoDB

Demo and
Walkthrough


Wrapping Up


These systems can
(and should) live and
work together


Most important step
is to actually think
about data design


When you have a
whole bag of tools,
things stop looking
like nails


@mbleigh


Questions?


Persistence Smoothie: Blending SQL and NoSQL (RubyNation Edition)

More Related Content

Viewers also liked

Similar to Persistence Smoothie: Blending SQL and NoSQL (RubyNation Edition)

More from Michael Bleigh

Recently uploaded

Persistence Smoothie: Blending SQL and NoSQL (RubyNation Edition)