Practical Ruby Projects with MongoDB - MongoSF

Practical Ruby Projects with

Who am I? Alex Sharp Lead Developer at OptimisDev @ajsharp alexjsharp.tumblr.com github.com/ajsharp

We’re going to talk about two things.

1. Why Mongo makes sense

2. Using MongoDB with Ruby

But ﬁrst...

This talk is not about shiny objects

It’s not about the new hotness

It’s not about speed...

Or performance...

Or even scalability...

Or boring acronyms like...

Or ACID

It’s about practicality.

This talk is about creating software.

Migrating to another technology is inefﬁcient

Most people are reluctant to move off RDBMS-es

Relational DBs are both 1.Antiquated 2.Inefﬁcient

The Problems of SQL

A brief history of SQL Developed at IBM in early 1970’s. Designed to manipulate and retrieve data stored in relational databases

A brief history of SQL Developed at IBM in early 1970’s. Designed to manipulate and retrieve data stored in relational databases this is a problem

We don’t work with data...

We work with objects

We don’t care about storing data

We care about persisting state

Data != Objects

Therefore...

Relational DBs are an antiquated tool for our needs

Ok, so what? SQL schemas are designed for storing and querying data, not persisting objects.

To reconcile this mismatch, we have ORM’s

Object Relational Mappers

Ok, so what? We need ORMs to bridge the gap (i.e. map) between SQL and native objects (in our case, Ruby objects)

We’re forced to create relationships for data when what we really want is properties for our objects.

This is NOT efﬁcient

@alex = Person.new( :name => "alex", :stalkings => [Friend.new("Jim"), Friend.new("Bob")] )

Native ruby object <Person:0x10017d030 @name="alex", @stalkings= [#<Friend:0x10017d0a8 @name="Jim">, #<Friend:0x10017d058 @name="Bob"> ]>

JSON/Mongo Representation @alex.to_json { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] }

SQL Schema Representation # in a SQL schema people: - name stalkings: - name - stalkee_id

SQL Schema Representation # in a SQL schema people: - name stalkings: Wha Wha!? - name - stalkee_id

SQL Schema Representation # in a SQL schema people: - name stalkings: stalkings ain’t got - name no stalkee_id - stalkee_id

SQL Schema Representation # in a SQL schema people: - name stalkings: stalkee_id is how we map our - name object to ﬁt in a SQL schema - stalkee_id

SQL Schema Representation # in a SQL schema people: - name stalkings: i.e. the “pointless join” - name - stalkee_id

Ruby -> JSON -> SQL <Person:0x10017d030 @name="alex", @stalkings= Ruby [#<Friend:0x10017d0a8 @name="Jim">, #<Friend:0x10017d058 @name="Bob"> ]> @alex.to_json JSON { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } people: - name SQL stalkings: - name - stalkee_id

Ruby -> JSON -> SQL <Person:0x10017d030 @name="alex", @stalkings= Ruby [#<Friend:0x10017d0a8 @name="Jim">, #<Friend:0x10017d058 @name="Bob"> ]> @alex.to_json JSON { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } people: - name Feels like we’re having SQL to work too hard here stalkings: - name - stalkee_id

Ruby -> JSON -> SQL <Person:0x10017d030 @name="alex", @stalkings= Ruby [#<Friend:0x10017d0a8 @name="Jim">, #<Friend:0x10017d058 @name="Bob"> ]> @alex.to_json JSON { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } people: - name SQL stalkings: - name - stalkee_id

This may seem trivial for simple object models

But consider a tree-type object graph

Example: a sentence builder

Ok, so what? You’re probably thinking... “Listen GUY, SQL isn’t that bad”

Ok, so what? Maybe.

Ok, so what? But we can do much, much better.

Summary of Problems mysql is both antiquated and inefﬁcient

Needs for a persistence layer:

Needs for a persistence layer: 1.To persist the native state of our objects

Needs for a persistence layer: 1.To persist the native state of our objects 2.Should NOT interfere w/ application development

Needs for a persistence layer: 1.To persist the native state of our objects 2.Should NOT interfere w/ application development 3.Provides features necessary to build modern web apps

Mongo to the Rescue!

Mongo was made for web apps

Mongo goes the 80/20 route

Document storage

Mongo stores everything in binary JSON

Simplifying schema design Maps/hashes/associative arrays are arguably among the best object serialization formats

Mongo Representation { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] }

Mongo Representation { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } In Mongo, stalkings is an “embedded document”

Mongo Representation { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } Great for one-to-many associations

Mongo Representation { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } No JOINS required.

Mongo Representation { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } This is a good thing for scalability, because JOINS limit our ability to horizontally scale.

Mongo Representation { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } But more importantly, this is more intuitive than messing with foreign keys

Mongo allows us to store objects in a near-native format

This is awesome.

Much less schema design.

Much less brain pain.

Embedded Documents

Lots of 1-to-many relationships can be implemented as an embedded doc

This results in way fewer “pointless JOINs”

Scalability

Ok, so I lied.

The ability to easily scale out is an important part of the philosophy behind Mongo

Also has implications for in how we store our objects

If scaling out is easy, who cares about the DB getting “too large”?

Just ﬁre up another EC2 instance and be done with it.

Auto-sharding is going to make this stupid easy.

So stay tuned...

Other features necessary for building web apps

indexes

replication/redundancy

rich query syntax

Rich query syntax

Practical Projects

Four Examples 1.Accounting Application 2.Capped Collection Logging 3.Blogging Application 4.Storing binary data w/ GridFS

A simple general ledger accounting application

The object model ledger * transactions * entries

The object model # Credits Debits Transaction { 1 { :account => “Cash”, :amount => 100.00 } { :account => “Notes Pay.”, :amount => 100.00 } Ledger Entries Transaction { 2 { :account => “A/R”, :amount => 25.00 } { :account => “Gross Revenue”, :amount => 25.00 }

Object model summary

Object model summary Each ledger transaction belongs to a ledger. Each ledger transaction has two ledger entries which must balance.

Object Model with ActiveRecord @credit_entry = LedgerEntry.new :account => "Cash", :amount => 100.00, :type => "credit" @debit_entry = LedgerEntry.new :account => "Notes Pay.", :amount => 100.00, :type => "debit" @ledger_transaction = LedgerTransaction.new :ledger_id => 1, :ledger_entries => [@credit_entry, @debit_entry]

Object Model with ActiveRecord @credit_entry = LedgerEntry.new :account => "Cash", :amount => 100.00, :type => "credit" @debit_entry = LedgerEntry.new :account => "Notes Pay.", :amount => 100.00, :type => "debit" @ledger_transaction = LedgerTransaction.new :ledger_id => 1, :ledger_entries => [@credit_entry, @debit_entry] In a SQL schema, we need a database transaction to ensure atomicity

Object Model with Mongo @ledger_transaction = LedgerTransaction.new :ledger_id => 1, :ledger_entries => [ { :account => 'Cash', :type => "credit", :amount => 100.00 }, { :account => 'Notes Pay.', :type => "debit", :amount => 100.00 } ]

Object Model with Mongo @ledger_transaction = LedgerTransaction.new :ledger_id => 1, :ledger_entries => [ { :account => 'Cash', :type => "credit", :amount => 100.00 }, { :account => 'Notes Pay.', :type => "debit", :amount => 100.00 } ] This is the perfect case for embedded documents.

Object Model with Mongo @ledger_transaction = LedgerTransaction.new :ledger_id => 1, :ledger_entries => [ { :account => 'Cash', :type => "credit", :amount => 100.00 }, { :account => 'Notes Pay.', :type => "debit", :amount => 100.00 } ] We would never have a ledger entry w/o a ledger transaction.

Object Model with Mongo @ledger_transaction = LedgerTransaction.new :ledger_id => 1, :ledger_entries => [ { :account => 'Cash', :type => "credit", :amount => 100.00 }, { :account => 'Notes Pay.', :type => "debit", :amount => 100.00 } ] In this case, we don’t even need database transactions.

Object Model with Mongo @ledger_transaction = LedgerTransaction.new :ledger_id => 1, :ledger_entries => [ { :account => 'Cash', :type => "credit", :amount => 100.00 }, { :account => 'Notes Pay.', :type => "debit", :amount => 100.00 } ] Sweet.

Logging with Capped Collections

Baseless statistic 95% of the time logs are NEVER used.

Most logs are only used when something goes wrong.

Capped collections Fixed-sized, limited operation, auto age-out collections (kinda like memcached) Fixed insertion order Super fast (faster than normal writes) Ideal for logging and caching

Great. What’s that mean? We can log additional pieces of arbitrary data effortlessly due to Mongo’s schemaless nature.

This is awesome.

Now we have logs we can query, analyze and use!

Also a really handy troubleshooting tool

Scenario User experiences weird, hard to reproduce error.

Scenario Wouldn’t it be useful to see the complete click-path?

Bunyan http://github.com/ajsharp/bunyan Thin ruby layer around a MongoDB capped collection

require 'bunyan' # put in config/initializers for rails Bunyan::Logger.configure do |c| c.database 'my_bunyan_db' c.collection 'development_log' # == 100.megabytes if using rails c.size 104857600 end

class BunyanMiddleware def initialize(app) @app = app end def call(env) @status, @headers, @response = @app.call(env) Bunyan::Logger.insert(prepare_extra_fields) [@status, @headers, @response] end end

Bunyan::Logger.find('user.email' => 'ajsharp@gmail.com')

Stay tuned for a Sinatra-based client front-end.

Mongolytics Drop-in to a rails app http://github.com/tpitale/mongolytics.git

Blogging Application

Blogging Application Much easier to model with Mongo than a relational database

Blogging Application A post has an author

Blogging Application A post has an author A post has many tags

Blogging Application A post has an author A post has many tags A post has many comments

Blogging Application A post has an author A post has many tags A post has many comments Instead of JOINing separate tables, we can use embedded documents.

require 'mongo' conn = Mongo::Connection.new.db('bloggery') posts = conn.collection('posts') authors = conn.collection('authors')

# returns a Mongo::ObjectID object alex = authors.save :name => "Alex" post = posts.save( :title => 'Post title', :body => 'Massive pontification...', :tags => ['mongosf', 'omg', 'lolcats'], :comments => [ { :name => "Loudmouth McGee", :email => 'loud@mouth.edu', :body => "Something really ranty..." } ], :author_id => alex )

# returns a Mongo::ObjectID object alex = authors.save :name => "Alex" post = posts.save( :title => 'Post title', :body => 'Massive pontification...', :tags => ['mongosf', 'omg', 'lolcats'], :comments => [ { :name => "Loudmouth McGee", :email => 'loud@mouth.edu', :body => "Something really ranty..." } ], Joins not necessary. Sweet. :author_id => alex )

MongoMapper

MongoMapper • MongoDB “ORM” developed by John Nunemaker

MongoMapper • MongoDB “ORM” developed by John Nunemaker • author of HttpParty

MongoMapper • MongoDB “ORM” developed by John Nunemaker • author of HttpParty • Very similar syntax to DataMapper

MongoMapper • MongoDB “ORM” developed by John Nunemaker • author of HttpParty • Very similar syntax to DataMapper • Declarative rather than inheritance-based

MongoMapper • MongoDB “ORM” developed by John Nunemaker • author of HttpParty • Very similar syntax to DataMapper • Declarative rather than inheritance-based • Very easy to drop into rails

MongoMapper class Post include MongoMapper::Document belongs_to :author, :class_name => "User" key :title, String, :required => true key :body, String key :author_id, Integer, :required => true key :published_at, Time key :published, Boolean, :default => false timestamps! many :tags end class Tag include MongoMapper::EmbeddedDocument key :name, String, :required => true end

Storing binary data w/ GridFS and Joint

Joint 1.MongoMapper plugin for accessing GridFS 2.Built on top of the Ruby driver GridFS API

class Bio include MongoMapper::Document plugin Joint key :name, String, :required => true key :title, String key :description, String timestamps! attachment :headshot attachment :video_bio end

class Bio include MongoMapper::Document plugin Joint key :name, String, :required => true key :title, String key :description, String timestamps! Can be served directly attachment :headshot from Mongo attachment :video_bio end

Using mongo w/ Ruby Ruby mongo driver MongoMapper MongoID MongoRecord (http://github.com/mongodb/mongorecord) MongoDoc (http://github.com/leshill/mongodoc) Many other ORM/ODM’s under active development

Questions?

Thanks!

Practical Ruby Projects with MongoDB - MongoSF

by Alex Sharp on Apr 30, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 34

Statistics

Practical Ruby Projects with MongoDB - MongoSF — Presentation Transcript