Practical Ruby Projects with MongoDB - Ruby Midwest
Upcoming SlideShare
Loading in...5
×
 

Practical Ruby Projects with MongoDB - Ruby Midwest

on

  • 10,606 views

 

Statistics

Views

Total Views
10,606
Views on SlideShare
7,767
Embed Views
2,839

Actions

Likes
26
Downloads
172
Comments
0

25 Embeds 2,839

http://www.mongodb.org 2648
http://www.nosqldatabases.com 75
http://speakerrate.com 20
http://ru.wiki.mongodb.org 19
http://pt.wiki.mongodb.org 15
http://vinublogs.blogspot.com 15
http://es.wiki.mongodb.org 12
http://billchristian.posterous.com 6
http://mongodb.onconfluence.com 4
http://www.slideshare.net 4
http://vinublogs.blogspot.in 3
http://www.opencracy.com 3
http://wiki.mongodb.org 2
http://8564857796999208448_8d4f07db92fca5f89eaedf41855461a8f07f2ae3.blogspot.com 2
http://vinublogs.blogspot.ca 1
http://feeds.feedburner.com 1
http://vinublogs.blogspot.kr 1
https://wiki.10gen.com 1
http://67.207.142.125 1
http://htmledit.squarefree.com 1
http://translate.googleusercontent.com 1
http://10gen.com 1
https://p.yammer.com 1
http://api.mongodb.org 1
http://www.linkedin.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Practical Ruby Projects with MongoDB - Ruby Midwest Practical Ruby Projects with MongoDB - Ruby Midwest Presentation Transcript

  • PRACTICAL PROJECTS WITH Saturday, July 17, 2010
  • WHO AM I? Alex Sharp Lead Developer at OptimisDev @ajsharp (on the twitters) alexjsharp.com (on the ‘tubes) github.com/ajsharp (on the ‘hubs) Saturday, July 17, 2010
  • MAJOR POINTS • Brief intro to MongoDB • Practical Projects • Making the case for Mongo (time permitting) Saturday, July 17, 2010
  • BRIEF INTRO Saturday, July 17, 2010
  • WHAT IS MONGODB? mongodb is a high-performance, schema-less, document-oriented database Saturday, July 17, 2010
  • STRENGTHS Saturday, July 17, 2010
  • SCHEMA-LESS Saturday, July 17, 2010
  • SCHEMA-LESS • Great for rapid, iterative, agile development Saturday, July 17, 2010
  • SCHEMA-LESS • Great for rapid, iterative, agile development • Makes things possible that in an rdbms are either a. impossibly hard b. virtually impossible c. way harder than they should be Saturday, July 17, 2010
  • DOCUMENT-ORIENTED Saturday, July 17, 2010
  • DOCUMENT-ORIENTED • Mongo stores documents, not rows • documents are stored as binary json Saturday, July 17, 2010
  • DOCUMENT-ORIENTED • Mongo stores documents, not rows • documents are stored as binary json • Allows for a rich query syntax Saturday, July 17, 2010
  • DOCUMENT-ORIENTED • Mongo stores documents, not rows • documents are stored as binary json • Allows for a rich query syntax • Embedded documents eliminate many modeling headaches Saturday, July 17, 2010
  • BIG DATA Saturday, July 17, 2010
  • BIG DATA • Built to scale horizontally into the cloudz Saturday, July 17, 2010
  • BIG DATA • Built to scale horizontally into the cloudz • Auto-sharding • set a shard-key, a cluster of nodes, go • still in alpha, but production-ready soon Saturday, July 17, 2010
  • FAST WRITES Saturday, July 17, 2010
  • FAST WRITES • Mongo writes are “fire-and-forget” • you can get a response with getLastError() Saturday, July 17, 2010
  • FAST WRITES • Mongo writes are “fire-and-forget” • you can get a response with getLastError() • In-place updates Saturday, July 17, 2010
  • FAST WRITES • Mongo writes are “fire-and-forget” • you can get a response with getLastError() • In-place updates • Lot’s of db commands for fast updates Saturday, July 17, 2010
  • AGGREGATION Map/reduce for aggregation Saturday, July 17, 2010
  • READS FAST TOO Optimize reads with indexes, just like an rdbms Saturday, July 17, 2010
  • WEAKNESSES Saturday, July 17, 2010
  • JOINS nope. Saturday, July 17, 2010
  • MULTI-DOCUMENT TRANSACTIONS no-sir-ee. Saturday, July 17, 2010
  • RELATIONAL INTEGRITY not a chance. Saturday, July 17, 2010
  • PRACTICAL PROJECTS Saturday, July 17, 2010
  • 1. Accounting Application 2. Capped Collection Logging 3. Blogging app 4. Reporting app Saturday, July 17, 2010
  • GENERAL LEDGER ACCOUNTING APPLICATION Saturday, July 17, 2010
  • THE OBJECT MODEL ledger * transactions * entries Saturday, July 17, 2010
  • THE OBJECT MODEL # Credits Debits Line item { 1 { :account => “Cash”, :amount => 100.00 } { :account => “Notes Pay.”, :amount => 100.00 } Ledger Entries Line item { 2 { :account => “A/R”, :amount => 25.00 } { :account => “Gross Revenue”, :amount => 25.00 } Saturday, July 17, 2010
  • THE OBJECT MODEL Saturday, July 17, 2010
  • THE OBJECT MODEL • Each ledger line item belongs to a ledger Saturday, July 17, 2010
  • THE OBJECT MODEL • Each ledger line item belongs to a ledger • Each ledger line item has two ledger entries • must have at least one credit and one debit • credits and debits must balance Saturday, July 17, 2010
  • THE OBJECT MODEL • Each ledger line item belongs to a ledger • Each ledger line item has two ledger entries • must have at least one credit and one debit • credits and debits must balance • These objects must be transactional Saturday, July 17, 2010
  • SQL-BASED OBJECT MODEL @debit = Entry.new(:account => 'Cash', :amount => 100.00, :type => 'credit') @credit = Entry.new(:account => 'Notes Pay.', :amount => 100.00, :type => 'debit') @line_item = LineItem.new :ledger_id => 1, :entries => [@debit, @credit] Saturday, July 17, 2010
  • SQL-BASED OBJECT MODEL @debit = Entry.new(:account => 'Cash', :amount => 100.00, :type => 'credit') @credit = Entry.new(:account => 'Notes Pay.', :amount => 100.00, :type => 'debit') @line_item = LineItem.new :ledger_id => 1, :entries => [@debit, @credit] In a SQL schema, we need a database transaction to ensure multi-row atomicity Saturday, July 17, 2010
  • SQL-BASED OBJECT MODEL @debit = Entry.new(:account => 'Cash', :amount => 100.00, :type => 'credit') @credit = Entry.new(:account => 'Notes Pay.', :amount => 100.00, :type => 'debit') @line_item = LineItem.new :ledger_id => 1, :entries => [@debit, @credit] Remember: we have to create 3 new records in the database here Saturday, July 17, 2010
  • MONGO-FIED MODELING @line_item = LineItem.new(:ledger_id => 1, :entries => [ {:account => 'Cash', :type => 'debit', :amount => 100}, {:account => 'Notes pay.', :type => 'debit', :amount => 100} ] ) Saturday, July 17, 2010
  • MONGO-FIED MODELING @line_item = LineItem.new(:ledger_id => 1, :entries => [ {:account => 'Cash', :type => 'debit', :amount => 100}, {:account => 'Notes pay.', :type => 'debit', :amount => 100} ] ) This is the perfect case for embedded documents. Saturday, July 17, 2010
  • MONGO-FIED MODELING @line_item = LineItem.new(:ledger_id => 1, :entries => [ {:account => 'Cash', :type => 'debit', :amount => 100}, {:account => 'Notes pay.', :type => 'debit', :amount => 100} ] ) We will never have more than a few entries (usually two) Saturday, July 17, 2010
  • MONGO-FIED MODELING @line_item = LineItem.new(:ledger_id => 1, :entries => [ {:account => 'Cash', :type => 'debit', :amount => 100}, {:account => 'Notes pay.', :type => 'debit', :amount => 100} ] ) Embedded docs are perfect for “contains” many relationships Saturday, July 17, 2010
  • MONGO-FIED MODELING @line_item = LineItem.new(:ledger_id => 1, :entries => [ {:account => 'Cash', :type => 'debit', :amount => 100}, {:account => 'Notes pay.', :type => 'debit', :amount => 100} ] ) DB-level transactions no-longer needed. Mongo has single document atomicity. Saturday, July 17, 2010
  • LESSONS Saturday, July 17, 2010
  • LESSONS • Simplified, de-normalized data model • Objects modeled differently in Mongo than SQL Saturday, July 17, 2010
  • LESSONS • Simplified, de-normalized data model • Objects modeled differently in Mongo than SQL • Simpler data model + no transactions = WIN Saturday, July 17, 2010
  • LOGGING WITH CAPPED COLLECTIONS Saturday, July 17, 2010
  • BASELESS STATISTIC 95% of the time logs are NEVER used. Saturday, July 17, 2010
  • MOST LOGS ARE ONLY USED WHEN SOMETHING GOES WRONG Saturday, July 17, 2010
  • CAPPED COLLECTIONS • Fixed-sized, limitedoperation, auto age-out collections (kinda like memcached, except persistent) • Fixed insertion order • Super fast (faster than normal writes) • Ideal for logging and caching Saturday, July 17, 2010
  • GREAT. WHAT’S THAT MEAN? We can log additional pieces of arbitrary data effortlessly due to Mongo’s schemaless nature. Saturday, July 17, 2010
  • THIS IS AWESOME Saturday, July 17, 2010
  • QUERY. ANALYZE. PROFIT. Saturday, July 17, 2010
  • ALSO A REALLY HANDY TROUBLESHOOTING TOOL Saturday, July 17, 2010
  • User-reported errors are difficult to diagnose Saturday, July 17, 2010
  • Wouldn’t it be useful to see the complete click- path? Saturday, July 17, 2010
  • BUNYAN http://github.com/ajsharp/bunyan Thin ruby layer around a MongoDB capped collection Saturday, July 17, 2010
  • BUNYAN http://github.com/ajsharp/bunyan Still needs a middleware api. Want to contribute? Saturday, July 17, 2010
  • require 'bunyan' Bunyan::Logger.configure do |c| c.database 'my_bunyan_db' c.collection "#{Rails.env}_bunyan_log" c.size 1073741824 # 1.gigabyte end Saturday, July 17, 2010
  • PAPERMILL http://github.com/ajsharp/papermill SINATRA FRONT-END TO BUNYAN Saturday, July 17, 2010
  • BLOGGING APPLICATION Saturday, July 17, 2010
  • BLOGGING APPLICATION Much easier to model with Mongo than a relational database Saturday, July 17, 2010
  • BLOGGING APPLICATION A post has an author Saturday, July 17, 2010
  • BLOGGING APPLICATION A post has an author A post has many tags Saturday, July 17, 2010
  • BLOGGING APPLICATION A post has an author A post has many tags A post has many comments Saturday, July 17, 2010
  • BLOGGING APPLICATION A post has an author A post has many tags A post has many comments Instead of JOINing separate tables, we can use embedded documents. Saturday, July 17, 2010
  • MONGOMAPPER Saturday, July 17, 2010
  • MONGOMAPPER •MongoDB “ORM” developed by John Nunemaker Saturday, July 17, 2010
  • MONGOMAPPER •MongoDB “ORM” developed by John Nunemaker •Very similar syntax to DataMapper Saturday, July 17, 2010
  • MONGOMAPPER •MongoDB “ORM” developed by John Nunemaker •Very similar syntax to DataMapper •Declarative rather than inheritance-based Saturday, July 17, 2010
  • MONGOMAPPER •MongoDB “ORM” developed by John Nunemaker •Very similar syntax to DataMapper •Declarative rather than inheritance-based •Very easy to drop into rails Saturday, July 17, 2010
  • MONGOMAPPER require 'mongo' connection = Mongo::Connection.new.db("#{Rails.env}_app") class Post include MongoMapper::Document belongs_to :author, :class_name => "User" key :title, String, :reuqired => true key :body, String, :required => true key :author_id, Integer, :required => true key :published_at, Time key :published, Boolean, :default => false key :tags, Array, :default => [] timestamps! end Saturday, July 17, 2010
  • REPORTING APP Saturday, July 17, 2010
  • Saturday, July 17, 2010
  • REPORTING IS HARD • Synchronization • Data, application API, schema • Schema synchronization is the hard one Saturday, July 17, 2010
  • WHY MONGO? • Aggregation support with Mongo’s map/reduce support • Mongo is good (and getting better) at handling big data sets • Schemaless is perfect for a flattened data set Saturday, July 17, 2010
  • Amazon EC2 Rackspace Transform MongoDB Server Reporting App WAN Main App MySQL Saturday, July 17, 2010
  • CLIENT REQUESTS Amazon EC2 MongoDB Server map/reduce Reporting App User requests Client request GET /reports/some_report Saturday, July 17, 2010
  • TRANFORMATION Data needs to extracted from the the main app, transormed (i.e. flattened) into some other form, and loaded into the reporting app for aggregation Saturday, July 17, 2010
  • TRANSFORMATION Rackspace Amazon EC2 1. GET /object_dependencies Transform MongoDB Server { :visit => [ :insurances, :patients ] } 2. read from active record MySQL 2. write to mongo Reporting App Main App Saturday, July 17, 2010
  • MAKING THE CASE Saturday, July 17, 2010
  • THE PROBLEMS OF SQL Saturday, July 17, 2010
  • A BRIEF HISTORY OF SQL • Developed at IBM in early 1970’s. • Designed to manipulate and retrieve data stored in relational databases Saturday, July 17, 2010
  • A BRIEF HISTORY OF SQL • Developed at IBM in early 1970’s. • Designed to manipulate and retrieve data stored in relational databases this is a problem Saturday, July 17, 2010
  • WHY?? Saturday, July 17, 2010
  • WE DON’T WORK WITH DATA... Saturday, July 17, 2010
  • WE WORK WITH OBJECTS Saturday, July 17, 2010
  • WE DON’T CARE ABOUT STORING DATA Saturday, July 17, 2010
  • WE CARE ABOUT PERSISTING STATE Saturday, July 17, 2010
  • DATA != OBJECTS Saturday, July 17, 2010
  • THEREFORE... Saturday, July 17, 2010
  • RELATIONAL DBS ARE AN ANTIQUATED TOOL FOR OUR NEEDS Saturday, July 17, 2010
  • OK, SO WHAT? SQL schemas are designed for storing and querying data, not persisting objects. Saturday, July 17, 2010
  • To reconcile this mismatch, we have ORM’s Saturday, July 17, 2010
  • Object Relational Mappers Saturday, July 17, 2010
  • OK, SO WHAT? We need ORMs to bridge the gap (map) between our native Ruby object model and a SQL schema Saturday, July 17, 2010
  • We’re forced to create relationships for data when what we really want is properties for our objects. Saturday, July 17, 2010
  • THIS IS NOT EFFICIENT Saturday, July 17, 2010
  • @alex = Person.new( :name => "alex", :stalkings => [ Friend.new("Jim"), Friend.new("Bob")] ) Saturday, July 17, 2010
  • Native ruby object <Person:0x10017d030 @name="alex", @stalkings= [#<Friend:0x10017d0a8 @name="Jim">, #<Friend:0x10017d058 @name="Bob"> ]> Saturday, July 17, 2010
  • JSON/Mongo Representation @alex.to_json { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } Saturday, July 17, 2010
  • SQL Schema Representation people: - name stalkings: - name - stalker_id Saturday, July 17, 2010
  • SQL Schema Representation people: - name stalkings: What!? - name - stalker_id Saturday, July 17, 2010
  • SQL Schema Representation people: - name stalkings: what’s a stalker_id? - name - stalker_id Saturday, July 17, 2010
  • SQL Schema Representation people: - name stalkings: this is our sql mapping - name - stalker_id Saturday, July 17, 2010
  • SQL Schema Representation people: - name stalkings: this is the - name “pointless” join - stalker_id Saturday, July 17, 2010
  • Ruby -> JSON -> SQL <Person:0x10017d030 @name="alex", @stalkings= Ruby [#<Friend:0x10017d0a8 @name="Jim">, #<Friend:0x10017d058 @name="Bob"> ]> @alex.to_json JSON { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } people: - name SQL stalkings: - name - stalker_id Saturday, July 17, 2010
  • Ruby -> JSON -> SQL <Person:0x10017d030 @name="alex", @stalkings= Ruby [#<Friend:0x10017d0a8 @name="Jim">, #<Friend:0x10017d058 @name="Bob"> ]> @alex.to_json JSON { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } people: - name Feels like we’re having SQL stalkings: to work too hard here - name - stalker_id Saturday, July 17, 2010
  • You’re probably thinking... “Listen GUY, SQL isn’t that bad” Saturday, July 17, 2010
  • Maybe. Saturday, July 17, 2010
  • But we can do much, much better. Saturday, July 17, 2010
  • NEEDS FOR A PERSISTENCE LAYER: Saturday, July 17, 2010
  • NEEDS FOR A PERSISTENCE LAYER: 1. To persist the native state of our objects Saturday, July 17, 2010
  • NEEDS FOR A PERSISTENCE LAYER: 1. To persist the native state of our objects 2. Should NOT interfere w/ application development Saturday, July 17, 2010
  • NEEDS FOR A PERSISTENCE LAYER: 1. To persist the native state of our objects 2. Should NOT interfere w/ application development 3. Provides features necessary to build modern web apps Saturday, July 17, 2010
  • NEXT STEPS Saturday, July 17, 2010
  • USING MONGO W/ RUBY • Ruby mongo driver • MongoMapper (github.com/jnunemaker/mongomapper) • MongoID (github.com/durran/mongoid) • Many other ORM/ODM’s • mongo-sequel (github.com/jeremyevans/sequel-mongo) • moneta (github.com/wycats/moneta) Saturday, July 17, 2010
  • MONGOMAPPER • DataMapper-like API • good for moving from a sql schema to mongo • easy to drop into rails • works with rails 3 Saturday, July 17, 2010
  • MONGOID • DataMapper-like API • uses ActiveModel, so familiar validation syntax • more opinionated embedded docs • easy to drop into rails • rails 2 - mongoid 1.x • rails 3 - mongoid 2.x Saturday, July 17, 2010
  • QUESTIONS? Saturday, July 17, 2010
  • THANKS! Saturday, July 17, 2010