Practical Ruby Projects (Alex Sharp)

3,816 views

Published on

Published in: Technology

Practical Ruby Projects (Alex Sharp)

  1. 1. Practical Ruby Projects with
  2. 2. Who am I? Alex Sharp Lead Developer at OptimisDev @ajsharp alexjsharp.tumblr.com github.com/ajsharp
  3. 3. We’re going to talk about two things.
  4. 4. 1. Why Mongo makes sense
  5. 5. 2. Using MongoDB with Ruby
  6. 6. But first...
  7. 7. This talk is not about shiny objects
  8. 8. It’s not about the new hotness
  9. 9. It’s not about speed...
  10. 10. Or performance...
  11. 11. Or even scalability...
  12. 12. Or boring acronyms like...
  13. 13. CAP
  14. 14. Or ACID
  15. 15. It’s about practicality.
  16. 16. This talk is about creating software.
  17. 17. Migrating to another technology is inefficient
  18. 18. Most people are reluctant to move off RDBMS-es
  19. 19. Relational DBs are both 1.Antiquated 2.Inefficient
  20. 20. The Problems of SQL
  21. 21. A brief history of SQL Developed at IBM in early 1970’s. Designed to manipulate and retrieve data stored in relational databases
  22. 22. A brief history of SQL Developed at IBM in early 1970’s. Designed to manipulate and retrieve data stored in relational databases this is a problem
  23. 23. Why??
  24. 24. We don’t work with data...
  25. 25. We work with objects
  26. 26. We don’t care about storing data
  27. 27. We care about persisting state
  28. 28. Data != Objects
  29. 29. Therefore...
  30. 30. Relational DBs are an antiquated tool for our needs
  31. 31. Ok, so what? SQL schemas are designed for storing and querying data, not persisting objects.
  32. 32. To reconcile this mismatch, we have ORM’s
  33. 33. Object Relational Mappers
  34. 34. Ok, so what? We need ORMs to bridge the gap (i.e. map) between SQL and native objects (in our case, Ruby objects)
  35. 35. We’re forced to create relationships for data when what we really want is properties for our objects.
  36. 36. This is NOT efficient
  37. 37. @alex = Person.new( :name => "alex", :stalkings => [Friend.new("Jim"), Friend.new("Bob")] )
  38. 38. Native ruby object <Person:0x10017d030 @name="alex", @stalkings= [#<Friend:0x10017d0a8 @name="Jim">, #<Friend:0x10017d058 @name="Bob"> ]>
  39. 39. JSON/Mongo Representation @alex.to_json { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] }
  40. 40. SQL Schema Representation # in a SQL schema people: - name stalkings: - name - stalkee_id
  41. 41. SQL Schema Representation # in a SQL schema people: - name stalkings: Wha Wha!? - name - stalkee_id
  42. 42. SQL Schema Representation # in a SQL schema people: - name stalkings: stalkings ain’t got - name no stalkee_id - stalkee_id
  43. 43. SQL Schema Representation # in a SQL schema people: - name stalkings: stalkee_id is how we map our - name object to fit in a SQL schema - stalkee_id
  44. 44. SQL Schema Representation # in a SQL schema people: - name stalkings: i.e. the “pointless join” - name - stalkee_id
  45. 45. Ruby -> JSON -> SQL <Person:0x10017d030 @name="alex", @stalkings= Ruby [#<Friend:0x10017d0a8 @name="Jim">, #<Friend:0x10017d058 @name="Bob"> ]> @alex.to_json JSON { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } people: - name SQL stalkings: - name - stalkee_id
  46. 46. Ruby -> JSON -> SQL <Person:0x10017d030 @name="alex", @stalkings= Ruby [#<Friend:0x10017d0a8 @name="Jim">, #<Friend:0x10017d058 @name="Bob"> ]> @alex.to_json JSON { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } people: - name Feels like we’re having SQL to work too hard here stalkings: - name - stalkee_id
  47. 47. Ruby -> JSON -> SQL <Person:0x10017d030 @name="alex", @stalkings= Ruby [#<Friend:0x10017d0a8 @name="Jim">, #<Friend:0x10017d058 @name="Bob"> ]> @alex.to_json JSON { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } people: - name SQL stalkings: - name - stalkee_id
  48. 48. This may seem trivial for simple object models
  49. 49. But consider a tree-type object graph
  50. 50. Example: a sentence builder
  51. 51. Ok, so what? You’re probably thinking... “Listen GUY, SQL isn’t that bad”
  52. 52. Ok, so what? Maybe.
  53. 53. Ok, so what? But we can do much, much better.
  54. 54. Summary of Problems mysql is both antiquated and inefficient
  55. 55. Needs for a persistence layer:
  56. 56. Needs for a persistence layer: 1.To persist the native state of our objects
  57. 57. Needs for a persistence layer: 1.To persist the native state of our objects 2.Should NOT interfere w/ application development
  58. 58. Needs for a persistence layer: 1.To persist the native state of our objects 2.Should NOT interfere w/ application development 3.Provides features necessary to build modern web apps
  59. 59. Mongo to the Rescue!
  60. 60. Mongo was made for web apps
  61. 61. Mongo goes the 80/20 route
  62. 62. Document storage
  63. 63. Mongo stores everything in binary JSON
  64. 64. Simplifying schema design Maps/hashes/associative arrays are arguably among the best object serialization formats
  65. 65. Mongo Representation { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] }
  66. 66. Mongo Representation { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } In Mongo, stalkings is an “embedded document”
  67. 67. Mongo Representation { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } Great for one-to-many associations
  68. 68. Mongo Representation { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } No JOINS required.
  69. 69. Mongo Representation { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } This is a good thing for scalability, because JOINS limit our ability to horizontally scale.
  70. 70. Mongo Representation { name: "alex", stalkings: [{ name: "Jim" }, { name: "Bob" }] } But more importantly, this is more intuitive than messing with foreign keys
  71. 71. Mongo allows us to store objects in a near-native format
  72. 72. This is awesome.
  73. 73. Much less schema design.
  74. 74. Much less brain pain.
  75. 75. Embedded Documents
  76. 76. Lots of 1-to-many relationships can be implemented as an embedded doc
  77. 77. This results in way fewer “pointless JOINs”
  78. 78. Scalability
  79. 79. Ok, so I lied.
  80. 80. ;)
  81. 81. The ability to easily scale out is an important part of the philosophy behind Mongo
  82. 82. Also has implications for in how we store our objects
  83. 83. If scaling out is easy, who cares about the DB getting “too large”?
  84. 84. Just fire up another EC2 instance and be done with it.
  85. 85. Auto-sharding is going to make this stupid easy.
  86. 86. So stay tuned...
  87. 87. Other features necessary for building web apps
  88. 88. indexes
  89. 89. replication/redundancy
  90. 90. rich query syntax
  91. 91. Rich query syntax
  92. 92. Practical Projects
  93. 93. Four Examples 1.Accounting Application 2.Capped Collection Logging 3.Blogging Application 4.Storing binary data w/ GridFS
  94. 94. A simple general ledger accounting application
  95. 95. The object model ledger * transactions * entries
  96. 96. The object model # Credits Debits Transaction { 1 { :account => “Cash”, :amount => 100.00 } { :account => “Notes Pay.”, :amount => 100.00 } Ledger Entries Transaction { 2 { :account => “A/R”, :amount => 25.00 } { :account => “Gross Revenue”, :amount => 25.00 }
  97. 97. Object model summary
  98. 98. Object model summary Each ledger transaction belongs to a ledger. Each ledger transaction has two ledger entries which must balance.
  99. 99. Object Model with ActiveRecord @credit_entry = LedgerEntry.new :account => "Cash", :amount => 100.00, :type => "credit" @debit_entry = LedgerEntry.new :account => "Notes Pay.", :amount => 100.00, :type => "debit" @ledger_transaction = LedgerTransaction.new :ledger_id => 1, :ledger_entries => [@credit_entry, @debit_entry]
  100. 100. Object Model with ActiveRecord @credit_entry = LedgerEntry.new :account => "Cash", :amount => 100.00, :type => "credit" @debit_entry = LedgerEntry.new :account => "Notes Pay.", :amount => 100.00, :type => "debit" @ledger_transaction = LedgerTransaction.new :ledger_id => 1, :ledger_entries => [@credit_entry, @debit_entry] In a SQL schema, we need a database transaction to ensure atomicity
  101. 101. Object Model with Mongo @ledger_transaction = LedgerTransaction.new :ledger_id => 1, :ledger_entries => [ { :account => 'Cash', :type => "credit", :amount => 100.00 }, { :account => 'Notes Pay.', :type => "debit", :amount => 100.00 } ]
  102. 102. Object Model with Mongo @ledger_transaction = LedgerTransaction.new :ledger_id => 1, :ledger_entries => [ { :account => 'Cash', :type => "credit", :amount => 100.00 }, { :account => 'Notes Pay.', :type => "debit", :amount => 100.00 } ] This is the perfect case for embedded documents.
  103. 103. Object Model with Mongo @ledger_transaction = LedgerTransaction.new :ledger_id => 1, :ledger_entries => [ { :account => 'Cash', :type => "credit", :amount => 100.00 }, { :account => 'Notes Pay.', :type => "debit", :amount => 100.00 } ] We would never have a ledger entry w/o a ledger transaction.
  104. 104. Object Model with Mongo @ledger_transaction = LedgerTransaction.new :ledger_id => 1, :ledger_entries => [ { :account => 'Cash', :type => "credit", :amount => 100.00 }, { :account => 'Notes Pay.', :type => "debit", :amount => 100.00 } ] In this case, we don’t even need database transactions.
  105. 105. Object Model with Mongo @ledger_transaction = LedgerTransaction.new :ledger_id => 1, :ledger_entries => [ { :account => 'Cash', :type => "credit", :amount => 100.00 }, { :account => 'Notes Pay.', :type => "debit", :amount => 100.00 } ] Sweet.
  106. 106. Logging with Capped Collections
  107. 107. Baseless statistic 95% of the time logs are NEVER used.
  108. 108. Most logs are only used when something goes wrong.
  109. 109. Capped collections Fixed-sized, limited operation, auto age-out collections (kinda like memcached) Fixed insertion order Super fast (faster than normal writes) Ideal for logging and caching
  110. 110. Great. What’s that mean? We can log additional pieces of arbitrary data effortlessly due to Mongo’s schemaless nature.
  111. 111. This is awesome.
  112. 112. Now we have logs we can query, analyze and use!
  113. 113. Also a really handy troubleshooting tool
  114. 114. Scenario User experiences weird, hard to reproduce error.
  115. 115. Scenario Wouldn’t it be useful to see the complete click-path?
  116. 116. Bunyan http://github.com/ajsharp/bunyan Thin ruby layer around a MongoDB capped collection
  117. 117. require 'bunyan' # put in config/initializers for rails Bunyan::Logger.configure do |c| c.database 'my_bunyan_db' c.collection 'development_log' # == 100.megabytes if using rails c.size 104857600 end
  118. 118. class BunyanMiddleware def initialize(app) @app = app end def call(env) @status, @headers, @response = @app.call(env) Bunyan::Logger.insert(prepare_extra_fields) [@status, @headers, @response] end end
  119. 119. Bunyan::Logger.find('user.email' => 'ajsharp@gmail.com')
  120. 120. Stay tuned for a Sinatra-based client front-end.
  121. 121. Mongolytics Drop-in to a rails app http://github.com/tpitale/mongolytics.git
  122. 122. Blogging Application
  123. 123. Blogging Application Much easier to model with Mongo than a relational database
  124. 124. Blogging Application A post has an author
  125. 125. Blogging Application A post has an author A post has many tags
  126. 126. Blogging Application A post has an author A post has many tags A post has many comments
  127. 127. Blogging Application A post has an author A post has many tags A post has many comments Instead of JOINing separate tables, we can use embedded documents.
  128. 128. require 'mongo' conn = Mongo::Connection.new.db('bloggery') posts = conn.collection('posts') authors = conn.collection('authors')
  129. 129. # returns a Mongo::ObjectID object alex = authors.save :name => "Alex" post = posts.save( :title => 'Post title', :body => 'Massive pontification...', :tags => ['mongosf', 'omg', 'lolcats'], :comments => [ { :name => "Loudmouth McGee", :email => 'loud@mouth.edu', :body => "Something really ranty..." } ], :author_id => alex )
  130. 130. # returns a Mongo::ObjectID object alex = authors.save :name => "Alex" post = posts.save( :title => 'Post title', :body => 'Massive pontification...', :tags => ['mongosf', 'omg', 'lolcats'], :comments => [ { :name => "Loudmouth McGee", :email => 'loud@mouth.edu', :body => "Something really ranty..." } ], Joins not necessary. Sweet. :author_id => alex )
  131. 131. MongoMapper
  132. 132. MongoMapper • MongoDB “ORM” developed by John Nunemaker
  133. 133. MongoMapper • MongoDB “ORM” developed by John Nunemaker • author of HttpParty
  134. 134. MongoMapper • MongoDB “ORM” developed by John Nunemaker • author of HttpParty • Very similar syntax to DataMapper
  135. 135. MongoMapper • MongoDB “ORM” developed by John Nunemaker • author of HttpParty • Very similar syntax to DataMapper • Declarative rather than inheritance-based
  136. 136. MongoMapper • MongoDB “ORM” developed by John Nunemaker • author of HttpParty • Very similar syntax to DataMapper • Declarative rather than inheritance-based • Very easy to drop into rails
  137. 137. MongoMapper class Post include MongoMapper::Document belongs_to :author, :class_name => "User" key :title, String, :required => true key :body, String key :author_id, Integer, :required => true key :published_at, Time key :published, Boolean, :default => false timestamps! many :tags end class Tag include MongoMapper::EmbeddedDocument key :name, String, :required => true end
  138. 138. Storing binary data w/ GridFS and Joint
  139. 139. Joint 1.MongoMapper plugin for accessing GridFS 2.Built on top of the Ruby driver GridFS API
  140. 140. class Bio include MongoMapper::Document plugin Joint key :name, String, :required => true key :title, String key :description, String timestamps! attachment :headshot attachment :video_bio end
  141. 141. class Bio include MongoMapper::Document plugin Joint key :name, String, :required => true key :title, String key :description, String timestamps! Can be served directly attachment :headshot from Mongo attachment :video_bio end
  142. 142. Bing.
  143. 143. Bang.
  144. 144. Boom.
  145. 145. Using mongo w/ Ruby Ruby mongo driver MongoMapper MongoID MongoRecord (http://github.com/mongodb/mongo- record) MongoDoc (http://github.com/leshill/mongodoc) Many other ORM/ODM’s under active development
  146. 146. Questions?
  147. 147. Thanks!

×