10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Rubyfile:///Users/gjm/10gen/...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                           Introduction to MongoDB and Ruby                       ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                     FUL...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                          Introduction to MongoDB and Ruby                        ...
10/19/12                                                           Introduction to MongoDB and Ruby                       ...
Upcoming SlideShare
Loading in …5
×

Introduction to MongoDB and Ruby

3,821 views

Published on

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

Introduction to MongoDB and Ruby

  1. 1. 10/19/12 Introduction to MongoDB and Ruby INTRODUCTION TO MONGODB AND RUBY Gary J. Murakami, Ph.D. Lead Engineer & Ruby Evangelist 10gen (the MongoDB company) gary.murakami@10gen.com Tuesday, September 25, 2012 https://github.com/gjmurakami-10gen/talk-mother-lodefile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 1/26
  2. 2. 10/19/12 Introduction to MongoDB and Ruby THE CALIFORNIA GOLD RUSH OF 1849file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 2/26
  3. 3. 10/19/12 Introduction to MongoDB and Rubyfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 3/26
  4. 4. 10/19/12 Introduction to MongoDB and Ruby MONGODB: RUBYS MOTHER LODE Ruby is a gem of a programming language dynamic, multi-paradigm expressive, powerful, enjoyable MongoDB is the mother lode of DBs BSON documents, high-performance indexing, replication, sharding flexible, powerful, enjoyable Ruby and MongoDB are well matchedfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 4/26
  5. 5. 10/19/12 Introduction to MongoDB and Ruby OUTFITTING BEYOND YOUR PICKAXE BOOK MAC OS X - MONGODB QUICKSTART $be udt rw pae $be isalmnob rw ntl ogd $mno ogd RUBY DRIVER SETUP - RUBY LANGUAGE CENTER $gmisalmno e ntl og $gmisalbo_x e ntl snet Gemfile gmog e mno gmsnet e bo_x RUBYMINE IDE includes support for Mongoid and MongoDBfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 5/26
  6. 6. 10/19/12 Introduction to MongoDB and Ruby LETS DIG IN TWITTER FRIENDS - LOAD USERS rqieog eur mno rqietplet eur htcin rqiesn eur jo sre_ae=AG[]| ayuaai cennm RV0 | grmrkm finsisui="tp:/p.wte.o//red/d.sncro=1sre_ae#sre_ae" red_d_r hts/aititrcm1finsisjo?usr-&cennm={cennm} fin_d =JO.as(TPletgtfinsisui.oy[is] redis SNpreHTCin.e(red_d_r)bd)d a few lines of Ruby gets and parses JSON data from the web - twitter API libraries/gems mongo, bson, bson_ext httpclient, json language capability method call chainingfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 6/26
  7. 7. 10/19/12 Introduction to MongoDB and Ruby TWITTER FRIENDS - LOAD USERS (CONTINUED) cneto =Mno:oncinnw oncin og:Cneto.e d =cneto[titr] b oncinwte cleto =d[ues] olcin bsr fin_d.ahsie10 d |d|#bs patcs redisec_lc(0) o is et rcie ueslou_r ="tp:/p.wte.o//sr/okpjo?sri={d.on} sr_okpui hts/aititrcm1ueslou.snue_d#isji(,)" rsos =HTCin.e(sr_okpui epne TPletgtueslou_r) dc =JO.as(epnebd)#hg-ee ojcs-Aryo Hse os SNprersos.oy ihlvl bet ra f ahs dc.ah|o|dci =dcd] #ue sple _d osec{dc o[_d] o[i} sr upid i cleto.netdc,:ae= tu)#n shm!-bl isr -bs patcs olcinisr(os sf > re o cea uk net et rcie ed n pt "sr:{olcincut" us ues#cleto.on} a small number of lines load JSON data from the web into a collection DB connection, database, collection docs - Ruby Array of Hashes from JSON parsing doc primary _id set by user or driver collection bulk insert and safe mode collection countfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 7/26
  8. 8. 10/19/12 Introduction to MongoDB and Ruby HITTING PAYDIRT TWITTER TOP 10 USERS BY FOLLOWERS COUNT ues=Mno:oncinnwwte[ues] sr og:Cneto.e[titr]sr uesfn(} :ot= {floescut - } :ii = 1)ec d |o| sr.id{, sr > olwr_on: 1 , lmt > 0.ah o dc pt dcvle_tolwr_on cennm.on"t) us o.ausa(floescut, sre_ae)ji(" ed n a simple query with sort and limit find - cursor enumerable expressed in Ruby code driver serializes into BSON and Mongo Wire Protocol no text language generation or parsing 1023 491 Dobx rpo 202 59 MnoB ogD 97 04 ocn so 84 18 risof alcn 70 32 sf3 p1 47 34 1gn 0e 25 29 khdrw cooo 12 95 rt i 19 25 der mr 14 18 arnekan aohcmnfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 8/26
  9. 9. 10/19/12 Introduction to MongoDB and Ruby DIGGING DEEPER TWITTER - LOAD TWEETS (SIMPLIFIED) d =Mno:oncinnwwte b og:Cneto.e[titr] ues=d[ues] sr bsr tet =d[tet wes bwes] tet.nueidxsri #dtntto t seiysbil wesesr_ne(ue.d) o oain o pcf ufed uesfn(} {ils {d tu,sre_ae tu,snei:tu})ec d |sr sr.id{, fed: i: re cennm: re ic_d re}.ah o ue| titrue_ieieui= "tp:/p.wte.o//ttssue_ieiejo?sri={s wte_srtmln_r hts/aititrcm1saue/srtmln.snue_d#u e[i}cut20icuerstu&otiuo_eal=re rd]&on=0&nld_t=recnrbtrdtistu" titrue_ieieui+ "ic_d#ue[snei} i ue[snei wte_srtmln_r = snei={sric_d]" f sric_d] rsos =HTCin.e(wte_srtmln_r) epne TPletgttitrue_ieieui dc =JO.as(epnebd)#hg-ee ojcs os SNprersos.oy ihlvl bet dc.ah|o|dci =dcd] #ue sple _d osec{dc o[_d] o[i} sr upid i tet.netdc,:otneo_ro = tu)#bl isr wesisr(os cniu_nerr > re uk net uesudt(_d ue[_d],{$e= {snei= dc.atd]} sr.pae{i: sri} st > ic_d > osls[i}) pt tet.on(qey= {ue.d = ue[i} us wescut:ur > sri > srd]) ed n pt "wes#tet.on} us tet:{wescut" a small number of lines loads and records tweets indexing - secondary indexes retrieving a subset of fields update - record since_id count with query selectorfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 9/26
  10. 10. 10/19/12 Introduction to MongoDB and Ruby TWITTER TOP 10 TWEETS FOR MONGODB BY RETWEET_COUNT tet =Mno:oncinnwwte[tet wes og:Cneto.e[titr]wes] sre_ae=ogD cennm MnoB tet.id{ue.cennm= sre_ae,:ot= {rtetcut - } :ii = 1)ec wesfn(srsre_ae > cennm} sr > ewe_on: 1 , lmt > 0.ah d |o| o dc pt dcvle_tewe_on et)ji(" us o.ausa(rtetcut, tx.on"t) ed n a simple query with a query selector, sort and limit MongoDB query language dot notation for nested sub-document fields 12#ogD v. rlae ht:/.os6z7 7 MnoB 22 eesd tp/tc/NRcD 7 #ogD2o2ofcal rlae,w Avne AgeainFaeok MliDt Cne Dpomn 7 mnoBdt fiily eesd / dacd grgto rmwr, ut-aa etr elyet +60nwfaue ht:/.oDSWw 0 e etrs tp/tc/MdGN 5 Anucn fe oln MnoBcass ht:/.oRobl 9 nonig re nie ogD lse. tp/tc/IA77 3 R @mkl:Itouigmnop afotn fr#ogD qeypoie hts/tc/OSsW 0 T jioa nrdcn ogq: rned o MnoBs ur rflr tp:/.oRCz6 2 Sg u frfe oln MnoBcusssatn i Otbrht:/.oI6QN 9 in p o re nie ogD ore trig n coe tp/tc/m84M 2 100 hv sge u frfe MnoBtann snetecusswr anucdtdyht:/.o 4 ,0+ ae ind p o re ogD riig ic h ore ee none oa tp/tc /IxHt oNf1 2 HwDse bitabgdt pafr o asatpbde uigMnoB ht:/.o8Wy7 2 o iny ul i aa ltom n tru ugt sn ogD. tp/tc/q5cT 2 #ogD2o2 aalbefrdwla.ht:/.okGUh 1 mnoBdt, vial o onod tp/tc/chDI 1 MnoBSadn Vsaie ht:/.ooI0j aohrpouto #0eLb 9 ogD hrig iulzr tp/tc/nG8v nte rdc f 1gnas 1 MnoBJv Die 290rlae ht:/.oUvoY 6 ogD aa rvr .. eesd tp/tc/DZAVfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 10/26
  11. 11. 10/19/12 Introduction to MongoDB and Ruby STRIKING GOLD BSON - BINARY JSON http://bsonspec.org/ {BO" [aeoe,50,18] "SN: "wsm" .5 96} → "x1x0x0x0 3000 0BO0 x4SNx0 &x0x0x0 000 00x0x8x0x0x0wsm0 x200000aeoex0 01x0333x4 x103331@ 12x0x2x7x0x0 x00c000 0 x0 0" x0 1. base types extended to match modern programming languages 2. transfer over wire - Mongo Wire Protocol request payload includes query and documents to server response and documents to client 3. store on server disk 4. index on server fast - no parsing, no server-side data repackaging to diskfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 11/26
  12. 12. 10/19/12 Introduction to MongoDB and Ruby REPLICA SETS multiple database server nodes one primary member, many secondary membersfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 12/26
  13. 13. 10/19/12 Introduction to MongoDB and Ruby high availability auto-failover via election by majority write concern read scaling - read preferences backups upgrades use (hidden) replica for secondary workload analytics data-processing integration with external systemsfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 13/26
  14. 14. 10/19/12 Introduction to MongoDB and Ruby SHARDING horizontal scaling across multiple nodesfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 14/26
  15. 15. 10/19/12 Introduction to MongoDB and Ruby horizontal scaling across multiple nodes automatic balancing failover - each shard can be a replica set shard key - partition by range chunks up to 64MB routing processes - mongosfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 15/26
  16. 16. 10/19/12 Introduction to MongoDB and Ruby NUGGETS - MORE MONGODB FEATURES mongo shell JavaScript, native for server-side processing highly recommended for development and administration map/reduce batch processing of data and aggregation operations aggregation framework calculate aggregated values without having to use map-reduce pipeline, operators, expressions GridFS specification for storing large files in MongoDB chunks usually 256KB Capped collections fixed sized collections that have a very high performance auto-FIFO age- out feature Hadoop connector reading MongoDB data into Hadoop as well as writing map/reduce results back outfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 16/26
  17. 17. 10/19/12 Introduction to MongoDB and Ruby FULL-BORE MINING WITH WEB FRAMEWORKS Ruby on Rails, Sinatra, etc. MONGOMAPPER ODM - JOHN NUNEMAKER casUe ls sr icueMnoapr:ouet nld ogMpe:Dcmn ky:ae Srn e nm, tig ky:g, Itgr e ae nee mn :obe ay hbis ed n casHby ls ob icueMnoapr:meddouet nld ogMpe:EbdeDcmn ky:ae e nm, Srn tig ky:tre,Tm e satd ie ed n ue =Ue.e(nm = rno sr srnw:ae > Badn) ue.obe.ul(nm = rgamn :tre = 1.er.g) srhbisbid:ae > Pormig, satd > 0yasao ue.ae srsv! Ue.hr(nm = rno.is srwee:ae > Badn)frt simplified model different from ActiveRecord good example for Ruby metaprogramming small is beautifulfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 17/26
  18. 18. 10/19/12 Introduction to MongoDB and Ruby MONGOID ODM - DURRAN JORDAN casAts ls rit icueMnod:ouet nld ogi:Dcmn fed:ae tp:Srn il nm, ye tig ebd_ay:ntuet mesmn isrmns ed n casIsrmn ls ntuet icueMnod:ouet nld ogi:Dcmn fed:ae tp:Srn il nm, ye tig ebde_n:rit meddi ats ed n sd=Ats.hr(ae "y Vcos)btenae 1.2)frt y ritweenm: Sd iiu".ewe(g: 8.5.is sdisrmnscet(ae "as) y.ntuet.raenm: Bs" sdwt(aaae "ad" ssin "akp)sv! y.ihdtbs: bns, eso: bcu".ae documents persistence querying ActiveModel relations (belongs_to, has_one, has_many, habtm) nested attributes validation indexing extrasfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 18/26
  19. 19. 10/19/12 Introduction to MongoDB and Ruby MOPED DRIVER - BERNERD SCHAEFER ssin=Mpd:eso.e( "2...:71"] eso oe:Ssinnw[ 17001207 ) ssinue"cots" eso.s eh_et ssinwt(ae tu)d |ae eso.ihsf: re o sf| sf[atss.netnm:"y Vcos) ae:rit]isr(ae Sd iiu" ed n ssin:rit]fn(ae "y Vcos)udt($uh= {isrmns {nm:"as }) eso[atss.idnm: Sd iiu".pae:ps > ntuet: ae Bs" } a MongoDB driver for Ruby which exposes a simple, elegant, and fast API fast, thread-safe, uses sidekiq pure Ruby - no C extensions metaprogramming - extends standard classes Ruby > 1.9 used by Mongoid 3file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 19/26
  20. 20. 10/19/12 Introduction to MongoDB and Ruby JRUBY The Best of the JVM High performance Real threading - multi-core parallelism Vast array of libraries Its Just Ruby Ruby 1.8.7 & 1.9.2 compatible Both versions in single install bson_java Mongo Hadoop connectorfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 20/26
  21. 21. 10/19/12 Introduction to MongoDB and Ruby FOOLS GOLD - RDBMS CASE STUDY ADOBE PHOTOSHOP LIGHTROOM "Lightroom 3 Catalog.lrcat" 0.5 GB 66 Sqlite3 tables 20 tables with column image INTEGER can be loaded with 40 lines of Ruby can be reduced to 1 image collection THE COST OF USING RDBMS FOR PERSISTENCE RDBMS flattened and normalized our data into rigid tables RDBMS costs O(10**6) - O(1000000) and more with many joins Memory O(10**9/sec) - O(1000000000) RDBMS O(10**3/sec) - O(1000)file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 21/26
  22. 22. 10/19/12 Introduction to MongoDB and Ruby THE MOTHER LODE MONGODB SUMMARY replica sets for high availability sharded clusters for scaling very high performance secondary indices BSON is fast for transfer, storage, queries server-side processing map-reduce, aggregation Hadoop connector very low cost - open source - communityfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 22/26
  23. 23. 10/19/12 Introduction to MongoDB and Ruby MONGODB CONCLUSION enjoyable software development 10gen and Moped Ruby drivers MongoMapper and Mongoid ODMs no schema migrations forced well-matched to programming languages bigger data Ruby and objectsfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 23/26
  24. 24. 10/19/12 Introduction to MongoDB and Ruby ANNOUNCEMENT AND QUESTIONS MONGODB AND RUBY BLOGGING CONTEST Objective Write about best practices for using MongoDB with Ruby. Post your blog posts in the comments section of this blog post by October 10 to be entered to win. http://blog.10gen.com/post/32158163299/mongodb-and-ruby-blogging- contest Google: mongodb and ruby blogging contestfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 24/26
  25. 25. 10/19/12 Introduction to MongoDB and Ruby THANK YOU AND MAY YOU STRIKE GOLD { nm:"ayJ Mrkm,P.., ae Gr . uaai hD" tte "edEgne adRb Eagls" il: La nier n uy vneit, cmay "0e (h MnoBcmay" opn: 1gn te ogD opn), poe "-6-3-85x05, hn: 1862781 81" mbl:"-0-8-61, oie 1987762" eal "aymrkm@0e.o" mi: gr.uaai1gncm, i:"juaai(I), m gmrkm AM" titr "GrMrkm" wte: @ayuaai, bo:"ryhswie.lgptcm, lg gagotrtrboso.o" wbie "w.oelog, est: wwnbl.r" lnei:"w.ikdncmpbgr-uaai13a37, ikdn wwlnei.o/u/aymrkm//6/2" fcbo:"aeokcmgr..uaai aeok fcbo.o/ayjmrkm" } https://github.com/gjmurakami-10gen/talk-mother-lode git://github.com/gjmurakami-10gen/talk-mother-lode.git MongoDB Ruby Language Center Github mongodb/mongo-ruby-driverfile:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 25/26

×