Your SlideShare is downloading. ×
0
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Big and Fat: Using MongoDB with Deep and Diverse Data
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Big and Fat: Using MongoDB with Deep and Diverse Data

1,428

Published on

Delivered at MongoDC

Delivered at MongoDC

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,428
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
12
Comments
0
Likes
3
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Big and Fat Using MongoDB with deep and diverse datasets: A case study
  • 2. About me • My name is Jeremy McAnally • “Software architect” at Intridea • Write a lot of books, OSS, etc. • http://github.com/jm • http://twitter.com/jm • http://railsupgradehandbook.com • http://wickhamhousebrand.com
  • 3. MongoDB analytics just like mom used to make. http://mongobase.com
  • 4. Preface The Application™
  • 5. Disclaimer We moved to (mostly) sql.
  • 6. YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE YAK SHAVE
  • 7. Lesson 1 Abstraction is a double- edged sword.
  • 8. Abstract away! Talking to all data (no matter the source) the same way will keep you sane.
  • 9. !"#$"%&%'()*+,,*!#$(-#.#/!0#12)3+345%6%789'%!"#$":2; !"#$"-#</=%>?%@!@ %%A?"0"%&%>B-/?CC#/0D?E1FA?"0"F;-GDE>1,!"#$HD>%&I%!JFD>FK; %%L%J---K %%/?MM#E0"%&%>B-/?CC#/0D?E1F/?MM#E0"F;-GDE>12NO=#$#2%&I%2"!M 10=D"-<>MDEH/?!E0P%0=D"-M?>#$<0?$H/?!E0;%&&%Q2; #E>
  • 10. !"#$"%&%R"#$-<CC !"#$"-#</=%>?%@!@ %%A?"0"%&%S?"0-GDE>1,!"#$HD>%&I%!-D>; %%L%J---K %%/?MM#E0"%&%4?MM#E0-O=#$#12"!M10=D"-<>MDEH/?!E0P% %%%%%%%%%%%%%%%%%%%%%%%%%%%%0=D"-M?>#$<0?$H/?!E0;%&&%Q2; #E>
  • 11. !"#$"%&%R"#$-<CC !"#$"-#</=%>?%@!@ %%A?"0"%&%S?"0-GDE>1,!"#$HD>%&I%!-D>; %%L%J---K %%/?MM#E0"%&%4?MM#E0-OD0=HGDT#H0=DEU" #E>
  • 12. ...but wait! MongoDB has a lot of features that will perform better and be less (and often better) code.
  • 13. A=<$M</D"0"%&%VW S<0D#E0-<CC-#</=%>?%@A<0D#E0@ %%A<0D#E0-A$#"/$DA0D?E"-#</=%>?%@A$#"/$DA0D?E@ %%%%A=<$M</D"0"JA$#"/DA0D?E-E<M#K%@@&%X %%%%A=<$M</D"0"JA$#"/DA0D?E-E<M#K%Y&%Z %%#E> #E>
  • 14. A=<$M</D"0"%&%VW S<0D#E0-<CC-#</=%>?%@A<0D#E0@ %%A<0D#E0-A$#"/$DA0D?E"-#</=%>?%@A$#"/$DA0D?E@ %%%%A=<$M</D"0"JA$#"/DA0D?E-E<M#K%@@&%X %%%%A=<$M</D"0"JA$#"/DA0D?E-E<M#K%Y&%Z %%#E> #E>SLOW AS CRAP
  • 15. M<A%&%2G!E/0D?E1;V %%%%0=D"-A$#"/$DA0D?E"-G?$3</=1 %%%%%%G!E/0D?E1A;%V% %%%%%%%%#MD01A-E<M#P%V%/?!E0%,%Z%W;: %%%%W;W2% %% $#>!/#%&%2G!E/0D?E1[P%T;%V %%T<$%E!MB#$%&%X: %%G?$%T-G?$3</=1G!E/0D?E1;%V %%%%E!MB#$%Y&%TJDK-/?!E0: %%W;: %%$#0!$E%V%/?!E0%,%E!MB#$%W:% W2% %% A=<$M"%&%A<0D#E0"-M<AH$#>!/#1M<AP%$#>!/#;
  • 16. M<A%&%2G!E/0D?E1;V %%%%0=D"-A$#"/$DA0D?E"-G?$3</=1 %%%%%%G!E/0D?E1A;%V% %%%%%%%%#MD01A-E<M#P%V%/?!E0%,%Z%W;: %%%%W;W2% %% $#>!/#%&%2G!E/0D?E1[P%T;%V %%T<$%E!MB#$%&%X: %%G?$%T-G?$3</=1G!E/0D?E1;%V %%%%E!MB#$%Y&%TJDK-/?!E0: %%W;: %%$#0!$E%V%/?!E0%,%E!MB#$%W:% W2% %% A=<$M"%&%A<0D#E0"-M<AH$#>!/#1M<AP%$#>!/#;
  • 17. Lesson 2 Schema design matters.
  • 18. Lesson 2 Schema design matters. DATAMODEL
  • 19. Embedding works. Embedding documents is a smart decision in a lot of cases.
  • 20. )3+345%6%789'%A<0D#E0"%]^383%D>&_Z_: )3+345%6%789'%A$#"/$DA0D?E"%]^383%A<0D#E0HD>&_Z_: )3+345%6%789'%<AA?DE0M#E0"%]^383%A<0D#E0HD>&_Z_: )3+345%6%789'%/?E0</0"%]^383%A<0D#E0HD>&_Z_: )3+345%6%789'%/C<DM"%]^383%A<0D#E0HD>&_Z_: - - -
  • 21. ...but watch it. You can also hit a ton of performance and design issues.
  • 22. OUR GIANT DOCUMENT Mongo’s Pre-Allocated Space
  • 23. Patient Pharmacy “Reference” Pharmacy Search, listing, etc.
  • 24. Lesson 3 Don’t go nuts.
  • 25. Schemaless is fun! Having schemaless data has its own battery of advantages.
  • 26. Schemaless Joy
  • 27. Schemaless Joy • Transforming data models is a delight
  • 28. Schemaless Joy • Transforming data models is a delight • Formless data isn’t awkward
  • 29. Schemaless Joy • Transforming data models is a delight • Formless data isn’t awkward • Arbitrary embedding is awesome
  • 30. Schemaless Joy • Transforming data models is a delight • Formless data isn’t awkward • Arbitrary embedding is awesome • Building to work with schemaless data can lead to some really powerful app concepts
  • 31. ...but be wary. Going nuts will create headaches for you.
  • 32. Schemaless Pain
  • 33. Schemaless Pain • Weird app behavior
  • 34. Schemaless Pain • Weird app behavior • Huge, long-running data transformations
  • 35. Schemaless Pain • Weird app behavior • Huge, long-running data transformations • Annoying data transforms for development env’s
  • 36. Schemaless Pain • Weird app behavior • Huge, long-running data transformations • Annoying data transforms for development env’s • Difficult to version data models
  • 37. Lesson 4 Dig deep.
  • 38. I%>B-$!E4?MM<E>1V2"#$T#$)0<0!"2%,%ZW; V % 2T#$"D?E2%,%2Z-`-a2P % 2!A0DM#2%,%bcP % 2C?/<C5DM#2%,%25=!%d?T%Ze%_XZX%XZ,`b,ae% f'5gXQXX%13)5;2P % 2UC?B<C+?/[2%,%V % % 20?0<C5DM#2%,%bcXXQ_bXP % % 2C?/[5DM#2%,%Zh`X`XP % % 2$<0D?2%,%X-XXZeZ_eZchh_bXbXhc_ % WP % 2M#M2%,%V % % 2BD0"2%,%c`P % % 2$#"D>#E02%,%_P % % 2TD$0!<C2%,%_abcP % % 2"!AA?$0#>2%,%0$!#P % % 2M<AA#>2%,%X % WP % 2/?EE#/0D?E"2%,%V % % 2/!$$#E02%,%ZP % % 2<T<DC<BC#2%,%Zbbbb % WP
  • 39. 2?A/?!E0#$"2%,%V %%2DE"#$02%,%XP %%2i!#$(2%,%ZP %%2!A><0#2%,%XP %%2>#C#0#2%,%XP %%2U#0M?$#2%,%XP %%2/?MM<E>2%,%a W
  • 40. 2/?EE#/0D?E"2%,%V %%2/!$$#E02%,%ZP %%2<T<DC<BC#2%,%Zbbbb W
  • 41. j#$#M(g'/kE<CC("g'</l??[gS$?,m%n#$#M(M/<E<CC(N%M?EU?"0<0 /?EE#/0#>%0?,%Z_h-X-X-Z DE"#$0o"%i!#$(o"%!A><0#o"%>#C#0#o"%U#0M?$#o"%/?MM<E>o"%M<AA#>%%T"Dp#%%%%$#"%q%C?/[#>%q%D>.%MD""%%/?EE% %%%%%%%X%%%%%%%X%%%%%%%%X%%%%%%%%X%%%%%%%%%X%%%%%%%%%Z%%%%%%X%%%_abc%%%%%%a%%%%%%%%X%%%%%%%%%%X%%%%%Z% %%%%%%%X%%%%%%%X%%%%%%%%X%%%%%%%%X%%%%%%%%%X%%%%%%%%%Z%%%%%%X%%%_abc%%%%%%a%%%%%%%%X%%%%%%%%%%X%%%%%Z% %%%%%%%X%%%%%%%X%%%%%%%%X%%%%%%%%X%%%%%%%%%X%%%%%%%%%Z%%%%%%X%%%_abc%%%%%%a%%%%%%%%X%%%%%%%%%%X%%%%%Z% %%%%%%%X%%%%%%%X%%%%%%%%X%%%%%%%%X%%%%%%%%%X%%%%%%%%%Z%%%%%%X%%%_abc%%%%%%a%%%%%%%%X%%%%%%%%%%X%%%%%Z% %%%%%%%X%%%%%%%X%%%%%%%%X%%%%%%%%X%%%%%%%%%X%%%%%%%%%Z%%%%%%X%%%_abc%%%%%%a%%%%%%%%X%%%%%%%%%%X%%%%%Z% %%%%%%%X%%%%%%%X%%%%%%%%X%%%%%%%%X%%%%%%%%%X%%%%%%%%%Z%%%%%%X%%%_abc%%%%%%a%%%%%%%%X%%%%%%%%%%X%%%%%Z% %%%%%%%X%%%%%%%X%%%%%%%%X%%%%%%%%X%%%%%%%%%X%%%%%%%%%Z%%%%%%X%%%_abc%%%%%%a%%%%%%%%X%%%%%%%%%%X%%%%%Z% %%%%%%%X%%%%%%%X%%%%%%%%X%%%%%%%%X%%%%%%%%%X%%%%%%%%%Z%%%%%%X%%%_abc%%%%%%a%%%%%%%%X%%%%%%%%%%X%%%%%Z%
  • 42. >B-H<>MDE4?MM<E>1V%>D<U+?UUDEU%,%Z%W;
  • 43. >B-/!$$#E09A1; V%DEA$?U,%J%V%2?AD>2%,%aQ%P%2?A2%,%2i!#$(2%P%2E"2%,% 2G!E>B-A<$0D#"2%P %%%%%%%%%%%%%%2i!#$(2%,%2V%"/?$#%,%Z-X%W2%P%2DE+?/[2%,%Z%W %%%%%%%%%%K W
  • 44. I%>B-?AC?U-NM<DE-GDE>1; V%20"2%,%V%202%,%Z_bXXcaQccXXXP%2D2%,%Z%WP%2?A2%,%2D2P%2E"2%,%2MDEU V%20"2%,%V%202%,%Z_bXXcaQcbXXXP%2D2%,%Z%WP%2?A2%,%2E2P%2E"2%,%22P%2 V%20"2%,%V%202%,%Z_bXXcaQhbXXXP%2D2%,%Z%WP%2?A2%,%2E2P%2E"2%,%22P%2 V%20"2%,%V%202%,%Z_bXXcaQeZXXXP%2D2%,%Z%WP%2?A2%,%2D2P%2E"2%,%2MDEU V%20"2%,%V%202%,%Z_bXXcaQeZXXXP%2D2%,%_%WP%2?A2%,%2D2P%2E"2%,%2MDEU
  • 45. V%20"2%,% %%V%202%,%Z_bXXcaQccXXXP% %%%%2D2%,%Z% %%WP% %%2?A2%,%2D2P% %%2E"2%,%2MDEU-G??2P% %%2?2%,%V% %%%%%2HD>2%,%9Bn#/0r>12`/#`/#/#<BBZBcQZQeXXXXXZ2;P% %%%%%2GD#C>2%,%_% %%W% W
  • 46. That’s all I got. Questions?

×