MongoDB                 At MercadoLibreWh at h ave we le a rned
OverviewAbout MeLiWhy MongoDB?Use casesLessons LearnedWrapping Up
MeLi - Platform
MeLi - Facts!20M requests / minute 50K requests / sec (peaks) 4 Gb bandwidth / sec
MeLi - Facts!20M requests / minute 50K requests / sec (peaks) 4 Gb bandwidth / secOur farm +1000 physical server +6000 vir...
MeLi - InsideLot of fun here playing with new stuff :P
MeLi - InsideLot of fun here playing with new stuff :POur NoSQL stack Redis Memcached Hadoop
MeLi - InsideLot of fun here playing with new stuff :POur NoSQL stack Redis Memcached Hadoop .... and of course
Why MongoDB?Horizontal ScalingHigh AvailabilityPerformanceFlexible Data!
Why MongoDB?Horizontal ScalingHigh AvailabilityPerformanceFlexible Data!
Why MongoDB?(... and more)   JavaScript(‘ing)   Polyglot   Open Source   Awesome Community   Support
Support :-)
Use Cases   DB      Documents Social     +700M Orders     +23MFeedback     +3M Emails     +61M Items     +200MLogging     ...
Use Case: SocialStore Meli’s users social dataLOT of data (friends, likes, ...)> 500 Gb> 700 M documents12 shards servers ...
Use Case: Social                                     Sp                                        ec                         ...
Use Case: Items1 Item!
Use Case: Items1 Item!
Use Case: Items1 Item!
Use Case: Items     1 Item!N x N x N ... > 100k rows
Use Case: Items1 Item!
Use Case: Logging                            Capped Collections!Info logs  RabbitMQ messages  Track certain use casesError...
Lessons LearnedStorage  File System    NSF: Bad choice, Poor results    Ext4: Far much better  Mount Options  Reducing Dis...
Lessons LearnedVM’s  Shard instances   4 cores   >= 16 Gb ram  Config server   low cpu consumption   1Gb ram / 1Gb storage...
Lessons LearnedMonitoring Disk usage (with zenoss) Ping health check    curl --connect-timeout 3          --write-out %{ht...
Lessons LearnedMonitoring                MMS - It’s free!
Lessons LearnedPlan your actions!   Shard key   Upgrades   Cfg Servers                 +   bad decisions = Lot of rework  ...
Lessons LearnedPlan your actions!   Shard key   Upgrades   Cfg Servers                 +   bad decisions = Lot of rework  ...
Wrapping Up!Meli loves MongoDB!
The End            Questions?   {         name     : "Pablo Molnar",          title    : "Software Engineer",        work ...
Upcoming SlideShare
Loading in …5
×

MongoDB at MercadoLibre

1,953 views

Published on

Presentation for MongoDB Buenos Aires 2012

0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,953
On SlideShare
0
From Embeds
0
Number of Embeds
20
Actions
Shares
0
Downloads
19
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • MongoDB at MercadoLibre

    1. 1. MongoDB At MercadoLibreWh at h ave we le a rned
    2. 2. OverviewAbout MeLiWhy MongoDB?Use casesLessons LearnedWrapping Up
    3. 3. MeLi - Platform
    4. 4. MeLi - Facts!20M requests / minute 50K requests / sec (peaks) 4 Gb bandwidth / sec
    5. 5. MeLi - Facts!20M requests / minute 50K requests / sec (peaks) 4 Gb bandwidth / secOur farm +1000 physical server +6000 virtual instances
    6. 6. MeLi - InsideLot of fun here playing with new stuff :P
    7. 7. MeLi - InsideLot of fun here playing with new stuff :POur NoSQL stack Redis Memcached Hadoop
    8. 8. MeLi - InsideLot of fun here playing with new stuff :POur NoSQL stack Redis Memcached Hadoop .... and of course
    9. 9. Why MongoDB?Horizontal ScalingHigh AvailabilityPerformanceFlexible Data!
    10. 10. Why MongoDB?Horizontal ScalingHigh AvailabilityPerformanceFlexible Data!
    11. 11. Why MongoDB?(... and more) JavaScript(‘ing) Polyglot Open Source Awesome Community Support
    12. 12. Support :-)
    13. 13. Use Cases DB Documents Social +700M Orders +23MFeedback +3M Emails +61M Items +200MLogging +332M
    14. 14. Use Case: SocialStore Meli’s users social dataLOT of data (friends, likes, ...)> 500 Gb> 700 M documents12 shards servers : 4 replica setEach replica set = 3 servers80Gb per instance
    15. 15. Use Case: Social Sp ec @T ial ioB thk or stStore Meli’s users social data rac o hoLOT of data (friends, likes, ...)> 500 Gb> 700 M documents12 shards servers : 4 replica setEach replica set = 3 servers80Gb per instance
    16. 16. Use Case: Items1 Item!
    17. 17. Use Case: Items1 Item!
    18. 18. Use Case: Items1 Item!
    19. 19. Use Case: Items 1 Item!N x N x N ... > 100k rows
    20. 20. Use Case: Items1 Item!
    21. 21. Use Case: Logging Capped Collections!Info logs RabbitMQ messages Track certain use casesErrors logs Something bad happen... Recuperable errors with retry handler
    22. 22. Lessons LearnedStorage File System NSF: Bad choice, Poor results Ext4: Far much better Mount Options Reducing Disk IO by mounting partitions with noatime and nodiratime Track disk performance iostat -xm 2 (%util column < 100%)
    23. 23. Lessons LearnedVM’s Shard instances 4 cores >= 16 Gb ram Config server low cpu consumption 1Gb ram / 1Gb storage Mongos runs along with the webserver (client) 1 mongos x webserver
    24. 24. Lessons LearnedMonitoring Disk usage (with zenoss) Ping health check curl --connect-timeout 3 --write-out %{http_code} --silent --output /dev/null http://localhost:28017/_status
    25. 25. Lessons LearnedMonitoring MMS - It’s free!
    26. 26. Lessons LearnedPlan your actions! Shard key Upgrades Cfg Servers + bad decisions = Lot of rework ...
    27. 27. Lessons LearnedPlan your actions! Shard key Upgrades Cfg Servers + bad decisions = Lot of rework ... Study Think Plan Hands On!
    28. 28. Wrapping Up!Meli loves MongoDB!
    29. 29. The End Questions? {  name     : "Pablo Molnar",   title    : "Software Engineer", work    : "MercadoLibre", location : "Buenos Aires", twitter  : "@pablomolnar" }

    ×