Mongo db at_customink

1,731 views
1,629 views

Published on

Presented at MongoDC in June, 2011. This talk describes how we use MongoDB at CustomInk, some challenges we faced introducing the technology, and some lessons learned.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,731
On SlideShare
0
From Embeds
0
Number of Embeds
458
Actions
Shares
0
Downloads
20
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Tempfile https://github.com/mongodb/mongo-ruby-driver/blob/master/lib/bson/bson_ruby.rb#L153MongoDB maintains deleted lists of space within the datafiles when objects or collections are deleted. This space is reused but never freed to the operating system.http://www.mongodb.org/display/DOCS/Durability+and+Repair
  • Mongo db at_customink

    1. 1. @<br />@nathenharvey<br />Nathen Harvey<br />
    2. 2. Hello!<br />Nathen Harvey<br />Web Operations at CustomInk<br />nharvey@customink.com<br />@nathenharvey<br />Organize MongoDCMeetups<br />See you there next month!<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    3. 3. Thanks for attending<br />In the other track<br />Eliot Horowitz, CTO & Co-Founder, 10gen<br />Schema Design at Scale<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    4. 4. CustomInk<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    5. 5. CustomInk<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    6. 6. CustomInk<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    7. 7. CustomInk Customers<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    8. 8. CustomInk Customers<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    9. 9. CustomInk Customers<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    10. 10. CustomInk Customers<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    11. 11. Customer Reviews<br />Nathen Harvey<br />Nathen Harvey<br />
    12. 12. CustomInk Technology<br />Divide our technology into two classes:<br />ECommerce – pre-checkout and shopping cart<br />Operations Technology – post-check-out, fulfillment, etc.<br />Ruby on Rails<br />With some legacy Java<br />Approximately 30 different applications running in production<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    13. 13. CustomInk Databases<br />Primarily Oracle<br />Was the “right” choice 11 years ago when the company was started<br />No longer a good solution for us<br />MySQL<br />Percona Server with XtraDB<br />MongoDB<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    14. 14. MongoDB<br />Champions<br />Easy to set-up and configure<br />“Greenfield” projects with loose schema definitions<br />Makes development easier and developers happy<br />Limited amount of time between releases<br />Challenges<br />No prior experience with running production<br />Limited amount of time between releases<br />Unproven solution<br />Nathen Harvey<br />Nathen Harvey<br />
    15. 15. Problem<br />Complex Production Environments<br />OrchestratedServices<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    16. 16. Why Mongo?<br />Introduces company to MongoDB<br />Easily to modify document structure<br />Lightweight development<br />Easy to install and deploy<br />Capped Collections<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    17. 17. Central Logger<br />Subclasses Buffered Logger<br />It’s a gem loaded into Rails applications as a Railtie<br />Captures useful metadata<br />Use the MongoDB Ruby Driver<br />http://github.com/customink/central_logger<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    18. 18. Central Logger<br />
    19. 19. Record Structure<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    20. 20. Log Viewer<br />
    21. 21. Product Catalog<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    22. 22. Why Mongo?<br />Frequently changing requirements<br />GridFS<br />Wide variety of products with varied characteristics<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    23. 23. Case for Document-based Schema<br />@nathenharvey<br />Nathen Harvey<br />
    24. 24. Application Architecture<br />Rails application for managing the product catalog<br />Writes to MongoDB as it’s database<br />Utilizes GridFS for storing files<br />Uses Mongoid as the ODM<br />@nathenharvey<br />Nathen Harvey<br />
    25. 25. Implementation Flow<br />ECommerce<br />Application<br />Catalog<br />Application<br />MongoDB<br />Oracle<br />@nathenharvey<br />Nathen Harvey<br />
    26. 26. Next Steps<br />Continue to adopt MongoDB in the Operations Technology applications<br />Begin using MongoDB with ECommerce applications<br />Reporting artwork usage<br />Capturing JavaScript logs<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    27. 27. Automated Deployment<br />Chef is an open source systems integration framework built to bring the benefits of configuration management to your entire infrastructure.<br />We use Chef to manage our infrastructure and deploy MongoDB<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    28. 28. MongoDB Cookbook<br />Use Vagrant and VirtualBox as the development environment for our Cookbooks<br />Cookbook shared on Github and, soon, OpsCode Community Site.<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    29. 29. Monitoring Production<br />http://github.com/mzupan/nagios-plugin-mongodb<br />
    30. 30. Monitoring Production<br />Monit<br />
    31. 31. Lessons Learned<br />Capped Collections<br />Testing replica set failover<br />Schema free != design free<br />JSON in the DB, JQuery in the browser == happy developers<br />mongo_import could not handle carriage returns in CSV files<br />Unserializable data types<br />Data transformations on a large dataset still painful<br />Unless you’re sharding, map reduce probably slower and more complex than what you’re used to<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    32. 32. MongoDCMeetup<br />Join fellow MongoDB enthusiasts for a monthly meetup<br />Meetings are held at CustomInk in McLean, VA<br />http://www.meetup.com/Washington-DC-MongoDB-Users-Group/<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />
    33. 33. Thank You!<br />Nathen Harvey<br />Web Operations at CustomInk<br />nharvey@customink.com<br />@nathenharvey<br />Organize MongoDCMeetups<br />See you there next month!<br />@nathenharvey<br />Nathen Harvey<br />@nathenharvey<br />Nathen Harvey<br />

    ×