Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Mongodb at-gilt-groupe-seattle-2012-09-14-final

1,547 views

Published on

Published in: Technology, Education
  • Be the first to comment

Mongodb at-gilt-groupe-seattle-2012-09-14-final

  1. 1. @September 2012 Sean Sullivan
  2. 2. About me• software engineer• ~18 months at Gilt• work in Gilt’s Portland office• back office applications
  3. 3. Agenda• Gilt Groupe• Gilt’s technology stack• MongoDB at Gilt• Q&A
  4. 4. Gilt Groupe http://www.gilt.com
  5. 5. flash sales
  6. 6. Everyday at 12 noon
  7. 7. what does Gilt sell?
  8. 8. Apparel
  9. 9. Kids merchandise
  10. 10. Home furnishings
  11. 11. Food
  12. 12. Local deals
  13. 13. Travel
  14. 14. Gilt technology stack
  15. 15. Gilt architecture
  16. 16. 2007 2012monolithic service-orientedapplication approach
  17. 17. 2007
  18. 18. 2012 service A service B service C legacy web app service Dgilt.com service E
  19. 19. Data storage @ Gilt
  20. 20. @
  21. 21. Why MongoDB?• Ease of use• Horizontal scaling• High availability• Automatic failover
  22. 22. Why MongoDB?• Stability• Support• Drivers
  23. 23. • MongoDB 2.0• sharded and non-sharded data• Solid State Drives• MMS for monitoring
  24. 24. Use case #1: user profiles
  25. 25. Challenges• user data in Postgres• legacy Rails app expects to find user data in Postgres• we want Gilt’s customer facing application code to retrieve user data from MongoDB
  26. 26. Solution• keep user data in both MongoDB and Postgres• replicate table changes from Postgres to MongoDB
  27. 27. Replicating datauser service legacy web app replication
  28. 28. Use case #2: feature configurationGoal manage the release of new features on gilt.comHow feature configuration persisted in MongoDB
  29. 29. Rolling out a new feature 1. deploy new application code to production 2. enable feature for Gilt Tech employees 3. ... then enable for all Gilt employees 4. ... then enable for a subset of users 5. gradually ramp up to 100% of users
  30. 30. Feature configuration Feature config servicegilt.com
  31. 31. Use case #3:favorite brands
  32. 32. Favorite brandsAJAX user preference service
  33. 33. Application development
  34. 34. mongo-java-driver CasbahMorphia
  35. 35. MongoDB Java driver• main class: com.mongodb.Mongo• Mongo object maintains a pool of connections• Latest version: 2.9.1
  36. 36. MongoDB Java driverconnection pool tuningMongoOptions opts = new MongoOptions();// example valuesopts.connectionsPerHost = 50;opts.threadsAllowedToBlockForConnectionMultiplier = 5;
  37. 37. com.mongodb.WriteConcern • NORMAL • SAFE • REPLICAS_SAFE • MAJORITY • FSYNC_SAFE • JOURNAL_SAFE
  38. 38. Morphia library
  39. 39. Morphia• object-document mapper for Java• built on top of mongo-java-driver• map fields using Java annotations
  40. 40. // Morphia exampleimport com.google.code.morphia.annotations.*;@Entity(value="features", noClassnameStored = true)public class Feature { @Id ObjectId featureId; @Property("feature_key") @Indexed(unique=true) String featureKey; @Property("release_to_percentage") int releaseToPercentage;}
  41. 41. http://code.google.com/p/morphia/
  42. 42. Casbah library
  43. 43. Casbah• Scala toolkit for MongoDB• built on top of the mongo-java-driver
  44. 44. Casbah• Scala idioms• Scala collections• fluid query syntax
  45. 45. https://twitter.com/max4f/status/230503836958199808
  46. 46. Gilt Scala code
  47. 47. trait MongoWriteConcern { def withSession[T](f: => T)(implicit mongoDb: MongoDB) { mongoDb.requestStart() mongoDb.requestEnsureConnection() try { f } finally { mongoDb.requestDone() } }}
  48. 48. Best Practices
  49. 49. Best practicesConnection tuningexplicitly configure Mongo Java Driverconnection pool size
  50. 50. Best practices Use caution when creating new indexes “creating a new index on a production mongo server canbasically cause it to stop working while it builds the index” (source: Gilt production incident report)
  51. 51. Minimizing impact of building a new index • remove one secondary from replica set • create/rebuild index on this instance • rejoin replica set • repeat on all remaining secondaries • run rs.stepDown() on primary memberhttp://docs.mongodb.org/manual/administration/indexes/#index-building-replica-sets
  52. 52. Best practices use short names for fields to avoid wasted space{ { city: “Portland”, ci: “Portland”, state: “Oregon”, vs st: “Oregon”, country: “US” co: “US”} } http://christophermaier.name/blog/2011/05/22/MongoDB-key-names
  53. 53. Best practicesuse explain() during developmentdb.collection.find(query).explain()
  54. 54. Best practicesuse caution when choosing a shard key“It is generally not a good idea to use the defaultObjectId as the shard key”source: http://stackoverflow.com/questions/9164356/sharding-by-objectid-is-it-the-right-way
  55. 55. FutureMongoDB 2.2 • standardized read preferences • tag aware shardingMongo Java driver 2.9.x • enhanced read preference support (see: com.mongodb.ReadPreference)
  56. 56. Gilt Groupe is hiring! http://techjobs.gilt.com
  57. 57. Questions?
  58. 58. ssullivan@gilt.com@tinyrobots
  59. 59. The end
  60. 60. Bonus slides
  61. 61. Gilt tech talks• Apache Camel and MongoDB @ Gilt http://bit.ly/OYO37K• Deploying new features @ Gilt http://slidesha.re/OoCYfd• Voldemort @ Gilt http://bit.ly/b9Qhib

×