- Gilt Groupe is a flash sales company that sells apparel, home goods, and other items through daily deals. - Gilt has transitioned from a monolithic architecture to a service-oriented approach using microservices like user, feature configuration, and favorite brands services. - MongoDB is used at Gilt for user profiles, feature flag configuration, and storing favorite brands. The Java driver and Morphia/Casbah libraries help with development. - Best practices include connection pool tuning, minimizing impact of index builds, using short field names, and using explain() during development.

1. @
September 2012
Sean Sullivan

2. About me
• software engineer
• ~18 months at Gilt
• work in Gilt’s Portland office
• back office applications

3. Agenda
• Gilt Groupe
• Gilt’s technology stack
• MongoDB at Gilt
• Q&A

4. Gilt Groupe
http://www.gilt.com

5. flash sales

6. Everyday at 12 noon

7. what does Gilt sell?

8. Apparel

11. Kids merchandise

13. Home furnishings

15. Food

17. Local deals

19. Travel

22. Gilt technology stack

24. Gilt architecture

25. 2007 2012
monolithic service-oriented
application approach

26. 2007

27. 2012
service A
service B
service C
legacy
web app
service D
gilt.com service E

28. Data storage @ Gilt

29. @

30. Why MongoDB?
• Ease of use
• Horizontal scaling
• High availability
• Automatic failover

31. Why MongoDB?
• Stability
• Support
• Drivers

32. • MongoDB 2.0
• sharded and non-sharded data
• Solid State Drives
• MMS for monitoring

33. Use case #1: user profiles

34. Challenges
• user data in Postgres
• legacy Rails app expects to find user data in
Postgres
• we want Gilt’s customer facing application code
to retrieve user data from MongoDB

35. Solution
• keep user data in both MongoDB and
Postgres
• replicate table changes from Postgres to
MongoDB

36. Replicating data
user service
legacy
web app
replication

37. Use case #2:
feature configuration
Goal
manage the release of new features on gilt.com
How
feature configuration persisted in MongoDB

38. Rolling out a new feature
1. deploy new application code to production
2. enable feature for Gilt Tech employees
3. ... then enable for all Gilt employees
4. ... then enable for a subset of users
5. gradually ramp up to 100% of users

39. Feature configuration
Feature
config
service
gilt.com

40. Use case #3:
favorite brands

41. Favorite brands
AJAX user
preference
service

42. Application development

43. mongo-java-driver Casbah
Morphia

44. MongoDB Java driver
• main class: com.mongodb.Mongo
• Mongo object maintains a pool of connections
• Latest version: 2.9.1

45. MongoDB Java driver
connection pool tuning
MongoOptions opts = new MongoOptions();
// example values
opts.connectionsPerHost = 50;
opts.threadsAllowedToBlockForConnectionMultiplier = 5;

46. com.mongodb.WriteConcern
• NORMAL
• SAFE
• REPLICAS_SAFE
• MAJORITY
• FSYNC_SAFE
• JOURNAL_SAFE

47. Morphia library

48. Morphia
• object-document mapper for Java
• built on top of mongo-java-driver
• map fields using Java annotations

49. // Morphia example
import com.google.code.morphia.annotations.*;
@Entity(value="features", noClassnameStored = true)
public class Feature {
@Id
ObjectId featureId;
@Property("feature_key") @Indexed(unique=true)
String featureKey;
@Property("release_to_percentage")
int releaseToPercentage;
}

50. http://code.google.com/p/morphia/

51. Casbah library

52. Casbah
• Scala toolkit for MongoDB
• built on top of the mongo-java-driver

53. Casbah
• Scala idioms
• Scala collections
• fluid query syntax

54. https://twitter.com/max4f/status/230503836958199808

55. Gilt Scala code

56. trait MongoWriteConcern {
def withSession[T](f: => T)(implicit mongoDb: MongoDB) {
mongoDb.requestStart()
mongoDb.requestEnsureConnection()
try {
f
} finally {
mongoDb.requestDone()
}
}
}

57. Best Practices

58. Best practices
Connection tuning
explicitly configure Mongo Java Driver
connection pool size

59. Best practices
Use caution when creating new indexes
“creating a new index on a production mongo server can
basically cause it to stop working while it builds the index”
(source: Gilt production incident report)

60. Minimizing impact of
building a new index
• remove one secondary from replica set
• create/rebuild index on this instance
• rejoin replica set
• repeat on all remaining secondaries
• run rs.stepDown() on primary member
http://docs.mongodb.org/manual/administration/indexes/#index-building-replica-sets

61. Best practices
use short names for fields to avoid wasted space
{ {
city: “Portland”, ci: “Portland”,
state: “Oregon”, vs st: “Oregon”,
country: “US” co: “US”
} }
http://christophermaier.name/blog/2011/05/22/MongoDB-key-names

62. Best practices
use explain() during development
db.collection.find(query).explain()

63. Best practices
use caution when choosing a shard key
“It is generally not a good idea to use the default
ObjectId as the shard key”
source: http://stackoverflow.com/questions/9164356/sharding-by-objectid-is-it-the-right-way

64. Future
MongoDB 2.2
• standardized read preferences
• tag aware sharding
Mongo Java driver 2.9.x
• enhanced read preference support
(see: com.mongodb.ReadPreference)

65. Gilt Groupe is hiring!
http://techjobs.gilt.com

66. Questions?

67. ssullivan@gilt.com
@tinyrobots

68. The end

69. Bonus slides

70. Gilt tech talks
• Apache Camel and MongoDB @ Gilt
http://bit.ly/OYO37K
• Deploying new features @ Gilt
http://slidesha.re/OoCYfd
• Voldemort @ Gilt
http://bit.ly/b9Qhib