e-Com Product Catalog using
                   MongoDB


                               - Vishwas Bhagath
                   vbhagath@equalexperts.com



http://www.equalexperts.com/
Why MongoDB ?

   ●   Agile incremental releases with a short time to market
   ●   Feeds from multiple vendors containing products with
       varying schema
   ●   Complex queries on product attributes
   ●   Rich Content user experience
   ●   Capital risk mitigation


http://www.equalexperts.com/
How we approached...
    ●   Modeling and schema design
    ●   Querying
    ●   Referencing one document in another
    ●   Rich content using GridFS
    ●   Indexes
    ●   MongoDB challenges
    ➢    Locking mechanism
    ➢    Managing Schema and fat fingers


http://www.equalexperts.com/
Domain model
                                     {"_id" : ObjectId("5082626144ae3a687919c094"),
                                     "name" : "iPhone 5 Pop Blue Case",
                                     "canonicalName" : "iphone-5-pop-blue-case",
                                     "retailPrice" : 19.99,
                                     "productCode" : "G4IC542G",
                                     "category" : {
                                             "categoryCode" : "CAS",
                                             "name" : "Cases",
                                             "canonicalName" : "cases"
                                     },
                                     "compatibleHandsets" : [{
                                                      "manufacturer" : {
                                                       "name" : "Apple",
                                                      "canonicalName" : "apple"
                                                       },
                                                  "model" : "iPhone 5 16GB",
                                                  "name" : "Apple_iPhone_5_16GB",
                                                      "canonicalName" : "apple_iphone_5_16gb"
                                     }],
                                     review_ids : ["review_id1", "review_id2"]
                                     }




http://www.equalexperts.com/
Querying
  ●    Querying for a accesory by a category :
      db.accessory.find({"category.canonicalName" : "cases"})

  ● Querying for a accessory by list of categories :
   db.accessory.find({"category.canonicalName" : {"$in" : ["cases",
  "memory"]}})

  ●    Getting distinct categories of products:
      db.accessory.distinct("category");




http://www.equalexperts.com/
Querying contd...
   ●   Querying all cases between a price range sorted by price
       db.accessory.find( { $and : [{"category.canonicalName" : "cases"},
       {"retailPrice" : {$gt : 15, $lt : 25}}]}).sort({“retailPrice” : 1})

   ●   Querying accessories by text:
       db.accessory.find( {"keywords" : { $regex : "iphone", $options : 'I'}})

   ●   Querying embedded arrays:
       db.accessory.find({"compatibleHandsets" : {$elemMatch :
       { "manufacturer.canonicalName" : "nokia"}}})




http://www.equalexperts.com/
Referencing documents
   ●   Manual Referencing – Save the _id field of one
       document in another as a reference manually.
       Eg : Accessory - Reviews

   ●   DBRefs – A standard convention for referencing a
       document in another. It stores the name of the
       collection and the _id of the parent document.
       Eg : Basket - Accessory


http://www.equalexperts.com/
GridFS
   ●   Storage specification for storing large binary objects in
       mongoDB as BSON documents
   ●   Easy to use and can store files upto 16MB
       GridFS gfs = new GridFS(dbName);
       //create file
       gfs.createFile(inputStream, “myFileName”);
       //retrieve file
       gfs.findOne(“myFileName”);



http://www.equalexperts.com/
Indexes
   ●   Indexes
       db.accessory.ensureIndex({“name” : 1})
   ●   Unique Keys
       db.accessory.ensureIndex({“canonicalName” :
       1}, {unique : true})
   ●   Sparse Keys
       db.order.ensureIndex({“uniqueActiveOrder” :
       1}, {unique : true, sparse : true})


http://www.equalexperts.com/
Optimistic Locking mechanism
   ●   Set ReadPreference on these collections to
       Primary
   ●   Give the lock key to each client who reads the
       collection
   ●   Each time you update generate a unique lock key
   ●   Before update check if the lock key from client
       matches the one in the DB, if not do not update
       since the object is stale.


http://www.equalexperts.com/
Manging schemas
   ●   DB to Object Convertors and vice versa
       should be written to manually manage the old
       schema to the current one as we release code
       to production
   ●   Requires high developer discipline and good
       knowledge of previous object model
   ●   Delayed replication node



http://www.equalexperts.com/
Questions




http://www.equalexperts.com/

Product catalog using MongoDB

  • 1.
    e-Com Product Catalogusing MongoDB - Vishwas Bhagath vbhagath@equalexperts.com http://www.equalexperts.com/
  • 2.
    Why MongoDB ? ● Agile incremental releases with a short time to market ● Feeds from multiple vendors containing products with varying schema ● Complex queries on product attributes ● Rich Content user experience ● Capital risk mitigation http://www.equalexperts.com/
  • 3.
    How we approached... ● Modeling and schema design ● Querying ● Referencing one document in another ● Rich content using GridFS ● Indexes ● MongoDB challenges ➢ Locking mechanism ➢ Managing Schema and fat fingers http://www.equalexperts.com/
  • 4.
    Domain model {"_id" : ObjectId("5082626144ae3a687919c094"), "name" : "iPhone 5 Pop Blue Case", "canonicalName" : "iphone-5-pop-blue-case", "retailPrice" : 19.99, "productCode" : "G4IC542G", "category" : { "categoryCode" : "CAS", "name" : "Cases", "canonicalName" : "cases" }, "compatibleHandsets" : [{ "manufacturer" : { "name" : "Apple", "canonicalName" : "apple" }, "model" : "iPhone 5 16GB", "name" : "Apple_iPhone_5_16GB", "canonicalName" : "apple_iphone_5_16gb" }], review_ids : ["review_id1", "review_id2"] } http://www.equalexperts.com/
  • 5.
    Querying ● Querying for a accesory by a category : db.accessory.find({"category.canonicalName" : "cases"}) ● Querying for a accessory by list of categories : db.accessory.find({"category.canonicalName" : {"$in" : ["cases", "memory"]}}) ● Getting distinct categories of products: db.accessory.distinct("category"); http://www.equalexperts.com/
  • 6.
    Querying contd... ● Querying all cases between a price range sorted by price db.accessory.find( { $and : [{"category.canonicalName" : "cases"}, {"retailPrice" : {$gt : 15, $lt : 25}}]}).sort({“retailPrice” : 1}) ● Querying accessories by text: db.accessory.find( {"keywords" : { $regex : "iphone", $options : 'I'}}) ● Querying embedded arrays: db.accessory.find({"compatibleHandsets" : {$elemMatch : { "manufacturer.canonicalName" : "nokia"}}}) http://www.equalexperts.com/
  • 7.
    Referencing documents ● Manual Referencing – Save the _id field of one document in another as a reference manually. Eg : Accessory - Reviews ● DBRefs – A standard convention for referencing a document in another. It stores the name of the collection and the _id of the parent document. Eg : Basket - Accessory http://www.equalexperts.com/
  • 8.
    GridFS ● Storage specification for storing large binary objects in mongoDB as BSON documents ● Easy to use and can store files upto 16MB GridFS gfs = new GridFS(dbName); //create file gfs.createFile(inputStream, “myFileName”); //retrieve file gfs.findOne(“myFileName”); http://www.equalexperts.com/
  • 9.
    Indexes ● Indexes db.accessory.ensureIndex({“name” : 1}) ● Unique Keys db.accessory.ensureIndex({“canonicalName” : 1}, {unique : true}) ● Sparse Keys db.order.ensureIndex({“uniqueActiveOrder” : 1}, {unique : true, sparse : true}) http://www.equalexperts.com/
  • 10.
    Optimistic Locking mechanism ● Set ReadPreference on these collections to Primary ● Give the lock key to each client who reads the collection ● Each time you update generate a unique lock key ● Before update check if the lock key from client matches the one in the DB, if not do not update since the object is stale. http://www.equalexperts.com/
  • 11.
    Manging schemas ● DB to Object Convertors and vice versa should be written to manually manage the old schema to the current one as we release code to production ● Requires high developer discipline and good knowledge of previous object model ● Delayed replication node http://www.equalexperts.com/
  • 12.