Your SlideShare is downloading. ×
Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory

765

Published on

During this session we will cover the best practices for implementing a real-time inventory with MongoDB. This includes properly model quantities and stores to avoid large numbers of documents being …

During this session we will cover the best practices for implementing a real-time inventory with MongoDB. This includes properly model quantities and stores to avoid large numbers of documents being indexed, how to efficiently use geo-indexing to find the closest store with a specific item available and how to run aggregation to gather interesting inventory stats. We will also cover operational considerations, like how to make inventory queries and updates from anywhere be low-latency and resilient to network partitions via tag-aware sharding.

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
765
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
49
Comments
0
Likes
4
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Senior Consulting Engineer, MongoDB Norman Graham #MongoDBWorld Retail Reference Architecture: Real-Time, Geo-Distributed Inventory
  • 2. Inventory Inventory MongoDB External Inventory Internal Inventory Regional Inventory Purchase Orders Fulfillment Promotions
  • 3. Inventory – Traditional Architecture Relational DB System of Records Nightly Batches Analytics, Aggregations, Reports Caching Layer Field Inventory Internal & External Apps Point-in-time Loads
  • 4. Inventory – Opportunities Missed • Can’t reliably detect availability • Can't redirect purchasers to in-store pickup • Can’t do intra-day replenishment • Degraded customer experience • Higher internal expense
  • 5. Inventory – Principles • Single view of the inventory • Used by most services and channels • Read-dominated workload • Local, real-time writes • Bulk writes for refresh • Geographically distributed • Horizontally scalable
  • 6. Requirement Challenge MongoDB Single view of inventory Ensure availability of inventory information on all channels and services Developer-friendly, document-oriented storage High volume, low latency reads Anytime, anywhere access to inventory data without overloading the system of record Fast, indexed reads Local reads Horizontal scaling Bulk updates, intra-day deltas Provide window-in-time consistency for highly available services Bulk writes Fast, in-place updates Horizontal scaling Rapid application development cycles Deliver new services rapidly to capture new opportunities Flexible schema Rich query language Agile-friendly iterations Inventory – Requirements
  • 7. Inventory – Target Architecture Relational DB System of Records Analytics, Aggregations, Reports Field Inventory Internal & External Apps Inventory Assortments Shipments Audits Products Stores Point-in-time Loads Nightly Refresh Real-time Updates
  • 8. Horizontal Scaling Inventory – Technical Decisions Store Inventory Schema Indexing
  • 9. Inventory – Collections Stores Inventory Products Audits AssortmentsShipments
  • 10. > db.stores.findOne() { "_id" : ObjectId("53549fd3e4b0aaf5d6d07f35"), "className" : "catalog.Store", "storeId" : "store0", "name" : "Bessemer store", "address" : { "addr1" : "1st Main St", "city" : "Bessemer", "state" : "AL", "zip" : "12345", "country" : "US" }, "location" : [ -86.95444, 33.40178 ], ... } Stores – Sample Document
  • 11. Stores – Sample Queries • Get a store by storeId db.stores.find({ "storeId" : "store0" }) • Get a store by zip code db.stores.find({ "address.zip" : "12345" })
  • 12. What’s near me?
  • 13. Stores – Sample Geo Queries • Get nearby stores sorted by distance db.runCommand({ geoNear : "stores", near : { type : "Point", coordinates : [-82.8006, 40.0908] }, maxDistance : 10000.0, spherical : true })
  • 14. Stores – Sample Geo Queries • Get the five nearest stores within 10 km db.stores.find({ location : { $near : { $geometry : { type : "Point", coordinates : [-82.80, 40.09] }, $maxDistance : 10000.0 } } }).limit(5)
  • 15. Stores – Indices • { "storeId" : 1 }, { "unique" : true } • { "name" : 1 } • { "address.zip" : 1 } • { "location" : "2dsphere" }
  • 16. > db.inventory.findOne() { "_id": "5354869f300487d20b2b011d", "storeId": "store0", "location": [-86.95444, 33.40178], "productId": "p0", "vars": [ { "sku": "sku1", "q": 14 }, { "sku": "sku3", "q": 7 }, { "sku": "sku7", "q": 32 }, { "sku": "sku14", "q": 65 }, ... ] } Inventory – Sample Document
  • 17. Inventory – Sample Queries • Get all items in a store db.inventory.find({ storeId : "store100" }) • Get quantity for an item at a store db.inventory.find({ "storeId" : "store100", "productId" : "p200" })
  • 18. Inventory – Sample Queries • Get quantity for a sku at a store db.inventory.find( { "storeId" : "store100", "productId" : "p200", "vars.sku" : "sku11736" }, { "vars.$" : 1 } )
  • 19. Inventory – Sample Update • Increment / decrement inventory for an item at a store db.inventory.update( { "storeId" : "store100", "productId" : "p200", "vars.sku" : "sku11736" }, { "$inc" : { "vars.$.q" : 20 } } )
  • 20. Inventory – Sample Aggregations • Aggregate total quantity for a product db.inventory.aggregate( [ { $match : { productId : "p200" } }, { $unwind : "$vars" }, { $group : { _id : "result", count : { $sum : "$vars.q" } } } ] ) { "_id" : "result", "count" : 101752 }
  • 21. Inventory – Sample Aggregations • Aggregate total quantity for a store db.inventory.aggregate( [ { $match : { storeId : "store100" } }, { $unwind : "$vars" }, { $match : { "vars.q" : { $gt : 0 } } }, { $group : { _id : "result", count : { $sum : 1 } } } ] ) { "_id" : "result", "count" : 29347 }
  • 22. Inventory – Sample Aggregations • Aggregate total quantity for a store db.inventory.aggregate( [ { $match : { storeId : "store100" } }, { $unwind : "$vars" }, { $group : { _id : "result", count : { $sum : "$vars.q" } } } ] ) { "_id" : "result", "count" : 29347 }
  • 23. Inventory – Sample Geo-Query • Get inventory for an item near a point db.runCommand( { geoNear : "inventory", near : { type : "Point", coordinates : [-82.8006, 40.0908] }, maxDistance : 10000.0, spherical : true, limit : 10, query : { "productId" : "p200", "vars.sku" : "sku11736" } } )
  • 24. Inventory – Sample Geo-Query • Get closest store with available sku db.runCommand( { geoNear : "inventory", near : { type : "Point", coordinates : [-82.800672, 40.090844] }, maxDistance : 10000.0, spherical : true, limit : 1, query : { productId : "p200", vars : { $elemMatch : { sku : "sku11736", q : { $gt : 0 } } } } } )
  • 25. Inventory – Sample Geo- Aggregation • Get count of inventory for an item near a point db.inventory.aggregate([ { $geoNear: { near : { type : "Point", coordinates : [-82.800672, 40.090844] }, distanceField: "distance", maxDistance: 10000.0, spherical : true, query: { productId: "p200", vars : { $elemMatch : { sku : "sku11736", q : {$gt : 0} } } }, includeLocs: "dist.location", num: 5 } }, { $unwind: "$vars" }, { $match: { "vars.sku" : "sku11736" } }, { $group: { _id: "result", count: {$sum: "$vars.q"} } }])
  • 26. Inventory – Sample Indices • { storeId : 1 } • { productId : 1, storeId : 1 } • { productId : 1, location : "2dsphere" } • Why not "vars.sku"? – { productId : 1, storeId : 1, "vars.sku" : 1 }
  • 27. Horizontal Scaling Inventory – Technical Decisions Store Inventory Schema Indexing
  • 28. Shard East Shard Central Shard West East DC Inventory – Sharding Topology West DC Central DC Legacy Inventory Primary Primary Primary
  • 29. Inventory – Shard Key • Choose shard key – { storeId : 1 } ? – { productId : 1, storeId : 1 }? – { storeId : 1, productId : 1 } ? • Set up sharding – sh.enableSharding("inventoryDB") – sh.shardCollection( "inventoryDB.inventory", { storeId : 1, productId : 1 })
  • 30. Inventory – Shard Tags • Set up shard tags – sh.addShardTag("shard0000", "west") – sh.addShardTag("shard0001", "central") – sh.addShardTag("shard0002", "east") • Set up tag ranges – sh.addTagRange("inventoryDB.inventory", { storeId : 0 }, { storeId : 100}, "west" ) – sh.addTagRange("inventoryDB.inventory", { storeId : 100 }, { storeId : 200 }, "central" ) – sh.addTagRange("inventoryDB.inventory", { storeId : 200 }, { storeId : 300 }, "east" )
  • 31. Senior Consulting Engineer, MongoDB Norman Graham #MongoDBWorld Thank You

×