Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory
 

Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory

on

  • 334 views

During this session we will cover the best practices for implementing a real-time inventory with MongoDB. This includes properly model quantities and stores to avoid large numbers of documents being ...

During this session we will cover the best practices for implementing a real-time inventory with MongoDB. This includes properly model quantities and stores to avoid large numbers of documents being indexed, how to efficiently use geo-indexing to find the closest store with a specific item available and how to run aggregation to gather interesting inventory stats. We will also cover operational considerations, like how to make inventory queries and updates from anywhere be low-latency and resilient to network partitions via tag-aware sharding.

Statistics

Views

Total Views
334
Views on SlideShare
258
Embed Views
76

Actions

Likes
1
Downloads
19
Comments
0

3 Embeds 76

http://www.mongodb.com 52
https://www.mongodb.com 23
https://comwww-drupal.10gen.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory Presentation Transcript

  • Senior Consulting Engineer, MongoDB Norman Graham #MongoDBWorld Retail Reference Architecture: Real-Time, Geo-Distributed Inventory
  • Inventory Inventory MongoDB External Inventory Internal Inventory Regional Inventory Purchase Orders Fulfillment Promotions
  • Inventory – Traditional Architecture Relational DB System of Records Nightly Batches Analytics, Aggregations, Reports Caching Layer Field Inventory Internal & External Apps Point-in-time Loads View slide
  • Inventory – Opportunities Missed • Can’t reliably detect availability • Can't redirect purchasers to in-store pickup • Can’t do intra-day replenishment • Degraded customer experience • Higher internal expense View slide
  • Inventory – Principles • Single view of the inventory • Used by most services and channels • Read-dominated workload • Local, real-time writes • Bulk writes for refresh • Geographically distributed • Horizontally scalable
  • Requirement Challenge MongoDB Single view of inventory Ensure availability of inventory information on all channels and services Developer-friendly, document-oriented storage High volume, low latency reads Anytime, anywhere access to inventory data without overloading the system of record Fast, indexed reads Local reads Horizontal scaling Bulk updates, intra-day deltas Provide window-in-time consistency for highly available services Bulk writes Fast, in-place updates Horizontal scaling Rapid application development cycles Deliver new services rapidly to capture new opportunities Flexible schema Rich query language Agile-friendly iterations Inventory – Requirements
  • Inventory – Target Architecture Relational DB System of Records Analytics, Aggregations, Reports Field Inventory Internal & External Apps Inventory Assortments Shipments Audits Products Stores Point-in-time Loads Nightly Refresh Real-time Updates
  • Horizontal Scaling Inventory – Technical Decisions Store Inventory Schema Indexing
  • Inventory – Collections Stores Inventory Products Audits AssortmentsShipments
  • > db.stores.findOne() { "_id" : ObjectId("53549fd3e4b0aaf5d6d07f35"), "className" : "catalog.Store", "storeId" : "store0", "name" : "Bessemer store", "address" : { "addr1" : "1st Main St", "city" : "Bessemer", "state" : "AL", "zip" : "12345", "country" : "US" }, "location" : [ -86.95444, 33.40178 ], ... } Stores – Sample Document
  • Stores – Sample Queries • Get a store by storeId db.stores.find({ "storeId" : "store0" }) • Get a store by zip code db.stores.find({ "address.zip" : "12345" })
  • What’s near me?
  • Stores – Sample Geo Queries • Get nearby stores sorted by distance db.runCommand({ geoNear : "stores", near : { type : "Point", coordinates : [-82.8006, 40.0908] }, maxDistance : 10000.0, spherical : true })
  • Stores – Sample Geo Queries • Get the five nearest stores within 10 km db.stores.find({ location : { $near : { $geometry : { type : "Point", coordinates : [-82.80, 40.09] }, $maxDistance : 10000.0 } } }).limit(5)
  • Stores – Indices • { "storeId" : 1 }, { "unique" : true } • { "name" : 1 } • { "address.zip" : 1 } • { "location" : "2dsphere" }
  • > db.inventory.findOne() { "_id": "5354869f300487d20b2b011d", "storeId": "store0", "location": [-86.95444, 33.40178], "productId": "p0", "vars": [ { "sku": "sku1", "q": 14 }, { "sku": "sku3", "q": 7 }, { "sku": "sku7", "q": 32 }, { "sku": "sku14", "q": 65 }, ... ] } Inventory – Sample Document
  • Inventory – Sample Queries • Get all items in a store db.inventory.find({ storeId : "store100" }) • Get quantity for an item at a store db.inventory.find({ "storeId" : "store100", "productId" : "p200" })
  • Inventory – Sample Queries • Get quantity for a sku at a store db.inventory.find( { "storeId" : "store100", "productId" : "p200", "vars.sku" : "sku11736" }, { "vars.$" : 1 } )
  • Inventory – Sample Update • Increment / decrement inventory for an item at a store db.inventory.update( { "storeId" : "store100", "productId" : "p200", "vars.sku" : "sku11736" }, { "$inc" : { "vars.$.q" : 20 } } )
  • Inventory – Sample Aggregations • Aggregate total quantity for a product db.inventory.aggregate( [ { $match : { productId : "p200" } }, { $unwind : "$vars" }, { $group : { _id : "result", count : { $sum : "$vars.q" } } } ] ) { "_id" : "result", "count" : 101752 }
  • Inventory – Sample Aggregations • Aggregate total quantity for a store db.inventory.aggregate( [ { $match : { storeId : "store100" } }, { $unwind : "$vars" }, { $match : { "vars.q" : { $gt : 0 } } }, { $group : { _id : "result", count : { $sum : 1 } } } ] ) { "_id" : "result", "count" : 29347 }
  • Inventory – Sample Aggregations • Aggregate total quantity for a store db.inventory.aggregate( [ { $match : { storeId : "store100" } }, { $unwind : "$vars" }, { $group : { _id : "result", count : { $sum : "$vars.q" } } } ] ) { "_id" : "result", "count" : 29347 }
  • Inventory – Sample Geo-Query • Get inventory for an item near a point db.runCommand( { geoNear : "inventory", near : { type : "Point", coordinates : [-82.8006, 40.0908] }, maxDistance : 10000.0, spherical : true, limit : 10, query : { "productId" : "p200", "vars.sku" : "sku11736" } } )
  • Inventory – Sample Geo-Query • Get closest store with available sku db.runCommand( { geoNear : "inventory", near : { type : "Point", coordinates : [-82.800672, 40.090844] }, maxDistance : 10000.0, spherical : true, limit : 1, query : { productId : "p200", vars : { $elemMatch : { sku : "sku11736", q : { $gt : 0 } } } } } )
  • Inventory – Sample Geo- Aggregation • Get count of inventory for an item near a point db.inventory.aggregate([ { $geoNear: { near : { type : "Point", coordinates : [-82.800672, 40.090844] }, distanceField: "distance", maxDistance: 10000.0, spherical : true, query: { productId: "p200", vars : { $elemMatch : { sku : "sku11736", q : {$gt : 0} } } }, includeLocs: "dist.location", num: 5 } }, { $unwind: "$vars" }, { $match: { "vars.sku" : "sku11736" } }, { $group: { _id: "result", count: {$sum: "$vars.q"} } }])
  • Inventory – Sample Indices • { storeId : 1 } • { productId : 1, storeId : 1 } • { productId : 1, location : "2dsphere" } • Why not "vars.sku"? – { productId : 1, storeId : 1, "vars.sku" : 1 }
  • Horizontal Scaling Inventory – Technical Decisions Store Inventory Schema Indexing
  • Shard East Shard Central Shard West East DC Inventory – Sharding Topology West DC Central DC Legacy Inventory Primary Primary Primary
  • Inventory – Shard Key • Choose shard key – { storeId : 1 } ? – { productId : 1, storeId : 1 }? – { storeId : 1, productId : 1 } ? • Set up sharding – sh.enableSharding("inventoryDB") – sh.shardCollection( "inventoryDB.inventory", { storeId : 1, productId : 1 })
  • Inventory – Shard Tags • Set up shard tags – sh.addShardTag("shard0000", "west") – sh.addShardTag("shard0001", "central") – sh.addShardTag("shard0002", "east") • Set up tag ranges – sh.addTagRange("inventoryDB.inventory", { storeId : 0 }, { storeId : 100}, "west" ) – sh.addTagRange("inventoryDB.inventory", { storeId : 100 }, { storeId : 200 }, "central" ) – sh.addTagRange("inventoryDB.inventory", { storeId : 200 }, { storeId : 300 }, "east" )
  • Senior Consulting Engineer, MongoDB Norman Graham #MongoDBWorld Thank You