Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
MongoDB  E-commerceand Transactions
My name isSteve Francia     @spf13
• 15+ years building the  internet• Father, husband, skateboarder• Chief Solutions Architect @  10gen• Author of upcoming ...
Before 10gen Iworked    for     http://opensky.com
OpenSky was the firste-commerce site built    on MongoDB ... also the first e-commerce site built on                  Symfony2
Introduction to   MongoDB
Why MongoDB?
MongoDB Goals• Open source• Designed for today • Today’s hardware / environments • Today’s challenges• Great developer exp...
• Company behind MongoDB • AGPL license, own copyrights, engineering    team  • support, consulting, commercial license   ...
A bit of history
1974The relational database is created
1979
1979   1982-1996
1979   1982-1996   1995
Computers in 1995•Pentium 100 mhz•10base T•16 MB ram•200 MB HD
Cloud in 1995(Windows 95 cloud wallpaper)
Cell Phones in 2011•Dual core 1.5 Ghz•WiFi 802.11n (300+ Mbps)•1 GB ram•64GB Solid State
How about a DBdesigned for today?
MongoDB philosophy• Keep functionality when we can (key/  value stores are great, but we need more)• Non-relational (no jo...
MongoDB is:          Application      Document                           Oriented                           { author: “ste...
Under the hood•Written in C++•Runs on nearly anything•Data serialized to BSON•Extensive use of memory-mapped files
Database Landscape
This has led    some to say“MongoDB has the bestfeatures of key/ valuesstores, document databasesand relational databases ...
Use Cases
CMS / BlogNeeds:• Business needed modern data store for rapid development and  scaleSolution:• Use PHP & MongoDBResults:• ...
Photo Meta-DataProblem:• Business needed more flexibility than Oracle could deliverSolution:• Use MongoDB instead of Oracle...
Customer AnalyticsProblem:• Deal with massive data volume across all customer sitesSolution:• Use MongoDB to replace Googl...
Online DictionaryProblem:• MySQL could not scale to handle their 5B+ documentsSolution:• Switched from MySQL to MongoDBRes...
E-commerceProblem:• Multi-vertical E-commerce impossible to model (efficiently) in  RDBMSSolution:• Switched from MySQL to ...
Tons morePretty much if you can use a RDMBS or Key/Value             MongoDB is a great fit
In Good Company
Why NoSQL for   e-commerce?Using the right solution for each situation
Data dilemma of  e-commerce     Pick One
Data dilemma of     e-commerce              Pick One•Stick to one vertical (Sane schema)•Flexibility (Insane schema)
Sane schema
Sane schema•Works ... for a while•Fine for a few types of products•Not possible when more product types  introduced
Let’s Use an Example
Let’s Use an Example   How about we start with books
Book Product SchemaProduct {id:sku:                                    General Productproduct dimensions:shipping weight: ...
Seems simple enoughWhat happens when we add another vertical...             say music albums
Album Product SchemaProduct {id:sku:                               General Productproduct dimensions:                attri...
Okay, it’s getting  hairy but is stillmanageable, right?  Now the business want to sell jeans
Jeans Product SchemaProduct {id:                           General Productsku:                          attributes stay th...
Now we’re screwed
Now we’re screwed
We need a flexibleschema in RDBMS
We need a flexibleschema in RDBMS    We got this ... right?
Many approachesdealing with unknown unknowns in RDBMS
Many approachesdealing with unknown unknowns in RDBMS      None work well
EAV         as popularized by Magento“For purposes of flexibility, the Magento database heavilyutilizes an Entity-Attribute...
EAV• Crazy SQL queries• Hundreds of joins in a  query... or• Hundreds of queries joined  in the application• No database e...
Did I say crazy SQL(this is a single query)
Did I say crazy SQL(this is a single query)You may have trouble reading this in the back
Selecting a single     product
Single Table Inheritance         (insanely wide tables)• No data integrity enforcement• Only can use FK for common  elemen...
Generic Columns• No data integrity enforcement• No data type enforcement• Only can use FK for common  elements• Wasteful (...
Serialized in Blob• Not searchable• No integrity• All the disadvantages of a  document store, but none of the  advantages•...
Concrete Table Inheritance  (a table for each product attribute set)• Allows for data integrity• Querying across attribute...
Class table inheritance              (single product table,         each attribute set in own table)• Likely best of SQL w...
MongoDB to the   Rescue
MongoDB to the      Rescue•Flexible (and sane) Schema
MongoDB to the      Rescue•Flexible (and sane) Schema•Easily searchable
MongoDB to the      Rescue•Flexible (and sane) Schema•Easily searchable•Easily accessible
MongoDB to the      Rescue•Flexible (and sane) Schema•Easily searchable•Easily accessible•Fast
Flexible schema
{                                 {    sku: "00e8da9c",                  sku: "00e8da9d",    type: "Audio Album",         ...
pct_savings: 20                      pct_savings: 8.5},                                   },details: {                    ...
Queries
db.products.find( { name: "The Matrix" } );
db.products.find( { name: "The Matrix" } ); {     "_id": ObjectId("4d8ad78b46b731a22943d3d3"),     "sku": "00e8da9d",     ...
db.products.find( { details.actor: "Groucho Marx" } );
db.products.find( { details.actor: "Groucho Marx" } ); }, "pricing": {     "list": 1000,     "retail": 800,     "savings":...
db.products.find( {     details.genre: "Jazz", details.format: "CD"} );
db.products.find( {     details.genre: "Jazz", details.format: "CD"} );     "list": 1200,     "retail": 1100,     "savings...
db.products.find( { details.actor:     { $all: [James Stewart, Donna Reed] }} );
db.products.find( { details.actor:     { $all: [James Stewart, Donna Reed] }} ); }, "details": {     "title": "Its a Wonde...
Wanna Play?• grab products.js from  http://github.com/spf13/  mongoProducts• mongo   --shell products.js• > use   mongoPro...
Embedded documents are great for orders•Ordered items need to be fixed at the  time of purchase•Embed them right in the ord...
What about     transactions?Using the right solution for each situation
Data (like people) arereally sensitive when  it comes to money
Stricter datarequirements for $$
Stricter data requirements for $$•For financial systems any data  inconsistency is unacceptable
Stricter data requirements for $$•For financial systems any data  inconsistency is unacceptable•Perhaps you’ve heard of ACID?
What about ACID?
What about ACID?Q: Is MongoDB ACID?
What about ACID?Q: Is MongoDB ACID?A: Kinda
Atomicity
Atomicity•MongoDB does atomic writes
Atomicity•MongoDB does atomic writes  ... for single document changesets
Atomicity•MongoDB does atomic writes  ... for single document changesets• $set, $unset, $inc, $push,  $pushAll, $pull, $pu...
Consistency
Consistency•MongoDB can enforce unique keys
Consistency•MongoDB can enforce unique keys•MongoDB cant enforce referential  integrity
Isolation
Isolation•   // Pseudo-isolated updates    db.foo.update( { x : 1 } , { $inc : { y : 1 } } , false , true );
Isolation•   // Pseudo-isolated updates    db.foo.update( { x : 1 } , { $inc : { y : 1 } } , false , true );•   // Isolate...
Isolation•   // Pseudo-isolated updates    db.foo.update( { x : 1 } , { $inc : { y : 1 } } , false , true );•   // Isolate...
Isolation•   // Pseudo-isolated updates    db.foo.update( { x : 1 } , { $inc : { y : 1 } } , false , true );•   // Isolate...
Isolation•   // Pseudo-isolated updates    db.foo.update( { x : 1 } , { $inc : { y : 1 } } , false , true );•   // Isolate...
Isolation•   // Pseudo-isolated updates    db.foo.update( { x : 1 } , { $inc : { y : 1 } } , false , true );•   // Isolate...
Durability
Durability•Mongo has this one covered
What doesMongoDB Support?
• Atomic single document writes • If you need atomic writes across multi-document     transactions dont use Mongo  • Many ...
• Atomic single document writes • If you need atomic writes across multi-document     transactions dont use Mongo  • Many ...
• Atomic single document writes • If you need atomic writes across multi-document     transactions dont use Mongo  • Many ...
• Atomic single document writes • If you need atomic writes across multi-document     transactions dont use Mongo  • Many ...
There are ways to   guarantee ACIDproperties in MongoDB Here are 2 good approaches useful for       E-commerce transactions
OptimisticConcurrency
Optimistic       Concurrency•Read the current state of a product
Optimistic       Concurrency•Read the current state of a product•Make your changes with the assertion  that your product h...
Optimistic concurrency     in MongoDB
Optimistic concurrency     in MongoDB We’ll use an update-if-current strategy.
Optimistic concurrency     in MongoDB We’ll use an update-if-current strategy. This example is straight from the documenta...
Optimistic concurrency     in MongoDB    We’ll use an update-if-current strategy.    This example is straight from the doc...
Optimistic concurrency      in MongoDB      We’ll use an update-if-current strategy.      This example is straight from th...
Optimistic       concurrency•Read the current state of a product.•Make your changes with the assertion  that your product ...
Optimistic         concurrency•Read the current state of a product.•Make your changes with the assertion    that your prod...
Optimistic concurrency control assumes anenvironment with low   data contention
OCC works great forcompanies like Amazon•Amazon has a long-tail catalog•A long tail catalog lends itself well to  optimist...
OCC fails miserably        for
OCC fails miserably        for•eBay
OCC fails miserably        for•eBay•Gilt
OCC fails miserably         for•eBay•Gilt•Groupon
OCC fails miserably         for•eBay•Gilt•Groupon•OpenSky
OCC fails miserably             for•eBay•Gilt•Groupon•OpenSky•Living Social
OCC fails miserably             for•eBay•Gilt•Groupon•OpenSky•Living Social•InsertFlashSaleSiteOfTheMinute
Flash sales andauctions are defined by high data contention
Flash sales andauctions are defined by high data contention•The model doesnt work otherwise
Flash sales andauctions are defined by high data contention•The model doesnt work otherwise•They cant afford to be optimistic
Flash sales andauctions are defined by high data contention•The model doesnt work otherwise•They cant afford to be optimist...
What about high  contentionenvironments?
If we can avoidconcurrency we’ve    got it made
Commerce is ACID   In Real Life
1. I go to Barneys and see a pair of shoes I just have to   buy.2. I call “dibs” (by grabbing them off the shelf).3. I tak...
All of this is   accomplishedwithout concurrency
Each item can only be held by a consumer
We follow the samemodel for e-commerce
1. Select a product.
1. Select a product.2. Update the document to hold inventory.
1. Select a product.2. Update the document to hold inventory. • Store inventory has been    decremented.
1. Select a product.2. Update the document to hold inventory. • Store inventory has been    decremented.3. Purchase the pr...
1. Select a product.2. Update the document to hold inventory. • Store inventory has been    decremented.3. Purchase the pr...
1. Select a product.2. Update the document to hold inventory. • Store inventory has been    decremented.3. Purchase the pr...
MongoDB e-commerce    transactions
MongoDB e-commerce    transactions• Each Item (not SKU) has it’s own document• Document consists of... • a reference to th...
Transactionsin MongoDB
Transactions      in MongoDBWe’ll use a simple update statementhere.
Transactions            in MongoDB We’ll use a simple update statement here.> t = db.inventory> sku = sku.findOne({sku:abc...
Transactions              in MongoDB   We’ll use a simple update statement   here.  > t = db.inventory  > sku = sku.findOn...
Cart in Cart Action
Cart in Cart ActionAn added benefit, it can easily provideinventory hold in cart.
Cart in Cart Action An added benefit, it can easily provide inventory hold in cart.> t = db.inventory> sku = sku.findOne({s...
Cart in Cart Action An added benefit, it can easily provide inventory hold in cart.> t = db.inventory> sku = sku.findOne({s...
http://spf13.com                                http://github.com/spf13                                @spf13           Qu...
MongoDB, E-commerce and Transactions
MongoDB, E-commerce and Transactions
MongoDB, E-commerce and Transactions
MongoDB, E-commerce and Transactions
MongoDB, E-commerce and Transactions
MongoDB, E-commerce and Transactions
Upcoming SlideShare
Loading in …5
×

MongoDB, E-commerce and Transactions

79,348 views

Published on

Why MongoDB is the perfect solution for the challenges of E-Commerce.

Published in: Technology, Business

MongoDB, E-commerce and Transactions

  1. 1. MongoDB E-commerceand Transactions
  2. 2. My name isSteve Francia @spf13
  3. 3. • 15+ years building the internet• Father, husband, skateboarder• Chief Solutions Architect @ 10gen• Author of upcoming O’Reilly publication “MongoDB and PHP”
  4. 4. Before 10gen Iworked for http://opensky.com
  5. 5. OpenSky was the firste-commerce site built on MongoDB ... also the first e-commerce site built on Symfony2
  6. 6. Introduction to MongoDB
  7. 7. Why MongoDB?
  8. 8. MongoDB Goals• Open source• Designed for today • Today’s hardware / environments • Today’s challenges• Great developer experience• Reliable• Scalable
  9. 9. • Company behind MongoDB • AGPL license, own copyrights, engineering team • support, consulting, commercial license revenue• Management • Google/DoubleClick, Oracle, Apple, NetApp • Funding: Sequoia, Union Square, Flybridge • Offices in NYC, Redwood Shores & London • 60+ employees
  10. 10. A bit of history
  11. 11. 1974The relational database is created
  12. 12. 1979
  13. 13. 1979 1982-1996
  14. 14. 1979 1982-1996 1995
  15. 15. Computers in 1995•Pentium 100 mhz•10base T•16 MB ram•200 MB HD
  16. 16. Cloud in 1995(Windows 95 cloud wallpaper)
  17. 17. Cell Phones in 2011•Dual core 1.5 Ghz•WiFi 802.11n (300+ Mbps)•1 GB ram•64GB Solid State
  18. 18. How about a DBdesigned for today?
  19. 19. MongoDB philosophy• Keep functionality when we can (key/ value stores are great, but we need more)• Non-relational (no joins) makes scaling horizontally practical• Document data models are good• Database technology should run anywhere VMs, cloud, metal, etc
  20. 20. MongoDB is: Application Document Oriented { author: “steve”, High date: new Date(), text: “About MongoDB...”,Performance tags: [“tech”, “database”]} Fully Consistent Horizontally Scalable
  21. 21. Under the hood•Written in C++•Runs on nearly anything•Data serialized to BSON•Extensive use of memory-mapped files
  22. 22. Database Landscape
  23. 23. This has led some to say“MongoDB has the bestfeatures of key/ valuesstores, document databasesand relational databases inone. John Nunemaker
  24. 24. Use Cases
  25. 25. CMS / BlogNeeds:• Business needed modern data store for rapid development and scaleSolution:• Use PHP & MongoDBResults:• Real time statistics• All data, images, etc stored together, easy access, easy deployment, easy high availability• No need for complex migrations• Enabled very rapid development and growth
  26. 26. Photo Meta-DataProblem:• Business needed more flexibility than Oracle could deliverSolution:• Use MongoDB instead of OracleResults:• Developed application in one sprint cycle• 500% cost reduction compared to Oracle• 900% performance improvement compared to Oracle
  27. 27. Customer AnalyticsProblem:• Deal with massive data volume across all customer sitesSolution:• Use MongoDB to replace Google Analytics / Omniture optionsResults:• Less than one week to build prototype and prove business case• Rapid deployment of new features
  28. 28. Online DictionaryProblem:• MySQL could not scale to handle their 5B+ documentsSolution:• Switched from MySQL to MongoDBResults:• Massive simplification of code base• Eliminated need for external caching system• 20x performance improvement over MySQL
  29. 29. E-commerceProblem:• Multi-vertical E-commerce impossible to model (efficiently) in RDBMSSolution:• Switched from MySQL to MongoDBResults:• Massive simplification of code base• Rapidly build, halving time to market (and cost)• Eliminated need for external caching system• 50x+ improvement over MySQL
  30. 30. Tons morePretty much if you can use a RDMBS or Key/Value MongoDB is a great fit
  31. 31. In Good Company
  32. 32. Why NoSQL for e-commerce?Using the right solution for each situation
  33. 33. Data dilemma of e-commerce Pick One
  34. 34. Data dilemma of e-commerce Pick One•Stick to one vertical (Sane schema)•Flexibility (Insane schema)
  35. 35. Sane schema
  36. 36. Sane schema•Works ... for a while•Fine for a few types of products•Not possible when more product types introduced
  37. 37. Let’s Use an Example
  38. 38. Let’s Use an Example How about we start with books
  39. 39. Book Product SchemaProduct {id:sku: General Productproduct dimensions:shipping weight: attributesMSRP:price:description:...author: Orson Scott Cardtitle: Enders Gamebinding: Hardcoverpublication date: July 15, 1994 Book Specificpublisher name: Tor Science Fiction attributesnumber of pages: 352ISBN: 0812550706language: English...
  40. 40. Seems simple enoughWhat happens when we add another vertical... say music albums
  41. 41. Album Product SchemaProduct {id:sku: General Productproduct dimensions: attributes stayshipping weight:MSRP: the sameprice:description:...artist: MxPxtitle: Panic Album Specificrelease date: June 7, 2005label: Side One Dummy attributes aretrack listing: [ The Darkest ... differentlanguage: Englishformat: CD...
  42. 42. Okay, it’s getting hairy but is stillmanageable, right? Now the business want to sell jeans
  43. 43. Jeans Product SchemaProduct {id: General Productsku: attributes stay theproduct dimensions: sameshipping weight:MSRP:price:description:...brand: Lucky Jeans specificgender: Mens attributes aremake: Vintage totally different ...style: Straight Cutlength: 34 and not consistentwidth: 34 across brands &color: Hipster makematerial: Cotten Blend...
  44. 44. Now we’re screwed
  45. 45. Now we’re screwed
  46. 46. We need a flexibleschema in RDBMS
  47. 47. We need a flexibleschema in RDBMS We got this ... right?
  48. 48. Many approachesdealing with unknown unknowns in RDBMS
  49. 49. Many approachesdealing with unknown unknowns in RDBMS None work well
  50. 50. EAV as popularized by Magento“For purposes of flexibility, the Magento database heavilyutilizes an Entity-Attribute-Value (EAV) data model.As is often the case, the cost of flexibility is complexity -Magento is no exception.The process of manipulating data in Magento is oftenmore “involved” than that typically experienced usingtraditional relational tables.” - Varien
  51. 51. EAV• Crazy SQL queries• Hundreds of joins in a query... or• Hundreds of queries joined in the application• No database enforced integrity
  52. 52. Did I say crazy SQL(this is a single query)
  53. 53. Did I say crazy SQL(this is a single query)You may have trouble reading this in the back
  54. 54. Selecting a single product
  55. 55. Single Table Inheritance (insanely wide tables)• No data integrity enforcement• Only can use FK for common elements• Very wasteful (but disk is cheap!)• Can’t effectively index
  56. 56. Generic Columns• No data integrity enforcement• No data type enforcement• Only can use FK for common elements• Wasteful (but disk is cheap!)• Can’t index
  57. 57. Serialized in Blob• Not searchable• No integrity• All the disadvantages of a document store, but none of the advantages• Never should be used• One exception is Oracle XML which operates similar to a document store
  58. 58. Concrete Table Inheritance (a table for each product attribute set)• Allows for data integrity• Querying across attribute sets quite hard to do (lots of joins, OR statements and full table scanning)• New table needs to be created for each new attribute set
  59. 59. Class table inheritance (single product table, each attribute set in own table)• Likely best of SQL within the constraint solution• Supports data type enforcement• No data integrity enforcement• Easybrowse pages) since (for querying across categories common data in single table• Every set needs a new table• Requiresare very complicated changes a ton of forsight, as
  60. 60. MongoDB to the Rescue
  61. 61. MongoDB to the Rescue•Flexible (and sane) Schema
  62. 62. MongoDB to the Rescue•Flexible (and sane) Schema•Easily searchable
  63. 63. MongoDB to the Rescue•Flexible (and sane) Schema•Easily searchable•Easily accessible
  64. 64. MongoDB to the Rescue•Flexible (and sane) Schema•Easily searchable•Easily accessible•Fast
  65. 65. Flexible schema
  66. 66. { { sku: "00e8da9c", sku: "00e8da9d", type: "Audio Album", type: "Film", title: "Hoss", title: "The Matrix", description: "by Lagwagon", description: "Set in the 22nd century, Th asin: "B0000007QG", asin: "B000P0J0AQ", shipping: { shipping: { weight: 6, weight: 6, dimensions: { dimensions: { width: 10, width: 10, height: 10, height: 10, depth: 1 depth: 1 }, }, }, }, pricing: { pricing: { list: 1000, list: 1200, retail: 800, retail: 1100, savings: 200, savings: 100, pct_savings: 20 pct_savings: 8.5 }, }, details: { details: { title: "Hoss", title: "The Matrix",
  67. 67. pct_savings: 20 pct_savings: 8.5}, },details: { details: { title: "Hoss", title: "The Matrix", artist: "Lagwagon", director: [ "Andy Wachowski", "Larry Wa genre: [ "Punk", "Hardcore", "Indie Rock" ], [ "Andy Wachowski", "Larry Wach writer: label: "Fat Wreck Chords", actor: [ "Keanu Reeves" , "Lawrence Fis number_of_discs: 1, genre: [ "Science Fiction", "Action" ], issue_date: "November 21, 1995", number_of_discs: 1, format: "CD", issue_date: "May 15 2007", alternate_formats: [ Vinyl, MP3 ],original_release_date: "1999", tracks: [ disc_format: "DVD", "Kids Dont Like To Share", rating: "R", "Violins", alternate_formats: [ VHS, Bluray ], "Name Dropping", run_time: "136", "Bombs Away", studio: "Warner Bros", "Move The Car", language: "English", "Sleep", format: [ "AC-3", "Closed-captioned", " "Sick", aspect_ratio: "1.66:1" "Rifle", }, "Weak", } "Black Eye", "Bro Dependent", "Razor Burn", "Shaving Your Head", "Ride The Snake", ],
  68. 68. Queries
  69. 69. db.products.find( { name: "The Matrix" } );
  70. 70. db.products.find( { name: "The Matrix" } ); { "_id": ObjectId("4d8ad78b46b731a22943d3d3"), "sku": "00e8da9d", "type": "Film", "name": "The Matrix", "description": "Set in the 22nd century, The Matrix...", "asin": "B000P0J0AQ", "shipping": { "weight": 6, "dimensions": { "width": 10, "height": 10, "depth": 1 } }, "pricing": {
  71. 71. db.products.find( { details.actor: "Groucho Marx" } );
  72. 72. db.products.find( { details.actor: "Groucho Marx" } ); }, "pricing": { "list": 1000, "retail": 800, "savings": 200, "pct_savings": 20 }, "details": { "title": "A Night at the Opera", "director": "Sam Wood", "actor": ["Groucho Marx", "Chico Marx", "Harpo Marx"], "genre": "Comedy", "number_of_discs": 1, "issue_date": "May 4 2004", "original_release_date": "1935", "disc_format": "DVD",
  73. 73. db.products.find( { details.genre: "Jazz", details.format: "CD"} );
  74. 74. db.products.find( { details.genre: "Jazz", details.format: "CD"} ); "list": 1200, "retail": 1100, "savings": 100, "pct_savings": 8 }, "details": { "title": "A Love Supreme [Original Recording Reissued]", "artist": "John Coltrane", "genre": ["Jazz", "General"], "format": "CD", "label": "Impulse Records", "number_of_discs": 1, "issue_date": "December 9, 1964", "alternate_formats": ["Vinyl", "MP3"], "tracks": [ "A Love Supreme Part I: Acknowledgement",
  75. 75. db.products.find( { details.actor: { $all: [James Stewart, Donna Reed] }} );
  76. 76. db.products.find( { details.actor: { $all: [James Stewart, Donna Reed] }} ); }, "details": { "title": "Its a Wonderful Life", "director": "Frank Capra", "actor": ["James Stewart", "Donna Reed", "Lionel Barrymore"], "writer": [ "Frank Capra", "Albert Hackett", "Frances Goodrich", "Jo Swerling", "Michael Wilson" ], "genre": "Drama", "number_of_discs": 1, "issue_date": "Oct 31 2006", "original_release_date": "1947",
  77. 77. Wanna Play?• grab products.js from http://github.com/spf13/ mongoProducts• mongo --shell products.js• > use mongoProducts
  78. 78. Embedded documents are great for orders•Ordered items need to be fixed at the time of purchase•Embed them right in the orderdb.order.find( { items.sku: 00e8da9f } );db.order.find( { items.details.actor: James Stewart} ).count();
  79. 79. What about transactions?Using the right solution for each situation
  80. 80. Data (like people) arereally sensitive when it comes to money
  81. 81. Stricter datarequirements for $$
  82. 82. Stricter data requirements for $$•For financial systems any data inconsistency is unacceptable
  83. 83. Stricter data requirements for $$•For financial systems any data inconsistency is unacceptable•Perhaps you’ve heard of ACID?
  84. 84. What about ACID?
  85. 85. What about ACID?Q: Is MongoDB ACID?
  86. 86. What about ACID?Q: Is MongoDB ACID?A: Kinda
  87. 87. Atomicity
  88. 88. Atomicity•MongoDB does atomic writes
  89. 89. Atomicity•MongoDB does atomic writes ... for single document changesets
  90. 90. Atomicity•MongoDB does atomic writes ... for single document changesets• $set, $unset, $inc, $push, $pushAll, $pull, $pullAll, $bit
  91. 91. Consistency
  92. 92. Consistency•MongoDB can enforce unique keys
  93. 93. Consistency•MongoDB can enforce unique keys•MongoDB cant enforce referential integrity
  94. 94. Isolation
  95. 95. Isolation• // Pseudo-isolated updates db.foo.update( { x : 1 } , { $inc : { y : 1 } } , false , true );
  96. 96. Isolation• // Pseudo-isolated updates db.foo.update( { x : 1 } , { $inc : { y : 1 } } , false , true );• // Isolated updates db.foo.update( { x : 1 , $atomic : 1 } , { $inc : { y : 1 } } , false , true );
  97. 97. Isolation• // Pseudo-isolated updates db.foo.update( { x : 1 } , { $inc : { y : 1 } } , false , true );• // Isolated updates db.foo.update( { x : 1 , $atomic : 1 } , { $inc : { y : 1 } } , false , true );• But there are caveats...
  98. 98. Isolation• // Pseudo-isolated updates db.foo.update( { x : 1 } , { $inc : { y : 1 } } , false , true );• // Isolated updates db.foo.update( { x : 1 , $atomic : 1 } , { $inc : { y : 1 } } , false , true );• But there are caveats... • Despite the $atomic keyword, this is not an atomic update, since atomicity implies “all or nothing”
  99. 99. Isolation• // Pseudo-isolated updates db.foo.update( { x : 1 } , { $inc : { y : 1 } } , false , true );• // Isolated updates db.foo.update( { x : 1 , $atomic : 1 } , { $inc : { y : 1 } } , false , true );• But there are caveats... • Despite the $atomic keyword, this is not an atomic update, since atomicity implies “all or nothing” • $atomic here means update is done without an interference from any other operation (isolated)
  100. 100. Isolation• // Pseudo-isolated updates db.foo.update( { x : 1 } , { $inc : { y : 1 } } , false , true );• // Isolated updates db.foo.update( { x : 1 , $atomic : 1 } , { $inc : { y : 1 } } , false , true );• But there are caveats... • Despite the $atomic keyword, this is not an atomic update, since atomicity implies “all or nothing” • $atomic here means update is done without an interference from any other operation (isolated) • An isolated update can only act on a single collection. Multi- collection updates are not transactional, thus not isolatable.
  101. 101. Durability
  102. 102. Durability•Mongo has this one covered
  103. 103. What doesMongoDB Support?
  104. 104. • Atomic single document writes • If you need atomic writes across multi-document transactions dont use Mongo • Many if not most e-commerce transactions could be accomplished within a single document write
  105. 105. • Atomic single document writes • If you need atomic writes across multi-document transactions dont use Mongo • Many if not most e-commerce transactions could be accomplished within a single document write• Unique indexes • This only works on keys used by the entire collection
  106. 106. • Atomic single document writes • If you need atomic writes across multi-document transactions dont use Mongo • Many if not most e-commerce transactions could be accomplished within a single document write• Unique indexes • This only works on keys used by the entire collection• Isolated (not atomic) single collection updates. • Mongo does not support locking • There are ways to work around this
  107. 107. • Atomic single document writes • If you need atomic writes across multi-document transactions dont use Mongo • Many if not most e-commerce transactions could be accomplished within a single document write• Unique indexes • This only works on keys used by the entire collection• Isolated (not atomic) single collection updates. • Mongo does not support locking • There are ways to work around this• It’s durable
  108. 108. There are ways to guarantee ACIDproperties in MongoDB Here are 2 good approaches useful for E-commerce transactions
  109. 109. OptimisticConcurrency
  110. 110. Optimistic Concurrency•Read the current state of a product
  111. 111. Optimistic Concurrency•Read the current state of a product•Make your changes with the assertion that your product has the same state as it did when you last read it
  112. 112. Optimistic concurrency in MongoDB
  113. 113. Optimistic concurrency in MongoDB We’ll use an update-if-current strategy.
  114. 114. Optimistic concurrency in MongoDB We’ll use an update-if-current strategy. This example is straight from the documentation:
  115. 115. Optimistic concurrency in MongoDB We’ll use an update-if-current strategy. This example is straight from the documentation:> t = db.inventory> p = t.findOne({sku:abc})> t.update({_id:p._id, qty:p.qty}, {$inc: {qty: -1}});> db.$cmd.findOne({getlasterror:1});{"err" : , "updatedExisting" : true , "n" : 1 , "ok" : 1}// it worked
  116. 116. Optimistic concurrency in MongoDB We’ll use an update-if-current strategy. This example is straight from the documentation: > t = db.inventory > p = t.findOne({sku:abc}) > t.update({_id:p._id, qty:p.qty}, {$inc: {qty: -1}}); > db.$cmd.findOne({getlasterror:1}); {"err" : , "updatedExisting" : true , "n" : 1 , "ok" : 1} // it worked... If that didnt work, try again until it does.
  117. 117. Optimistic concurrency•Read the current state of a product.•Make your changes with the assertion that your product has the same state as it did when you last read it.
  118. 118. Optimistic concurrency•Read the current state of a product.•Make your changes with the assertion that your product has the same state as it did when you last read it.• It is also possible to use OCC to bootstrap pessimistic concurrency and fake row level locking
  119. 119. Optimistic concurrency control assumes anenvironment with low data contention
  120. 120. OCC works great forcompanies like Amazon•Amazon has a long-tail catalog•A long tail catalog lends itself well to optimistic concurrency, because it has low data contention
  121. 121. OCC fails miserably for
  122. 122. OCC fails miserably for•eBay
  123. 123. OCC fails miserably for•eBay•Gilt
  124. 124. OCC fails miserably for•eBay•Gilt•Groupon
  125. 125. OCC fails miserably for•eBay•Gilt•Groupon•OpenSky
  126. 126. OCC fails miserably for•eBay•Gilt•Groupon•OpenSky•Living Social
  127. 127. OCC fails miserably for•eBay•Gilt•Groupon•OpenSky•Living Social•InsertFlashSaleSiteOfTheMinute
  128. 128. Flash sales andauctions are defined by high data contention
  129. 129. Flash sales andauctions are defined by high data contention•The model doesnt work otherwise
  130. 130. Flash sales andauctions are defined by high data contention•The model doesnt work otherwise•They cant afford to be optimistic
  131. 131. Flash sales andauctions are defined by high data contention•The model doesnt work otherwise•They cant afford to be optimistic•Order really matters
  132. 132. What about high contentionenvironments?
  133. 133. If we can avoidconcurrency we’ve got it made
  134. 134. Commerce is ACID In Real Life
  135. 135. 1. I go to Barneys and see a pair of shoes I just have to buy.2. I call “dibs” (by grabbing them off the shelf).3. I take them up to the cash register and purchase them: • Store inventory has been manually decremented. • I pay for them with my trusty AmEx.4. If all goes according to plan, I walk out of the store.5. If my card was declined, the shoes are “rolled back” ... out onto the shelves and sold to the next customer who wants them.
  136. 136. All of this is accomplishedwithout concurrency
  137. 137. Each item can only be held by a consumer
  138. 138. We follow the samemodel for e-commerce
  139. 139. 1. Select a product.
  140. 140. 1. Select a product.2. Update the document to hold inventory.
  141. 141. 1. Select a product.2. Update the document to hold inventory. • Store inventory has been decremented.
  142. 142. 1. Select a product.2. Update the document to hold inventory. • Store inventory has been decremented.3. Purchase the product(s)
  143. 143. 1. Select a product.2. Update the document to hold inventory. • Store inventory has been decremented.3. Purchase the product(s) • Process payment
  144. 144. 1. Select a product.2. Update the document to hold inventory. • Store inventory has been decremented.3. Purchase the product(s) • Process payment4. Roll back if anything went wrong.
  145. 145. MongoDB e-commerce transactions
  146. 146. MongoDB e-commerce transactions• Each Item (not SKU) has it’s own document• Document consists of... • a reference to the SKU (product) • a state ( available / sold / ... ) • potentially other data (timestamp, order ref)
  147. 147. Transactionsin MongoDB
  148. 148. Transactions in MongoDBWe’ll use a simple update statementhere.
  149. 149. Transactions in MongoDB We’ll use a simple update statement here.> t = db.inventory> sku = sku.findOne({sku:abc})> t.update({ref_id:sku._id, state: available}, {$set:{state: ordered}});> db.$cmd.findOne({getlasterror:1});{"err" : , "updatedExisting" : true , "n" : 1 , "ok" : 1}// it worked
  150. 150. Transactions in MongoDB We’ll use a simple update statement here. > t = db.inventory > sku = sku.findOne({sku:abc}) > t.update({ref_id:sku._id, state: available}, {$set: {state: ordered}}); > db.$cmd.findOne({getlasterror:1}); {"err" : , "updatedExisting" : true , "n" : 1 , "ok" : 1} // it worked... If that didnt work, no inventory available
  151. 151. Cart in Cart Action
  152. 152. Cart in Cart ActionAn added benefit, it can easily provideinventory hold in cart.
  153. 153. Cart in Cart Action An added benefit, it can easily provide inventory hold in cart.> t = db.inventory> sku = sku.findOne({sku:abc})> t.update({ref_id:sku._id, state: available}, {$set:{state: in cart}});> db.$cmd.findOne({getlasterror:1});{"err" : , "updatedExisting" : true , "n" : 1 , "ok" : 1}// it worked
  154. 154. Cart in Cart Action An added benefit, it can easily provide inventory hold in cart.> t = db.inventory> sku = sku.findOne({sku:abc})> t.update({ref_id:sku._id, state: available}, {$set:{state: in cart}});> db.$cmd.findOne({getlasterror:1});{"err" : , "updatedExisting" : true , "n" : 1 , "ok" : 1}// it worked just like reality, each item is either available, in a cart, or purchased
  155. 155. http://spf13.com http://github.com/spf13 @spf13 Questions? download at mongodb.orgPS: We’re hiring!! Contact us at jobs@10gen.com

×