Migrating one of the most popular e commerce platforms to mongodb
 

Migrating one of the most popular e commerce platforms to mongodb

on

  • 4,251 views

 

Statistics

Views

Total Views
4,251
Views on SlideShare
2,211
Embed Views
2,040

Actions

Likes
8
Downloads
36
Comments
3

11 Embeds 2,040

http://java.dzone.com 2018
http://groovy.dzone.com 5
http://php.dzone.com 4
https://twitter.com 4
http://css.dzone.com 3
http://ruby.dzone.com 1
http://www.dzone.com 1
http://architects.dzone.com 1
https://www.google.com 1
http://news.google.com 1
http://translate.googleusercontent.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • @AlexMcauley It's definitely an interesting thought. I think in theory if you were to use MongoDB as the primary database for your e-commerce site, you are asking for trouble. Based on my experience and others, MongoDB doesn't work well for applications that use a lot of JOIN's like Magento and other similar sized projects use.

    With the improvements in MySQL 5.7 with the in-built Memcache layer on-top and engine improvements, you'd be crazy to not use MySQL or at the very least PostgreSQL over Mongo. Don't get me wrong, I've had good experiences with MongoDB, but I know its limits and a data-critical app like in production should never use it.
    Are you sure you want to
    Your message goes here
    Processing…
  • Dwayne you're correct. We can only assume this was done as a proof of concept or an experiment. Magento has so many joins in its inner workings that I am surprised if there is any performance gain at all
    Are you sure you want to
    Your message goes here
    Processing…
  • Considering the whole premise of fast-writes means data isn't checked, isn't MongoDB not very well suited out-of-the-box for an e-commerce site where a transaction finishing is of utmost importance? MongoDB doesn't support transactions like other RDBMS's like MySQL do. Is there a feature to enable transactions in MongoDB?
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Migrating one of the most popular e commerce platforms to mongodb Migrating one of the most popular e commerce platforms to mongodb Presentation Transcript

  • Migrating One of the Most Popular eCommerce Platforms to MongoDB MongoDB Munich 2013, October 14th Aron Spohr, fashion4home GmbH, Berlin aron.spohr@fashionforhome.de
  • Will it blend?
  • What is Magento? ● ● ● ● ● ● ● Open eCommerce platform Serves all primary features of a Webshop Written in PHP Works on top of MySQL out of the box Extensible architecture Runs on over 180,000 sites eBay owned since 2011
  • The Shopping Cart ● Quote ● Items ● Addresses ● one Model for each ● one MySQL-Table for each
  • Data Structure quote_id discount_amount grand_total 560 100.00 299.00 561 0.00 quote 1028.40 quote_item item_id quote_id product_id qty price 100 560 1257 1 39.90 101 560 1349 2 140.10 quote_address address_id quote_id city street type 388 560 Munich Hauptstr. 33a shipping 389 560 Berlin Zahlstrasse 12 billing 390 561 Hamburg Geheimweg 3 shipping, billing
  • as a developer in // Loading a quote from database $quote = Mage::getModel(‘sales/quote’)->load(560); SELECT * FROM sales_quote WHERE quote_id=560; // Loading a filtered collection of quote items from database $items = $quote->getItemCollection(); $items->addFieldToFilter(‘product_id’, 1257); $items->load(); SELECT * FROM sales_quote_item WHERE quote_id=560 AND product_id=1257;
  • Persistence in Application every Model has ● a Resource Model to load/save one record from/to DB ● a Collection Model to load multiple records from DB Model Resource Model DB Collection Model Model Model Model
  • Resource Model basics of function load($object, $id) { $stmt = “SELECT * FROM “ . $this->getTableName() . ” WHERE “ . $this->getIdFieldname() . ”=$id”; $data = $sqlAdapter->fetchRow( $stmt ); $object->setData( $data ); }
  • Collection Model basics of function load() { $stmt = “SELECT * FROM “ . $this->getTableName() . ” WHERE “ . $this->renderFilters(); foreach($sqlAdapter->fetchRows( $stmt ) as $row) { $object = new Object(); $object->setData($data); $this->addItem($object); } }
  • Why should we change that? ● You don’t have to ● It always depends on your application Reasons we had: ● Have more/other scalability options ● Make it easier to work with raw data ● Be more flexible with your data schema ● Learn more about the software you are using
  • Let’s get started ● ● ● ● not change the way Magento works a very simple, self-explainable data schema make it easy to migrate old data not lose any existing features
  • Data Structure - mongoDB { quote_id: 560, discount_amount: 100.00, grand_total: 299, item: [ {item_id: 100, product_id: 1257, qty: 1, price: 39.9}, {item_id: 101, product_id: 1349, qty: 2, price: 140.10} ], address: [ {address_id: 388, city: ‘Munich’, street: ‘Hauptstr. 33a’, ...}, {address_id: 389, city: ‘Berlin’, street: ‘Zahlstrasse 12’, ...} ] }
  • Do we still have a table? SELECT * FROM quote_item; db.quote.find( {}, { ‘item’: 1 } ); ● { ‘item’:[ {item_id: 100, product_id: 1257, qty: 1 }, {item_id: 101, product_id: 1349, qty: 2 } ] } ● { ‘item’:[ {item_id: 102, product_id: 4421, qty: 1 } ] } ● { ‘item’:[ {item_id: 103, product_id: 2301, qty: 1 }, {item_id: 104, product_id: 5511, qty: 1 } ] } ● ...
  • a Tool to simplify our work with mongoDB...
  • Loading a collection of things loadDataCollection(‘/quote/item’, array()); ● ● ● ● { item_id: 100, product_id: 1257, qty: 1 } { item_id: 101, product_id: 1349, qty: 2 } { item_id: 102, product_id: 4421, qty: 1 } ... db.quote.find( {}, { ‘item’: 1 } ); ● { ‘item’:[ ● { ‘item’:[ ● ... {item_id: 100, product_id: 1257, qty: 1}, {item_id: 101, product_id: 1349, qty: 2} ] } {item_id: 102, product_id: 4421, qty: 1} ] }
  • The path to our data Name of Collection /quote /item Name of Array in document ● Tells us where to find the data ● Is very similar to a table name
  • We rewire all Table names in quote /quote quote_item /quote/item quote_address /quote/address
  • The new Collection Model $rows = $newAdapter->loadDataCollection( $this->getTableName(), $this->renderFilters() ); foreach($rows as $row) { $object = new Object(); $object->setData($data); $this->addItem($object); }
  • Loading a collection of things (filtered) loadDataCollection(‘/quote/item’, array(product_id => 1257)); ● { item_id: 100, product_id: 1257, qty: 1 } ● { item_id: 342, product_id: 1257, qty: 2 } ● ... db.quote.find( { ‘item.product_id’: 1257 }, { ‘item’: 1 } ); ● { ‘item’:[ ● { ‘item’:[ ● ... {item_id: 100, product_id: 1257, qty: 1} ] } {item_id: 342, product_id: 1257, qty: 2} ] }
  • as a developer in // loading a filtered collection of quote items from database $items = Mage::getModel(‘sales/quote_item’)->getCollection(); $items->addFieldToFilter(‘quote_id’, 560); $items->addFieldToFilter(‘product_id’, 1257); $items->load();
  • Loading a collection of things (filtered) ● This is not a relational database anymore ● Quote Items may not have a quote_id in our schema loadDataCollection( ‘/quote/item’, array( ‘quote_id’ => 560, ‘product_id’ => 1257) ); - no result db.quote.find( { ‘item.quote_id’: 560, ‘item.product_id’: 1257 }, { ‘item’: 1 } ); - no result
  • Loading a collection of things (filtered) loadDataCollection( ‘/quote{quote_id:560}/item’, array(product_id=> 1257)); ● { item_id: 100, product_id: 1257, qty: 1 } db.quote.find( ● { ‘item’:[ {‘quote_id’: 560, ‘item.product_id’: 1257}, { ‘item’: 1 } ); {item_id: 100, product_id: 1257, qty: 1} ] }
  • On-the-fly Tablename completion getTablename() /quote{quote_id:$quote_id}/item completePath(filters, properties) /quote{quote_id:560}/item
  • as a developer in // load a single item from db, change qty, save it $item = Mage::getModel(‘sales/quote_item’)->load(101); $item->setQty(2); $item->save(); // add a product to cart $item = Mage::getModel(‘sales/quote_item’); $item->setQuoteId(560)->setProductId(1566)->setQty(1); $item->save();
  • Loading a single record loadData( ‘/quote{quote_id:560}/item’, ‘item_id’, 100); findOne({ ‘quote_id’: 560, ‘item.item_id’: 100}, {‘item.$’: 1}); loadData( ‘/quote/item’, ‘item_id’, 100); loadData( ‘/quote{quote_id:$quote_id}/item’, ‘item_id’, 100); findOne({ ‘item.item_id’: 100}, {‘item.$’: 1}); Result for all three { item_id: 100, product_id: 1257, qty: 1 }
  • Saving a single record Inserting saveData( ‘/quote{quote_id:560}/item’, array(‘item_id’ => 732, ‘product_id’ => 1257, ‘qty’ => 1)); db.quote.update( { quote_id: 560 }, { $push : {‘item’ => { item_id: 732, product_id: 1257, qty: 1 }} }); Updating saveData( ‘/quote/item’, array(‘item_id’ => 732, ‘qty’ => 2)); saveData( ‘/quote{quote_id:$quote_id}/item’, ...); db.quote.update( { item.item_id: 732 }, { $set : { item.$.qty: 2 } } );
  • The new resource model function load($object, $id) { $data = $adapter->loadData( $this->getTablename(), $this->getIdFieldname(), $id); $object->setData( $data ); }
  • The new resource model function save($object) { $id = $this->getNewId(); $data = $adapter->saveData( $this->getTablename(), $this->getIdFieldname(), $id, $object->getData()); $object->setId($id); }
  • Migration of old data in Application create a simple application to ● load using the old resource model ● save using the new resource model Old Resource Model Model New Resource Model DB
  • Thanks a lot Firefly Glow Cube Dining Table Campagna