• Like

Migrating One of the Most Popular eCommerce Platforms to MongoDB

  • 1,720 views
Uploaded on

Is it worthwhile to migrate a heavy SQL application to MongoDB? In this talk we show our insights and positive outcomes on the migration of a popular open source eCommerce platform (Magento) to …

Is it worthwhile to migrate a heavy SQL application to MongoDB? In this talk we show our insights and positive outcomes on the migration of a popular open source eCommerce platform (Magento) to MongoDB. The talk covers interesting migration challenges and techniques both from a data and software point of view.

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,720
On Slideshare
0
From Embeds
0
Number of Embeds
5

Actions

Shares
Downloads
30
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Migrating One of the Most Popular eCommerce Platforms to MongoDB MongoDB Munich 2013, October 14th Aron Spohr, fashion4home GmbH, Berlin aron.spohr@fashionforhome.de
  • 2. Will it blend?
  • 3. What is Magento? ● ● ● ● ● ● ● Open eCommerce platform Serves all primary features of a Webshop Written in PHP Works on top of MySQL out of the box Extensible architecture Runs on over 180,000 sites eBay owned since 2011
  • 4. The Shopping Cart ● Quote ● Items ● Addresses ● one Model for each ● one MySQL-Table for each
  • 5. Data Structure quote_id discount_amount grand_total 560 100.00 299.00 561 0.00 quote 1028.40 quote_item item_id quote_id product_id qty price 100 560 1257 1 39.90 101 560 1349 2 140.10 quote_address address_id quote_id city street type 388 560 Munich Hauptstr. 33a shipping 389 560 Berlin Zahlstrasse 12 billing 390 561 Hamburg Geheimweg 3 shipping, billing
  • 6. as a developer in // Loading a quote from database $quote = Mage::getModel(‘sales/quote’)->load(560); SELECT * FROM sales_quote WHERE quote_id=560; // Loading a filtered collection of quote items from database $items = $quote->getItemCollection(); $items->addFieldToFilter(‘product_id’, 1257); $items->load(); SELECT * FROM sales_quote_item WHERE quote_id=560 AND product_id=1257;
  • 7. Persistence in Application every Model has ● a Resource Model to load/save one record from/to DB ● a Collection Model to load multiple records from DB Model Resource Model DB Collection Model Model Model Model
  • 8. Resource Model basics of function load($object, $id) { $stmt = “SELECT * FROM “ . $this->getTableName() . ” WHERE “ . $this->getIdFieldname() . ”=$id”; $data = $sqlAdapter->fetchRow( $stmt ); $object->setData( $data ); }
  • 9. Collection Model basics of function load() { $stmt = “SELECT * FROM “ . $this->getTableName() . ” WHERE “ . $this->renderFilters(); foreach($sqlAdapter->fetchRows( $stmt ) as $row) { $object = new Object(); $object->setData($data); $this->addItem($object); } }
  • 10. Why should we change that? ● You don’t have to ● It always depends on your application Reasons we had: ● Have more/other scalability options ● Make it easier to work with raw data ● Be more flexible with your data schema ● Learn more about the software you are using
  • 11. Let’s get started ● ● ● ● not change the way Magento works a very simple, self-explainable data schema make it easy to migrate old data not lose any existing features
  • 12. Data Structure - mongoDB { quote_id: 560, discount_amount: 100.00, grand_total: 299, item: [ {item_id: 100, product_id: 1257, qty: 1, price: 39.9}, {item_id: 101, product_id: 1349, qty: 2, price: 140.10} ], address: [ {address_id: 388, city: ‘Munich’, street: ‘Hauptstr. 33a’, ...}, {address_id: 389, city: ‘Berlin’, street: ‘Zahlstrasse 12’, ...} ] }
  • 13. Do we still have a table? SELECT * FROM quote_item; db.quote.find( {}, { ‘item’: 1 } ); ● { ‘item’:[ {item_id: 100, product_id: 1257, qty: 1 }, {item_id: 101, product_id: 1349, qty: 2 } ] } ● { ‘item’:[ {item_id: 102, product_id: 4421, qty: 1 } ] } ● { ‘item’:[ {item_id: 103, product_id: 2301, qty: 1 }, {item_id: 104, product_id: 5511, qty: 1 } ] } ● ...
  • 14. a Tool to simplify our work with mongoDB...
  • 15. Loading a collection of things loadDataCollection(‘/quote/item’, array()); ● ● ● ● { item_id: 100, product_id: 1257, qty: 1 } { item_id: 101, product_id: 1349, qty: 2 } { item_id: 102, product_id: 4421, qty: 1 } ... db.quote.find( {}, { ‘item’: 1 } ); ● { ‘item’:[ ● { ‘item’:[ ● ... {item_id: 100, product_id: 1257, qty: 1}, {item_id: 101, product_id: 1349, qty: 2} ] } {item_id: 102, product_id: 4421, qty: 1} ] }
  • 16. The path to our data Name of Collection /quote /item Name of Array in document ● Tells us where to find the data ● Is very similar to a table name
  • 17. We rewire all Table names in quote /quote quote_item /quote/item quote_address /quote/address
  • 18. The new Collection Model $rows = $newAdapter->loadDataCollection( $this->getTableName(), $this->renderFilters() ); foreach($rows as $row) { $object = new Object(); $object->setData($data); $this->addItem($object); }
  • 19. Loading a collection of things (filtered) loadDataCollection(‘/quote/item’, array(product_id => 1257)); ● { item_id: 100, product_id: 1257, qty: 1 } ● { item_id: 342, product_id: 1257, qty: 2 } ● ... db.quote.find( { ‘item.product_id’: 1257 }, { ‘item’: 1 } ); ● { ‘item’:[ ● { ‘item’:[ ● ... {item_id: 100, product_id: 1257, qty: 1} ] } {item_id: 342, product_id: 1257, qty: 2} ] }
  • 20. as a developer in // loading a filtered collection of quote items from database $items = Mage::getModel(‘sales/quote_item’)->getCollection(); $items->addFieldToFilter(‘quote_id’, 560); $items->addFieldToFilter(‘product_id’, 1257); $items->load();
  • 21. Loading a collection of things (filtered) ● This is not a relational database anymore ● Quote Items may not have a quote_id in our schema loadDataCollection( ‘/quote/item’, array( ‘quote_id’ => 560, ‘product_id’ => 1257) ); - no result db.quote.find( { ‘item.quote_id’: 560, ‘item.product_id’: 1257 }, { ‘item’: 1 } ); - no result
  • 22. Loading a collection of things (filtered) loadDataCollection( ‘/quote{quote_id:560}/item’, array(product_id=> 1257)); ● { item_id: 100, product_id: 1257, qty: 1 } db.quote.find( ● { ‘item’:[ {‘quote_id’: 560, ‘item.product_id’: 1257}, { ‘item’: 1 } ); {item_id: 100, product_id: 1257, qty: 1} ] }
  • 23. On-the-fly Tablename completion getTablename() /quote{quote_id:$quote_id}/item completePath(filters, properties) /quote{quote_id:560}/item
  • 24. as a developer in // load a single item from db, change qty, save it $item = Mage::getModel(‘sales/quote_item’)->load(101); $item->setQty(2); $item->save(); // add a product to cart $item = Mage::getModel(‘sales/quote_item’); $item->setQuoteId(560)->setProductId(1566)->setQty(1); $item->save();
  • 25. Loading a single record loadData( ‘/quote{quote_id:560}/item’, ‘item_id’, 100); findOne({ ‘quote_id’: 560, ‘item.item_id’: 100}, {‘item.$’: 1}); loadData( ‘/quote/item’, ‘item_id’, 100); loadData( ‘/quote{quote_id:$quote_id}/item’, ‘item_id’, 100); findOne({ ‘item.item_id’: 100}, {‘item.$’: 1}); Result for all three { item_id: 100, product_id: 1257, qty: 1 }
  • 26. Saving a single record Inserting saveData( ‘/quote{quote_id:560}/item’, array(‘item_id’ => 732, ‘product_id’ => 1257, ‘qty’ => 1)); db.quote.update( { quote_id: 560 }, { $push : {‘item’ => { item_id: 732, product_id: 1257, qty: 1 }} }); Updating saveData( ‘/quote/item’, array(‘item_id’ => 732, ‘qty’ => 2)); saveData( ‘/quote{quote_id:$quote_id}/item’, ...); db.quote.update( { item.item_id: 732 }, { $set : { item.$.qty: 2 } } );
  • 27. The new resource model function load($object, $id) { $data = $adapter->loadData( $this->getTablename(), $this->getIdFieldname(), $id); $object->setData( $data ); }
  • 28. The new resource model function save($object) { $id = $this->getNewId(); $data = $adapter->saveData( $this->getTablename(), $this->getIdFieldname(), $id, $object->getData()); $object->setId($id); }
  • 29. Migration of old data in Application create a simple application to ● load using the old resource model ● save using the new resource model Old Resource Model Model New Resource Model DB
  • 30. Thanks a lot Firefly Glow Cube Dining Table Campagna