Migrating One of the Most Popular eCommerce Platforms to MongoDB

5,081 views

Published on

Is it worthwhile to migrate a heavy SQL application to MongoDB? In this talk we show our insights and positive outcomes on the migration of a popular open source eCommerce platform (Magento) to MongoDB. The talk covers interesting migration challenges and techniques both from a data and software point of view.

Published in: Technology, Business
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,081
On SlideShare
0
From Embeds
0
Number of Embeds
1,122
Actions
Shares
0
Downloads
86
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Migrating One of the Most Popular eCommerce Platforms to MongoDB

  1. 1. Migrating One of the Most Popular eCommerce Platforms to MongoDB MongoDB Munich 2013, October 14th Aron Spohr, fashion4home GmbH, Berlin aron.spohr@fashionforhome.de
  2. 2. Will it blend?
  3. 3. What is Magento? ● ● ● ● ● ● ● Open eCommerce platform Serves all primary features of a Webshop Written in PHP Works on top of MySQL out of the box Extensible architecture Runs on over 180,000 sites eBay owned since 2011
  4. 4. The Shopping Cart ● Quote ● Items ● Addresses ● one Model for each ● one MySQL-Table for each
  5. 5. Data Structure quote_id discount_amount grand_total 560 100.00 299.00 561 0.00 quote 1028.40 quote_item item_id quote_id product_id qty price 100 560 1257 1 39.90 101 560 1349 2 140.10 quote_address address_id quote_id city street type 388 560 Munich Hauptstr. 33a shipping 389 560 Berlin Zahlstrasse 12 billing 390 561 Hamburg Geheimweg 3 shipping, billing
  6. 6. as a developer in // Loading a quote from database $quote = Mage::getModel(‘sales/quote’)->load(560); SELECT * FROM sales_quote WHERE quote_id=560; // Loading a filtered collection of quote items from database $items = $quote->getItemCollection(); $items->addFieldToFilter(‘product_id’, 1257); $items->load(); SELECT * FROM sales_quote_item WHERE quote_id=560 AND product_id=1257;
  7. 7. Persistence in Application every Model has ● a Resource Model to load/save one record from/to DB ● a Collection Model to load multiple records from DB Model Resource Model DB Collection Model Model Model Model
  8. 8. Resource Model basics of function load($object, $id) { $stmt = “SELECT * FROM “ . $this->getTableName() . ” WHERE “ . $this->getIdFieldname() . ”=$id”; $data = $sqlAdapter->fetchRow( $stmt ); $object->setData( $data ); }
  9. 9. Collection Model basics of function load() { $stmt = “SELECT * FROM “ . $this->getTableName() . ” WHERE “ . $this->renderFilters(); foreach($sqlAdapter->fetchRows( $stmt ) as $row) { $object = new Object(); $object->setData($data); $this->addItem($object); } }
  10. 10. Why should we change that? ● You don’t have to ● It always depends on your application Reasons we had: ● Have more/other scalability options ● Make it easier to work with raw data ● Be more flexible with your data schema ● Learn more about the software you are using
  11. 11. Let’s get started ● ● ● ● not change the way Magento works a very simple, self-explainable data schema make it easy to migrate old data not lose any existing features
  12. 12. Data Structure - mongoDB { quote_id: 560, discount_amount: 100.00, grand_total: 299, item: [ {item_id: 100, product_id: 1257, qty: 1, price: 39.9}, {item_id: 101, product_id: 1349, qty: 2, price: 140.10} ], address: [ {address_id: 388, city: ‘Munich’, street: ‘Hauptstr. 33a’, ...}, {address_id: 389, city: ‘Berlin’, street: ‘Zahlstrasse 12’, ...} ] }
  13. 13. Do we still have a table? SELECT * FROM quote_item; db.quote.find( {}, { ‘item’: 1 } ); ● { ‘item’:[ {item_id: 100, product_id: 1257, qty: 1 }, {item_id: 101, product_id: 1349, qty: 2 } ] } ● { ‘item’:[ {item_id: 102, product_id: 4421, qty: 1 } ] } ● { ‘item’:[ {item_id: 103, product_id: 2301, qty: 1 }, {item_id: 104, product_id: 5511, qty: 1 } ] } ● ...
  14. 14. a Tool to simplify our work with mongoDB...
  15. 15. Loading a collection of things loadDataCollection(‘/quote/item’, array()); ● ● ● ● { item_id: 100, product_id: 1257, qty: 1 } { item_id: 101, product_id: 1349, qty: 2 } { item_id: 102, product_id: 4421, qty: 1 } ... db.quote.find( {}, { ‘item’: 1 } ); ● { ‘item’:[ ● { ‘item’:[ ● ... {item_id: 100, product_id: 1257, qty: 1}, {item_id: 101, product_id: 1349, qty: 2} ] } {item_id: 102, product_id: 4421, qty: 1} ] }
  16. 16. The path to our data Name of Collection /quote /item Name of Array in document ● Tells us where to find the data ● Is very similar to a table name
  17. 17. We rewire all Table names in quote /quote quote_item /quote/item quote_address /quote/address
  18. 18. The new Collection Model $rows = $newAdapter->loadDataCollection( $this->getTableName(), $this->renderFilters() ); foreach($rows as $row) { $object = new Object(); $object->setData($data); $this->addItem($object); }
  19. 19. Loading a collection of things (filtered) loadDataCollection(‘/quote/item’, array(product_id => 1257)); ● { item_id: 100, product_id: 1257, qty: 1 } ● { item_id: 342, product_id: 1257, qty: 2 } ● ... db.quote.find( { ‘item.product_id’: 1257 }, { ‘item’: 1 } ); ● { ‘item’:[ ● { ‘item’:[ ● ... {item_id: 100, product_id: 1257, qty: 1} ] } {item_id: 342, product_id: 1257, qty: 2} ] }
  20. 20. as a developer in // loading a filtered collection of quote items from database $items = Mage::getModel(‘sales/quote_item’)->getCollection(); $items->addFieldToFilter(‘quote_id’, 560); $items->addFieldToFilter(‘product_id’, 1257); $items->load();
  21. 21. Loading a collection of things (filtered) ● This is not a relational database anymore ● Quote Items may not have a quote_id in our schema loadDataCollection( ‘/quote/item’, array( ‘quote_id’ => 560, ‘product_id’ => 1257) ); - no result db.quote.find( { ‘item.quote_id’: 560, ‘item.product_id’: 1257 }, { ‘item’: 1 } ); - no result
  22. 22. Loading a collection of things (filtered) loadDataCollection( ‘/quote{quote_id:560}/item’, array(product_id=> 1257)); ● { item_id: 100, product_id: 1257, qty: 1 } db.quote.find( ● { ‘item’:[ {‘quote_id’: 560, ‘item.product_id’: 1257}, { ‘item’: 1 } ); {item_id: 100, product_id: 1257, qty: 1} ] }
  23. 23. On-the-fly Tablename completion getTablename() /quote{quote_id:$quote_id}/item completePath(filters, properties) /quote{quote_id:560}/item
  24. 24. as a developer in // load a single item from db, change qty, save it $item = Mage::getModel(‘sales/quote_item’)->load(101); $item->setQty(2); $item->save(); // add a product to cart $item = Mage::getModel(‘sales/quote_item’); $item->setQuoteId(560)->setProductId(1566)->setQty(1); $item->save();
  25. 25. Loading a single record loadData( ‘/quote{quote_id:560}/item’, ‘item_id’, 100); findOne({ ‘quote_id’: 560, ‘item.item_id’: 100}, {‘item.$’: 1}); loadData( ‘/quote/item’, ‘item_id’, 100); loadData( ‘/quote{quote_id:$quote_id}/item’, ‘item_id’, 100); findOne({ ‘item.item_id’: 100}, {‘item.$’: 1}); Result for all three { item_id: 100, product_id: 1257, qty: 1 }
  26. 26. Saving a single record Inserting saveData( ‘/quote{quote_id:560}/item’, array(‘item_id’ => 732, ‘product_id’ => 1257, ‘qty’ => 1)); db.quote.update( { quote_id: 560 }, { $push : {‘item’ => { item_id: 732, product_id: 1257, qty: 1 }} }); Updating saveData( ‘/quote/item’, array(‘item_id’ => 732, ‘qty’ => 2)); saveData( ‘/quote{quote_id:$quote_id}/item’, ...); db.quote.update( { item.item_id: 732 }, { $set : { item.$.qty: 2 } } );
  27. 27. The new resource model function load($object, $id) { $data = $adapter->loadData( $this->getTablename(), $this->getIdFieldname(), $id); $object->setData( $data ); }
  28. 28. The new resource model function save($object) { $id = $this->getNewId(); $data = $adapter->saveData( $this->getTablename(), $this->getIdFieldname(), $id, $object->getData()); $object->setId($id); }
  29. 29. Migration of old data in Application create a simple application to ● load using the old resource model ● save using the new resource model Old Resource Model Model New Resource Model DB
  30. 30. Thanks a lot Firefly Glow Cube Dining Table Campagna

×