Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Our way to a Performant,
Self Healing, Highly Available
Magento Stack
Homework
• Which features are shipped by default with Magento?
• Which features are really needed for our Shop?
• How does they sca...
First Step
Making the core lightweight and more scaleable.
First Step
Making the core lightweight and more scaleable.
First Step
Making the core lightweight and more scaleable.
Making the core lightweight and more scaleable.
Removed not required core modules and also the ones
which brings poor perf...
Making the core lightweight and more scaleable.
Added great community modules
• Zookal_Mock
• Ecomdev_LayoutCompiler
• Inc...
Making the core lightweight and more scaleable.
Varien_Db_Select
• Introduced SQL_USE_CACHE AND SQL_NO_CACHE support
• Int...
Making the core lightweight and more scaleable.
Magento Indexer
• Rewrite all indexer to use SQL_NO_CACHE
• use CREATE TEM...
Making the core lightweight and more scaleable.
indexer for url rewrites
• Mysql implementation based on EcomDev_UrlRewrit...
Making the core lightweight and more scaleable.
Redis Cache Backend
• removed Credis libary and directly speak over pecl_r...
Making the core lightweight and more scaleable.
Magento Admin
• all SQL Queries in the admin interface by default use SQL_...
Making the core lightweight and more scaleable.
Block caching / layout.xml
• Introduced caching of html and also of tags a...
Making the core lightweight and more scaleable.
Block caching for products
• created a common product block
• template and...
Making the core lightweight and more scaleable.
Navigation Block
• Removed Usage of Varien_Data_Tree which is horrible slo...
Making the core lightweight and more scaleable.
Cache Locking
• add cache locking during config generationa
• add block ca...
Making the core lightweight and more scaleable.
Mage_Core_Controller_Varien_Front::dispatch
• added fastcgi_finish_request...
Infrastructure
Infrastructure
• Stackformation by AOE Media
• all nodes are based on gentoo compiled for their use case
• Provisioned wit...
Upcoming SlideShare
Loading in …5
×

Magento performance feat. core Hacks

439 views

Published on

A small presentation about our stuff we did to get an performant magento core

Published in: Engineering
  • Be the first to like this

Magento performance feat. core Hacks

  1. 1. Our way to a Performant, Self Healing, Highly Available Magento Stack
  2. 2. Homework
  3. 3. • Which features are shipped by default with Magento? • Which features are really needed for our Shop? • How does they scale? • How does the default implementation working with high load scenarios like tv-ads? • How does the default implementation is cacheable? • Which features are not shipped per default we need to develop on our own? • Which knowledge we have in house to deliver all the required features?
  4. 4. First Step Making the core lightweight and more scaleable.
  5. 5. First Step Making the core lightweight and more scaleable.
  6. 6. First Step Making the core lightweight and more scaleable.
  7. 7. Making the core lightweight and more scaleable. Removed not required core modules and also the ones which brings poor performance. • AdminNotification • Authorizenet • Backup • Captcha • Compiler • ConfigurableSwatches • Connect • Downloadable • Google* • Newsletter • Oauth • PageCache • .... and even more
  8. 8. Making the core lightweight and more scaleable. Added great community modules • Zookal_Mock • Ecomdev_LayoutCompiler • Inchoo_PHP7 (Directly in the affected core files)
  9. 9. Making the core lightweight and more scaleable. Varien_Db_Select • Introduced SQL_USE_CACHE AND SQL_NO_CACHE support • Introduced FORCE INDEX • Introduced IGNORE INDEX • Introduced USE INDEX
  10. 10. Making the core lightweight and more scaleable. Magento Indexer • Rewrite all indexer to use SQL_NO_CACHE • use CREATE TEMPOARY TABLE to avoid growing transaction log • Implemented Table rotation to reclaim table space • all index events are processed asyncronius via rabbitmq (except stock after checkout) • Array_map(‘intval‘, $entityId) to avoid mixed data types in queries • process full reindex in chunks to reduce possibility of slave replication lag • Implemented ORDER BY NULL for indexers to avoid tmp_tables_on_disc
  11. 11. Making the core lightweight and more scaleable. indexer for url rewrites • Mysql implementation based on EcomDev_UrlRewrite with some parts as UDF in C • add new unsigned integer column rewrite_hash • Crc32 over store_id and request path • add index over rewrite_hash and first 10 characters of request_path
  12. 12. Making the core lightweight and more scaleable. Redis Cache Backend • removed Credis libary and directly speak over pecl_redis • add the possibility to make cache tags persistent (max memory policy) • implemented master / slave connections
  13. 13. Making the core lightweight and more scaleable. Magento Admin • all SQL Queries in the admin interface by default use SQL_NO_CACHE • Long running tasks are handled by rabbitmq via worker
  14. 14. Making the core lightweight and more scaleable. Block caching / layout.xml • Introduced caching of html and also of tags and object id (required for varnish tagging) • Cache tags are also applied to parent block • removed symlink check • make layout handles for category and product configurable if needed
  15. 15. Making the core lightweight and more scaleable. Block caching for products • created a common product block • template and caching configurable over layout.xml • holepunching possible for price block • accepts product id or full product data • fetches data from collection with LIMIT 1 and also caches collection response • implemented Observer AddStockStatusToCollection within product load • product urls are unique and can be directly joined • use column target_path and store_id with CONCAT
  16. 16. Making the core lightweight and more scaleable. Navigation Block • Removed Usage of Varien_Data_Tree which is horrible slow • $this->getCategoryCollection()->getItemById($categoryId) • Cache query result for $this->getCategoryCollection()
  17. 17. Making the core lightweight and more scaleable. Cache Locking • add cache locking during config generationa • add block cache locking confgurable via layout.xml
  18. 18. Making the core lightweight and more scaleable. Mage_Core_Controller_Varien_Front::dispatch • added fastcgi_finish_request (php-fpm only)
  19. 19. Infrastructure
  20. 20. Infrastructure • Stackformation by AOE Media • all nodes are based on gentoo compiled for their use case • Provisioned with Packer and Managed by Chef • All Services HA • Galera Cluster • Cache Replication • Varnish Cache with Tagging and Segmentation • mostly spot instances • different instances for night day and peak • Scaling handled by Lamda and a C Daemon which watches for termination notification • Fast upscaling / slow downscaling • Downscaling happends only if instances are reaching the next payment iteration • all instance related metrics calculated down to one cpu core (scaling over spot price) • Elasticsearch for catalog and search • Google Analytics API for feeding dynamic weighting for products

×