Varnish your Magento store, make it fly!

Ibiza, June 4th – 7th 2011Monday, 6 June 2011

VARNISH YOUR MAGENTO STORE Make it fly!Monday, 6 June 2011

WHO AM I • Alistair Stead - @alistairstead • Technical Team Lead @ Ibuildings UK / Session Digital • Lead Magento projects for Dreams, 3663, Kookai and others • Author of MageTool and MageTest • Zend Certiﬁed Engineer • Over 11 years commercial experience developing in PHPMonday, 6 June 2011

PERFORMANCE AND SCALABILITY Is a hot topic for clients and developers alike.Monday, 6 June 2011

PERFORMANCE Vs CONVERSION RATE These are directly linked!Monday, 6 June 2011

PERCEIVED PERFORMANCE Vs PHYSICAL PERFORMANCE Is there an easy route to performance?Monday, 6 June 2011

VARNISH http://www.varnish-cache.org/Monday, 6 June 2011

PRELIMINARY LOAD TESTS What can Varnish do for us?Monday, 6 June 2011

+4,000,000 Requests per hourMonday, 6 June 2011

REVERSE PROXY? Yeah okay... tell me more!Monday, 6 June 2011

CACHING PROXY? It is...Monday, 6 June 2011

LOAD BALANCER? It can do that too!Monday, 6 June 2011

HTTP ACCELERATOR It may just be magic!Monday, 6 June 2011

HOW TO CONTROL CACHING Additional caching layers need to be coordinatedMonday, 6 June 2011

RFC 2616 HTTP 1.1 http://www.w3.org/Protocols/rfc2616/rfc2616.html http://tools.ietf.org/wg/httpbis/Monday, 6 June 2011

HTTP 1.1 $ curl -I magentocommerce.comMonday, 6 June 2011

$ curl -I magentocommerce.com HTTP/1.1 200 OK Server: nginx Date: Sun, 05 Jun 2011 17:40:56 GMT Content-Type: text/html; charset=UTF-8 Connection: keep-alive X-Powered-By: PHP/5.2.17 Set-Cookie: PHPSESSID=j1cdbj2617abbl8ccsbkd2mt44; path=/ Expires: Mon, 26 Jul 1997 05:00:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post- check=0, pre-check=0 Pragma: no-cache Last-Modified: Sun, 05 Jun 2011 17:40:56 GMTMonday, 6 June 2011

HTTP 1.1 HEADERS Request modiﬁers / Response parametersMonday, 6 June 2011

CACHEING WITH HTTP 1.1Monday, 6 June 2011

EXPIRATION MODEL How long may I use this resource for?Monday, 6 June 2011

EXPIRATION HEADERS • Cache-Control • ExpiresMonday, 6 June 2011

VALIDATION MODEL Is this resource still okay to use?Monday, 6 June 2011

VALIDATION HEADERS • Last-Modiﬁed • If-Modiﬁed-Since • ETag • If-None-MatchMonday, 6 June 2011

HTTP CACHE HEADERS Only work with safe HTTP verbs GET & HEADMonday, 6 June 2011

HTTP 1.1 REQUEST FLOWMonday, 6 June 2011

GET / HTTP1.1 Host: mystore.com GET / HTTP1.1 Host: mystore.com Client Cache Application Client HTTP1.1 200 OK HTTP1.1 200 OKMonday, 6 June 2011

GET / HTTP1.1 Host: mystore.com GET / HTTP1.1 Host: mystore.com Client Cache Application Client HTTP1.1 200 OK Cache-Control: max- age=20 HTTP1.1 200 OK Cache-Control: max- HTTP1.1 200 OK age=20 Cache-Control: max- age=20Monday, 6 June 2011

GET / HTTP1.1 Host: mystore.com Client Cache Application Client HTTP1.1 200 OK Cache-Control: max- age=20 HTTP1.1 200 OK Cache-Control: max- age=20Monday, 6 June 2011

GET / HTTP1.1 Host: mystore.com GET / HTTP1.1 Host: mystore.com Client Cache Application Client HTTP1.1 200 OK Etag: abcde HTTP1.1 200 OK Etag: abcde HTTP1.1 200 OK Etag: abcdeMonday, 6 June 2011

GET / HTTP1.1 Host: mystore.com GET / HTTP1.1 Host: mystore.com If-None-Match: abcde Client Cache Application Client HTTP1.1 200 OK Etag: abcde HTTP1.1 304 Not Modiﬁed HTTP1.1 304 Not ModiﬁedMonday, 6 June 2011

EXPIRATION ALLOWS YOU TO SCALE VALIDATION SAVES BANDWIDTH Fewer requests hitting your server and client speed is better too!Monday, 6 June 2011

OUR GOAL IS TO NEVER GENERATE THE SAME RESPONSE TWICE HTTP caching can helpMonday, 6 June 2011

LEVERAGE HTTP CACHE IN MAGENTO With an event observer this is simple.Monday, 6 June 2011

<?php class Disclosure_Varnish_Model_Observer extends Disclosure_Varnish_Model_Observer_Abstract { /** * Update the headers transmitted to allow greater varnish cache hits * * @param $observer */ public function processPreDispatch(Varien_Event_Observer $observer) { if (!$this->isVarnishEnabled()) { return $this; } $routeTtls = $this->_config->getRouteTtls(); $routeName = Mage::app()->getRequest()->getRequestedRouteName(); if (isset($routeTtls[$routeName])) { $observer->getResponse()->setHeader(Cache-Control, $routeTtls[$routeName][cache_control], false); } return $this; } }Monday, 6 June 2011

<?xml version="1.0" encoding="UTF-8"?> <config> <ttls> <routes> <cms> <name>cms</name> <cache_control>public, max-age=3600</cache_control> </cms> <catalog> <name>catalog</name> <cache_control>public, must-revalidate, max-age=0</cache_control> <actions> <index> <cache_control>private, max-age=0</cache_control> </index> </actions> </catalog> </routes> </ttls>Monday, 6 June 2011

USE VARNISH WITH MAGENTO Add shared cache to reduce the load on the application Backend Client Varnish Apache Browser Port 80 Port 8080Monday, 6 June 2011

VARNISH RESPECTS CACHE CONTROL Dispatch the headers that ﬁt your applicationMonday, 6 June 2011

GET / HTTP1.1 Host: mystore.com GET / HTTP1.1 Host: mystore.com Reverse Proxy Cache Client Cache Application Client HTTP1.1 200 OK Cache-Control: max- age=20 HTTP1.1 200 OK Cache-Control: max- HTTP1.1 200 OK age=20 Cache-Control: max- age=20Monday, 6 June 2011

GET / HTTP1.1 Host: mystore.com Reverse Proxy Cache Client Cache Application Client HTTP1.1 200 OK Cache-Control: max- age=20 HTTP1.1 200 OK Cache-Control: max- age=20Monday, 6 June 2011

GET / HTTP1.1 GET / HTTP1.1 Client Cache Host: mystore.com Host: mystore.com HTTP1.1 200 OK Client Cache-Control: max- age=20 HTTP1.1 200 OK HTTP1.1 200 OK Reverse Proxy Cache Cache-Control: max- Cache-Control: max- age=20 Application age=20 GET / HTTP1.1 Host: mystore.com Client Cache Client HTTP1.1 200 OK Cache-Control: max- age=20Monday, 6 June 2011

Client ClientMonday, 6 June 2011 Client Cache Client Cache max-age max-age Network Proxy Cache max-age s-maxage Gateway Cache max-age s-maxage Reverse Proxy Cache max-age s-maxage Application

VARNISH IS CONFIGURED USING VCL Send the correct headers you should not need to do much.Monday, 6 June 2011

sub vcl_recv { set req.http.Surrogate-Capability = "magento=ESI/1.0"; return (lookup); } sub vcl_fetch { if (beresp.http.Surrogate-Control ~ "ESI/1.0") { # unset beresp.http.Surrogate-Control; esi; } return (deliver); } sub vcl_deliver { # Set a cache header to allow us to inspect the response # headers during testing if (obj.hits > 0) { set resp.http.X-Cache = "HIT"; set resp.http.X-Cache-Hits = obj.hits; } else { set resp.http.X-Cache = "MISS"; } }Monday, 6 June 2011

EDGE SIDE INCLUDES http://www.w3.org/TR/esi-langMonday, 6 June 2011

max-age=3600 max-age=86400Monday, 6 June 2011

ESI TAGS <esi:include src="/varnish/esi?attributes=%s"/>Monday, 6 June 2011

/** * Render ESI tags for the block if needed * * @param Varien_Event_Observer $observer */ public function renderBlockEsiTag(Varien_Event_Observer $observer) { if (!$this->isEsiEnabled() || !$this->isVarnishEnabled()) { return $this; } $block = $observer->getEvent()->getBlock(); $transport = $observer->getEvent()->getTransport(); $tag = Mage::getModel(varnish/esi_tag_factory)->create($block); if ($transport && $tag) { $transport->setHtml($tag->getHtml()); } return $this; }Monday, 6 June 2011

/** * Render the html for the ESI tag * * @return string * @author Alistair Stead **/ public function getHtml() { return sprintf( <esi:include src="/varnish/esi?attributes=%s"/>, $this->getAttributeHash() ); }Monday, 6 June 2011

class Disclosure_Varnish_EsiController extends Mage_Core_Controller_Front_Action { public function indexAction() { $attributes = unserialize(base64_decode($this->getRequest()->getParam(attributes))); $context = Mage::getModel(varnish/esi_context); $context->setData($attributes); $processor = Mage::getModel(varnish/esi_processor, $context); // Initialize the layout $this->loadLayout(); Mage::dispatchEvent(varnish_controller_esi_appendbody_before, array("response" => $this->getResponse())); $this->getResponse()->appendBody($processor->render()); Mage::dispatchEvent(varnish_controller_esi_appendbody_after, array("response" => $this->getResponse())); } }Monday, 6 June 2011

FINAL PERFORMANCE RESULTS How much trafﬁc did these modiﬁcations let us serveMonday, 6 June 2011

TRAFFIC SAMPLE • Product Details page: 58k views per hour • Login: 11k requests per hour • Homepage: 50k requests per hour • Category Pages: 46k requests per hour • Main Search : 70k requests per hour • Checkout: 6k requests per hour • Reﬁned Search (via facets): 20k requests per hour • Add to Basket: 11k requests per hourMonday, 6 June 2011

FINAL PERFORMANCE RESULTS Homepage Category Listings Product Detail Add to cart Login Search Purchase 0 375000 750000 1125000 1500000 Client Baseline Magento Baseline Magento Compiler Magento + VarnishMonday, 6 June 2011

NON CACHEABLE TRANSACTIONS Add to cart Purchase 0 7500 15000 22500 30000 Client Baseline Magento Baseline Magento Compiler Magento + VarnishMonday, 6 June 2011

Monday, 6 June 2011

THE TARGET ~ 300K REQUEST We achieved almost 14 x that with little effort.Monday, 6 June 2011

IS VARNISH THE BEST SOLUTION? What do we need to consider?Monday, 6 June 2011

VARNISH LIMITATIONS • Does not currently support GZIP • Cookies need to be managed carefully • Still need SSL endpoint • Routing and URL generation can be tricky • Beware cache stampede • ESI must have a TTL greater than the parent page • Shared cache consider using s-maxage with max-ageMonday, 6 June 2011

ADDITIONAL OPTIONS Other HTTP caching options and solutionsMonday, 6 June 2011

CLIENT SIDE INCLUDES A common pattern using AjaxMonday, 6 June 2011

STALE WHILE Maintain a compromised UI while resolving contentMonday, 6 June 2011

ALTERNATE WHILE Provide alternate content will content is being resolvedMonday, 6 June 2011

REFERENCES • http://blog.sessiondigital.com/post/5764403845/high-performance-magento • http://www.varnish-cache.org/ • http://www.w3.org/Protocols/rfc2616/rfc2616.html • http://tools.ietf.org/wg/httpbis/ • http://www.w3.org/TR/esi-lang • http://www.fabrizio-branca.de/make-your-magento-store-ﬂy-using-varnish.htmlMonday, 6 June 2011

IMAGE CREDITS http://www.ﬂickr.com/photos/bondgirly/4520060395/sizes/o/in/photostream/Monday, 6 June 2011

THANK YOU! • Email: astead@ibuildings.com • Skype: astead-ibuildings • Twitter: @alistairsteadMonday, 6 June 2011

QUESTIONS?Monday, 6 June 2011

WE ARE HIRING! http://www.ibuildings.co.uk/about/careers/Monday, 6 June 2011

Monday, 6 June 2011

http://lanyrd.com	88
http://a0.twimg.com	1
https://twimg0-a.akamaihd.net	1

Varnish your Magento store, make it fly!

by astead on Jun 06, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 90

Statistics

Varnish your Magento store, make it fly! — Presentation Transcript