Nice performance using Sf2 cache wrapping Sf1 application
•
3 likes•1,915 views
In collaboration with Emmanuel Cohen. At a key moment for online press in France, a major French news company chooses PHP and Symfony to extend its popular web site. We will present the architecture we designed at Sensio Labs to meet a very good performance requirement. We used Symfony2 kernel wrapping symfony 1.4 and relied on loose-coupled applications serving content from heterogeneous backend sources.
![Who we are ,[object Object],[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Nice performance using Sf2 cache wrapping Sf1 application
Similar to Nice performance using Sf2 cache wrapping Sf1 application (20)
Recently uploaded
Recently uploaded (20)
Nice performance using Sf2 cache wrapping Sf1 application
- 1. Nice performance using SF2 cache wrapping sf1 application Marc Weistroff Emmanuel Cohen Sf Live SF 2011 Tuesday, February 8 th 2011
- 13. How does this app work ?
- 22. ESI : A page in lexpress.fr
- 24. How does ESI work?
- 26. /movie/dogma
- 27. /movie/dogma
- 30. First request ever Client Proxy Application /movie/dogma /movie/dogma miss Cache-Control: s-maxage=600 /movie/dogma/casting /movie/dogma/critics Cache-Control: s-maxage=3600 Cache-Control: s-maxage=300
- 31. Request at t+200 Client Proxy Application /movie/dogma hit
- 32. The application is never hit! As if the complete page was cached in the reverse proxy
- 33. Request at t+500 Client Proxy Application /movie/dogma /movie/dogma/critics Cache-Control: s-maxage=300 hit
- 34. The application is partially hit And has to build only a small fragment of the page
- 37. But our app is symfony 1 right ?
- 39. Wrapping symfony 1 with Symfony2
- 41. symfony 1 needs tweaking in order to work around a few obstacles
- 46. Final app architecture EsiCacheKernel SymfonyWrapperKernel symfony 1 application
- 48. Then… You have to design your app to serve HTML fragments!
- 50. And now where happy
- 51. What to do next ?
- 52. Proxy ESI New response Or 304 URL-> {ETag} ETag-> lastmodified If Etag absent or stale Response with HTTP headers Etag LastModified Client Request with HTTP headers If-None-Match If-Modified-Since App New response Or 304 If cache entry is not fresh enough Cache Validation Optimizer
- 55. Thank you! Please rate this talk at http://joind.in/2596
Editor's Notes
- I’m a project manager at Sensio Labs. I worked more than 5 years with Java, as a developper and architect. Then I switched to PHP 4 years ago.
- I’m a web developer at sensio labs and I’m the lead developer on the whole project. I started programming with Amos Basic on Amiga 500 and then discovered C and PHP. You can find me on twitter, but don’t bother following me as I mostly tweet about stupid stuff and pandas.
- This talk is about Online media, Caching and architecture, Edge Side Includes, symfony 1 and of course Symfony 2. Marc and I would like to talk to you about a project that we did for a client last year, back in Paris, with the precious help of Fabien. At that time, Fabien had already implemented, very conveniently indeed, the concepts of Edge Side Include and HTTP cache in Symfony2. Which fitted exactly our client need, who was looking for a way to improve its cache handling for the next version of its web site. It is also a use case of the talk Fabien gave earlier about HTTP Cache. I will present our client, the project and the architecture we designed for him. Marc will explain the implementation.
- Our client’s name is L’Express Magazine. It’s a general-interest magazine in France. It was created in 1953 . Due to its long and rich history, it is quite a famous media,both offline and online, in France. Today its online version, lexpress.fr is the 3 rd online news web site, which makes it reasonably in need for performance.
- This is from the existing web site, lexpress.fr It’s a popular site. Talks about hot topics…
- … Hot people (French cancan)
- unknown people
- Our project is an extension of the existing web site. It is a knowledge base of the cultural scene in France a way for the visitor to browse hot news and link it to cultural knowledge
- And this is what it looks like Here is a page about a movie that is about a web site that you may have heard of… this is a typical page of our site displaying a pile of raw information about directors, publishers, news, critics, movie ratings… The user can browse all kinds of information linked to the movie, and relate it to articles of lexpress.fr.
- From the very beginning of the project, our objectives are : * Performance We want to deliver as fast as possible while keeping their cost of hosting low. For that, we will use a little dose of Symfony2. * Our client has a vision for the future of the site : So they want our application to serve as a full-scale proof of concept that they will use as a basis to build the next version of their web site. We have multiple entry points, that we do not entirely control, we have a need for performance But we need to keep it simple (and loosely-coupled) (we, at Sensio, are required to say simple as often as we can)
- We need to take a few things into account 1. Various sources delivering XML content And we need to aggregate those sources 2. Today Symfony 2 can natively support the whole architecture. But we started the project in June last year. At that time Symfony 2 was not stable yet. So we decided to implement it in Sf1 while taking advantage of some of Sf2 features. 3. We need to stay as generic as possible, so as to be able to support future developments of our client
- At the bottom, you have multiple sources of information, including a content-management system that was developped by the team at L'express. At L’Express, they have a technical team that works on web R&D and innovation. Ocari is the in-house content manager system. It is based on Symfony 1 and generates xml content. Last year at the Paris Symfony Live L'Express came to present us the way Ocari, their content manager is working, based on XML content management and schedulers that handle what should be displayed and when. They also have the project to port it on Symfony2 and to make it open source. 2. Ocari Culture is a backoffice, which journalists use to build their knowledge base. Both of these applications provide content in XML formats. And the contents can be linked : they can say for instance that the movie ‘The social network’ described in Ocari Culture is linked to this or this article from Ocari Content. --------------------- Then we need a layer that will aggregate contents. --------------------- The XML Server gathers all XML content,. * stores them, so the backoffices are not reached at runtime * Normalizes content using XSL stylesheets, since we got contents from different places * aggregates XML, if one content references another * serves them to the HTML renderer --------------------------- Each of the content is being used to render a fragment of a page That's HTML Renderer's job. Basically what HTML Renderer does is to request a content from the XML Server with its URI, and then render it with XSLT. Delivers HTML in fragments, to best benefit from cache and ESI mechanisms. We will see that later. ----------------------------- We have actually four distinct applications in our architecture . ----------------------------- This way any data source can be smoothly integrated into a front that is dedicated to high performance page delivery.
- What have we got ? 4 different apps that do very specialized things They are loosely coupled and we have a front layer that has its own logic and can be entirely independant of the admin
- although we don't hit the admin at runtime, the whole fetching xml contents and the xslt processing is still heavy... So we need to address the performance issue, it is one of our goals we use two components from Symfony 2 to do that : One is the new cache component of Symfony 2 It is based on the HTTP RFC and that’s a good news the other is a component that deals with ESI ESI an Akamai mechanism that was made a RFC and is used to deliver fragments of pages instead of the whole page at once Those mechanisms are not brand new (HTTP RFC is 1998 and ESI RFC is from 2001).
- The cache in Sf2 is based on HTTP cache. I invite you to visit http://tools.ietf.org/html/draft-ietf-httpbis-p6-cache-12 SF2 handles HTTP headers of the request and response to assess the freshness of the cache In our case, as we only need expiration, we use the Cache-Control: s-maxage header.
- So you’re using a standard (and an old and famous one, at that) and I will not come back to the good of using standards. And the good thing is Symfony cache loads the application only if needed. With sf1, the application is started first so the cache can be handled. --- With Sf2, only the HttpKernel is hit and can deliver the response according to its HTTP header, without the application being lit. if you response is in cache, the request will not hit your application. I will stop here with HTTP cache because Fabien will talk about it at length tomorrow.
- But that’s not all. In fact we undergo two more inconvenients with cache : We are supposed to know what each page is composed of, in order to invalidate the right way, which, you probably know, is a hard thing to do. So for each fragment of the page that is invalid, you have to know which page it belongs to in order to invalidate the whole thing The second thing is that if you have elements that stay fresh longer, you do not want to regenerate them It's a pity that the layout should be re-delivered when it's still valid.
- Then comes ESI in Symfony2. Edge Side Includes (ESI) is a simple markup language used to define Web page components for dynamic assembly. ESI is a standard, that was proposed by Akamai in the first place, in 2001. So you see it’s not really the latest news, but it’s extremely efficient.
- Here is a typical page rendered by our architecture You can easily cut down the page into smaller fragments ( a box of information = component ) that may have different times to live, or expiration dates, or validation process. You can use ESI to decompose your whole page into those fragments, and you handle each fragment as if it was a standalone HTTP response. One include is in charge of rendering one fragment in the page. So what we use here is : HTML static Layouts in which we place esi includes Each include will be solved by the ESI component of Symfony 2 You can then manage different time to live in the same page .
- Each fragment has its own ttl or validation rule => the fragments are selectively refreshed You do not have to worry about dependances. They are defined once and for all.
- So How Esi Works ?
- The client can be anything. The ESIs are processed on the client side You will need a proxy that handle ESI. Varnish handles it, Symfony2 can too.
- So let’s see what happen when the client issues the first request ever on a web page ? Let’s say we want some informations about the Dogma movie. The client issues a request. The proxy get it and check if it has something in cache. As it is the first request, there is nothing in cache and the request is forwarded to the application. The app sends the response, composed of a HTML layout with esi tags inside. The response is also sent with a Cache-Control header setting the time to live of this response of 600 seconds. The proxy analyse the responds body and for each ESI tags found, checks the cache if there is a cached version of the response. As this is the first request, none of these ESI tags are stored in the cache and for each one, a request has to be issued to the application For each request, the application answer with an HTML response and optionnally Cache-Control header. In this example, the TTL is set differently for each modules of the page.
- Massive performance gain !
- Granularity: You can use expiration and/or validation for different parts of your website.
- How to benefit from Sf 2 capabilities, in terms of cacheing ? Imagine you have your sf1 application ready, and then, blam ! , sf2 comes in. You think about rebuild the whole thing, but you're lazy, like me, and your client has given you 3 days to finish the whole thing. Well. You wrap it. In our case, we only need to wrap our HTML Renderer application. Most of the time, it will not even be loaded. It’s the Symfony2 Kernel that handles client requests and solves ESI includes. In order to do that : * we need to tweak the Symfony2 controller. * You disable sf1 cache * you prevent it from rendering its response to the client * allow for multiple reentrance * and you use HttpKernel from Sf2 up front to handle it.
- Symfony 1 is based on a filter chain execution. At the end, the sfRenderingFilter class sends the response directly to the client.
- Symfony2 is divided into serveral components you can exploit individually
- Symfony 1 is not able to give the response as a string, so we have to catch it with output buffering Finally, we build a Symfony2 response object and we specify that this content has to be parsed with ESI. Préciser que ça vient d’HttpKernel This method requires the ProjectConfiguration class of symfony 1.x We instanciate a sfApplicationConfiguration object with the given parameters from the front controller
- In symfony 1, the strategy was to divide your page into partials. This is not longer valid when you want to leverage ESI. You have to transform your partial into standalone actions that will responds HTML
- And now we’re happy
- Or are we ? Because there’s still a long way to go