Nice performance using Sf2 cache wrapping Sf1 application - Paris


Published on

In collaboration with Emmanuel Cohen.

At a key moment for online press in France, a major French news company chooses PHP and Symfony to extend its popular web site. We will present the architecture we designed at Sensio Labs to meet a very good performance requirement. We used Symfony2 kernel wrapping symfony 1.4 and relied on loose-coupled applications serving content from heterogeneous backend sources.

Published in: Technology

Nice performance using Sf2 cache wrapping Sf1 application - Paris

  1. 1. Nice performance using SF2 cache wrapping sf1 application Marc Weistroff Emmanuel Cohen Sf Live SF 2011 Thursday, March 3 2011
  2. 2. Who we are <ul><li>Emmanuel Cohen </li></ul><ul><li>Project manager at Sensio Labs </li></ul><ul><li>4 year experience with PHP5 </li></ul><ul><li>5+ year experience with Java EE </li></ul><ul><li>Learned Basic on Amstrad CPC 6128 </li></ul>
  3. 3. Who we are <ul><li>Marc Weistroff </li></ul><ul><li>Developer at Sensio Labs since 11/2009 </li></ul><ul><li>Lead developer on this project </li></ul><ul><li>Experiences with C, and PHP from version 3 </li></ul><ul><li>Encountered programming with Amos Basic on Amiga 500 </li></ul>
  4. 4. This talk <ul><li>Symfony2 </li></ul><ul><li>symfony 1 </li></ul><ul><li>HTTP Cache </li></ul><ul><li>Edge Side Includes (ESI) </li></ul><ul><li>Web architecture </li></ul><ul><li>Online media </li></ul>
  5. 5. Our client <ul><li>L’Express Magazine </li></ul><ul><li>Express-Roularta Group </li></ul><ul><li>Magazine created in 1953 </li></ul><ul><li> </li></ul><ul><li>Top 3 French online news </li></ul><ul><li>Need for speed! </li></ul>
  6. 9. The project <ul><li>A cultural knowledge base </li></ul><ul><li>A bridge between hot news and cultural knowledge </li></ul><ul><li>An extension of </li></ul>
  7. 12. Technical objectives <ul><li>Performance </li></ul><ul><li>A full scale Proof-of-Concept for the future </li></ul><ul><li>Keep it Simple </li></ul>
  8. 13. Constraints <ul><li>Heterogeneous XML Sources </li></ul><ul><li>Solution based on symfony 1 </li></ul><ul><li>Adaptability (for future extensions) </li></ul>
  9. 14. How does this app work ?
  10. 15. Our symfony 1 application design HTML Renderer XML Server OCARI Culture OCARI Content <ul><li>Stores </li></ul><ul><li>Normalizes </li></ul><ul><li>Aggregates </li></ul><ul><li>Serves XML </li></ul>Sources deliver heterogeneous data <ul><li>Renders HTML with XSL </li></ul><ul><li>Renders static layouts </li></ul>
  11. 16. Our symfony 1 application design <ul><li>Highly specialized applications </li></ul><ul><li>Loose coupling </li></ul><ul><li>Front dedicated to delivering content fast </li></ul>
  12. 17. Addressing performance with Symfony2
  13. 18. Addressing performance with Sf2 <ul><li>With Symfony2 built-in Gateway : </li></ul><ul><li>HTTP cache </li></ul><ul><li>ESI : Edge Side Includes </li></ul>
  14. 19. Performance with Symfony2 : HTTP Cache <ul><li>HTTP RFC </li></ul><ul><li> </li></ul><ul><li> </li></ul>
  15. 20. Performance with Symfony2 : HTTP Cache <ul><li>Symfony2 uses HTTP cache headers to handle cache </li></ul><ul><li>s-maxage or max-age </li></ul><ul><li>Etag, Last-Modified, If-Modified-Since </li></ul><ul><li> </li></ul>
  16. 21. Performance with Symfony2 : HTTP Cache <ul><li>symfony 1 </li></ul><ul><li>Applicative cache </li></ul><ul><li>Symfony2 </li></ul><ul><li>Light-weight HttpKernel </li></ul><ul><li>The application is not hit </li></ul><ul><li>Supports a standard </li></ul>
  17. 22. <ul><li>Find out what’s stale </li></ul><ul><li>The problem of invalidation </li></ul>Performance with Symfony2: Cache issues
  18. 23. <ul><li>Selective caching </li></ul><ul><li>Do not regenerate the whole page when you just need to regenerate small parts of it </li></ul>Performance with Symfony2: Cache issues
  19. 24. Performance with Symfony2: ESI <ul><li>Edge-Side include: A markup language </li></ul><ul><li>W3C note from Akamai </li></ul><ul><li>http:// </li></ul><ul><li>http:// </li></ul>
  20. 25. ESI : A page in
  21. 26. Performance with Symfony2: ESI <ul><li>Each fragment has its own ttl or validation rule => the fragments are selectively refreshed </li></ul>
  22. 27. Performance with Symfony2: ESI <ul><li>Dependance is defined once and for all </li></ul><ul><li>In expiration: simply set your ttl for each fragment </li></ul>
  23. 28. How does ESI work?
  24. 29. Performance with Symfony2: ESI <ul><li>Represented by an HTML Tag </li></ul><ul><ul><li><esi:include src=“/movie/dogma/critics” /> </li></ul></ul><ul><li>Processed by a proxy </li></ul><ul><li>Transformed into HTTP Request to the application </li></ul><ul><li>Response is inserted in place of the esi tag </li></ul><ul><li>HTTP cache is handled independently for each esi tag </li></ul>
  25. 30. /movie/dogma
  26. 31. /movie/dogma
  27. 32. Html Fragment <ul><li>Pure HTML </li></ul><ul><li>Inserted in lieu of the <esi /> tag </li></ul><ul><li>No <html>, <body> or <head> tag in this case </li></ul>
  28. 33. What you need <ul><li>Any client! </li></ul><ul><li>A proxy that handles ESI (ie: Varnish or Symfony2) </li></ul><ul><li>An application that delivers HTML and HTTP cache headers </li></ul><ul><li>Serve HTML pages that contain ESI tags </li></ul>
  29. 34. First request ever Client Proxy Application /movie/dogma /movie/dogma miss Cache-Control: s-maxage=600 /movie/dogma/casting /movie/dogma/critics Cache-Control: s-maxage=3600 Cache-Control: s-maxage=300
  30. 35. Request at t+200 Client Proxy Application /movie/dogma hit
  31. 36. The application is never hit! As if the complete page was cached in the reverse proxy
  32. 37. Request at t+500 Client Proxy Application /movie/dogma /movie/dogma/critics Cache-Control: s-maxage=300 hit
  33. 38. The application is partially hit And has to build only a small fragment of the page
  34. 39. Pros <ul><li>You use HTTP cache, not the application one. </li></ul><ul><li>Performance improvements! </li></ul><ul><li>Granularity: Different cache strategy or TTL on different parts of your page </li></ul>
  35. 40. Cons <ul><li>Your app will be hit n times upon the very first request </li></ul><ul><li>Your app have to be designed to support the rendering of fragments </li></ul>
  36. 41. But our app is symfony 1 right ?
  37. 42. <ul><li>What to do with your symfony 1 application when you are dying to use Symfony2? </li></ul>Wrap it. Our app is symfony 1
  38. 43. Our symfony 1 application design HTML Renderer XML Server OCARI Culture OCARI Content Symfony1 wrapper Edge side Symfony2 <ul><li>Stores </li></ul><ul><li>Normalizes </li></ul><ul><li>Aggregates </li></ul><ul><li>Serves XML </li></ul>Sources deliver heterogeneous data <ul><li>Renders HTML with XSL </li></ul><ul><li>Renders static layouts </li></ul>
  39. 44. Wrapping symfony 1 with Symfony2
  40. 45. Constraint <ul><li>We use the Symfony2 ESI/Cache proxy </li></ul><ul><li>All the call to our app is done in the same PHP process </li></ul>
  41. 46. symfony 1 needs tweaking in order to work around a few obstacles
  42. 47. Which ones ? <ul><li>It is not reentrant </li></ul><ul><ul><li>sfConfig singleton </li></ul></ul><ul><ul><li>sfContext singleton </li></ul></ul><ul><li>It sends the response directly to the client </li></ul><ul><ul><li>Filter chain execution </li></ul></ul><ul><ul><li>At the end, sfRenderingFilter sends the complete response </li></ul></ul>
  43. 48. What about Symfony2? <ul><li>Better architecture: Symfony2 is reentrant </li></ul><ul><li>Symfony2 is heavily based on interfaces </li></ul><ul><li>Symfony2 is divided into several components </li></ul><ul><li>HttpKernelHttpKernelInterface </li></ul>
  44. 49. HttpKernelHttpKernelInterface <ul><li>Lightweight (1 method) </li></ul><ul><li>Forces to implement a “handle” method that accepts a Symfony2 Request object and returns a Symfony2 Response object </li></ul><ul><li>Used by all the classes that act as a Kernel in Symfony2 (ie: HttpKernelHttpCache) </li></ul>
  45. 50. Addressing the issues <ul><li>Reentrance </li></ul><ul><ul><li>Override the super globals </li></ul></ul><ul><ul><li>Create a fresh new context each time </li></ul></ul><ul><li>Filter chain </li></ul><ul><ul><li>Replace sfRenderingFilter by a noRenderingFilter class </li></ul></ul><ul><li>Create a Symfony2 Response object and returns it </li></ul>
  46. 51. Final app architecture EsiCacheKernel SymfonyWrapperKernel symfony 1 application
  47. 52.
  48. 53. Then… You have to design your app to serve HTML fragments!
  49. 54. Designing your app <ul><li>Don’t think partials or components, think actions! </li></ul><ul><li>Actions render HTML fragments </li></ul><ul><li>Actions MUST define explicitly the response cache headers. </li></ul>
  50. 55. And now where happy
  51. 56. Or are we ?
  52. 57. Our choices <ul><li>The choice of symfony 1 </li></ul><ul><li>The choice of Symfony2 </li></ul><ul><li>The choice to mix </li></ul><ul><li>Expiration over validation </li></ul>
  53. 58. This is symfony1 : DO NOT TRY ANY OF THIS AT HOME!
  54. 59. Our results <ul><li>It works! </li></ul><ul><li>The frontend is scalable and extensible </li></ul><ul><li>Performance is nice, but we needed to improve the Store.php class </li></ul>
  55. 60. What to do next ?
  56. 61. Implementing cache validation <ul><li>The application needs to garantee freshness </li></ul><ul><li>How to avoid hitting the application ? </li></ul>
  57. 62. Proxy ESI New response Or 304 URL-> {ETag} ETag-> lastmodified If Etag absent or stale Response with HTTP headers Etag LastModified Client Request with HTTP headers If-None-Match If-Modified-Since App New response Or 304 If cache entry is not fresh enough Cache Validation Optimizer
  58. 63. Contact @ L’Express <ul><li>Sébastien Angèle </li></ul><ul><li>[email_address] </li></ul><ul><li>Jérôme Macias </li></ul><ul><li>[email_address] </li></ul>
  59. 64. Questions ? <ul><li>Marc Weistroff </li></ul><ul><li>@futurecat </li></ul><ul><li>[email_address] </li></ul><ul><li> </li></ul><ul><li>Emmanuel Cohen </li></ul><ul><li>@emmanuelcohen </li></ul><ul><li>[email_address] </li></ul>
  60. 65. Thank you! Please rate this talk at