Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
When dynamic becomes static 
(the next step in web caching techniques) 
Wim Godden 
Cu.be Solutions 
@wimgtr
Disclaimer 
The next step 
As in : what you will be doing in the future 
Not as in : go home and run it ;-) 
Language of c...
Who am I ? 
Wim Godden (@wimgtr)
Where I'm from
Where I'm from
Where I'm from
Where I'm from
Where I'm from
Where I'm from
My town
My town
Belgium – the traffic
Who am I ? 
Wim Godden (@wimgtr) 
Founder of Cu.be Solutions (http://cu.be) 
Open Source developer since 1997 
Developer o...
Who are you ? 
Developers ? 
System/network engineers ? 
Managers ?
To understand the present 
Understand the past
The Stone Age 
New blog post by : caveman003
Pre-dynamic : draw it and make html
The Egyptian Era
Old-school dynamic : 'rebuild-every-time'
The Industrial Revolution
Dynamic : let's cache
Extra ! Extra !
Dynamic content in static content
The Modern Era
More load, more webservers
Pushing updates to cache
Today
Adding reverse proxy caching
Typical website structure 
Header 
Latest news 
Article content page 
Page content 
Navigation
Caching blocks with individual TTLs 
Top header 
(TTL = 2h) 
Latest news 
Article content page 
Page content 
Navigation 
...
Caching blocks with individual TTLs 
Top header 
(TTL = 2h) 
Latest news (TTL = 2m) 
Article content page 
Page content (T...
ESI – how it works 
GET /page GET /page
ESI – how it works 
GET /page GET /page 
<html> 
... 
<esi:include src="/top"/> 
<esi:include src="/nav"/> 
<div id=”somet...
ESI – how it works 
GET /top 
<div id=”top-part”> 
<a href=”/login”>Login</a> 
</div>
ESI – how it works 
GET /page GET /page 
<html> 
... 
<esi:include src="/top"/> 
<esi:include src="/nav"/> 
<div id=”somet...
ESI – how it works 
GET /page GET /page 
<html> 
... 
<div id=”top-part”> 
<a href=”/login”>Login</a> 
</div> 
<esi:includ...
Varnish - what can/can't be cached ? 
Can : 
Static pages 
Images, js, css 
Static parts of pages that don't change often ...
ESI → no caching on user-specific content ? 
Logged in as : Wim Godden 
5 messages 
TTL = 0s ? 
TTL=1h TTL = 5min
Error... does not compute !
Back in 2010
No more backend 
GET /page 
DB
No more backend 
GET /page
Nginx 
Web server 
Reverse proxy 
Lightweight, fast 
Low memory footprint 
14.54% of all Websites
Nginx 
No threads, event-driven 
Uses epoll / kqueue 
20000 active connections = normal 
20000 req/sec = normal
ESI on Nginx 
Logged in as : Wim Godden 
5 messages 
Menu NEWS
ESI SLIC on Nginx 
Logged in as : Wim Godden 
5 messages 
Menu NEWS
Requesting /page (1st time) 
Nginx 
Shared memory 
1 
2 
3 
4 
/page 
/page
Requesting /page SLIC subrequests (1st time) 
Nginx 
1 
2 
3 
/menu 
/news 
/top (with session cookie) 
/menu 
/news 
/top...
Requesting /page (next time) 
Nginx 
Shared memory 1 
2 
/page 
/page 
/menu 
/news 
/top (in SLIC session)
SLIC on Nginx 
<slic:include key="top" src="/top" session="true" /> 
<slic:include key="news" src="/news" /> 
<slic:includ...
New message is sent... 
POST /send 
DB 
insert into... 
set(...) 
top (in SLIC session)
Advantages 
No repeated GET hits to webserver anymore ! 
At login : POST → warm up the cache ! 
No repeated hits for user-...
News added 
addnews() method 
DB 
insert into... 
set(...) 
Memcache key /news
How many Memcached requests ? 
Logged in as : Wim Godden 
5 messages 
<slic:include key="top" src="/top" session="true" />...
First release : ESI 
Part of the ESI 1.0 spec 
Only relevant features implemented 
Extension for dynamic session support 
...
Rebuilt from scratch : SLIC 
Control structures : if/else, switch/case, foreach 
Variable handling 
Strings : concatenatio...
SLIC code samples 
You are logged in as : <slic:session_var("person_name") /> 
You are logged in as : <@s("person_name") /...
SLIC code samples 
<slic:switch var="session_var('isAdmin')"> 
<slic:case value="1"> 
<slic:include key="admin-buttons" sr...
SLIC code samples 
<slic:foreach item="messageId" src="global_var('thread' + query_var('threadId'))"> 
<slic:include key="...
Approaches – full block 
Logged in as : Wim Godden 
<p id=”LoggedInAs”> 
You are logged in as : <slic:session_var("person_...
Approaches – individual variables 
Logged in as : Wim Godden 
<p id=”LoggedInAs”> 
You are logged in as : <slic:session_va...
Approaches – JSON 
Logged in as : Wim Godden 
5 messages 
<slic:include key="top" src="/top" session="true" /> 
<p id=”Log...
Identifying the user 
In Nginx configuration : 
slic_session_cookie <name> → Defined by language (or configurable) 
slic_s...
Identifying the user 
Cookie : 
PHPSESSID = 
jpsidc1po35sq9q3og4f3hi6e2 
Nginx + SLIC 
4g3e2t UserID_jpsidc1po35sq9q3og4f3...
Retrieving user specific content 
Nginx + SLIC 
get userData_432 
Cookie : 
PHPSESSID = 
jpsidc1po35sq9q3og4f3hi6e2
Why Nginx ? 
Native Memcached support 
Excellent and superfast subrequest system 
Including parallel subrequests 
Handles ...
What's the result ?
Figures 
2nd customer : 
No. of web servers : 72 → 8 
No. of db servers : 15 → 4 
Total : 87 → 12 (86% reduction !) 
Last ...
Why is it so much faster ?
A real example : vBulletin
A real example : vBulletin 
Post 
isAdmin session variable 
isModerator session variable
A real example : vBulletin 
DB Server Load Web Server Load Max Requests/sec (1 = 282) 
35 
30 
25 
20 
15 
10 
5 
0 
Stand...
Code changes 
Required 
Template conversion 
Push-to-DB → Push-to-DB + Push-to-Cache 
Choice : 
If user is logged in → pus...
Availability 
Good news : 
It will become Open Source 
The concept is solid : ESI version stable at 4 customers 
Bad news ...
So...
Questions ?
Questions ?
Contact 
Twitter @wimgtr 
Web http://techblog.wimgodden.be 
Slides http://www.slideshare.net/wimg 
E-mail wim.godden@cu.be
Upcoming SlideShare
Loading in …5
×

When dynamic becomes static : the next step in web caching techniques

792 views

Published on

Although tools like Varnish can improve performance and scalability for static sites, when user-specific content is needed, a hit to the PHP/Ruby/Python/.Net backend is still required, causing scalability issues. We’ll look at a brand-new Nginx module which implements an ultra-fast and scalable solution to this problem, changing the way developers think about designing sites with user-specific content.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

When dynamic becomes static : the next step in web caching techniques

  1. 1. When dynamic becomes static (the next step in web caching techniques) Wim Godden Cu.be Solutions @wimgtr
  2. 2. Disclaimer The next step As in : what you will be doing in the future Not as in : go home and run it ;-) Language of choice : PHP But : think Perl, Python, Ruby, Java, .Net, …
  3. 3. Who am I ? Wim Godden (@wimgtr)
  4. 4. Where I'm from
  5. 5. Where I'm from
  6. 6. Where I'm from
  7. 7. Where I'm from
  8. 8. Where I'm from
  9. 9. Where I'm from
  10. 10. My town
  11. 11. My town
  12. 12. Belgium – the traffic
  13. 13. Who am I ? Wim Godden (@wimgtr) Founder of Cu.be Solutions (http://cu.be) Open Source developer since 1997 Developer of OpenX, PHPCompatibility, PHPConsistent, ... Speaker at Open Source conferences
  14. 14. Who are you ? Developers ? System/network engineers ? Managers ?
  15. 15. To understand the present Understand the past
  16. 16. The Stone Age New blog post by : caveman003
  17. 17. Pre-dynamic : draw it and make html
  18. 18. The Egyptian Era
  19. 19. Old-school dynamic : 'rebuild-every-time'
  20. 20. The Industrial Revolution
  21. 21. Dynamic : let's cache
  22. 22. Extra ! Extra !
  23. 23. Dynamic content in static content
  24. 24. The Modern Era
  25. 25. More load, more webservers
  26. 26. Pushing updates to cache
  27. 27. Today
  28. 28. Adding reverse proxy caching
  29. 29. Typical website structure Header Latest news Article content page Page content Navigation
  30. 30. Caching blocks with individual TTLs Top header (TTL = 2h) Latest news Article content page Page content Navigation (TTL = 1h)
  31. 31. Caching blocks with individual TTLs Top header (TTL = 2h) Latest news (TTL = 2m) Article content page Page content (TTL = 30m) Navigation (TTL = 1h)
  32. 32. ESI – how it works GET /page GET /page
  33. 33. ESI – how it works GET /page GET /page <html> ... <esi:include src="/top"/> <esi:include src="/nav"/> <div id=”something”> <esi:include src="/latest-news"/> </div> <esi:include src="/article/id/732"/> ... </html>
  34. 34. ESI – how it works GET /top <div id=”top-part”> <a href=”/login”>Login</a> </div>
  35. 35. ESI – how it works GET /page GET /page <html> ... <esi:include src="/top"/> <esi:include src="/nav"/> <div id=”something”> <esi:include src="/latest-news"/> </div> <esi:include src="/article/id/732"/> ... </html>
  36. 36. ESI – how it works GET /page GET /page <html> ... <div id=”top-part”> <a href=”/login”>Login</a> </div> <esi:include src="/nav"/> <div id=”something”> <esi:include src="/latest-news"/> </div> <esi:include src="/article/id/732"/> ... </html>
  37. 37. Varnish - what can/can't be cached ? Can : Static pages Images, js, css Static parts of pages that don't change often (ESI) Can't : POST requests Very large files (it's not a file server !) Requests with Set-Cookie User-specific content
  38. 38. ESI → no caching on user-specific content ? Logged in as : Wim Godden 5 messages TTL = 0s ? TTL=1h TTL = 5min
  39. 39. Error... does not compute !
  40. 40. Back in 2010
  41. 41. No more backend GET /page DB
  42. 42. No more backend GET /page
  43. 43. Nginx Web server Reverse proxy Lightweight, fast Low memory footprint 14.54% of all Websites
  44. 44. Nginx No threads, event-driven Uses epoll / kqueue 20000 active connections = normal 20000 req/sec = normal
  45. 45. ESI on Nginx Logged in as : Wim Godden 5 messages Menu NEWS
  46. 46. ESI SLIC on Nginx Logged in as : Wim Godden 5 messages Menu NEWS
  47. 47. Requesting /page (1st time) Nginx Shared memory 1 2 3 4 /page /page
  48. 48. Requesting /page SLIC subrequests (1st time) Nginx 1 2 3 /menu /news /top (with session cookie) /menu /news /top (in SLIC session)
  49. 49. Requesting /page (next time) Nginx Shared memory 1 2 /page /page /menu /news /top (in SLIC session)
  50. 50. SLIC on Nginx <slic:include key="top" src="/top" session="true" /> <slic:include key="news" src="/news" /> <slic:include key="menu" src="/menu" /> Logged in as : Wim Godden 5 messages ???
  51. 51. New message is sent... POST /send DB insert into... set(...) top (in SLIC session)
  52. 52. Advantages No repeated GET hits to webserver anymore ! At login : POST → warm up the cache ! No repeated hits for user-specific content Not even for non-specific content
  53. 53. News added addnews() method DB insert into... set(...) Memcache key /news
  54. 54. How many Memcached requests ? Logged in as : Wim Godden 5 messages <slic:include key="top" src="/top" session="true" /> <slic:include key="news" src="/news" /> <slic:include key="menu" src="/menu" />
  55. 55. First release : ESI Part of the ESI 1.0 spec Only relevant features implemented Extension for dynamic session support But : unavailable for copyright reasons
  56. 56. Rebuilt from scratch : SLIC Control structures : if/else, switch/case, foreach Variable handling Strings : concatenation, substring, … Exception handling, header manipulation, … JSON support !
  57. 57. SLIC code samples You are logged in as : <slic:session_var("person_name") /> You are logged in as : <@s("person_name") />
  58. 58. SLIC code samples <slic:switch var="session_var('isAdmin')"> <slic:case value="1"> <slic:include key="admin-buttons" src="/admin-buttons.php" /> </slic:case> <slic:default> <div id="just-a-user"> <slic:include key="user-buttons" src="/user-buttons.php" /> </div> </slic:default> </slic:switch>
  59. 59. SLIC code samples <slic:foreach item="messageId" src="global_var('thread' + query_var('threadId'))"> <slic:include key="'thread-message_' + messageId" src="'/thread/message.php?id=' + messageId" /> </slic:foreach>
  60. 60. Approaches – full block Logged in as : Wim Godden <p id=”LoggedInAs”> You are logged in as : <slic:session_var("person_name") /> </p> <p id=”MessageCount”> You have 5 messages </p> 5 messages <slic:include key="top" src="/top" session="true" /> top_31
  61. 61. Approaches – individual variables Logged in as : Wim Godden <p id=”LoggedInAs”> You are logged in as : <slic:session_var("person_name") /> </p> <p id=”MessageCount”> You have <slic:session_var(“messages”) /> messages </p> 5 messages <slic:include key="top" src="/top" session="true" />
  62. 62. Approaches – JSON Logged in as : Wim Godden 5 messages <slic:include key="top" src="/top" session="true" /> <p id=”LoggedInAs”> You are logged in as : <slic:session_var("userData").person_name /> </p> <p id=”MessageCount”> You have <slic:session_var(“userData”).message_count /> messages </p>
  63. 63. Identifying the user In Nginx configuration : slic_session_cookie <name> → Defined by language (or configurable) slic_session_identifier <string> → Defined by you Example for PHP : slic_session_cookie PHPSESSID slic_session_identifier UserID
  64. 64. Identifying the user Cookie : PHPSESSID = jpsidc1po35sq9q3og4f3hi6e2 Nginx + SLIC 4g3e2t UserID_jpsidc1po35sq9q3og4f3hi6e2
  65. 65. Retrieving user specific content Nginx + SLIC get userData_432 Cookie : PHPSESSID = jpsidc1po35sq9q3og4f3hi6e2
  66. 66. Why Nginx ? Native Memcached support Excellent and superfast subrequest system Including parallel subrequests Handles thousands of connections per worker With minimal memory footprint Integrates with php-fpm Additional features (chroot, slow request log, offline processing, ...) Graceful rolling upgrades
  67. 67. What's the result ?
  68. 68. Figures 2nd customer : No. of web servers : 72 → 8 No. of db servers : 15 → 4 Total : 87 → 12 (86% reduction !) Last customer : No. of total servers : +/- 1350 Expected reduction : 1350 → 380 Expected savings : €1.5 Million per year
  69. 69. Why is it so much faster ?
  70. 70. A real example : vBulletin
  71. 71. A real example : vBulletin Post isAdmin session variable isModerator session variable
  72. 72. A real example : vBulletin DB Server Load Web Server Load Max Requests/sec (1 = 282) 35 30 25 20 15 10 5 0 Standard install With Memcached Nginx + SCL + memcached
  73. 73. Code changes Required Template conversion Push-to-DB → Push-to-DB + Push-to-Cache Choice : If user is logged in → push updates to cache If user is not logged in → warm up cache on login
  74. 74. Availability Good news : It will become Open Source The concept is solid : ESI version stable at 4 customers Bad news : First customer holds copyrights Total rebuild → Open Source release No current projects, so spare time Anyone feel like sponsoring ? Beta : October ! Stable : January
  75. 75. So...
  76. 76. Questions ?
  77. 77. Questions ?
  78. 78. Contact Twitter @wimgtr Web http://techblog.wimgodden.be Slides http://www.slideshare.net/wimg E-mail wim.godden@cu.be

×