Although tools like Varnish can improve performance and scalability for static sites, when user-specific content is needed, a hit to the PHP/Ruby/Python/.Net backend is still required, causing scalability issues. We'll look at a brand-new Nginx module which implements an ultra-fast and scalable solution to this problem, changing the way developers think about designing sites with user-specific content.

1. When dynamic becomes static – the next
step in web caching techniques
Wim Godden
Cu.be Solutions

2. Disclaimer
The next step
As in : what you will be doing in the future
Not as in : go home and run it ;-)
Language of choice : PHP
But : think Perl, Python, Ruby, Java, .Net, …

3. Who am I ?
Wim Godden (@wimgtr)
Founder of Cu.be Solutions (http://cu.be)
Open source developer since 1997
Developer of OpenX, PHPCompatibility, ...
Speaker at PHP and Open Source conferences

4. Who are you ?
Developers ?
System/network engineers ?
Managers ?

5. To understand the present
Understand the past

6. The Stone Age
New blog post by : caveman003

7. Pre-PHP : draw it and make html

8. The Egyptian Era

9. Old-school PHP : 'rebuild-every-time'

10. The Industrial Revolution

11. PHP : let's cache

12. Extra ! Extra !

13. PHP : dynamic content in static content

14. The Modern Era

15. PHP : multiple webservers

16. PHP : push updates to cache

17. Today

18. Adding reverse proxy caching

19. Website X with ESI
Header
Latest news
Article content page
Navigation
Page content

20. Website X with ESI
Top header
(TTL = 2h)
Latest news
Article content page
Navigation
(TTL = 1h)
Page content

21. Website X with ESI
Top header
(TTL = 2h)
Latest news (TTL = 2m)
Article content page
Navigation
(TTL = 1h)
Page content (TTL = 30m)

22. Going to /page/id/732
<esi:include src="/top"/>
<esi:include src="/nav"/>
<esi:include src="/latest-news"/>
<esi:include src="/article/id/732"/>

23. Varnish - ESI
<esi:include src="/top"/>
<esi:include src="/news"/>
Article content page
<esi:include
src="/nav"/>
<esi:include src="/article/732"/>

24. A simple benchmark – 2KByte JPEG
Apache 2.2
4210
IIS 7.5
3960
Varnish 3.0
11400

25. A dynamically generated, but static page
Apache 2.2 + PHP (3 DB queries)
18
IIS 7.5 + .Net (3 DB queries)
16
Varnish 3.0
11400

26. Varnish - what can/can't be cached ?
Can :
Static pages
Images, js, css
Static parts of pages that don't change often (ESI)
Can't :
POST requests
Very large files (it's not a file server !)
Requests with Set-Cookie
User-specific content

27. ESI → no caching on user-specific content ?
Logged in as : Wim Godden
TTL = 0s ?
TTL=1h
5 messages
TTL = 5min

28. Nginx
Web server
Reverse proxy
Lightweight, fast
12.81% of all Websites

29. Nginx
No threads, event-driven
Uses epoll / kqueue
Low memory footprint
10000 active connections = normal

30. ESI on Nginx
Logged in as : Wim Godden
5 messages
Menu
NEWS

31. ESI SCL on Nginx
<scl:include key="top" src="/top" session="true" ttl="1h" />
<scl:include
key="menu"
src="/menu"
ttl="1h" />
<scl:include key="news" src="/news" ttl="5m" />

32. Requesting /page (1st time)
Nginx
1
Shared memory
/page
4
/page
3
2

33. Requesting /page ESI subrequests (1st time)
Nginx
1
2
3
/menu
/news
/top (in SCL session)

34. Requesting /page (next time)
Nginx
Shared memory
1
/page
2
/page
/menu
/news
/top (in SCL session)

35. New message is sent...
POST /send
o
i nt
rt
e
ins
...
se
t
(..
.)
DB
top (in SCL session)

36. Advantages
No repeated GET hits to webserver anymore !
At login : POST → warm up the cache !
No repeated hits for user-specific content
Not even for non-specific content

37. News added
addnews() method
o
i nt
rt
e
ins
...
se
t
(..
.)
DB
Memcache key /news

38. Advantages
No repeated GET hits to webserver anymore !
At login : POST → warm up the cache !
No repeated hits for user-specific content
Not even for non-specific content
No TTLs for non-specific content

39. How many Memcache requests ?
Logged in as : Wim Godden
<scl:include key="top" src="/top" session="true" ttl="1h" />
5 messages
<scl:include
key="menu"
src="/menu"
ttl="1h" />
<scl:include key="news" src="/news" ttl="5m" />

40. First release : ESI
Part of the ESI 1.0 spec
Only relevant features implemented
Extension for dynamic session support
But : unavailable for copyright reasons

41. Rebuilt from scratch : SCL
Session-specific Caching Language
Language details :
Control structures : if/else, switch/case, foreach
Variable handling
Strings : concatenation, substring, …
Exception handling, header manipulation, ...

42. SCL code samples
You are logged in as : <scl:session_var("person_name") />
You are logged in as : <@s("person_name") />

43. SCL code samples
<scl:switch var="session_var('isAdmin')">
<scl:case value="1">
<scl:include key="admin-buttons" src="/admin-buttons.php" />
</scl:case>
<scl:default>
<div id="just-a-user">
<scl:include key="user-buttons" src="/user-buttons.php" />
</div>
</scl:default>
</scl:switch>

44. SCL code samples
<scl:foreach item="messageId" src="global_var('thread' + query_var('threadId'))">
<scl:include key="'thread-message_' + messageId"
src="'/thread/message.php?id=' + messageId" />
</scl:foreach>

45. Identifying the user
In Nginx configuration :
scl_session_cookie <name> → Defined by language (or configurable)
scl_session_identifier <string> → Defined by you
Example for PHP :
scl_session_cookie PHPSESSID
scl_session_identifier UID

46. Identifying the user
Cookie :
PHPSESSID =
jpsidc1po35sq9q3og4f3hi6e2
Nginx + SCL
432
get UID_jpsidc1po35sq9q3og4f3hi6e2

47. Retrieving user specific content
Cookie :
PHPSESSID =
jpsidc1po35sq9q3og4f3hi6e2
Nginx + SCL
get top_432

48. Why Nginx ?
Native Memcached support
Excellent and superfast subrequest system
Including parallel subrequests
Handles thousands of connections per worker
With minimal memory footprint
Integrates with php-fpm
Additional features (chroot, slow request log, offline processing, ...)
Graceful rolling upgrades

49. What's the result ?

50. Figures
2nd customer :
No. of web servers : 72 → 8
No. of db servers : 15 → 4
Total : 87 → 12 (86% reduction !)
Last customer :
No. of total servers : +/- 1350
Expected reduction : 1350 → 300
Expected savings : €1.6 Million per year

51. Why is it so much faster ?

52. A real example : vBulletin

53. A real example : vBulletin
isAdmin session variable
Post
isModerator session variable

54. A real example : vBulletin
35
30
25
20
Standard install
With Memcached
Nginx + SCL + memcached
15
10
5
0
DB Server Load
Web Server Load
Max Requests/sec (1 = 282)

55. Availability
Good news :
It will become Open Source
It's solid : ESI version stable at 4 customers
Bad news :
First customer holds copyrights
Total rebuild
→ Open Source release
No current projects, so spare time
Beta : Jan-Feb
Stable : Q2 (on Github)

56. So...

57. Questions ?

58. Questions ?

59. Contact
Twitter
Web
Slides
E-mail
@wimgtr
http://techblog.wimgodden.be
http://www.slideshare.net/wimg
wim.godden@cu.be
Please provide feedback :
http://go.cu.be/velocity2013