A NU.nl case-study about how to handle large amounts of traffics.

1. Surviving a Plane Crash
a NU.nl case-study
Peter C. Verhage
June 12th 2009

2. Surviving a Plane Crash
NU.nl
• most popular Dutch news website
• 10th anniversary
• 4.5 million unique visitors each month
• 7-15 million page views each day
Me
The plane crash
• 20.2 million page views
• 1.83 Gbit/s traffic
2

3. Surviving a Plane Crash
CMS
Front-end
Performance
QA
3

4. CMS
4

5. CMS
100% custom
PHP 5.1 / MySQL 5
ATK framework
• in-house developed
• Open Source (LGPL)
• specializes in CMS-like applications
Prototype / Scriptaculous
5

6. CMS
vs
6

7. CMS
7

8. CMS
8

9. CMS
9

10. CMS
10

11. CMS
11

12. Front-end
12

13. Front-end
Multiple sites (NU, NUvideo, NUfoto, NUsport, ...)
CodeIgniter framework
• PHP 4 :(
• lightweight MVC
• extensible
Extensions
• Snippets
• Globalization (i18n / l10n)
• Code sharing between sites
13

14. Front-end
Snippets
14

15. Front-end: Snippets
15

16. Front-end: Snippets
header
16

17. Front-end: Snippets
navigation
17

18. Front-end: Snippets
sidebar
18

19. Front-end: Snippets
other news
19

20. Front-end: Snippets
article
20

21. Front-end: Snippets
snippets
• are like ZF partials, but more powerful
• contain logic that doesn’t belong in the template / view
• have full access to the model
• render a view for their result
abstract class Snippet {
public function __construct($params);
protected function getParam();
• so a lot like a controller action protected function renderView($name, $params);
public abstract function render();
public function getPath();
...
• but more re-usable }
21

22. Front-end
Globalization
22

23. Front-end: Globalization
Dnews.de; german version of NU.nl
Same code base, different configuration
Default CodeIgniter globalization not sufficient
Custom globalization consists of:
• translations
• template overrides
• snippet overrides
• css / javascript overrides
• routing overrides
23

24. Front-end
Code Sharing
24

25. Front-end: Code Sharing
Good developers are lazy developers
Lazy developers are good developers
Avoid code duplication
Each site has its own code base
One shared code base
Class not found in site code base? → Try shared code base
Small variations can be tweaked in the site configuration
25

26. Performance
26

27. Performance
Static HTML
Going more dynamic
A Layer of Varnish
Hardware setup
27

28. Performance
Static HTML
28

29. Performance: Static HTML
Nothing beats static HTML
Snippets; the problem
Task Scheduler
Degradation
29

30. Performance
Going more dynamic
30

31. Performance: Going more dynamic
NUfoto / NUvideo
Lists ordered by date
Problem
• latest photo / video is pushed on top
• pagination continuously changes
Solution
• use a proxy server
• use Memcache
• generate HTML for first few pages
31

32. Performance: Going more dynamic
Showing the logged in user
Problem
• every page is customized for the user
Solution
• set cookie with display data
• use JavaScript to inject
• don’t cache real user specific pages
• never trust the cookie!
32

33. Performance
A Layer of Varnish
33

34. Performance: A Layer of Varnish
Open Source reverse proxy server
Serves from memory
Blazingly fast
Snippets; the solution → ESI
• Edge Side Includes (W3C standard)
• In-memory
• Remember Snippet::getPath()?
• Example:
<esi:include src=”/includes/site/sections/algemeen.html”/>
34

35. Performance
Hardware setup
35

36. Performance: Hardware setup
2 back-end servers (apache / master db), only 1 active
4 front-end servers (apache / slave db)
4 front-end Varnish proxy servers
4 media Varnish proxy servers
Juniper load balancers
36

37. Questions ?
37

38. Thank you!
Contact details:
E-mail: peter@ibuildings.nl
Skype: peter-ibuildings

39. Links & sources
http://www.nu.nl
http://www.nufoto.nl
http://www.nuvideo.nl
http://www.nusport.nl
http://www.dnews.de
http://www.atk-framework.com
http://www.codeigniter.com
http://varnish.projects.linpro.no
39