0
Faster & more flexible CDN integration

Wim Leers ~ http://wimleers.com/

Drupal.org, IRC, Twitter, LinkedIn: wimleers
Bachelor thesis


“Improving Drupal’s page loading performance”


• Promotor: Prof. dr. Wim Lamotte
• Co-promotor: dr. Pet...
Goal

• Faster web sites


     • Speed = satisfaction = more & happier visitors = more revenue




source: http://www.sli...
Goal

• Faster web sites


     • Speed = satisfaction = more & happier visitors = more revenue


• Google: +0.5s → -20% s...
Terminology: page loading performance




              90%                10%




                    CSS, JS, images …
 ...
Drupal’s page loading performance

• One of the most effective measures:
   “Use a CDN”
Drupal’s page loading performance

• One of the most effective measures:
   “Use a CDN”


• Drupal: not yet possible!
Terminology: CDN
Key properties of a CDN
Key properties of a CDN

• Geographical spread
Key properties of a CDN

• Geographical spread


• Pull versus Push

                                          Pull
      ...
Key properties of a CDN

• Geographical spread


• Pull versus Push

                                     Pull            ...
Key properties of a CDN

• Geographical spread


• Pull versus Push


• Lock-in                            Pull           ...
Profiling: Episodes
Profiling: Episodes

• Measures “episodes” during page loading
Profiling: Episodes

• Measures “episodes” during page loading

• Real measurements: JS in browser, for each visitor

• No ...
Episodes module
Episodes module

• Drupal module that offers Episodes integration
Episodes module

• Drupal module that offers Episodes integration
Episodes Server module
Episodes Server module

• Drupal module that visualizes the collected measurements
Daemon: File Conveyor
Daemon: File Conveyor

1. Configuration: simple XML file
Daemon: File Conveyor

1. Configuration: simple XML file

2. Detection: instantaneous
Daemon: File Conveyor

1. Configuration: simple XML file

2. Detection: instantaneous

3. Processing: store image more effici...
Daemon: File Conveyor

1. Configuration: simple XML file

2. Detection: instantaneous

3. Processing: store image more effici...
Daemon: File Conveyor

1. Configuration: simple XML file

2. Detection: instantaneous

3. Processing: store image more effici...
Demo
Daemon’s capabilities: scenario 1

Company wants to switch from CDN provider X to Amazon S3.
- CDN X: FTP
- Amazon S3: cus...
Daemon’s capabilities: scenario 1

Company wants to switch from CDN provider X to Amazon S3.
- CDN X: FTP
- Amazon S3: cus...
Daemon’s capabilities: scenario 1

Company wants to switch from CDN provider X to Amazon S3.
- CDN X: FTP
- Amazon S3: cus...
Daemon’s capabilities: scenario 2

U.S. company expands to South-Korea
Daemon’s capabilities: scenario 2

U.S. company expands to South-Korea


• Setup:


  • North-American CDN


  • Static fil...
Daemon’s capabilities: scenario 2

U.S. company expands to South-Korea


• Setup:


  • North-American CDN


  • Static fil...
CDN integration module
CDN integration module

• Drupal core patch for Drupal 6, committed to Drupal 7
CDN integration module

• Drupal core patch for Drupal 6, committed to Drupal 7

• Basic mode: Origin Pull CDN
CDN integration module

• Drupal core patch for Drupal 6, committed to Drupal 7

• Basic mode: Origin Pull CDN

• Advanced...
Test case
Test case
Test case
Conclusion
Conclusion

1. Speed of web site very important
Conclusion

1. Speed of web site very important

2. Episodes → monitor page loading performance
Conclusion

1. Speed of web site very important

2. Episodes → monitor page loading performance

3. CDN integration module...
Conclusion

1. Speed of web site very important

2. Episodes → monitor page loading performance

3. CDN integration module...
Conclusion

1. Speed of web site very important

2. Episodes → monitor page loading performance

3. CDN integration module...
Upcoming SlideShare
Loading in...5
×

Faster & more flexible CDN integration

10,392

Published on

90% of the page loading time is spent on retrieving CSS, JavaScript and images. There are many techniques to reduce the page loading time — using a CDN is one of the most effective. Currently it's expensive to integrate with a CDN (especially if you want to avoid vendor lock-in) and it's hard to serve file A from a CDN, file B from a static file server and file C from neither. It's also impossible to process a file (e.g. compress JavaScript, optimize images, transcode videos …) before they are synced to the CDN.

This session will explain how a CDN (Content Delivery Network) improves page loading times and how you should continuously analyze the page loading performance of your web site. Older techniques for integrating a CDN with Drupal will be compared and my new, alternative, comprehensive solution will be presented.
All the research and work performed for this presentation was part of my bachelor thesis at Hasselt University, which got the highest score possible: 19/20 (they never hand out a perfect score). It was labeled as being equivalent to a master thesis. So at least, you can expect the concept to be solid.

For the web site that served as the test case of my bachelor thesis, which had less than 5 images per page, the results were dramatic: the time required to load CSS, JS and images was reduced to less than half the original time, globally (visitors from more than 150 countries)! A combination of a North-American CDN and a static file server in Belgium was used to achieve this.
The average web site today has 50 images per page, so it's likely your results will be even more impressive.

Agenda

What is page loading performance?
How a CDN improves page loading times
Continuous page loading performance analysis through Episodes
Older Drupal CDN integration techniques
The new CDN integration module
The new daemon to process & sync files with unlimited flexibility
Alternative uses for the daemon: create your own CDN, massive back-up tool, transcoding server (Media Mover integration?) …
Goals

You should have a good overview of the different techniques to integrate Drupal with a CDN.
You should have learned how you can continuously evaluate page loading performance to detect problems and bottlenecks.
You should have learned about the benefits of using the daemon: reduced cost, no more vendor lock-in …

Published in: Technology, Business
1 Comment
8 Likes
Statistics
Notes
No Downloads
Views
Total Views
10,392
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
139
Comments
1
Likes
8
Embeds 0
No embeds

No notes for slide
  • A content delivery network (CDN) is a collection of webservers that are distributed across multiple locations, to deliver data more efficiently to the end user. Typically, the server with the shortest distance to the end user is selected.
  • - geographical spread: more spread is better, but more expensive
    - lock-in:
    * push: supporting all transfer protocols is too hard
    * pull: not every CDN supports pulling (and unlikely that the same features are supported)
  • - geographical spread: more spread is better, but more expensive
    - lock-in:
    * push: supporting all transfer protocols is too hard
    * pull: not every CDN supports pulling (and unlikely that the same features are supported)
  • - geographical spread: more spread is better, but more expensive
    - lock-in:
    * push: supporting all transfer protocols is too hard
    * pull: not every CDN supports pulling (and unlikely that the same features are supported)
  • - geographical spread: more spread is better, but more expensive
    - lock-in:
    * push: supporting all transfer protocols is too hard
    * pull: not every CDN supports pulling (and unlikely that the same features are supported)
  • - geographical spread: more spread is better, but more expensive
    - lock-in:
    * push: supporting all transfer protocols is too hard
    * pull: not every CDN supports pulling (and unlikely that the same features are supported)
  • - geographical spread: more spread is better, but more expensive
    - lock-in:
    * push: supporting all transfer protocols is too hard
    * pull: not every CDN supports pulling (and unlikely that the same features are supported)
  • - geographical spread: more spread is better, but more expensive
    - lock-in:
    * push: supporting all transfer protocols is too hard
    * pull: not every CDN supports pulling (and unlikely that the same features are supported)
  • - Best of +- 10 evaluated tools and commercial services
  • - Best of +- 10 evaluated tools and commercial services
  • - Best of +- 10 evaluated tools and commercial services
  • Result: no switching cost → no lock-in!
  • Result: no switching cost → no lock-in!
  • Result: no switching cost → no lock-in!
  • Result: faster web site, same setup cost, lower maintenance cost
  • Result: faster web site, same setup cost, lower maintenance cost
  • Result: faster web site, same setup cost, lower maintenance cost
  • - own web site
    - > 100.000 visitors per month, >150 countries
    - makes it a good test case to show the effect of CDN integration
    - far less images than the average web site, thus less HTTP requests, thus harder to show the difference
    - Europe + Russia: server in Belgium, rest of the world: CDN with servers in North America
    - uses ip2country module to determine country of user
  • - information in the screenshots generated by the Episodes Server module
    - > 2.7 million episodes measured over > 260k page views
    - before June 21: large variability, especially globally
    - starting June 21: CDN integration active
    - starting June 22: also CSS & JS of delicious widget with CDN integration
    - very clear downward trend starting June 21, even stronger starting June 22 (partially thanks to caching in the browser)
  • Transcript of "Faster & more flexible CDN integration"

    1. 1. Faster & more flexible CDN integration Wim Leers ~ http://wimleers.com/ Drupal.org, IRC, Twitter, LinkedIn: wimleers
    2. 2. Bachelor thesis “Improving Drupal’s page loading performance” • Promotor: Prof. dr. Wim Lamotte • Co-promotor: dr. Peter Quax • Mentors: Stijn Agten & Maarten Wijnants
    3. 3. Goal • Faster web sites • Speed = satisfaction = more & happier visitors = more revenue source: http://www.slideshare.net/stubbornella/designing-fast-websites-presentation, Nicole Sullivan, Yahoo!
    4. 4. Goal • Faster web sites • Speed = satisfaction = more & happier visitors = more revenue • Google: +0.5s → -20% searches source: http://www.slideshare.net/stubbornella/designing-fast-websites-presentation, Nicole Sullivan, Yahoo!
    5. 5. Terminology: page loading performance 90% 10% CSS, JS, images … HTML
    6. 6. Drupal’s page loading performance • One of the most effective measures: “Use a CDN”
    7. 7. Drupal’s page loading performance • One of the most effective measures: “Use a CDN” • Drupal: not yet possible!
    8. 8. Terminology: CDN
    9. 9. Key properties of a CDN
    10. 10. Key properties of a CDN • Geographical spread
    11. 11. Key properties of a CDN • Geographical spread • Pull versus Push Pull F transfer automatically We protocol + virtually no setup – no flexibility
    12. 12. Key properties of a CDN • Geographical spread • Pull versus Push Pull Push FTP, SFTP, rsync, transfer automatically WebDAV, Amazon S3 protocol … + virtually no setup flexibility – no flexibility setup
    13. 13. Key properties of a CDN • Geographical spread • Pull versus Push • Lock-in Pull Push FTP, SFTP, rsync, transfer automatically WebDAV, Amazon S3 protocol … + virtually no setup flexibility – no flexibility setup
    14. 14. Profiling: Episodes
    15. 15. Profiling: Episodes • Measures “episodes” during page loading
    16. 16. Profiling: Episodes • Measures “episodes” during page loading • Real measurements: JS in browser, for each visitor • No simulation!
    17. 17. Episodes module
    18. 18. Episodes module • Drupal module that offers Episodes integration
    19. 19. Episodes module • Drupal module that offers Episodes integration
    20. 20. Episodes Server module
    21. 21. Episodes Server module • Drupal module that visualizes the collected measurements
    22. 22. Daemon: File Conveyor
    23. 23. Daemon: File Conveyor 1. Configuration: simple XML file
    24. 24. Daemon: File Conveyor 1. Configuration: simple XML file 2. Detection: instantaneous
    25. 25. Daemon: File Conveyor 1. Configuration: simple XML file 2. Detection: instantaneous 3. Processing: store image more efficiently … — extensible!
    26. 26. Daemon: File Conveyor 1. Configuration: simple XML file 2. Detection: instantaneous 3. Processing: store image more efficiently … — extensible! 4. Syncing: supports many protocols (FTP, Amazon S3 …) — extensible!
    27. 27. Daemon: File Conveyor 1. Configuration: simple XML file 2. Detection: instantaneous 3. Processing: store image more efficiently … — extensible! 4. Syncing: supports many protocols (FTP, Amazon S3 …) — extensible! 5. Result: SQLite DB with CDN URLs
    28. 28. Demo
    29. 29. Daemon’s capabilities: scenario 1 Company wants to switch from CDN provider X to Amazon S3. - CDN X: FTP - Amazon S3: custom protocol
    30. 30. Daemon’s capabilities: scenario 1 Company wants to switch from CDN provider X to Amazon S3. - CDN X: FTP - Amazon S3: custom protocol • Setup: • CDN X → Amazon S3 • Daemon
    31. 31. Daemon’s capabilities: scenario 1 Company wants to switch from CDN provider X to Amazon S3. - CDN X: FTP - Amazon S3: custom protocol • Setup: • CDN X → Amazon S3 • Daemon • Alternative: write a lot of code
    32. 32. Daemon’s capabilities: scenario 2 U.S. company expands to South-Korea
    33. 33. Daemon’s capabilities: scenario 2 U.S. company expands to South-Korea • Setup: • North-American CDN • Static file server in South-Korea • Daemon + language-/subdomain-based logic to pick CDN/server
    34. 34. Daemon’s capabilities: scenario 2 U.S. company expands to South-Korea • Setup: • North-American CDN • Static file server in South-Korea • Daemon + language-/subdomain-based logic to pick CDN/server • Alternative: global CDN or slower web site in South-Korea
    35. 35. CDN integration module
    36. 36. CDN integration module • Drupal core patch for Drupal 6, committed to Drupal 7
    37. 37. CDN integration module • Drupal core patch for Drupal 6, committed to Drupal 7 • Basic mode: Origin Pull CDN
    38. 38. CDN integration module • Drupal core patch for Drupal 6, committed to Drupal 7 • Basic mode: Origin Pull CDN • Advanced mode: any CDN (daemon)
    39. 39. Test case
    40. 40. Test case
    41. 41. Test case
    42. 42. Conclusion
    43. 43. Conclusion 1. Speed of web site very important
    44. 44. Conclusion 1. Speed of web site very important 2. Episodes → monitor page loading performance
    45. 45. Conclusion 1. Speed of web site very important 2. Episodes → monitor page loading performance 3. CDN integration module → best method to integrate Drupal with a CDN
    46. 46. Conclusion 1. Speed of web site very important 2. Episodes → monitor page loading performance 3. CDN integration module → best method to integrate Drupal with a CDN 4. Daemon → flexible CDN integration
    47. 47. Conclusion 1. Speed of web site very important 2. Episodes → monitor page loading performance 3. CDN integration module → best method to integrate Drupal with a CDN 4. Daemon → flexible CDN integration More info? See http://wimleers.com/talk/drupalcon-paris-2009
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×