Search

Drupal CDN integration: easier, more flexible and faster!

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    Notes on slide 1















    Popular: d.o usage statistics indicate >125,000 web sites

    Conclusion: important for:
    - Drupal project growth
    - More Happy People

    Popular: d.o usage statistics indicate >125,000 web sites

    Conclusion: important for:
    - Drupal project growth
    - More Happy People

    Popular: d.o usage statistics indicate >125,000 web sites

    Conclusion: important for:
    - Drupal project growth
    - More Happy People






    There are also CDNs out there that can serve content other than static files, but in this talk, only static files are concerned.

    - geographical spread: more spread is better, but more expensive
    - PoP = Point of Presence = connection with >= 1 ISPs
    - lock-in:
    * push: supporting all transfer protocols is too hard
    * pull: not every CDN supports pulling (and unlikely that the same features are supported)

    - geographical spread: more spread is better, but more expensive
    - PoP = Point of Presence = connection with >= 1 ISPs
    - lock-in:
    * push: supporting all transfer protocols is too hard
    * pull: not every CDN supports pulling (and unlikely that the same features are supported)

    - geographical spread: more spread is better, but more expensive
    - PoP = Point of Presence = connection with >= 1 ISPs
    - lock-in:
    * push: supporting all transfer protocols is too hard
    * pull: not every CDN supports pulling (and unlikely that the same features are supported)

    - geographical spread: more spread is better, but more expensive
    - PoP = Point of Presence = connection with >= 1 ISPs
    - lock-in:
    * push: supporting all transfer protocols is too hard
    * pull: not every CDN supports pulling (and unlikely that the same features are supported)

    - geographical spread: more spread is better, but more expensive
    - PoP = Point of Presence = connection with >= 1 ISPs
    - lock-in:
    * push: supporting all transfer protocols is too hard
    * pull: not every CDN supports pulling (and unlikely that the same features are supported)

    - geographical spread: more spread is better, but more expensive
    - PoP = Point of Presence = connection with >= 1 ISPs
    - lock-in:
    * push: supporting all transfer protocols is too hard
    * pull: not every CDN supports pulling (and unlikely that the same features are supported)

    - geographical spread: more spread is better, but more expensive
    - PoP = Point of Presence = connection with >= 1 ISPs
    - lock-in:
    * push: supporting all transfer protocols is too hard
    * pull: not every CDN supports pulling (and unlikely that the same features are supported)





    - industry standard goal is realistic because the author is Steve Souders

    - industry standard goal is realistic because the author is Steve Souders

    - industry standard goal is realistic because the author is Steve Souders

    - industry standard goal is realistic because the author is Steve Souders

    - industry standard goal is realistic because the author is Steve Souders

    - industry standard goal is realistic because the author is Steve Souders

    - industry standard goal is realistic because the author is Steve Souders

    - industry standard goal is realistic because the author is Steve Souders

    - industry standard goal is realistic because the author is Steve Souders

    - industry standard goal is realistic because the author is Steve Souders

    - industry standard goal is realistic because the author is Steve Souders


































    4 Favorites & 1 Group

    Drupal CDN integration: easier, more flexible and faster! - Presentation Transcript

    1. Drupal CDN integration: easier, more flexible and faster! Wim Leers ~ http://wimleers.com/ Drupal.org, IRC, Twitter, LinkedIn: wimleers Thursday, March 5, 2009
    2. Overview Thursday, March 5, 2009
    3. Overview • Page loading performance • Why does it matter? • What is it? • Why is it important to Drupal? Thursday, March 5, 2009
    4. Overview • Page loading performance • Why does it matter? • What is it? • Why is it important to Drupal? • Key Properties of a CDN Thursday, March 5, 2009
    5. Overview • Page loading performance • Why does it matter? • What is it? • Why is it important to Drupal? • Key Properties of a CDN • The CDN Effect • YSlow • Episodes Thursday, March 5, 2009
    6. Overview • Page loading performance • Why does it matter? • What is it? • Why is it important to Drupal? • Key Properties of a CDN • The CDN Effect • YSlow • Episodes • Drupal + CDN (where we are) • Basic Drupal core patch • Simple CDN module • CDN integration module Thursday, March 5, 2009
    7. Overview • Page loading performance • Bachelor thesis • Why does it matter? • What exactly? • What is it? • Daemon requirements • Why is it important to Drupal? • Collaborating companies • Key Properties of a CDN • The CDN Effect • YSlow • Episodes • Drupal + CDN (where we are) • Basic Drupal core patch • Simple CDN module • CDN integration module Thursday, March 5, 2009
    8. Overview • Page loading performance • Bachelor thesis • Why does it matter? • What exactly? • What is it? • Daemon requirements • Why is it important to Drupal? • Collaborating companies • Key Properties of a CDN • The Future (where we will be in 3-4 months) • Goals • The CDN Effect • Sample scenario 1 • YSlow • Sample scenario 2 • Episodes • Sample scenario 3 • Drupal + CDN (where we are) • Other use cases • Basic Drupal core patch • What else? • Simple CDN module • CDN integration module Thursday, March 5, 2009
    9. Overview • Page loading performance • Bachelor thesis • Why does it matter? • What exactly? • What is it? • Daemon requirements • Why is it important to Drupal? • Collaborating companies • Key Properties of a CDN • The Future (where we will be in 3-4 months) • Goals • The CDN Effect • Sample scenario 1 • YSlow • Sample scenario 2 • Episodes • Sample scenario 3 • Drupal + CDN (where we are) • Other use cases • Basic Drupal core patch • What else? • Simple CDN module • Questions? • CDN integration module Thursday, March 5, 2009
    10. Terminology: page loading performance Thursday, March 5, 2009
    11. Terminology: page loading performance page rendering performance the time the server needs to render a web page Thursday, March 5, 2009
    12. Terminology: page loading performance page rendering performance back- front- the time the server needs to render end end a web page Thursday, March 5, 2009
    13. Terminology: page loading performance page rendering performance back- front- the time the server needs to render end end a web page is included in page loading performance the time it takes to load a web page and all its components Thursday, March 5, 2009
    14. Terminology: page loading performance page rendering performance back- front- the time the server needs to render end end a web page is included in page loading performance back- front- the time it takes to load a web page end end and all its components Thursday, March 5, 2009
    15. Why does it matter? source: http://www.slideshare.net/stubbornella/designing-fast-websites-presentation Thursday, March 5, 2009
    16. Why does it matter? • Users care about performance! • Amazon: 100 ms of extra load time caused a 1% drop in sales • Google: 500 ms of extra load time caused 20% fewer searches source: http://www.slideshare.net/stubbornella/designing-fast-websites-presentation Thursday, March 5, 2009
    17. Why does it matter? • Users care about performance! • Amazon: 100 ms of extra load time caused a 1% drop in sales • Google: 500 ms of extra load time caused 20% fewer searches • Fast web sites are rewarded, slow web sites are punished source: http://www.slideshare.net/stubbornella/designing-fast-websites-presentation Thursday, March 5, 2009
    18. Why is it important to Drupal? Thursday, March 5, 2009
    19. Why is it important to Drupal? • Drupal’s numbers • Big, high-traffic web sites • Popular Thursday, March 5, 2009
    20. Why is it important to Drupal? • Drupal’s numbers • Big, high-traffic web sites • Popular • Drupal is international (i18n/L10n) Thursday, March 5, 2009
    21. Why is it important to Drupal? • Drupal’s numbers • Big, high-traffic web sites • Popular • Drupal is international (i18n/L10n) • The Drupal Experience: happier users & developers Thursday, March 5, 2009
    22. What is it? HTML Components • End user response time • 10-20%: HTML (mix of back-end + front-end) 90% • 80-90%: components (front-end only) 10% source: http://stevesouders.com/hpws/ Thursday, March 5, 2009
    23. What is it? HTML Components • End user response time • 10-20%: HTML (mix of back-end + front-end) 90% • 80-90%: components (front-end only) • More effective to focus on front-end 10% performance! source: http://stevesouders.com/hpws/ Thursday, March 5, 2009
    24. What is it? HTML Components • End user response time • 10-20%: HTML (mix of back-end + front-end) 90% • 80-90%: components (front-end only) • More effective to focus on front-end 10% performance! • CDNs have major impact source: http://stevesouders.com/hpws/ Thursday, March 5, 2009
    25. Terminology: CDN A content delivery network (CDN) is a collection of web servers distributed across multiple locations to deliver content more efficiently to users. The server selected for delivering content to a specific user is typically based on a measure of network proximity. source: http://developer.yahoo.com/performance/rules.html#cdn Thursday, March 5, 2009
    26. Key Properties of a CDN Thursday, March 5, 2009
    27. Key Properties of a CDN • Geographical spread (PoPs) Thursday, March 5, 2009
    28. Key Properties of a CDN • Geographical spread (PoPs) • Populating: Pull versus Push Pull F transfer automatically We protocol + virtually no setup no – no flexibility redundant traffic Thursday, March 5, 2009
    29. Key Properties of a CDN • Geographical spread (PoPs) • Populating: Pull versus Push Pull Push FTP, SFTP, rsync, transfer automatically WebDAV, Amazon S3 protocol … + flexibility virtually no setup no redundant traffic – no flexibility setup redundant traffic Thursday, March 5, 2009
    30. Key Properties of a CDN • Geographical spread (PoPs) • Populating: Pull versus Push Pull Push • Lock-in FTP, SFTP, rsync, transfer automatically WebDAV, Amazon S3 protocol … + flexibility virtually no setup no redundant traffic – no flexibility setup redundant traffic Thursday, March 5, 2009
    31. The CDN Effect: YSlow Thursday, March 5, 2009
    32. The CDN Effect: YSlow Thursday, March 5, 2009
    33. The CDN Effect: YSlow Thursday, March 5, 2009
    34. The CDN Effect: YSlow • YSlow does not reflect real-world page loading performance! Thursday, March 5, 2009
    35. The CDN Effect: Episodes Thursday, March 5, 2009
    36. The CDN Effect: Episodes • JS code that measures episodes of a web page’s loading sequence Thursday, March 5, 2009
    37. The CDN Effect: Episodes • JS code that measures episodes of a web page’s loading sequence Thursday, March 5, 2009
    38. The CDN Effect: Episodes • JS code that measures episodes of a web page’s loading sequence Thursday, March 5, 2009
    39. The CDN Effect: Episodes • JS code that measures episodes of a web page’s loading sequence Thursday, March 5, 2009
    40. The CDN Effect: Episodes • JS code that measures episodes of a web page’s loading sequence Thursday, March 5, 2009
    41. The CDN Effect: Episodes • JS code that measures episodes of a web page’s loading sequence • Reflects real-world page loading performance! Thursday, March 5, 2009
    42. The CDN Effect: Episodes • JS code that measures episodes of • Goal: industry standard! a web page’s loading sequence • Reflects real-world page loading performance! Thursday, March 5, 2009
    43. The CDN Effect: Episodes • JS code that measures episodes of • Goal: industry standard! a web page’s loading sequence • Reflects real-world page loading • Drupal module: available next week! performance! Thursday, March 5, 2009
    44. Drupal + CDN: Basic Drupal core patch http://tedserbinski.com/tags/drupal/getting-drupal-play-nice-with-your-cdn http://www.lullabot.com/articles/using-lighttpd-static-file-server Thursday, March 5, 2009
    45. Drupal + CDN: Basic Drupal core patch http://tedserbinski.com/tags/drupal/getting-drupal-play-nice-with-your-cdn http://www.lullabot.com/articles/using-lighttpd-static-file-server • Change all code that calls base_path(): • common.inc • file.inc • theme.inc • page.tpl.php Thursday, March 5, 2009
    46. Drupal + CDN: Basic Drupal core patch http://tedserbinski.com/tags/drupal/getting-drupal-play-nice-with-your-cdn http://www.lullabot.com/articles/using-lighttpd-static-file-server • Change all code that calls base_path(): • common.inc • file.inc • theme.inc • page.tpl.php • Cons • Very limited Thursday, March 5, 2009
    47. Drupal + CDN: Simple CDN module http://drupal.org/project/simplecdn Thursday, March 5, 2009
    48. Drupal + CDN: Simple CDN module http://drupal.org/project/simplecdn • simplecdn_rewrite_url($url, 'modulename') Thursday, March 5, 2009
    49. Drupal + CDN: Simple CDN module http://drupal.org/project/simplecdn • simplecdn_rewrite_url($url, 'modulename') • Cons • Very limited • Not Drupal core & more dependencies • Immature (released Feb 21, 2009), coding standards, no proper releases Thursday, March 5, 2009
    50. Drupal + CDN: CDN integration module http://drupal.org/project/cdn (by me) Thursday, March 5, 2009
    51. Drupal + CDN: CDN integration module http://drupal.org/project/cdn (by me) • Also includes a Drupal core patch Thursday, March 5, 2009
    52. Drupal + CDN: CDN integration module http://drupal.org/project/cdn (by me) • Also includes a Drupal core patch • Automatic synchronization via FTP Thursday, March 5, 2009
    53. Drupal + CDN: CDN integration module http://drupal.org/project/cdn (by me) • Also includes a Drupal core patch • Automatic synchronization via FTP • Cons • Still too limited (rsync, Amazon S3, image optimization …) • Drupal 5 only • Proof-of-concept: not scalable (cron and … serialized arrays!) Thursday, March 5, 2009
    54. Bachelor thesis “Improving Drupal’s page loading performance” • Promotor: Prof. dr. Wim Lamotte • Co-promotor: dr. Peter Quax • Mentors: Stijn Agten & Maarten Wijnants Thursday, March 5, 2009
    55. Bachelor thesis: What exactly? • Episodes.module • Basic reports (e.g. average page load time per browser) • CDN integration • Drupal core patch: unify file URL generation/alteration (sufficient for Pull CDNs) • Daemon for synchronization (Push CDNs) • Drupal module: administration & basic reports • JS at the bottom Thursday, March 5, 2009
    56. Bachelor thesis: Daemon requirements • Python • Full unit testing coverage • Very extensible • Plug-ins for transfer protocols • Plug-ins for processors • Very robust & scalable • Multi-threaded, must be able to handle any amount of files Thursday, March 5, 2009
    57. Bachelor thesis: Collaborating companies The following CDN companies The following companies will be will be providing their services evaluating the improvements to for free for testing purposes. Drupal or the daemon. A big thanks to all of them! Thursday, March 5, 2009
    58. The Future: Goals Thursday, March 5, 2009
    59. The Future: Goals • Transparency (transfer protocol irrelevant) Thursday, March 5, 2009
    60. The Future: Goals • Transparency (transfer protocol irrelevant) • Mixing CDNs and static file servers Thursday, March 5, 2009
    61. The Future: Goals • Transparency (transfer protocol irrelevant) • Mixing CDNs and static file servers • Processing before sync: image optimization, transcoding … Thursday, March 5, 2009
    62. The Future: Goals • Transparency (transfer protocol irrelevant) • Mixing CDNs and static file servers • Processing before sync: image optimization, transcoding … • Detect new files instantly (inotify/FSEvents/WMI) Thursday, March 5, 2009
    63. The Future: Goals • Transparency (transfer protocol irrelevant) • Mixing CDNs and static file servers • Processing before sync: image optimization, transcoding … • Detect new files instantly (inotify/FSEvents/WMI) … thanks to the daemon I’ll write. Thursday, March 5, 2009
    64. The Future: Sample scenario 1 Company wants to switch from CDN provider A to provider B. A only supports FTP, B only supports SFTP. Thursday, March 5, 2009
    65. The Future: Sample scenario 1 Company wants to switch from CDN provider A to provider B. A only supports FTP, B only supports SFTP. • Setup: • CDN A → CDN B • Daemon Thursday, March 5, 2009
    66. The Future: Sample scenario 1 Company wants to switch from CDN provider A to provider B. A only supports FTP, B only supports SFTP. • Setup: • CDN A → CDN B • Daemon • Alternative: write a lot of code Thursday, March 5, 2009
    67. The Future: Sample scenario 1 Company wants to switch from CDN provider A to provider B. A only supports FTP, B only supports SFTP. • Setup: • CDN A → CDN B • Daemon • Alternative: write a lot of code • Result: no switching cost → no lock-in! Thursday, March 5, 2009
    68. The Future: Sample scenario 2 U.S. company expands to South-Korea Thursday, March 5, 2009
    69. The Future: Sample scenario 2 U.S. company expands to South-Korea • Setup: • North-American CDN • Static file servers in South-Korea • Daemon + GeoIP API/Domain GeoLocalization module Thursday, March 5, 2009
    70. The Future: Sample scenario 2 U.S. company expands to South-Korea • Setup: • North-American CDN • Static file servers in South-Korea • Daemon + GeoIP API/Domain GeoLocalization module • Alternative: global CDN Thursday, March 5, 2009
    71. The Future: Sample scenario 2 U.S. company expands to South-Korea • Setup: • North-American CDN • Static file servers in South-Korea • Daemon + GeoIP API/Domain GeoLocalization module • Alternative: global CDN • Result: faster web site, same setup cost, lower maintenance cost Thursday, March 5, 2009
    72. The Future: Sample scenario 3 U.S. company hosts an event in Washington D.C. Thursday, March 5, 2009
    73. The Future: Sample scenario 3 U.S. company hosts an event in Washington D.C. • Setup: • North-American CDN • Static file servers in Washington D.C. • Daemon + GeoIP API/Domain GeoLocalization module Thursday, March 5, 2009
    74. The Future: Sample scenario 3 U.S. company hosts an event in Washington D.C. • Setup: • North-American CDN • Static file servers in Washington D.C. • Daemon + GeoIP API/Domain GeoLocalization module • Alternative: a slower web site Thursday, March 5, 2009
    75. The Future: Sample scenario 3 U.S. company hosts an event in Washington D.C. • Setup: • North-American CDN • Static file servers in Washington D.C. • Daemon + GeoIP API/Domain GeoLocalization module • Alternative: a slower web site • Result: faster web site, same setup cost, slightly higher maintenance cost Thursday, March 5, 2009
    76. The Future: Other use cases • Massive back-up tool • Transcoding server • Key component in creating your own CDN • Key component for a Dropbox-like tool Thursday, March 5, 2009
    77. The Future: What else? What else do you need? More use cases? More ideas? … Talk to me! Thursday, March 5, 2009
    78. Questions? Relevant links: • This presentation: http://wimleers.com/talk/drupalcon-dc-2009 • Bachelor thesis: http://wimleers.com/tags/bachelor-thesis • Questions & feedback: http://wimleers.com/contact • Modules: http://drupal.org/project/cdn & http://drupal.org/project/episodes Thursday, March 5, 2009

    + Wim LeersWim Leers, 8 months ago

    custom

    1670 views, 4 favs, 1 embeds more stats

    90% of the page loading time is spent on retrieving more

    More info about this document

    © All Rights Reserved

    Go to text version

    • Total Views 1670
      • 1553 on SlideShare
      • 117 from embeds
    • Comments 0
    • Favorites 4
    • Downloads 55
    Most viewed embeds
    • 117 views on http://wimleers.com

    more

    All embeds
    • 117 views on http://wimleers.com

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories

    Tags

    Groups / Events

    -->
    Search
    RSS Feed
    What's new?

    © 2009 SlideShare Inc. All Rights Reserved