Evolving Archetecture

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    6 Favorites & 1 Group

    Evolving Archetecture - Presentation Transcript

    1. Evolving architecture make development easy and your site faster Leo Lapworth @ YAPC::EU 2007 http://leo.cuckoo.org/
    2. Covering
    3. Covering How running websites has evolved
    4. Covering How running websites has evolved Servers improved
    5. Covering How running websites has evolved Servers improved Architecture improved
    6. Covering How running websites has evolved Servers improved Architecture improved Development tools improved
    7. Covering How running websites has evolved Servers improved Architecture improved Development tools improved Code (and code libraries) got better
    8. but not...
    9. but not... Covering anything in depth
    10. Concepts stolen from
    11. Concepts stolen from Mod_perl guide
    12. Concepts stolen from Mod_perl guide Léon Brocard
    13. Concepts stolen from Mod_perl guide Léon Brocard Six Apart
    14. Concepts stolen from Mod_perl guide Léon Brocard Six Apart Yahoo!
    15. Concepts stolen from Mod_perl guide Léon Brocard Six Apart Yahoo! + many other people...
    16. Concepts stolen from Mod_perl guide Léon Brocard Six Apart Yahoo! + many other people... ... this is social Perl
    17. Templates
    18. Templates An example of how things have improved...
    19. Recognise this?
    20. Recognise this? use CGI; $q = CGI->new(); $name = $q->param('name'); print $q->header('text/html'); $link = 'http://foo.com/'; print \"<html><head><title>HTML and code together</title></head><body>\"; print \"<a href=\\\"$link/$name\\\">\"; print \"Click here $name</a><br />\\n\"; print \"<table>\" . join(\"\\n\", map { \"<tr><td>$_</td></tr>\" } qw(this is hard to maintain)); print \"</table>\";
    21. How about...
    22. How about... Now, was it cellspacing, or cellpadding I need for ie4.02?
    23. and...
    24. and... I'm a developer, not a designer. I shouldn't have to know this £@$%!
    25. Use templates...
    26. Use templates... use Template;
    27. Code, not HTML
    28. Code, not HTML use CGI; use Template; $q = CGI->new(); print $q->header('text/html'); my %vals = ( name = $q->param('name'), ); $template = Template->new({ INCLUDE => '/path/' }); $template->process('hello.html',\\%vals);
    29. I just code, I have clean code and I don't have to know HTML or browser compatibility issues
    30. Simple changes make a huge difference
    31. Servers
    32. Single server
    33. Single server Internet Web server (static & dynamic) All editing Database server
    34. Lots of...
    35. and... I thought I only changed one thing, but it's still broken
    36. 1 live server 1 development server
    37. 1 live server 1 development server Internet Live server Development server Web server Web server All editing (static & dynamic) (static & dynamic) Database server Database server
    38. Does not solve: What did I change last? Who changed what / when?
    39. You need...
    40. You need...
    41. Version control & docs
    42. Version control & docs Use SVK with Subversion
    43. Version control & docs Use SVK with Subversion
    44. Version control & docs Use SVK with Subversion Clkao - SVK
    45. Version control & docs Use SVK with Subversion Use Trac (http://trac.edgewall.org) Clkao - SVK
    46. Version control & docs Use SVK with Subversion Use Trac (http://trac.edgewall.org) - wiki Clkao - SVK
    47. Version control & docs Use SVK with Subversion Use Trac (http://trac.edgewall.org) - wiki - task tracker Clkao - SVK
    48. Version control & docs Use SVK with Subversion Use Trac (http://trac.edgewall.org) - wiki - task tracker - web interface to Subversion Clkao - SVK
    49. Tests - all you need on one slide
    50. Tests - all you need on one slide Write tests, write tests often
    51. Tests - all you need on one slide Write tests, write tests often Run tests, run tests often
    52. Tests - all you need on one slide Write tests, write tests often Run tests, run tests often Tests are good
    53. Tests - all you need on one slide Write tests, write tests often Run tests, run tests often Tests are good Found a bug? - write the test that replicates it. You know it's fixed when your test passes
    54. Tests - all you need on one slide Write tests, write tests often Run tests, run tests often Tests are good Found a bug? - write the test that replicates it. You know it's fixed when your test passes Use Test::More, and then test some more!
    55. Making your site faster
    56. mod_perl
    57. mod_perl Code caching (loaded/compiled once)
    58. mod_perl Code caching (loaded/compiled once) Server only spends time running the code
    59. mod_perl Code caching (loaded/compiled once) Server only spends time running the code Apache::Registry for old/existing code
    60. mod_perl Code caching (loaded/compiled once) Server only spends time running the code Apache::Registry for old/existing code Apache::SizeLimit (safety net)
    61. mod_perl Code caching (loaded/compiled once) Server only spends time running the code Apache::Registry for old/existing code Apache::SizeLimit (safety net) Set it high
    62. mod_perl Code caching (loaded/compiled once) Server only spends time running the code Apache::Registry for old/existing code Apache::SizeLimit (safety net) Set it high Check it
    63. Front/back end split
    64. Front/back end split Internet Slow Back end Front end (application server) Fast Web server Web server (static & proxy to (dynamic) application server) Database server Development server
    65. or put another way...
    66. or put another way... Internet slow and demanding kids (users)
    67. or put another way... Canteen Internet Front end slow and Dinner-ladies - good at demanding serving precooked kids (static) food. Can order (users) from à la carte chefs and then serve
    68. or put another way... Canteen Kitchen Internet Front end Back end slow and Dinner-ladies - good at Chefs - good at demanding serving precooked making à la carte kids (static) food. Can order (dynamic) food (users) from à la carte chefs and then serve
    69. Internet Slow Back end Front end (application server) Fast Web server Web server (static & proxy to (dynamic) application server) Database server Development server
    70. Internet Slow Back end Front end (application server) Fast Web server Web server (static & proxy to (dynamic) application server) Database server Development server
    71. Speed up the frequent...
    72. Speed up the frequent... Are you re-creating the same objects/data structures again and again?
    73. Speed up the frequent... Are you re-creating the same objects/data structures again and again? Can this information persist for a period of time?
    74. Add caching
    75. Add caching Persisting data for a period of time
    76. Add caching Persisting data for a period of time Simple to implement:
    77. Add caching Persisting data for a period of time Simple to implement: Does cache exist?
    78. Add caching Persisting data for a period of time Simple to implement: Does cache exist? YES -> retrieve it and return it
    79. Add caching Persisting data for a period of time Simple to implement: Does cache exist? YES -> retrieve it and return it NO -> create data/object, store in cache and return it
    80. Things you might cache
    81. Things you might cache Search result sets (a list of ids)
    82. Things you might cache Search result sets (a list of ids) Individual items
    83. Things you might cache Search result sets (a list of ids) Individual items Look-ups of information from a database
    84. Things you might cache Search result sets (a list of ids) Individual items Look-ups of information from a database Fetching of information from external sources
    85. Centralise
    86. Centralise Put caching methods in one package
    87. Centralise Put caching methods in one package Put any generic methods in a package
    88. We now have a cache Internet Slow Back end Front end (application server) Fast Web server Web server (static & proxy to Cache (dynamic) application server) Database server Development server
    89. We now have a cache Internet Slow Back end Front end (application server) Fast Web server Web server (static & proxy to Cache Cache (dynamic) application server) Database server Development server
    90. Adding in the cache...
    91. Adding in the cache... Internet Kids
    92. Adding in the cache... Internet Front end Kids Dinner-ladies
    93. Adding in the cache... Internet Back end Front end Kids Chefs Dinner-ladies
    94. Adding in the cache... Cache Internet Back end Front end Magic Kids Chefs Dinner-ladies freezer
    95. Coping with more traffic
    96. Coping with more traffic Development Internet Stage server server (in the office) Front end Back end Web server (application server) Database (static & proxy server to application Web server server) (dynamic) Back end (application server) Web server (dynamic)
    97. Coping with more traffic Development Internet Stage server server (in the office) Front end Back end Web server (application server) Database (static & proxy server to application Web server server) (dynamic) Back end (application server) Web server (dynamic)
    98. That's two separate disks
    99. That's two separate disks Development Internet Stage server server (in the office) Front end Back end Web server (application server) Database (static & proxy server to application Web server Cache server) (dynamic) Back end (application server) Web server Cache (dynamic)
    100. Second kitchen, two freezers...
    101. Second kitchen, two freezers... Internet Kids
    102. Second kitchen, two freezers... Internet Front end Kids Dinner-ladies
    103. Second kitchen, two freezers... Kitchen 1 Kitchen 2 Internet Front end Back end Kids Dinner-ladies Chefs
    104. Second kitchen, two freezers... Kitchen 1 Kitchen 2 Internet Front end Back end Cache Kids Dinner-ladies Chefs Freezers
    105. We want a cache across servers...
    106. We want a cache across servers... Enter Memcache
    107. We want a cache across servers... Enter Memcache See Léon Brocard's talk at 16:20 for the details
    108. Development Internet Stage server server (in the office) Front end Back end Web server (application server) Database (static & proxy server to application Web server server) (dynamic) Cache Back end (application server) Web server (dynamic)
    109. Shared freezer
    110. Shared freezer Internet Kids
    111. Shared freezer Internet Front end Kids Dinner-ladies
    112. Shared freezer Kitchen 1 Kitchen 2 Internet Front end Kids Dinner-ladies Back end Chefs
    113. Shared freezer Kitchen 1 Kitchen 2 Internet Front end Kids Dinner-ladies Back end Cache Chefs Freezer
    114. Maintaining servers
    115. Maintaining servers Keep all servers identical where possible
    116. Maintaining servers Keep all servers identical where possible Package your code (e.g. .debs)
    117. Maintaining servers Keep all servers identical where possible Package your code (e.g. .debs) Centralise your crontabs and configuration files (and put in version control)
    118. Maintaining servers Keep all servers identical where possible Package your code (e.g. .debs) Centralise your crontabs and configuration files (and put in version control) Deploy with one script: rsync + ssh + apt-get update & apt-get upgrade (or your OS equivalent) to each server
    119. Simplify further Development Internet Stage server server (in the office) Front end Back end Perlbal Web server load balancer / (static+dynamic) proxy Database Cache server Back end Web server (static+dynamic)
    120. Simplify further Development Internet Stage server server (in the office) Front end Back end Perlbal Web server load balancer / (static+dynamic) proxy Database Cache server Back end Web server (static+dynamic)
    121. Simplify further Development Internet Stage server server (in the office) Front end Back end Perlbal Web server load balancer / (static+dynamic) proxy Database Cache server Back end Web server (static+dynamic)
    122. Waiters: cheaper, faster and less demanding
    123. Waiters: cheaper, faster and less demanding Internet Kids
    124. Waiters: cheaper, faster and less demanding Front end Internet Waiters - good at serving Kids quickly and dealing with annoying customers or Kitchens
    125. Waiters: cheaper, faster and less demanding Kitchen 1 Kitchen 2 Front end Internet Waiters - good at serving Kids Back end quickly and dealing with annoying customers or Chefs - simple and complex meals (static and dynamic) Kitchens
    126. Waiters: cheaper, faster and less demanding Kitchen 1 Kitchen 2 Front end Internet Waiters - good at serving Kids Back end quickly and dealing with Cache annoying customers or Chefs - simple and complex meals Freezer (static and dynamic) Kitchens
    127. More tips for faster user experience
    128. mod_gzip
    129. mod_gzip HTML/css/xml
    130. mod_gzip HTML/css/xml
    131. mod_gzip gzip HTML/css/xml
    132. mod_gzip gzip HTML/css/xml
    133. mod_gzip gzip HTML/css/xml Users
    134. mod_gzip gzip HTML/css/xml Users If a browser support compressions... compress (javascript can be tricky - at least minify)
    135. Cache headers (expiry)
    136. Cache headers (expiry)
    137. Cache headers (expiry) Let web browsers know how long to cache
    138. Cache somethings forever
    139. Cache somethings forever /includes/js/<VERSION>/common.js
    140. Cache somethings forever /includes/js/<VERSION>/common.js Ensures user has version which matches HTML (client could have cache of old HTML)
    141. Cache somethings forever /includes/js/<VERSION>/common.js Ensures user has version which matches HTML (client could have cache of old HTML) Use include file to update all pages
    142. Handling images: MogileFS
    143. Handling images: MogileFS Store images by group (replication levels)
    144. Handling images: MogileFS Store images by group (replication levels) Spread IO across disks & servers
    145. Handling images: MogileFS Store images by group (replication levels) Spread IO across disks & servers Retrieve from fastest server
    146. Handling images: MogileFS Store images by group (replication levels) Spread IO across disks & servers Retrieve from fastest server Integrate with Perlbal, hidden from the user
    147. Handling images: MogileFS Store images by group (replication levels) Spread IO across disks & servers Retrieve from fastest server Integrate with Perlbal, hidden from the user Automatic resilience
    148. Handling images: MogileFS Store images by group (replication levels) Spread IO across disks & servers Retrieve from fastest server Integrate with Perlbal, hidden from the user Automatic resilience
    149. Basic rules to consider
    150. Basic rules to consider Centralise (code/templates/configuration/ deployment etc)
    151. Basic rules to consider Centralise (code/templates/configuration/ deployment etc) Test, run lots of them
    152. Basic rules to consider Centralise (code/templates/configuration/ deployment etc) Test, run lots of them Cache if you need it; memcache is great
    153. Basic rules to consider Centralise (code/templates/configuration/ deployment etc) Test, run lots of them Cache if you need it; memcache is great Keep is simple (perlbal especially)
    154. Basic rules to consider Centralise (code/templates/configuration/ deployment etc) Test, run lots of them Cache if you need it; memcache is great Keep is simple (perlbal especially) Sometimes it's just easier to buy another server (this is not always true)
    155. We covered...
    156. We covered... perlbal
    157. We covered... perlbal memcached
    158. We covered... perlbal memcached svk (subversion)
    159. We covered... perlbal memcached svk (subversion) trac
    160. We covered... perlbal memcached svk (subversion) trac rsync
    161. We covered... perlbal memcached svk (subversion) trac rsync gzip
    162. We covered... perlbal mogileFS memcached svk (subversion) trac rsync gzip
    163. We covered... perlbal mogileFS server architecture memcached svk (subversion) trac rsync gzip
    164. We covered... perlbal mogileFS server architecture memcached Test::More svk (subversion) trac rsync gzip
    165. We covered... perlbal mogileFS server architecture memcached Test::More svk (subversion) mod_perl trac rsync gzip
    166. We covered... perlbal mogileFS server architecture memcached Test::More svk (subversion) mod_perl trac Template::Toolkit rsync gzip
    167. We covered... perlbal mogileFS server architecture memcached Test::More svk (subversion) mod_perl trac Template::Toolkit rsync HTTP Cache headers gzip
    168. We covered... perlbal mogileFS server architecture memcached Test::More svk (subversion) mod_perl trac Template::Toolkit rsync HTTP Cache headers gzip and...
    169. ...how a kitchen should run
    170. ...how a kitchen should run
    171. ...how a kitchen should run Any questions?
    172. Evolving architecture http://leo.cuckoo.org/projects/ea/ Leo Lapworth http://leo.cuckoo.org/

    + leo lapworthleo lapworth, 3 years ago

    custom

    5030 views, 6 favs, 0 embeds more stats


    Looking at historic, current and evolving approach more

    More info about this presentation

    © All Rights Reserved

    • Total Views 5030
      • 5030 on SlideShare
      • 0 from embeds
    • Comments 0
    • Favorites 6
    • Downloads 173
    Most viewed embeds

    more

    All embeds

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories

    Groups / Events