0628阙宏宇

2,692 views

Published on

Published in: Design, Technology
0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,692
On SlideShare
0
From Embeds
0
Number of Embeds
310
Actions
Shares
0
Downloads
143
Comments
0
Likes
11
Embeds 0
No embeds

No notes for slide

0628阙宏宇

  1. 1. Lighttpd & Modcache http://blog.quehy.com/ 2009/06/28 Beta 技术沙龙 http://club.blogbeta.com 官方 twitter : @betasalon
  2. 2. Lighttpd & Modcache 2009/06/28
  3. 3. Basic lighttpd info <ul><li>Event-driven, single process </li></ul><ul><li>Uses non-block io (network) + writev (memory) + sendfile (local files) </li></ul><ul><li>Light weighted, simple module structure </li></ul><ul><li>Homepage </li></ul><ul><ul><li>http://redmine.lighttpd.net/projects/lighttpd </li></ul></ul>
  4. 4. Lighttpd ’ s advantages <ul><li>No need to sync data and variables between modules </li></ul><ul><li>Modern OS/Hardware are so powerful that event-driven model works great </li></ul><ul><li>Handy conditional configuration </li></ul><ul><li>Easy to write modules ( http://redmine.lighttpd.net/projects/lighttpd/wiki/Devel ) </li></ul>
  5. 5. Lighttpd ’ s disadvantages <ul><li>Sometimes one process isn ’ t enough (try server.max-workers) </li></ul><ul><li>One block io operation may block whole http service </li></ul><ul><li>mod_fastcgi doesn ’ t work as fast as it should be because of poor request distribution algorithm </li></ul>
  6. 6. General lighttpd tuning <ul><li>server.event-handler = &quot;linux-sysepoll“ </li></ul><ul><li>server.max-fds = 16384 # set max open fd limit to 16k </li></ul><ul><li>server.max-keep-alive-idle = 0 #don’t use keep-alive </li></ul><ul><li>server.max-read-idle = 30 # smaller read timeout </li></ul><ul><li>server.max-write-idle = 180 # smaller write timeout </li></ul>
  7. 7. Mod_fastcgi Tuning <ul><li>server.max-request-size = 40960 #maximum 40M post content length </li></ul><ul><li>fastcgi.server = ( &quot;.php&quot; => (( </li></ul><ul><li>&quot;socket&quot; => &quot;/tmp/php-fastcgi.socket&quot;, </li></ul><ul><li>&quot;bin-path&quot; => &quot;/usr/bin/php-cgi -c /etc/php.ini&quot;, </li></ul><ul><li>&quot;max-procs&quot; => 20, </li></ul><ul><li>&quot;bin-environment&quot; => ( </li></ul><ul><li>&quot;PHP_FCGI_CHILDREN&quot; => &quot;9&quot;, #lighttpd will fork 20*10=200 php-cgi children </li></ul><ul><li>&quot;PHP_FCGI_MAX_REQUESTS&quot; => &quot;50000“ # php child restart after processing 50k requests </li></ul><ul><li>), </li></ul><ul><li>&quot;broken-scriptfilename&quot; => &quot;enable&quot;, </li></ul><ul><li>&quot;allow-x-send-file&quot; => &quot;enable&quot;, </li></ul><ul><li>&quot;check-local&quot; => &quot;disable&quot;, </li></ul><ul><li>&quot;bin-copy-environment&quot; => ( &quot;PATH&quot;, &quot;SHELL&quot;, &quot;USER&quot; ) </li></ul><ul><li>)) </li></ul><ul><li>) </li></ul>
  8. 8. Basic modcache info <ul><li>Project started when squid can ’ t fulfill need </li></ul><ul><li>v1.0 released at 10/2006, v1.8 released at 06/2009 </li></ul><ul><li>Open source, BSD License </li></ul><ul><li>Supports both disk-based cache and memory-based cache </li></ul><ul><li>Homepage </li></ul><ul><ul><li>http://www.linux.com.cn/modcache/ </li></ul></ul>
  9. 9. Modcache ’ s advantages <ul><li>Lighttpd based (it ’ s a powerful web server, far away beyond normal proxy server) </li></ul><ul><li>Works with other lighttpd modules, such as mod_compress / mod_access </li></ul><ul><li>Keep it as simple as possible </li></ul><ul><li>Small source code base (< 3000 lines) </li></ul><ul><li>Don ’ t use local db to store cache information </li></ul>
  10. 10. Modcache ’ s disadvantages <ul><li>No internal cache files management (try /usr/sbin/tmpwatch) </li></ul><ul><li>Rules of cache are controlled by modcache, not by backend servers </li></ul><ul><li>Memory cache doesn ’ t share between different lighttpd processes </li></ul><ul><li>It ’ s a patch, not included in official lighttpd release </li></ul>
  11. 11. Modcache vs Squid <ul><li>Modcache </li></ul><ul><li>Event based </li></ul><ul><li>Directories based </li></ul><ul><li>Without disk management </li></ul><ul><li>Reverse Cache only </li></ul><ul><li>No ICP </li></ul><ul><li>Expire time based on refresh.pattern </li></ul><ul><li>Conditional config /ACL </li></ul><ul><li>Small source codebase </li></ul><ul><li>Squid </li></ul><ul><li>Thread based </li></ul><ul><li>DB based </li></ul><ul><li>With disk management </li></ul><ul><li>Reverse/Forward Cache </li></ul><ul><li>Supports ICP </li></ul><ul><li>Expire time based on backend response and refresh-pattern </li></ul><ul><li>ACL </li></ul><ul><li>Large source codebase </li></ul>
  12. 12. Modcache config(1) <ul><li>cache.bases </li></ul><ul><ul><li>List of cache directories. For examples: cache.bases = (&quot;/data/cache&quot;, &quot;/data2/cache&quot;) </li></ul></ul><ul><li>cache.domains </li></ul><ul><ul><li>List of domains which modcache would try to cache for. When it isn’t set, modcache would cache files of every domains. for examples: cache.domains = (&quot;(^|.)linux.com.cn$&quot;) caches *.linux.com.cn. domains. </li></ul></ul><ul><li>cache.ignore-hostname . </li></ul><ul><ul><li>Includes hostname in saved filename or not. default is disable. </li></ul></ul><ul><li>cache.debug </li></ul><ul><ul><li>Default is disable </li></ul></ul><ul><li>cache.enable </li></ul><ul><ul><li>Default is enable </li></ul></ul>
  13. 13. Modcache config(2) <ul><li>cache.support-queries </li></ul><ul><ul><li>Try to cache query with '?'. default is 'disable'. When cache.support-queries = &quot;enable&quot; and cache.dynamic-mode = &quot;disable&quot;, modcache will treat &quot;/uri?q1&quot; or &quot;/uri?q2&quot; as &quot;/uri&quot; and save them to local cache file of &quot;/uri&quot;. When cache.support-queries and cache.dynamic-mode are both enabled, modcache will treat &quot;/uri?q1&quot; and &quot;/uri?q2&quot; as different resource and save them to different local cache files. </li></ul></ul><ul><li>cache.dynamic-mode </li></ul><ul><ul><li>To support dynamic generated web page such as forum/bbs. default is &quot;disable&quot;. Note: cache.support-queries has to be enabled when dynamic-mode is enabled. </li></ul></ul><ul><li>cache.programs-ext </li></ul><ul><ul><li>List of local program extension, such as &quot;.php&quot;. modcache will replace them with &quot;.cache.html&quot; to aviod confliction with other modules, such as mod_fastcgi. default is not set </li></ul></ul>
  14. 14. Modcache config(3) <ul><li>cache.max-memory-size </li></ul><ul><ul><li>Number of MBytes which modcache will use to save cached content in memory. default is 256Mbytes. </li></ul></ul><ul><li>cache.lru-remove-count </li></ul><ul><ul><li>Number of content removed from memory when memory is full. default is 256. </li></ul></ul><ul><li>cache.purge-host </li></ul><ul><ul><li>pcre regex hosts ip which are allowed to PURGE cache files. for examples: cache.purge-host=&quot;^200.100.1.&quot; allow 200.100.1.0/24 to purge cache. Default is not set </li></ul></ul>
  15. 15. Cache.refresh-pattern <ul><li>Key of modcache, idea from squid ’ s refresh-pattern </li></ul><ul><li>cache.refresh-pattern format is &quot;url_pcre_regex&quot; => &quot;minutes options&quot;. Note: zero 'minutes' means cache forever and minutes option is mandatory. </li></ul>
  16. 16. Refresh Pattern options(1) <ul><li>ignore-reload </li></ul><ul><ul><li>Don't update cache when browser sends 'Cache-Control: no-cache' header. It's default behaviour </li></ul></ul><ul><li>update-on-refresh </li></ul><ul><ul><li>Update cache when browser sends 'Cache-Control: no-cache' header. </li></ul></ul><ul><li>Nocache </li></ul><ul><ul><li>Don't cache matched url . </li></ul></ul><ul><li>fetchall-for-range-request </li></ul><ul><ul><li>Download all content of file if client sends 'Range: xxx-yyybytes' header. Useful for multi-thread downloaders(such as flashget). </li></ul></ul>
  17. 17. Refresh Pattern options(2) <ul><li>rfc-violated options: </li></ul><ul><li>override-expire </li></ul><ul><ul><li>Ignore backend's 'Expire' header while determining whether to cache. </li></ul></ul><ul><li>ignore-vary </li></ul><ul><ul><li>Igore backend’s ‘Vary’ header while determining whether to cache </li></ul></ul><ul><li>ignore-cache-control-header </li></ul><ul><ul><li>Ignore backend's 'Cache-Control' headers while determining whether to cache. </li></ul></ul>
  18. 18. Refresh Pattern options(3) <ul><li>flv-streaming </li></ul><ul><ul><li>to work with mod_flv_streaming. </li></ul></ul><ul><li>memory-compress </li></ul><ul><ul><li>compress memory-saved content for ‘Accept-Encoding: gzip, deflate’ request </li></ul></ul><ul><li>use-memory </li></ul><ul><ul><li>use memory to save caches instead of local disk </li></ul></ul>
  19. 19. Modcache Cook Books (1) <ul><li>Site with several millions image files </li></ul><ul><li>server.modules = ( </li></ul><ul><li># ...., # other modules </li></ul><ul><li>&quot;mod_cache&quot;, # make sure mod_cache loaded before mod_proxy </li></ul><ul><li>&quot;mod_proxy&quot; </li></ul><ul><li>) </li></ul><ul><li>proxy.worked-with-mod-cache = &quot;enable&quot; </li></ul><ul><li>cache.bases = (&quot;/data/cache&quot;) </li></ul><ul><li>cache.support-queries = &quot;enable&quot; </li></ul><ul><li>cache.max-memory = 2000 # 2000M suitable for 4G memory server </li></ul><ul><li>cache.refresh-pattern = ( </li></ul><ul><li>&quot;.(?i)(jpg|bmp|jpeg|gif|png|ico)$&quot; => &quot;0 use-memory&quot;, #cache for ever until removed from memory </li></ul><ul><li>&quot;.&quot; => &quot;120 &quot; # other uses disk based cache </li></ul><ul><li>) </li></ul>
  20. 20. Modcache Cook Books (2) <ul><li>Softwares/Music/Video download site </li></ul><ul><li># uses several separate disks </li></ul><ul><li>cache.bases = (&quot;/data/cache1&quot;, &quot;/data/cache2&quot;, &quot;/data/cache3&quot;, &quot;/data/cache4&quot;) </li></ul><ul><li>cache.support-queries = &quot;enable&quot; </li></ul><ul><li>cache.refresh-pattern = ( </li></ul><ul><li>&quot;.(?i)(rar|zip|exe|wmv|avi|mp3|ape|rm|mpeg|mpg|wma|asf|rmvb)$&quot; => &quot;10080 fetchall-for-range-request&quot;, </li></ul><ul><li>&quot;.&quot; => &quot;120&quot; </li></ul><ul><li>) </li></ul>
  21. 21. Modcache Cook Books (3) <ul><li>BBS Site </li></ul><ul><li>server.modules = ( </li></ul><ul><li>&quot;mod_compress&quot;, #use it to compress html/css files </li></ul><ul><li>&quot;mod_cache&quot;, &quot;mod_proxy&quot; </li></ul><ul><li>) </li></ul><ul><li>mimetype.assign = ( </li></ul><ul><li>&quot;.css&quot; => &quot;text/css&quot;, </li></ul><ul><li>&quot;.js&quot; => &quot;application/x-javascript&quot;, </li></ul><ul><li>&quot;.html&quot; => &quot;text/html&quot;, </li></ul><ul><li>&quot;.htm&quot; => &quot;text/html&quot; </li></ul><ul><li>) </li></ul>
  22. 22. Modcache Cook Books (3) <ul><li>BBS Site(continued) </li></ul><ul><li># compress output for css/js/html/htm </li></ul><ul><li>compress.filetype = (&quot;application/x-javascript&quot;, &quot;text/css&quot;, &quot;text/html&quot;) </li></ul><ul><li>cache.support-queries = &quot;enable&quot; </li></ul><ul><li>cache.dynamic-mode = &quot;enable&quot; </li></ul><ul><li>cache.bases = (&quot;/data/cache&quot;) </li></ul><ul><li>cache.refresh-pattern = ( </li></ul><ul><li>&quot;.(?i)(js|css|xml)$&quot; => &quot;240&quot;, </li></ul><ul><li>&quot;.(?i)(htm|html|shtml)$&quot; => &quot;30 ignore-vary&quot;, </li></ul><ul><li>&quot;.(?i)(/|php)$&quot; => &quot;5 use-memory update-on-refresh memory-compress&quot;, </li></ul><ul><li>&quot;.&quot; => &quot;30 update-on-refresh&quot; # default to update every 30 minutes and on refresh requests </li></ul><ul><li>) </li></ul>
  23. 23. Modcache Cook Books (4) <ul><li>Video share site </li></ul><ul><li>server.modules = ( </li></ul><ul><li>&quot;mod_flv_streaming&quot;, &quot;mod_cache&quot;, &quot;mod_proxy&quot; </li></ul><ul><li>) </li></ul><ul><li>flv-streaming.extensions = (&quot;.flv&quot;) </li></ul><ul><li>cache.bases = (&quot;/data/cache1&quot;, &quot;/data/cache2&quot;, &quot;/data/cache3“) </li></ul><ul><li>cache.support-queries = &quot;enable&quot; </li></ul><ul><li>cache.refresh-pattern = ( </li></ul><ul><li>&quot;.(?i)(flv)$&quot; => &quot;0 flv-streaming&quot;, # flv_streaming for flv files </li></ul><ul><li>&quot;.&quot; => &quot;120&quot; </li></ul><ul><li>) </li></ul>
  24. 24. Thanks for listening Q&A

×