Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.



Published on

Published in: Design, Technology
  • Be the first to comment


  1. 1. Lighttpd & Modcache 2009/06/28 Beta 技术沙龙 官方 twitter : @betasalon
  2. 2. Lighttpd & Modcache 2009/06/28
  3. 3. Basic lighttpd info <ul><li>Event-driven, single process </li></ul><ul><li>Uses non-block io (network) + writev (memory) + sendfile (local files) </li></ul><ul><li>Light weighted, simple module structure </li></ul><ul><li>Homepage </li></ul><ul><ul><li> </li></ul></ul>
  4. 4. Lighttpd ’ s advantages <ul><li>No need to sync data and variables between modules </li></ul><ul><li>Modern OS/Hardware are so powerful that event-driven model works great </li></ul><ul><li>Handy conditional configuration </li></ul><ul><li>Easy to write modules ( ) </li></ul>
  5. 5. Lighttpd ’ s disadvantages <ul><li>Sometimes one process isn ’ t enough (try server.max-workers) </li></ul><ul><li>One block io operation may block whole http service </li></ul><ul><li>mod_fastcgi doesn ’ t work as fast as it should be because of poor request distribution algorithm </li></ul>
  6. 6. General lighttpd tuning <ul><li>server.event-handler = &quot;linux-sysepoll“ </li></ul><ul><li>server.max-fds = 16384 # set max open fd limit to 16k </li></ul><ul><li>server.max-keep-alive-idle = 0 #don’t use keep-alive </li></ul><ul><li>server.max-read-idle = 30 # smaller read timeout </li></ul><ul><li>server.max-write-idle = 180 # smaller write timeout </li></ul>
  7. 7. Mod_fastcgi Tuning <ul><li>server.max-request-size = 40960 #maximum 40M post content length </li></ul><ul><li>fastcgi.server = ( &quot;.php&quot; => (( </li></ul><ul><li>&quot;socket&quot; => &quot;/tmp/php-fastcgi.socket&quot;, </li></ul><ul><li>&quot;bin-path&quot; => &quot;/usr/bin/php-cgi -c /etc/php.ini&quot;, </li></ul><ul><li>&quot;max-procs&quot; => 20, </li></ul><ul><li>&quot;bin-environment&quot; => ( </li></ul><ul><li>&quot;PHP_FCGI_CHILDREN&quot; => &quot;9&quot;, #lighttpd will fork 20*10=200 php-cgi children </li></ul><ul><li>&quot;PHP_FCGI_MAX_REQUESTS&quot; => &quot;50000“ # php child restart after processing 50k requests </li></ul><ul><li>), </li></ul><ul><li>&quot;broken-scriptfilename&quot; => &quot;enable&quot;, </li></ul><ul><li>&quot;allow-x-send-file&quot; => &quot;enable&quot;, </li></ul><ul><li>&quot;check-local&quot; => &quot;disable&quot;, </li></ul><ul><li>&quot;bin-copy-environment&quot; => ( &quot;PATH&quot;, &quot;SHELL&quot;, &quot;USER&quot; ) </li></ul><ul><li>)) </li></ul><ul><li>) </li></ul>
  8. 8. Basic modcache info <ul><li>Project started when squid can ’ t fulfill need </li></ul><ul><li>v1.0 released at 10/2006, v1.8 released at 06/2009 </li></ul><ul><li>Open source, BSD License </li></ul><ul><li>Supports both disk-based cache and memory-based cache </li></ul><ul><li>Homepage </li></ul><ul><ul><li> </li></ul></ul>
  9. 9. Modcache ’ s advantages <ul><li>Lighttpd based (it ’ s a powerful web server, far away beyond normal proxy server) </li></ul><ul><li>Works with other lighttpd modules, such as mod_compress / mod_access </li></ul><ul><li>Keep it as simple as possible </li></ul><ul><li>Small source code base (< 3000 lines) </li></ul><ul><li>Don ’ t use local db to store cache information </li></ul>
  10. 10. Modcache ’ s disadvantages <ul><li>No internal cache files management (try /usr/sbin/tmpwatch) </li></ul><ul><li>Rules of cache are controlled by modcache, not by backend servers </li></ul><ul><li>Memory cache doesn ’ t share between different lighttpd processes </li></ul><ul><li>It ’ s a patch, not included in official lighttpd release </li></ul>
  11. 11. Modcache vs Squid <ul><li>Modcache </li></ul><ul><li>Event based </li></ul><ul><li>Directories based </li></ul><ul><li>Without disk management </li></ul><ul><li>Reverse Cache only </li></ul><ul><li>No ICP </li></ul><ul><li>Expire time based on refresh.pattern </li></ul><ul><li>Conditional config /ACL </li></ul><ul><li>Small source codebase </li></ul><ul><li>Squid </li></ul><ul><li>Thread based </li></ul><ul><li>DB based </li></ul><ul><li>With disk management </li></ul><ul><li>Reverse/Forward Cache </li></ul><ul><li>Supports ICP </li></ul><ul><li>Expire time based on backend response and refresh-pattern </li></ul><ul><li>ACL </li></ul><ul><li>Large source codebase </li></ul>
  12. 12. Modcache config(1) <ul><li>cache.bases </li></ul><ul><ul><li>List of cache directories. For examples: cache.bases = (&quot;/data/cache&quot;, &quot;/data2/cache&quot;) </li></ul></ul><ul><li> </li></ul><ul><ul><li>List of domains which modcache would try to cache for. When it isn’t set, modcache would cache files of every domains. for examples: = (&quot;(^|.)$&quot;) caches * domains. </li></ul></ul><ul><li>cache.ignore-hostname . </li></ul><ul><ul><li>Includes hostname in saved filename or not. default is disable. </li></ul></ul><ul><li>cache.debug </li></ul><ul><ul><li>Default is disable </li></ul></ul><ul><li>cache.enable </li></ul><ul><ul><li>Default is enable </li></ul></ul>
  13. 13. Modcache config(2) <ul><li> </li></ul><ul><ul><li>Try to cache query with '?'. default is 'disable'. When = &quot;enable&quot; and cache.dynamic-mode = &quot;disable&quot;, modcache will treat &quot;/uri?q1&quot; or &quot;/uri?q2&quot; as &quot;/uri&quot; and save them to local cache file of &quot;/uri&quot;. When and cache.dynamic-mode are both enabled, modcache will treat &quot;/uri?q1&quot; and &quot;/uri?q2&quot; as different resource and save them to different local cache files. </li></ul></ul><ul><li>cache.dynamic-mode </li></ul><ul><ul><li>To support dynamic generated web page such as forum/bbs. default is &quot;disable&quot;. Note: has to be enabled when dynamic-mode is enabled. </li></ul></ul><ul><li>cache.programs-ext </li></ul><ul><ul><li>List of local program extension, such as &quot;.php&quot;. modcache will replace them with &quot;.cache.html&quot; to aviod confliction with other modules, such as mod_fastcgi. default is not set </li></ul></ul>
  14. 14. Modcache config(3) <ul><li>cache.max-memory-size </li></ul><ul><ul><li>Number of MBytes which modcache will use to save cached content in memory. default is 256Mbytes. </li></ul></ul><ul><li>cache.lru-remove-count </li></ul><ul><ul><li>Number of content removed from memory when memory is full. default is 256. </li></ul></ul><ul><li>cache.purge-host </li></ul><ul><ul><li>pcre regex hosts ip which are allowed to PURGE cache files. for examples: cache.purge-host=&quot;^200.100.1.&quot; allow to purge cache. Default is not set </li></ul></ul>
  15. 15. Cache.refresh-pattern <ul><li>Key of modcache, idea from squid ’ s refresh-pattern </li></ul><ul><li>cache.refresh-pattern format is &quot;url_pcre_regex&quot; => &quot;minutes options&quot;. Note: zero 'minutes' means cache forever and minutes option is mandatory. </li></ul>
  16. 16. Refresh Pattern options(1) <ul><li>ignore-reload </li></ul><ul><ul><li>Don't update cache when browser sends 'Cache-Control: no-cache' header. It's default behaviour </li></ul></ul><ul><li>update-on-refresh </li></ul><ul><ul><li>Update cache when browser sends 'Cache-Control: no-cache' header. </li></ul></ul><ul><li>Nocache </li></ul><ul><ul><li>Don't cache matched url . </li></ul></ul><ul><li>fetchall-for-range-request </li></ul><ul><ul><li>Download all content of file if client sends 'Range: xxx-yyybytes' header. Useful for multi-thread downloaders(such as flashget). </li></ul></ul>
  17. 17. Refresh Pattern options(2) <ul><li>rfc-violated options: </li></ul><ul><li>override-expire </li></ul><ul><ul><li>Ignore backend's 'Expire' header while determining whether to cache. </li></ul></ul><ul><li>ignore-vary </li></ul><ul><ul><li>Igore backend’s ‘Vary’ header while determining whether to cache </li></ul></ul><ul><li>ignore-cache-control-header </li></ul><ul><ul><li>Ignore backend's 'Cache-Control' headers while determining whether to cache. </li></ul></ul>
  18. 18. Refresh Pattern options(3) <ul><li>flv-streaming </li></ul><ul><ul><li>to work with mod_flv_streaming. </li></ul></ul><ul><li>memory-compress </li></ul><ul><ul><li>compress memory-saved content for ‘Accept-Encoding: gzip, deflate’ request </li></ul></ul><ul><li>use-memory </li></ul><ul><ul><li>use memory to save caches instead of local disk </li></ul></ul>
  19. 19. Modcache Cook Books (1) <ul><li>Site with several millions image files </li></ul><ul><li>server.modules = ( </li></ul><ul><li># ...., # other modules </li></ul><ul><li>&quot;mod_cache&quot;, # make sure mod_cache loaded before mod_proxy </li></ul><ul><li>&quot;mod_proxy&quot; </li></ul><ul><li>) </li></ul><ul><li>proxy.worked-with-mod-cache = &quot;enable&quot; </li></ul><ul><li>cache.bases = (&quot;/data/cache&quot;) </li></ul><ul><li> = &quot;enable&quot; </li></ul><ul><li>cache.max-memory = 2000 # 2000M suitable for 4G memory server </li></ul><ul><li>cache.refresh-pattern = ( </li></ul><ul><li>&quot;.(?i)(jpg|bmp|jpeg|gif|png|ico)$&quot; => &quot;0 use-memory&quot;, #cache for ever until removed from memory </li></ul><ul><li>&quot;.&quot; => &quot;120 &quot; # other uses disk based cache </li></ul><ul><li>) </li></ul>
  20. 20. Modcache Cook Books (2) <ul><li>Softwares/Music/Video download site </li></ul><ul><li># uses several separate disks </li></ul><ul><li>cache.bases = (&quot;/data/cache1&quot;, &quot;/data/cache2&quot;, &quot;/data/cache3&quot;, &quot;/data/cache4&quot;) </li></ul><ul><li> = &quot;enable&quot; </li></ul><ul><li>cache.refresh-pattern = ( </li></ul><ul><li>&quot;.(?i)(rar|zip|exe|wmv|avi|mp3|ape|rm|mpeg|mpg|wma|asf|rmvb)$&quot; => &quot;10080 fetchall-for-range-request&quot;, </li></ul><ul><li>&quot;.&quot; => &quot;120&quot; </li></ul><ul><li>) </li></ul>
  21. 21. Modcache Cook Books (3) <ul><li>BBS Site </li></ul><ul><li>server.modules = ( </li></ul><ul><li>&quot;mod_compress&quot;, #use it to compress html/css files </li></ul><ul><li>&quot;mod_cache&quot;, &quot;mod_proxy&quot; </li></ul><ul><li>) </li></ul><ul><li>mimetype.assign = ( </li></ul><ul><li>&quot;.css&quot; => &quot;text/css&quot;, </li></ul><ul><li>&quot;.js&quot; => &quot;application/x-javascript&quot;, </li></ul><ul><li>&quot;.html&quot; => &quot;text/html&quot;, </li></ul><ul><li>&quot;.htm&quot; => &quot;text/html&quot; </li></ul><ul><li>) </li></ul>
  22. 22. Modcache Cook Books (3) <ul><li>BBS Site(continued) </li></ul><ul><li># compress output for css/js/html/htm </li></ul><ul><li>compress.filetype = (&quot;application/x-javascript&quot;, &quot;text/css&quot;, &quot;text/html&quot;) </li></ul><ul><li> = &quot;enable&quot; </li></ul><ul><li>cache.dynamic-mode = &quot;enable&quot; </li></ul><ul><li>cache.bases = (&quot;/data/cache&quot;) </li></ul><ul><li>cache.refresh-pattern = ( </li></ul><ul><li>&quot;.(?i)(js|css|xml)$&quot; => &quot;240&quot;, </li></ul><ul><li>&quot;.(?i)(htm|html|shtml)$&quot; => &quot;30 ignore-vary&quot;, </li></ul><ul><li>&quot;.(?i)(/|php)$&quot; => &quot;5 use-memory update-on-refresh memory-compress&quot;, </li></ul><ul><li>&quot;.&quot; => &quot;30 update-on-refresh&quot; # default to update every 30 minutes and on refresh requests </li></ul><ul><li>) </li></ul>
  23. 23. Modcache Cook Books (4) <ul><li>Video share site </li></ul><ul><li>server.modules = ( </li></ul><ul><li>&quot;mod_flv_streaming&quot;, &quot;mod_cache&quot;, &quot;mod_proxy&quot; </li></ul><ul><li>) </li></ul><ul><li>flv-streaming.extensions = (&quot;.flv&quot;) </li></ul><ul><li>cache.bases = (&quot;/data/cache1&quot;, &quot;/data/cache2&quot;, &quot;/data/cache3“) </li></ul><ul><li> = &quot;enable&quot; </li></ul><ul><li>cache.refresh-pattern = ( </li></ul><ul><li>&quot;.(?i)(flv)$&quot; => &quot;0 flv-streaming&quot;, # flv_streaming for flv files </li></ul><ul><li>&quot;.&quot; => &quot;120&quot; </li></ul><ul><li>) </li></ul>
  24. 24. Thanks for listening Q&A