Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Nginx internals

5,580 views

Published on

Published in: Technology
  • Dating for everyone is here: ♥♥♥ http://bit.ly/2F4cEJi ♥♥♥
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Sex in your area is here: ❤❤❤ http://bit.ly/2F4cEJi ❤❤❤
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Nginx internals

  1. 1. Nginx Internals [email_address]
  2. 2. Agenda <ul><li>What </li></ul><ul><li>Why </li></ul><ul><li>Architecture </li></ul><ul><li>Infrastructure </li></ul><ul><li>Http Module </li></ul><ul><li>Conf & Location &Static </li></ul><ul><li>Upstream & fastcgi </li></ul><ul><li>Extension </li></ul>
  3. 3. What’s Nginx <ul><li>October 4, 2004 </li></ul><ul><li>6.8% of the top 1 million </li></ul><ul><li>HTTP Server </li></ul><ul><li>Reverse proxy server </li></ul><ul><li>Mail proxy server </li></ul><ul><li>http://nginx.org/en/ </li></ul>
  4. 4. Why Nginx <ul><li>Features : Nginx = transmit + apache </li></ul><ul><li>Stability : Nginx > Lighttpd </li></ul><ul><li>Performance : Nginx > Lighttpd > apache </li></ul><ul><li>Security : Acess & Limit Zone & Limit Requests </li></ul><ul><li>Community : sina/qq/tudou/taobao </li></ul><ul><li>Third party modules </li></ul><ul><li>more </li></ul>
  5. 5. Code Organization <ul><li>117590 </li></ul><ul><li>Core main/lib/data struct </li></ul><ul><li>Event epool/time/mutex </li></ul><ul><li>Http http server/upstream </li></ul><ul><li>Os & mail </li></ul>
  6. 6. Architecture <ul><li>Multi-process </li></ul><ul><li>Master+worker </li></ul><ul><li>Event driven </li></ul><ul><li>Non-blocking </li></ul><ul><li>Single-threaded </li></ul><ul><li>Highly modular </li></ul><ul><li>Cache management process </li></ul>
  7. 7. Infrastructure <ul><li>String </li></ul><ul><li>Array/List/Queue </li></ul><ul><li>Hash Table </li></ul><ul><li>Red black tree </li></ul><ul><li>Memory pool </li></ul><ul><li>Buff </li></ul><ul><li>Radix Tree / Slab / Spinlock / shmtx </li></ul><ul><li>http://code.google.com/p/nginxsrp/wiki/NginxCodeReview </li></ul>
  8. 8. Memory Management <ul><li>ngx_alloc , ngx_calloc , ngx_memalign </li></ul><ul><li>Memory pool </li></ul><ul><ul><li>Avoid memory fragmentation and memory leaks </li></ul></ul><ul><ul><li>Allocation once, deallocation once </li></ul></ul><ul><ul><li>lifetime : cycle 、 connection 、 Request </li></ul></ul><ul><ul><li>Optimizing large memory allocation </li></ul></ul><ul><ul><li>ngx_palloc ngx_pnalloc ngx_pcalloc ngx_pmemalign </li></ul></ul>
  9. 9. Memory pool
  10. 10. Master process
  11. 11. Worker process
  12. 12. Master & woker <ul><li>Socketpair </li></ul><ul><li>Accept Lock, avoid thundering herd </li></ul><ul><li>Multi Accept </li></ul><ul><li>Worker Processes Load-balance (7/8) </li></ul><ul><li>Connection List per process </li></ul><ul><li>Timeout handling. Red black tree </li></ul>
  13. 13. Config & command <ul><li>Command-style configuration files, if/echo </li></ul><ul><li>Configuration block, ‘{}’ </li></ul><ul><li>Support Include command </li></ul><ul><li>Support inheritance(Http main/Server/Location) </li></ul><ul><li>Multi-parameter </li></ul><ul><li>Variables </li></ul><ul><li>Conf demo </li></ul>
  14. 14. Config parse <ul><li>Type parse_file parse_block parse_param </li></ul><ul><li>Command handler </li></ul>
  15. 15. Modular
  16. 16. Http Module <ul><li>http command, http {} </li></ul><ul><li>ngx_http_block </li></ul><ul><li>ngx_http_module_t </li></ul>
  17. 17. Http request processing 1 、 phase checker 2 、 phase handler, array 3 、 sub request
  18. 18. http phase <ul><li>NGX_HTTP_POST_READ_PHASE (realip) </li></ul><ul><li>NGX_HTTP_SERVER_REWRITE_PHASE (Rewrite) </li></ul><ul><li>NGX_HTTP_FIND_CONFIG_PHASE Can not be extended </li></ul><ul><li>NGX_HTTP_REWRITE_PHASE (Rewite) </li></ul><ul><li>NGX_HTTP_POST_REWRITE_PHAS E Can not be extended </li></ul><ul><li>NGX_HTTP_PREACCESS_PHASE (limit_req 、 limit_zone 、 degradation 、 realip) </li></ul><ul><li>NGX_HTTP_ACCESS_PHASE (ACCESS 、 auth_basic) </li></ul><ul><li>NGX_HTTP_POST_ACCESS_PHASE Can not be extended </li></ul><ul><li>NGX_HTTP_TRY_FILES_PHASE Can not be extended </li></ul><ul><li>NGX_HTTP_CONTENT_PHASE (Autoindex 、 static 、 random_index 、 gzip_static) </li></ul><ul><li>NGX_HTTP_LOG_PHASE </li></ul>
  19. 19. content phase handler vs content_handler <ul><li>checker: ngx_http_core_content_phase </li></ul><ul><li>Fastcgi & autoindex </li></ul>
  20. 20. Nginx Filter <ul><li>Header Filter </li></ul><ul><li>Content Filter </li></ul><ul><li>ngx_http_send_header </li></ul><ul><li>ngx_http_output_filter </li></ul><ul><li>Example : gzip 、 more_set_header </li></ul>
  21. 21. Location features <ul><li>Exactly match </li></ul><ul><li>Regular(sensitive/ insensitive) </li></ul><ul><li>Location inheritance </li></ul><ul><li>How to find the right location? </li></ul>
  22. 22. Location internals <ul><li>Ngx_http_block loc_conf </li></ul><ul><li>ngx_http_init_locations Location inheritance check </li></ul><ul><li>ngx_http_init_static_location_trees Ternary Search Tree(better than trie) </li></ul><ul><li>Find location(ngx_http_core_find_location) first static location tree then regex locations </li></ul>
  23. 23. Static module <ul><li>Sendfile http://linux.die.net/man/2/sendfile </li></ul><ul><li>NGX_HTTP_CONTENT_PHASE ngx_http_static_handler </li></ul><ul><li>ngx_http_static_handler 1 、 method check. GET/HEAD/POST 2 、 Cache processing 3 、 ngx_http_discard_request_body 4 、 output </li></ul><ul><li>Only Cache fd </li></ul><ul><li>Red black tree </li></ul>
  24. 24. Nginx Upstream <ul><li>Protocol : Http/Fastcgi/Memcache/SCGI </li></ul><ul><li>Load-balance: hash/random/round_robin </li></ul><ul><li>Short connection </li></ul><ul><li>Long connection only for memcached </li></ul>
  25. 25. Nginx Upstream : fastcgi
  26. 26. Extension Develop <ul><li>Core Module Extension, such as nginx-php </li></ul><ul><li>Http Module Extension Handler : echo Filter : gzip, usertrack upstream : fastcgi Load-balancers : rand </li></ul><ul><li>Emiller's Guide To Nginx Module Development </li></ul>
  27. 27. Extension Develop—where? <ul><li>Just before the server reads the config file </li></ul><ul><li>For every configuration directive for the location and server for which it appears; </li></ul><ul><li>When Nginx initializes the main configuration </li></ul><ul><li>When Nginx initializes the server (i.e., host/port) configuration </li></ul><ul><li>When Nginx merges the server configuration with the main configuration </li></ul><ul><li>When Nginx initializes the location configuration </li></ul><ul><li>When Nginx merges the location configuration with its parent server configuration </li></ul><ul><li>When Nginx's master process starts </li></ul><ul><li>When a new worker process starts </li></ul><ul><li>When a worker process exits </li></ul><ul><li>When the master exits </li></ul><ul><li>Handling a request </li></ul><ul><li>Filtering response headers </li></ul><ul><li>Filtering the response body </li></ul><ul><li>Picking a backend server </li></ul><ul><li>Initiating a request to a backend server </li></ul><ul><li>Re-initiating a request to a backend server </li></ul><ul><li>Processing the response from a backend server </li></ul><ul><li>Finishing an interaction with a backend server </li></ul>
  28. 28. Extension Develop—where?
  29. 29. Extension Develop—where?
  30. 30. Extension Develop—where? <ul><li>Http Phase handler </li></ul><ul><li>Content_handler </li></ul><ul><li>Header Filter </li></ul><ul><li>Content Filter </li></ul>
  31. 31. Extension Develop—step <ul><li>“ Config” file </li></ul><ul><li>ngx_addon_name=ngx_http_baidu_usertrack_module </li></ul><ul><li>HTTP_AUX_FILTER_MODULES=&quot;$HTTP_AUX_FILTER_MODULES ngx_http_baidu_usertrack_module&quot; </li></ul><ul><li>NGX_ADDON_SRCS=&quot;$NGX_ADDON_SRCS $ngx_addon_dir/src/ngx_http_baidu_usertrack_module.c $ngx_addon_dir/src/baidu_des.c&quot; </li></ul><ul><li>NGX_ADDON_DEPS=&quot;$NGX_ADDON_DEPS $ngx_addon_dir/src/baidu_des.h&quot; </li></ul><ul><li>ORE_LIBS=&quot;$CORE_LIBS -lcrypto&quot; </li></ul><ul><li>Develop and build ./configure -add-module=path/to/your/new/module/directory Make Make install </li></ul>
  32. 32. Extension Develop—demo <ul><li>Demo </li></ul>
  33. 33. Q&A Q&A Thanks

×