Nginx internals

5,064 views

Published on

Published in: Technology
0 Comments
18 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,064
On SlideShare
0
From Embeds
0
Number of Embeds
843
Actions
Shares
0
Downloads
217
Comments
0
Likes
18
Embeds 0
No embeds

No notes for slide

Nginx internals

  1. 1. Nginx Internals [email_address]
  2. 2. Agenda <ul><li>What </li></ul><ul><li>Why </li></ul><ul><li>Architecture </li></ul><ul><li>Infrastructure </li></ul><ul><li>Http Module </li></ul><ul><li>Conf & Location &Static </li></ul><ul><li>Upstream & fastcgi </li></ul><ul><li>Extension </li></ul>
  3. 3. What’s Nginx <ul><li>October 4, 2004 </li></ul><ul><li>6.8% of the top 1 million </li></ul><ul><li>HTTP Server </li></ul><ul><li>Reverse proxy server </li></ul><ul><li>Mail proxy server </li></ul><ul><li>http://nginx.org/en/ </li></ul>
  4. 4. Why Nginx <ul><li>Features : Nginx = transmit + apache </li></ul><ul><li>Stability : Nginx > Lighttpd </li></ul><ul><li>Performance : Nginx > Lighttpd > apache </li></ul><ul><li>Security : Acess & Limit Zone & Limit Requests </li></ul><ul><li>Community : sina/qq/tudou/taobao </li></ul><ul><li>Third party modules </li></ul><ul><li>more </li></ul>
  5. 5. Code Organization <ul><li>117590 </li></ul><ul><li>Core main/lib/data struct </li></ul><ul><li>Event epool/time/mutex </li></ul><ul><li>Http http server/upstream </li></ul><ul><li>Os & mail </li></ul>
  6. 6. Architecture <ul><li>Multi-process </li></ul><ul><li>Master+worker </li></ul><ul><li>Event driven </li></ul><ul><li>Non-blocking </li></ul><ul><li>Single-threaded </li></ul><ul><li>Highly modular </li></ul><ul><li>Cache management process </li></ul>
  7. 7. Infrastructure <ul><li>String </li></ul><ul><li>Array/List/Queue </li></ul><ul><li>Hash Table </li></ul><ul><li>Red black tree </li></ul><ul><li>Memory pool </li></ul><ul><li>Buff </li></ul><ul><li>Radix Tree / Slab / Spinlock / shmtx </li></ul><ul><li>http://code.google.com/p/nginxsrp/wiki/NginxCodeReview </li></ul>
  8. 8. Memory Management <ul><li>ngx_alloc , ngx_calloc , ngx_memalign </li></ul><ul><li>Memory pool </li></ul><ul><ul><li>Avoid memory fragmentation and memory leaks </li></ul></ul><ul><ul><li>Allocation once, deallocation once </li></ul></ul><ul><ul><li>lifetime : cycle 、 connection 、 Request </li></ul></ul><ul><ul><li>Optimizing large memory allocation </li></ul></ul><ul><ul><li>ngx_palloc ngx_pnalloc ngx_pcalloc ngx_pmemalign </li></ul></ul>
  9. 9. Memory pool
  10. 10. Master process
  11. 11. Worker process
  12. 12. Master & woker <ul><li>Socketpair </li></ul><ul><li>Accept Lock, avoid thundering herd </li></ul><ul><li>Multi Accept </li></ul><ul><li>Worker Processes Load-balance (7/8) </li></ul><ul><li>Connection List per process </li></ul><ul><li>Timeout handling. Red black tree </li></ul>
  13. 13. Config & command <ul><li>Command-style configuration files, if/echo </li></ul><ul><li>Configuration block, ‘{}’ </li></ul><ul><li>Support Include command </li></ul><ul><li>Support inheritance(Http main/Server/Location) </li></ul><ul><li>Multi-parameter </li></ul><ul><li>Variables </li></ul><ul><li>Conf demo </li></ul>
  14. 14. Config parse <ul><li>Type parse_file parse_block parse_param </li></ul><ul><li>Command handler </li></ul>
  15. 15. Modular
  16. 16. Http Module <ul><li>http command, http {} </li></ul><ul><li>ngx_http_block </li></ul><ul><li>ngx_http_module_t </li></ul>
  17. 17. Http request processing 1 、 phase checker 2 、 phase handler, array 3 、 sub request
  18. 18. http phase <ul><li>NGX_HTTP_POST_READ_PHASE (realip) </li></ul><ul><li>NGX_HTTP_SERVER_REWRITE_PHASE (Rewrite) </li></ul><ul><li>NGX_HTTP_FIND_CONFIG_PHASE Can not be extended </li></ul><ul><li>NGX_HTTP_REWRITE_PHASE (Rewite) </li></ul><ul><li>NGX_HTTP_POST_REWRITE_PHAS E Can not be extended </li></ul><ul><li>NGX_HTTP_PREACCESS_PHASE (limit_req 、 limit_zone 、 degradation 、 realip) </li></ul><ul><li>NGX_HTTP_ACCESS_PHASE (ACCESS 、 auth_basic) </li></ul><ul><li>NGX_HTTP_POST_ACCESS_PHASE Can not be extended </li></ul><ul><li>NGX_HTTP_TRY_FILES_PHASE Can not be extended </li></ul><ul><li>NGX_HTTP_CONTENT_PHASE (Autoindex 、 static 、 random_index 、 gzip_static) </li></ul><ul><li>NGX_HTTP_LOG_PHASE </li></ul>
  19. 19. content phase handler vs content_handler <ul><li>checker: ngx_http_core_content_phase </li></ul><ul><li>Fastcgi & autoindex </li></ul>
  20. 20. Nginx Filter <ul><li>Header Filter </li></ul><ul><li>Content Filter </li></ul><ul><li>ngx_http_send_header </li></ul><ul><li>ngx_http_output_filter </li></ul><ul><li>Example : gzip 、 more_set_header </li></ul>
  21. 21. Location features <ul><li>Exactly match </li></ul><ul><li>Regular(sensitive/ insensitive) </li></ul><ul><li>Location inheritance </li></ul><ul><li>How to find the right location? </li></ul>
  22. 22. Location internals <ul><li>Ngx_http_block loc_conf </li></ul><ul><li>ngx_http_init_locations Location inheritance check </li></ul><ul><li>ngx_http_init_static_location_trees Ternary Search Tree(better than trie) </li></ul><ul><li>Find location(ngx_http_core_find_location) first static location tree then regex locations </li></ul>
  23. 23. Static module <ul><li>Sendfile http://linux.die.net/man/2/sendfile </li></ul><ul><li>NGX_HTTP_CONTENT_PHASE ngx_http_static_handler </li></ul><ul><li>ngx_http_static_handler 1 、 method check. GET/HEAD/POST 2 、 Cache processing 3 、 ngx_http_discard_request_body 4 、 output </li></ul><ul><li>Only Cache fd </li></ul><ul><li>Red black tree </li></ul>
  24. 24. Nginx Upstream <ul><li>Protocol : Http/Fastcgi/Memcache/SCGI </li></ul><ul><li>Load-balance: hash/random/round_robin </li></ul><ul><li>Short connection </li></ul><ul><li>Long connection only for memcached </li></ul>
  25. 25. Nginx Upstream : fastcgi
  26. 26. Extension Develop <ul><li>Core Module Extension, such as nginx-php </li></ul><ul><li>Http Module Extension Handler : echo Filter : gzip, usertrack upstream : fastcgi Load-balancers : rand </li></ul><ul><li>Emiller's Guide To Nginx Module Development </li></ul>
  27. 27. Extension Develop—where? <ul><li>Just before the server reads the config file </li></ul><ul><li>For every configuration directive for the location and server for which it appears; </li></ul><ul><li>When Nginx initializes the main configuration </li></ul><ul><li>When Nginx initializes the server (i.e., host/port) configuration </li></ul><ul><li>When Nginx merges the server configuration with the main configuration </li></ul><ul><li>When Nginx initializes the location configuration </li></ul><ul><li>When Nginx merges the location configuration with its parent server configuration </li></ul><ul><li>When Nginx's master process starts </li></ul><ul><li>When a new worker process starts </li></ul><ul><li>When a worker process exits </li></ul><ul><li>When the master exits </li></ul><ul><li>Handling a request </li></ul><ul><li>Filtering response headers </li></ul><ul><li>Filtering the response body </li></ul><ul><li>Picking a backend server </li></ul><ul><li>Initiating a request to a backend server </li></ul><ul><li>Re-initiating a request to a backend server </li></ul><ul><li>Processing the response from a backend server </li></ul><ul><li>Finishing an interaction with a backend server </li></ul>
  28. 28. Extension Develop—where?
  29. 29. Extension Develop—where?
  30. 30. Extension Develop—where? <ul><li>Http Phase handler </li></ul><ul><li>Content_handler </li></ul><ul><li>Header Filter </li></ul><ul><li>Content Filter </li></ul>
  31. 31. Extension Develop—step <ul><li>“ Config” file </li></ul><ul><li>ngx_addon_name=ngx_http_baidu_usertrack_module </li></ul><ul><li>HTTP_AUX_FILTER_MODULES=&quot;$HTTP_AUX_FILTER_MODULES ngx_http_baidu_usertrack_module&quot; </li></ul><ul><li>NGX_ADDON_SRCS=&quot;$NGX_ADDON_SRCS $ngx_addon_dir/src/ngx_http_baidu_usertrack_module.c $ngx_addon_dir/src/baidu_des.c&quot; </li></ul><ul><li>NGX_ADDON_DEPS=&quot;$NGX_ADDON_DEPS $ngx_addon_dir/src/baidu_des.h&quot; </li></ul><ul><li>ORE_LIBS=&quot;$CORE_LIBS -lcrypto&quot; </li></ul><ul><li>Develop and build ./configure -add-module=path/to/your/new/module/directory Make Make install </li></ul>
  32. 32. Extension Develop—demo <ul><li>Demo </li></ul>
  33. 33. Q&A Q&A Thanks

×