Nginx Internals

   Joshua Zhu
   09/19/2009
Agenda
   Source code layout
   Key concepts and infrastructure
   The event-driven architecture
   HTTP request handl...
Source Code Layout
   Files
       $ find . -name "*.[hc]" -print | wc –l
        234
       $ ls src
        core even...
Code Organization
   core/
         The backbone and infrastructure
   event/
         The event-driven engine and mod...
Nginx Architecture
   Non-blocking
   Event driven
   Single threaded[*]
   One master process and several worker
    ...
The Big Picture
Agenda
   Source code layout
   Key concepts and infrastructure
   The event-driven architecture
   HTTP request handl...
Memory Pool
   Avoid memory fragmentation
   Avoid memory leak
   Allocation and deallocation can be very
    fast
   ...
Memory Pool (cont’d)
   ngx_pool_t
       Small blocks
       Large blocks
       Free chain list
       Cleanup hand...
Memory Pool Example (1 Chunk)
Memory Pool Example (2 Chunks)
Buffer Management
   Buffer
        Pointers
             memory
                    start/pos/last/end
             ...
Buffer Management (cont’d)
   Buffer chain
       Singly-linked list of buffers
   Output chain
       Context
       ...
String Utilities
   ngx_str_t
        data
        len
        sizeof() - 1
   Memory related
   String formatting
...
Data Structures
   Abstract data types
       Array
       List
       Queue
       Hash table
       Red black tree...
Logging
   Error log
       Level
       Debug
   Access log
       Multiple logs
       Log format
            var...
Configuration File
   Directive
        name
        type
        set
        conf
        offset
        post
   ...
Configuration File (cont’d)
   Block
        events
        http
        server
        upstream
        location
  ...
Agenda
   Source code layout
   Key concepts and infrastructure
   The event-driven architecture
   HTTP request handl...
Master and Workers
   Master
       Monitor workers, respawn when a worker dies
       Handle signals and notify worker...
Master Process Cycle
Worker Process Cycle
Inter-process Communication
   Signals
       Channel
            socketpair
            command
   Shared memory
   ...
Event
   ngx_event_t
       Read
       Write
       Timeout
   Callbacks
   Handlers
       ngx_event_accept
    ...
Time Cache
   The overhead of gettimeofday()
   Time cache variables
       ngx_cached_time
       ngx_current_msec
  ...
Events and Timers Processing
Timer Management
   Actions
       Add a timer
       Delete a timer
       Get the minimum timer
   Red black tree[*...
Accept Mutex
   Thundering herd
   Serialize accept()
   Lock/unlock
   Listening sockets
   Delay
I/O
   Multiplexing
        kqueue/epoll
              NGX_USE_CLEAR_EVENT (edge triggered)
        select/poll/dev/po...
Agenda
   Source code layout
   Key concepts and infrastructure
   The event-driven architecture
   HTTP request handl...
Important Structures
   Connection
       ngx_connection_t
   HTTP connection
       ngx_http_connection_t
   HTTP re...
Virtual Servers
   Address
   Port
   Server names
   Core server conf
Locations
   Location tree
       Static
       Regex
            = ^~ ~ ~*
   Per-location configuration
       Val...
HTTP Contexts
   Types
        main_conf
        srv_conf
        loc_conf
   Request
        ngx_http_get_module_ma...
HTTP Handling
   Receive data
   Parse the request
   Find the virtual server
   Find the location
   Run phase handl...
Request Parsing
   Request line
   Headers
   Interesting tricks
       Finite state machine
       ngx_strX_cmp
Phases and Handlers
   Phases
        POST_READ
        SERVER_REWRITE
        FIND_CONFIG
        REWRITE
        P...
Phases and Handlers (cont’d)
   Phase engine
        Handlers
        server_rewrite_index
        location_rewrite_in...
Phases and Handlers (cont’d)
phase            modules
POST_READ        realip
SERVER_REWRITE   rewrite
REWRITE          re...
Filter Chain
   Singly-linked list like (CoR)
   Filter response only
        Header filter
        Body filter
   Se...
Filter Chain Example
HTTP Handling Example
   curl -i http://localhost/
HTTP Keep-Alive
   Request memory reuse
   Connection memory shrink
   Keep-alive timeout
   Request count
Subrequest
   Filters
       Addition filter
       SSI filter
   Maximum subrequests
Internal Redirect
   Return a different URL than originally
    requested
   Examples
       try_files
       index/ra...
Upstream
   Hooks
        input_filter_init
        input_filter
        create_request
        reinit_request
     ...
Agenda
   Source code layout
   Key concepts and infrastructure
   The event-driven architecture
   HTTP request handl...
Mail Proxy
   Sequence diagram
Mail Proxy (cont’d)
   Mail session
       Command parsing
       Packets relay
   Things you can do
       Load bala...
Agenda
   Source code layout
   Key concepts and infrastructure
   The event-driven architecture
   HTTP request handl...
General Module Interface
   Context
        index & ctx_index
   Directives
   Type
        core/event/http/mail
   ...
Core Module Interface
   Name
   Hooks
       create_conf
       init_conf
   Examples
       Core
       Events
  ...
Event Module Interface
   Name
   Hooks
       create_conf
       init_conf
       event_actions
            add
   ...
Mail Module Interface
   Protocol
       type
       init_session
       init_protocol
       parse_command
       a...
HTTP Module Interface
   Hooks
       preconfiguration
       postconfiguration
       create_main_conf
       init_m...
A “Hello World” HTTP Module
• Creating a hello world! module
      Files
           ngx_http_hello_module.c
           ...
Agenda
   Source code layout
   Key concepts and infrastructure
   The event-driven architecture
   HTTP request handl...
Auto Scripts
   Handle the differences
       OS
       Compiler
       Data types
       Libraries
   Module enable...
Reconfiguration
Hot Code Swapping
Thank You!
   My site: http://www.zhuzhaoyuan.com
   My blog: http://blog.zhuzhaoyuan.com
Upcoming SlideShare
Loading in...5
×

Nginx Internals

95,473
-1

Published on

Nginx Internals

Published in: Technology, Education
9 Comments
171 Likes
Statistics
Notes
No Downloads
Views
Total Views
95,473
On Slideshare
0
From Embeds
0
Number of Embeds
24
Actions
Shares
0
Downloads
2,447
Comments
9
Likes
171
Embeds 0
No embeds

No notes for slide

Nginx Internals

  1. 1. Nginx Internals Joshua Zhu 09/19/2009
  2. 2. Agenda  Source code layout  Key concepts and infrastructure  The event-driven architecture  HTTP request handling  Mail proxying process  Nginx module development  Misc. topics
  3. 3. Source Code Layout  Files  $ find . -name "*.[hc]" -print | wc –l 234  $ ls src core event http mail misc os  Lines of code  $ find . -name "*.[hc]" -print | xargs wc -l | tail -n1 110953 total
  4. 4. Code Organization  core/  The backbone and infrastructure  event/  The event-driven engine and modules  http/  The HTTP server and modules  mail/  The Mail proxy server and modules  misc/  C++ compatibility test and the Google perftools module  os/  OS dependent implementation files
  5. 5. Nginx Architecture  Non-blocking  Event driven  Single threaded[*]  One master process and several worker processes  Resource efficient  Highly modular
  6. 6. The Big Picture
  7. 7. Agenda  Source code layout  Key concepts and infrastructure  The event-driven architecture  HTTP request handling  Mail proxying process  Nginx module development  Misc. topics
  8. 8. Memory Pool  Avoid memory fragmentation  Avoid memory leak  Allocation and deallocation can be very fast  Lifetime and pool size  Cycle  Connection  Request
  9. 9. Memory Pool (cont’d)  ngx_pool_t  Small blocks  Large blocks  Free chain list  Cleanup handler list  API  ngx_palloc  memory aligned  ngx_pnalloc  ngx_pcalloc
  10. 10. Memory Pool Example (1 Chunk)
  11. 11. Memory Pool Example (2 Chunks)
  12. 12. Buffer Management  Buffer  Pointers  memory  start/pos/last/end  file  file_pos/file_last/file  Flags  last_buf  last_in_chain  flush  in_file  memory  …
  13. 13. Buffer Management (cont’d)  Buffer chain  Singly-linked list of buffers  Output chain  Context  in/free/busy chains  Output filter  Chain writer  Writer context
  14. 14. String Utilities  ngx_str_t  data  len  sizeof() - 1  Memory related  String formatting  String comparison  String search  Base64 encoding/decoding  URI escaping/unescaping  UTF-8 decoding  String-to-number conversion
  15. 15. Data Structures  Abstract data types  Array  List  Queue  Hash table  Red black tree  Radix tree  Characteristic  Set object values after added  keep interfaces clean  Chunked memory (part)  efficient
  16. 16. Logging  Error log  Level  Debug  Access log  Multiple logs  Log format  variables  Per location  Rotation
  17. 17. Configuration File  Directive  name  type  set  conf  offset  post  Parsing  ngx_conf_parse  Values  init  merge
  18. 18. Configuration File (cont’d)  Block  events  http  server  upstream  location  if  Variables  Buildins  Other types  http_  sent_http_  upstream_http_  cookie_  arg_
  19. 19. Agenda  Source code layout  Key concepts and infrastructure  The event-driven architecture  HTTP request handling  Mail proxying process  Nginx module development  Misc. topics
  20. 20. Master and Workers  Master  Monitor workers, respawn when a worker dies  Handle signals and notify workers  exit  reconfiguration  update  log rotation  …  Worker  Process client requests  handle connections  Get cmd from master
  21. 21. Master Process Cycle
  22. 22. Worker Process Cycle
  23. 23. Inter-process Communication  Signals  Channel  socketpair  command  Shared memory  Connection counter  Stat  Atomic & spinlock  Mutex
  24. 24. Event  ngx_event_t  Read  Write  Timeout  Callbacks  Handlers  ngx_event_accept  ngx_process_events_and_timers  ngx_handle_read_event  ngx_handle_write_event  Posted events  Posted accept events queue  Posted events queue
  25. 25. Time Cache  The overhead of gettimeofday()  Time cache variables  ngx_cached_time  ngx_current_msec  Time strings  ngx_cached_err_log_time  ngx_cached_http_time  ngx_cached_http_log_time  Timer resolution  Interval timer  setitimer()
  26. 26. Events and Timers Processing
  27. 27. Timer Management  Actions  Add a timer  Delete a timer  Get the minimum timer  Red black tree[*]  O(log n) complexity
  28. 28. Accept Mutex  Thundering herd  Serialize accept()  Lock/unlock  Listening sockets  Delay
  29. 29. I/O  Multiplexing  kqueue/epoll  NGX_USE_CLEAR_EVENT (edge triggered)  select/poll/dev/poll  NGX_USE_LEVEL_EVENT (level triggered)  …  Advanced I/O  sendfile()  writev()  direct I/O  mmap()  AIO  TCP/IP options  TCP_CORK/TCP_NODELAY/TCP_DEFER_ACCEPT
  30. 30. Agenda  Source code layout  Key concepts and infrastructure  The event-driven architecture  HTTP request handling  Mail proxying process  Nginx module development  Misc. topics
  31. 31. Important Structures  Connection  ngx_connection_t  HTTP connection  ngx_http_connection_t  HTTP request  ngx_http_request_t  headers_in  headers_out  …
  32. 32. Virtual Servers  Address  Port  Server names  Core server conf
  33. 33. Locations  Location tree  Static  Regex  = ^~ ~ ~*  Per-location configuration  Value  inheritance  override  Handler  Named location  try_files/post_action/error_page
  34. 34. HTTP Contexts  Types  main_conf  srv_conf  loc_conf  Request  ngx_http_get_module_main_conf  ngx_http_get_module_srv_conf  ngx_http_get_module_loc_conf  Parse conf file  ngx_http_conf_get_module_main_conf  ngx_http_conf_get_module_srv_conf  ngx_http_conf_get_module_loc_conf  Module context  ngx_http_get_module_ctx  ngx_http_set_ctx
  35. 35. HTTP Handling  Receive data  Parse the request  Find the virtual server  Find the location  Run phase handlers  Generate the response  Filter response headers  Filter the response body  Send out the output to the client
  36. 36. Request Parsing  Request line  Headers  Interesting tricks  Finite state machine  ngx_strX_cmp
  37. 37. Phases and Handlers  Phases  POST_READ  SERVER_REWRITE  FIND_CONFIG  REWRITE  POST_REWRITE  PREACCESS  ACCESS  POST_ACCESS  TRY_FILES  CONTENT  LOG  Phase handler  Checker  Handler  Next
  38. 38. Phases and Handlers (cont’d)  Phase engine  Handlers  server_rewrite_index  location_rewrite_index  r->phase_handler  Default checkers  ngx_http_core_generic_phase  ngx_http_core_find_config_phase  ngx_http_core_post_rewrite_phase  ngx_http_core_access_phase  ngx_http_core_post_access_phase  ngx_http_core_try_files_phase  ngx_http_core_content_phase
  39. 39. Phases and Handlers (cont’d) phase modules POST_READ realip SERVER_REWRITE rewrite REWRITE rewrite PREACCESS limit_req, limit_zone, realip ACCESS access, auth_basic CONTENT autoindex, dav, gzip, index, random_index, static LOG log
  40. 40. Filter Chain  Singly-linked list like (CoR)  Filter response only  Header filter  Body filter  Send out the response  ngx_http_send_header  top_header_filter  ngx_http_output_filter  ngx_http_top_body_filter  ngx_http_header_filter  ngx_http_copy_filter  ngx_http_write_filter  Process order
  41. 41. Filter Chain Example
  42. 42. HTTP Handling Example  curl -i http://localhost/
  43. 43. HTTP Keep-Alive  Request memory reuse  Connection memory shrink  Keep-alive timeout  Request count
  44. 44. Subrequest  Filters  Addition filter  SSI filter  Maximum subrequests
  45. 45. Internal Redirect  Return a different URL than originally requested  Examples  try_files  index/random_index  post_action  send_error_page  upstream_process_headers
  46. 46. Upstream  Hooks  input_filter_init  input_filter  create_request  reinit_request  process_header  abort_request  finalize_request  rewrite_redirect  Modules  FastCGI  Proxy  Memcached  Event pipe  Load balancer
  47. 47. Agenda  Source code layout  Key concepts and infrastructure  The event-driven architecture  HTTP request handling  Mail proxying process  Nginx module development  Misc. topics
  48. 48. Mail Proxy  Sequence diagram
  49. 49. Mail Proxy (cont’d)  Mail session  Command parsing  Packets relay  Things you can do  Load balancing  Authentication rewriting  Black lists/white lists
  50. 50. Agenda  Source code layout  Key concepts and infrastructure  The event-driven architecture  HTTP request handling  Mail proxying process  Nginx module development  Misc. topics
  51. 51. General Module Interface  Context  index & ctx_index  Directives  Type  core/event/http/mail  Hooks  init_master  called at master process initialization  init_module  called when the module is loaded  init_process  called at worker process initialization  exit_process  called at worker process termination  exit_master  called at master process termination
  52. 52. Core Module Interface  Name  Hooks  create_conf  init_conf  Examples  Core  Events  Log  HTTP
  53. 53. Event Module Interface  Name  Hooks  create_conf  init_conf  event_actions  add  del  enable  disable  add_conn  del_conn  process_changes  process_events  init  done
  54. 54. Mail Module Interface  Protocol  type  init_session  init_protocol  parse_command  auth_state  create_main_conf  init_main_conf  create_srv_conf  merge_srv_conf
  55. 55. HTTP Module Interface  Hooks  preconfiguration  postconfiguration  create_main_conf  init_main_conf  create_srv_conf  merge_srv_conf  create_loc_conf  merge_loc_conf
  56. 56. A “Hello World” HTTP Module • Creating a hello world! module  Files  ngx_http_hello_module.c  config  Build  ./configure –add-module=/path/to/hello/module  Configuration  location & directive
  57. 57. Agenda  Source code layout  Key concepts and infrastructure  The event-driven architecture  HTTP request handling  Mail proxying process  Nginx module development  Misc. topics
  58. 58. Auto Scripts  Handle the differences  OS  Compiler  Data types  Libraries  Module enable/disable  Modules order
  59. 59. Reconfiguration
  60. 60. Hot Code Swapping
  61. 61. Thank You!  My site: http://www.zhuzhaoyuan.com  My blog: http://blog.zhuzhaoyuan.com
  1. Gostou de algum slide específico?

    Recortar slides é uma maneira fácil de colecionar informações para acessar mais tarde.

×