Non Blocking Socket      Hu Zi Ming  hzmangel@gmail.com      201l-08-28                       Non Blocking Socket         ...
Outline1   Normal Server    Function Flow    Simple Stream Server2   The C10K Problem3   High Performance Web Server      ...
Function FlowS: Get available addressS: Create the socket file descriptorS: Bind the descriptor to specified portS: Listen o...
Function FlowS: Get available addressS: Create the socket file descriptorS: Bind the descriptor to specified portS: Listen o...
Function FlowS: Get available addressS: Create the socket file descriptorS: Bind the descriptor to specified portS: Listen o...
Function FlowS: Get available addressS: Create the socket file descriptorS: Bind the descriptor to specified portS: Listen o...
Simple Stream Serverserver.pyimport socketPORT = 8080HTTP_RESP = ’HTTP/1.1 200 OKrn’HTTP_RESP += ’Connection:close1rn’HTTP...
Outline1   Normal Server2   The C10K Problem    The C10K Problem    Programming Model3   High Performance Web Server      ...
Huston, We Have a ProblemAs web developed in high speedMore servers need to handle 10 thousand clients simultaneously     ...
Huston, We Have a ProblemAs web developed in high speedMore servers need to handle 10 thousand clients simultaneouslyWhich...
Huston, We Have a ProblemAs web developed in high speedMore servers need to handle 10 thousand clients simultaneouslyWhich...
Programming ModelOne client with each server thread, synchronous blocking IO.Classical model, used in Apache              ...
Programming ModelOne client with each server thread, synchronous blocking IO.Classical model, used in ApacheMany clients w...
Programming ModelOne client with each server thread, synchronous blocking IO.Classical model, used in ApacheMany clients w...
Programming ModelOne client with each server thread, synchronous blocking IO.Classical model, used in ApacheMany clients w...
Outline1   Normal Server2   The C10K Problem3   High Performance Web Server    Synchronous Blocking IO with Multi Thread/P...
Synchronous Blocking IO w/ Multi           Thread/ProcessEasy for programmingCan be used for complex logicGood for interac...
Synchronous Blocking IO w/ Multi           Thread/ProcessEasy for programmingCan be used for complex logicGood for interac...
Level-triggered (LT) Readiness Notification Triggered by state Notifications will be constantly sent if state not change    ...
Level-triggered (LT) Readiness Notification Triggered by state Notifications will be constantly sent if state not change sel...
select() & poll()Both are working in LT mode                                    Non Blocking Socket                       ...
select() & poll()Both are working in LT modeMax acceptable fd num of select() is FD SETSIZEWhich is defined in the kernelUs...
select() & poll()Both are working in LT modeMax acceptable fd num of select() is FD SETSIZEWhich is defined in the kernelUs...
select() & poll()Both are working in LT modeMax acceptable fd num of select() is FD SETSIZEWhich is defined in the kernelUs...
select() & poll()Both are working in LT modeMax acceptable fd num of select() is FD SETSIZEWhich is defined in the kernelUs...
Edge-triggered (ET) Readiness Notification Triggered by state change Notification will only be sent out when state change   ...
Edge-triggered (ET) Readiness Notification Triggered by state change Notification will only be sent out when state change Re...
epoll()Default mode is LT, can be configured to ET                                             Non Blocking Socket         ...
epoll()Default mode is LT, can be configured to ETMaximum acceptable fd num is the up limit of opened file                  ...
epoll()Default mode is LT, can be configured to ETMaximum acceptable fd num is the up limit of opened fileOnly handle activa...
epoll()Default mode is LT, can be configured to ETMaximum acceptable fd num is the up limit of opened fileOnly handle activa...
Asynchronous Non-Blocking IONot yet become popular in UnixThe programming model is different with classical                ...
Asynchronous Non-Blocking IONot yet become popular in UnixThe programming model is different with classicalAlmost the ONLY ...
AIOOverlapping process with IO                              Non Blocking Socket                                           ...
AIOOverlapping process with IOThe READ request return immediatelyWhich indicate the request has been initiated successfull...
AIOOverlapping process with IOThe READ request return immediatelyWhich indicate the request has been initiated successfull...
AIOOverlapping process with IOThe READ request return immediatelyWhich indicate the request has been initiated successfull...
Q AND A          Non Blocking Socket                                16 / 16
Upcoming SlideShare
Loading in …5
×

20110828 Non blocking socket

1,634 views

Published on

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,634
On SlideShare
0
From Embeds
0
Number of Embeds
150
Actions
Shares
0
Downloads
21
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

20110828 Non blocking socket

  1. 1. Non Blocking Socket Hu Zi Ming hzmangel@gmail.com 201l-08-28 Non Blocking Socket 1 / 16
  2. 2. Outline1 Normal Server Function Flow Simple Stream Server2 The C10K Problem3 High Performance Web Server Non Blocking Socket 2 / 16
  3. 3. Function FlowS: Get available addressS: Create the socket file descriptorS: Bind the descriptor to specified portS: Listen on the port (address) Non Blocking Socket 3 / 16
  4. 4. Function FlowS: Get available addressS: Create the socket file descriptorS: Bind the descriptor to specified portS: Listen on the port (address)C: Create the socketC: Connect to the server Non Blocking Socket 3 / 16
  5. 5. Function FlowS: Get available addressS: Create the socket file descriptorS: Bind the descriptor to specified portS: Listen on the port (address)C: Create the socketC: Connect to the serverS: Accept the connection Non Blocking Socket 3 / 16
  6. 6. Function FlowS: Get available addressS: Create the socket file descriptorS: Bind the descriptor to specified portS: Listen on the port (address)C: Create the socketC: Connect to the serverS: Accept the connectionData exchange Non Blocking Socket 3 / 16
  7. 7. Simple Stream Serverserver.pyimport socketPORT = 8080HTTP_RESP = ’HTTP/1.1 200 OKrn’HTTP_RESP += ’Connection:close1rn’HTTP_RESP += ’Content-Type: text/htmlrn’HTTP_RESP += ’Content-Length: 21rn’HTTP_RESP += ’rn<h1>hello, world</h1>’available_sock = socket.getaddrinfo(None, PORT, 0, socket.SOCK_STREAM)[0]s = socket.socket(available_sock[0], available_sock[1])s.bind((available_sock[3], PORT))print ’Binded at’, available_sock[4][0]s.listen(10)while True: conn, addr = s.accept() print ’Connected by’, addr conn.send(HTTP_RESP) conn.close() Non Blocking Socket 4 / 16
  8. 8. Outline1 Normal Server2 The C10K Problem The C10K Problem Programming Model3 High Performance Web Server Non Blocking Socket 5 / 16
  9. 9. Huston, We Have a ProblemAs web developed in high speedMore servers need to handle 10 thousand clients simultaneously Non Blocking Socket 6 / 16
  10. 10. Huston, We Have a ProblemAs web developed in high speedMore servers need to handle 10 thousand clients simultaneouslyWhich names this problem: C10K Non Blocking Socket 6 / 16
  11. 11. Huston, We Have a ProblemAs web developed in high speedMore servers need to handle 10 thousand clients simultaneouslyWhich names this problem: C10KClassical server can not handle this scenario wellHere comes the new models Non Blocking Socket 6 / 16
  12. 12. Programming ModelOne client with each server thread, synchronous blocking IO.Classical model, used in Apache Non Blocking Socket 7 / 16
  13. 13. Programming ModelOne client with each server thread, synchronous blocking IO.Classical model, used in ApacheMany clients with each threadAsynchronous blocking IO and level-triggered readinessnotification Non Blocking Socket 7 / 16
  14. 14. Programming ModelOne client with each server thread, synchronous blocking IO.Classical model, used in ApacheMany clients with each threadAsynchronous blocking IO and level-triggered readinessnotificationMany clients with each threadAsynchronous blocking IO and edge-triggered readinessnotification Non Blocking Socket 7 / 16
  15. 15. Programming ModelOne client with each server thread, synchronous blocking IO.Classical model, used in ApacheMany clients with each threadAsynchronous blocking IO and level-triggered readinessnotificationMany clients with each threadAsynchronous blocking IO and edge-triggered readinessnotificationMany clients with each threadAsynchronous non-blocking IO. Non Blocking Socket 7 / 16
  16. 16. Outline1 Normal Server2 The C10K Problem3 High Performance Web Server Synchronous Blocking IO with Multi Thread/Process Level-triggered (LT) Readiness Notification Edge-triggered (ET) Readiness Notification Asynchronous Non-Blocking IO Non Blocking Socket 8 / 16
  17. 17. Synchronous Blocking IO w/ Multi Thread/ProcessEasy for programmingCan be used for complex logicGood for interactive and persistent connection Non Blocking Socket 9 / 16
  18. 18. Synchronous Blocking IO w/ Multi Thread/ProcessEasy for programmingCan be used for complex logicGood for interactive and persistent connectionConsume too much memory Non Blocking Socket 9 / 16
  19. 19. Level-triggered (LT) Readiness Notification Triggered by state Notifications will be constantly sent if state not change Non Blocking Socket 10 / 16
  20. 20. Level-triggered (LT) Readiness Notification Triggered by state Notifications will be constantly sent if state not change select(2) and poll(2) are working in this mode epoll(4) can be configured to work in this mode Support both blocking and non-blocking socket Non Blocking Socket 10 / 16
  21. 21. select() & poll()Both are working in LT mode Non Blocking Socket 11 / 16
  22. 22. select() & poll()Both are working in LT modeMax acceptable fd num of select() is FD SETSIZEWhich is defined in the kernelUser defined array is used in poll() to break this limitation Non Blocking Socket 11 / 16
  23. 23. select() & poll()Both are working in LT modeMax acceptable fd num of select() is FD SETSIZEWhich is defined in the kernelUser defined array is used in poll() to break this limitationThe whole fd set will be scanned, and. . . Non Blocking Socket 11 / 16
  24. 24. select() & poll()Both are working in LT modeMax acceptable fd num of select() is FD SETSIZEWhich is defined in the kernelUser defined array is used in poll() to break this limitationThe whole fd set will be scanned, and. . .The whole fd set will be copied between user and kernel mode Non Blocking Socket 11 / 16
  25. 25. select() & poll()Both are working in LT modeMax acceptable fd num of select() is FD SETSIZEWhich is defined in the kernelUser defined array is used in poll() to break this limitationThe whole fd set will be scanned, and. . .The whole fd set will be copied between user and kernel modeSo, the performance will be reduced as set increased Non Blocking Socket 11 / 16
  26. 26. Edge-triggered (ET) Readiness Notification Triggered by state change Notification will only be sent out when state change Non Blocking Socket 12 / 16
  27. 27. Edge-triggered (ET) Readiness Notification Triggered by state change Notification will only be sent out when state change Recommended mode for epoll(4) Support only non-blocking socket Non Blocking Socket 12 / 16
  28. 28. epoll()Default mode is LT, can be configured to ET Non Blocking Socket 13 / 16
  29. 29. epoll()Default mode is LT, can be configured to ETMaximum acceptable fd num is the up limit of opened file Non Blocking Socket 13 / 16
  30. 30. epoll()Default mode is LT, can be configured to ETMaximum acceptable fd num is the up limit of opened fileOnly handle activate connectionsSo performance is better in normal scenario Non Blocking Socket 13 / 16
  31. 31. epoll()Default mode is LT, can be configured to ETMaximum acceptable fd num is the up limit of opened fileOnly handle activate connectionsSo performance is better in normal scenarioReturn fd id while trigged by eventSo no need to traverse the whole fd setShared memory is used to avoid copying Non Blocking Socket 13 / 16
  32. 32. Asynchronous Non-Blocking IONot yet become popular in UnixThe programming model is different with classical Non Blocking Socket 14 / 16
  33. 33. Asynchronous Non-Blocking IONot yet become popular in UnixThe programming model is different with classicalAlmost the ONLY and BEST solution on Windowz machineBut only file system IO is supported on Linux (?) Non Blocking Socket 14 / 16
  34. 34. AIOOverlapping process with IO Non Blocking Socket 15 / 16
  35. 35. AIOOverlapping process with IOThe READ request return immediatelyWhich indicate the request has been initiated successfully Non Blocking Socket 15 / 16
  36. 36. AIOOverlapping process with IOThe READ request return immediatelyWhich indicate the request has been initiated successfullyThe operation will be performed in background Non Blocking Socket 15 / 16
  37. 37. AIOOverlapping process with IOThe READ request return immediatelyWhich indicate the request has been initiated successfullyThe operation will be performed in backgroundAfter READ response readyA signal or thread-based callback will be generated Non Blocking Socket 15 / 16
  38. 38. Q AND A Non Blocking Socket 16 / 16

×