Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tornado in Depth

Tornado is a non-blocking light-weight web server and framework. There's been many introductory talks about it, and it's time to look deeper into it: not just what Tornado does, but how it does it and what can we learn from it when designing our own concurrent systems.

In this talk I go over the following topics. I cover them in two parts: first I present how to use a certain feature or approach in our applications; then, I dig into Tornado's source code to see how it really works.

- Getting Started: quickly get a simple Tornado application up and running. We'll keep digging into, changing and poking this Application for most of the talk.

- An Application Listens: what an Application is, how does Tornado start it and how does it process its requests.

- Application and IOLoop: we'll look at how the IOLoop receives the connections from the users and passes them on to the Applications.

- Scheduled Tasks: we'll see how to schedule tasks and how the IOLoop will run them.

- Generators: we'll learn to use generators to handle the responses of our asynchronous calls, and how they work with the IOLoop.


- Websockets: how to use them and how they work.

- IOStream: how do Tornado's non-blocking sockets work.

- Database: how to use non-blocking sockets to connect to databases.

- Process: how Tornado works with multiple processes.

I presented this talk at Europython 2012 and PyGrunn 2012.

Code examples:

Tornado in Depth

  2. 2. thor
  3. 3. thunder
  4. 4. tro
  5. 5. tronar
  6. 6. tronada
  7. 7. tornar
  8. 8. tornada
  9. 9. tornada
  10. 10. tronada + tornada
  11. 11. TORNADO
  12. 12. what is it? Non-blocking web server Based on epoll / kqueue Handles 1000s of connections Powers FriendFeed
  13. 13. why do you even care? Know what you use I can't trust what I don't know Learning and sharing: fun!
  14. 14. what do you use it for? We built a Scalable Distributed Fault tolerant High load ...thing
  16. 16.
  17. 17. TORNADO
  18. 18. what is it made of? IOLoop Callbacks, Tasks, Timeouts TCPServer Application RequestHandler Generators
  19. 19. can I extend it? Sure! It's easy How would we make a simple TCPServer?
  20. 20. show me the source!
  22. 22. how do I make the simplest app? Define a RequestHandler Implement get Define an Application Tell the application to listen Start the IOLoop
  23. 23. show me the source!
  24. 24. what is an Application? Collection of RequestHandlers Implements listen: Starts an HTTPServer Sets the Application as request callback Implements __call__: Handles the user requests
  25. 25. show me the source!
  26. 26. what does Application.__call__ do? Parses the URL Decides which handler to use and creates an instance of it (each connection gets one) _executes it (passing any defined transforms to it –e.g, Gzip compression)
  27. 27. show me the source!
  28. 28. what does RequestHandler._execute do? Calls the handler method Checks XSRF cookie Maybe closes the connection
  29. 29. t3_request_handler_ show me the source!
  31. 31. Core of Tornado Usable standalone or with WSGI Used for server and client Single instance per process Small! what is the IOLoop?
  32. 32. Loops forever Executes: Callbacks (asap) Timeouts (when due) Events (when occoured) what does the IOLoop do?
  33. 33. what is an Event? Something that happened on a socket (fd) (e.g. a user opened a conneciton) Applications define handlers for Events
  34. 34. how do I wait for an Event? add_handler(fd, handler, events) update_handler(fd, handler, events) remove_handler(fd, handler) “Notify me when I can READ or WRITE, or when there is an ERROR”
  35. 35. IOLOOP THE LOOP in four steps
  36. 36. Process Callbacks They were scheduled by previous: Callbacks Event handlers first process the Callbacks
  37. 37. For each Timeout: Is it due now? Run its callback Calculate the time till next Timeout second process the Timeouts
  38. 38. Poll timeout: Callbacks? Then 0 Timeouts? Then time until the next Timeout Neither? 1 hour Here the IOLoop blocks third poll for Events
  39. 39. For each (file descriptor, Event): Call its handler ...and repeat the loop and fourth process the Events
  40. 40. THE EXAMPLE WHAT it really does
  41. 41. show me the source!
  42. 42. Application.listen starts HTTPServer calls IOLoop.add_accept_handler (Application.__call__ will handle the ACCEPT event) and when a client connects... what does the example do?
  43. 43. what does the example do? IOLoop polls connection TCPServer calls HTTPServer which is HTTPConnection creates read and callsheaders body reads using callbacks request_callback and calls Application.__call__ which is RequestHandler ._execute calls (after parsing URL) RequestHandler calls get or post or ... calls HTTPConnection writes to (in chunks) _auto_finish? finishes yes
  44. 44. show me the source!
  45. 45. Scheduled tasks
  46. 46. how do we schedule a task? Call me back asap Ioloop.add_callback Call me back later Ioloop.add_timeout Call me often PeriodicCallback
  47. 47. show me the source!
  48. 48. how does add_callback work? Adds the callback to the list of callbacks. (and wraps it in a threadlocal-like stack context)
  49. 49. what do you mean, threadlocal-like? StackContext Keeps track of the socket connection Handles association between socket and Application classes
  50. 50. show me the source!
  51. 51. how does add_timeout work? Pushes the timeout to the heap of timeouts. (and wraps it in a threadlocal-like stack context too)
  52. 52. show me the source!
  53. 53. how do PeriodicCallbacks work? start Schedules the next timeout to call _run Marks the PeriodicCallback as running stop Removes the next timeout Marks the PeriodicCallback as stopped _run Calls the callback (unless stop was called)
  54. 54. show me the source!
  55. 55. what about Callback? Indeed.
  56. 56. Async handlers asynchronous@ & auto_ _finish
  57. 57. how does @asynchronous work? Sets _auto_finish to False (and does some Exception wrapping) The connection remains open after get, post... Close it yourself (whenever you want)
  58. 58. show me the source!
  59. 59. I put a callback on your callback Nested callbacks make ugly code.
  60. 60. what about Callback? Indeed.
  61. 61. generators
  62. 62. how do I avoid callbacks? Use Callback (finally!) and yield Or Task
  63. 63. show me the source!
  64. 64. Yield points
  65. 65. what is a YieldPoint? Something you yield Then stuff happens
  66. 66. what is a YieldPoint? Callback Sends a result for a key Wait Waits till a result for a key arrives WaitMany Same as Wait, for many keys
  67. 67. Task Wait + Callback (with an auto-generated key) Multi List of YeldPoints what is a YieldPoint?
  68. 68. how do we do async processing? callback=(yield Callback(“key”)) When a result arrives, send it for the key “key”
  69. 69. how do we do async processing? response=yield Wait(“key”) When the result is sent, read it into response.
  70. 70. show me the source!
  71. 71. show me the source!
  72. 72. show me the source!
  73. 73. websockets
  74. 74. how do we use websockets? Extend WebSocketHandler (instead of RequestHandler) Implement on_message
  75. 75. show me the source!
  76. 76. how do websockets work? Similar to @asynchronous (the connection is kept open) After writing, read for more (asynchronously—see IOStream) To Application, it looks like a RequestHandler
  77. 77. how does WebSocket work? _execute Accepts the connection Decides the version of the protocol. Instantiates a WebSocketProtocol class
  78. 78. how does WebSocketProtocol work? accept_connection Sends the required initial message Reads the next message (asynchronously) _write_response Writes a response on the socket Reads the next message (asynchronously)
  79. 79. show me the source!
  80. 80. iostream
  81. 81. what does IOStream do? Communicates with the socket Asynchronous Uses IOLoop Callbacks
  82. 82. how does IOStream work? _add_io_state “Notify me when I can READ or WRITE, or when ERROR” schedules Callback for an event (READ, WRITE, ...) _handle_events Can I read? Call _handle_read Can I write? Call _handle_write Handles errors
  83. 83. how does IOStream work? _handle_read Store data in read buffer Call the read callback (_read_from_buffer) _handle_write Write data from the buffer into the socket (handling funny circumstances) Call the write callback (if it exists)
  84. 84. how does IOStream work? connect socket.connect _add_io_state(WRITE) read Set callback Data in read buffer? Return it Read buffer empty? _add_io_state(READ) write Add data to write buffer _add_io_state(WRITE)
  85. 85. how do I use IOStream directly? read_until_regex read_until read_bytes read_until_close All take a callback
  86. 86. how do I use streaming callbacks? read_bytes read_until_close Param: streaming_callback Data is sent to callback as it arrives
  87. 87. show me the source!
  88. 88. database
  89. 89. how do I talk to a database? database.connection query(“SQL”) Returns an iterable Very simple
  90. 90. show me the source!
  91. 91. Not async!
  92. 92. asyncmongo
  93. 93. what is asyncmongo? Asynchronous MongoDB client Uses Tornado's IOLoop
  94. 94. how do I use asyncmongo? asyncmongo.Client db.find takes a callback parameter
  95. 95. show me the source!
  96. 96. how does asyncmongo work? Implements a Connection Many: ConnectionPool Sends data via Tornado's IOStream Commands send via Cursor Cursor.send_message Uses Connection to communicate asynchronously
  97. 97. can we write our own asyncmysql? Difficult C driver has blocking calls mysql_query mysql_store_result mysql_use_result Alternative Use Twisted's txMySQL with tornado.platform.twisted Slower
  98. 98. Q A&
  99. 99. GRAZIE! source: uncertain (sorry!)