Successfully reported this slideshow.

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Async programming and python

  1. 1. Async Programming and Python PyCon India, 2014 Chetan Giridhar
  2. 2. Basics • Programming tasks: o I/O bound o CPU bound • Say, you’re doing I/O o Will it complete immediately? When will it be done? o Wont they block you?
  3. 3. Blocking I/O: Example import requests r = requests.get(‘http://google.co.in’) r.status_code • What if the request takes a long time? • Operation blocks until all the data is recieved from the server Can we do something in the meanwhile? Can we run another task, concurrently?
  4. 4. Non Blocking / Async • Non-blocking means the ability to make continuous progress at all times • Resources needed for a response must not be monopolized • As such it can enable both lower latency, higher throughput
  5. 5. Programming Models Synchronous model time time time Threaded model Asynchronous model Task 1 Task 2 Task 3
  6. 6. Math • Task = make a call to http://ip.jsontest.com • Say, Task = Task1 = Task2 = Task 3 = 400ms • Sync Model o Time taken = Task1+ Task2 + Task3 = 1.2 sec • Threaded Model o Time taken = 510 ms • Async Model o Time taken = 460 ms What’s the magic here?
  7. 7. Async Paradigm • Clients requests the event driven web server; • requests are processed by event loop; • event handlers cater to events with callbacks Client Event driven server I/O loop Event driven I/O loop Request IO loop handles request Event Handlers
  8. 8. Reactor Pattern • Typical non blocking frameworks work on a philosophy of single threaded event loop o keeps polling for events Reactor Pattern Waiting for Events Handling Events
  9. 9. More Details! • Framework typically maintains a list of file descriptors(fd), events to monitor and corresponding event handlers for each of the fd • Listening to events on a fd is a kernel space task o epoll, [kqueue/select] – libraries provide event notifications in a non-blocking way • Epoll watches file descriptors (sockets) and returns needed (READ, WRITE & ERROR) events
  10. 10. Async way • Async strategy aims for: o Making I/O tasks non blocking o I/O tasks would run independently o generate an event when tasks are complete o with help of callbacks • Benefits o No need to wait till blocking I/O tasks are complete o More responsive real time applications o Thread safety isn't an issue • Can we solve any other Python problems with this mechanism? o Eliminating GIL?
  11. 11. Async in Python • Frameworks o Tornado o Twisted o Gevent • Modules o Tulip o Asyncio
  12. 12. Async in Python • Frameworks o Tornado o Twisted o Gevent • Modules o Tulip o Asyncio
  13. 13. Asyncio • Part of Python library o The latest module for async application development • Only for Python > 3.4 o Incompatible with prior versions • A whole new way to development o Let’s you write self contained, synchronous looking tasks o Run two infinite loops at the same time on the same thread • Works with other framework o Tornado, Twisted, GEvent
  14. 14. Asyncio… • Write single threaded concurrent code • Principle of Interleaved execution of subroutines • Co-operative scheduling o Only one task at a time • Based on libevent o Select, kpoll, kqueue
  15. 15. Asyncio: Components Event loop Co-routines, Futures, Tasks Transports, Protocols
  16. 16. Asyncio: Components • Event loop o Register, executing and cancelling calls o Schedule execution of a task (co-routine) o Creates transport (async client and server) o Runs I/O callbacks (Watches file descriptors) o Thread interface o [BaseEventLoop.create_task()] or async() o [asyncio.get_event_loop()]
  17. 17. Asyncio: Components • Co-routine o Generator (“yield from”) o suspended at preset execution points, and o resumed later by keeping track of local state o @coroutine decorator
  18. 18. Asyncio: Components • Task o responsible for executing a coroutine o If coroutine yields from a future, the task suspends the execution of the coroutine and waits for the future o coroutine restarts when future is done o Subclass of class Future o [async(coroutine)] o BaseEventLoop.create_task(coro)
  19. 19. Asyncio: Components • Future o A class o for results that are available later import asyncio @asyncio.coroutine def slow_operation(future): yield from asyncio.sleep(1) <- Co-routine suspend future.set_result('Future is done!') def got_result(future): print(future.result()) loop.stop() loop = asyncio.get_event_loop() <- Event loop future = asyncio.Future() <- Future object asyncio.async(slow_operation(future)) <- Task future.add_done_callback(got_result) try: loop.run_forever() finally: loop.close()
  20. 20. Asyncio: Components • transport o represent connections such as sockets, SSL connection and pipes o Async socket operations • Usually frameworks implement e.g. Tornado • protocols o represent applications such as HTTP client/server, SMTP, and FTP o Async http operation o [loop.create_connection()]
  21. 21. Example: Asyncio Redis import asyncio import asyncio_redis @asyncio.coroutine def my_subscriber(channels): connection = yield from asyncio_redis.Connection.create(host='localhost', port=6379) subscriber = yield from connection.start_subscribe() yield from subscriber.subscribe(channels) while True: reply = yield from subscriber.next_published() print('Received: ', repr(reply.value), 'on channel', reply.channel) loop = asyncio.get_event_loop() asyncio.async(my_subscriber('channel-1')) asyncio.async(my_subscriber('channel-2')) loop.run_forever()
  22. 22. Example: Asyncio ‘Tasks’ import asyncio @asyncio.coroutine def factorial(name, number): f = 1 for i in range(2, number+1): print("Task %s: Compute factorial(%s)..." % (name, i)) yield from asyncio.sleep(1) f *= i print("Task %s: factorial(%s) = %s" % (name, number, f)) loop = asyncio.get_event_loop() tasks = [ asyncio.async(factorial("A", 2)), asyncio.async(factorial("B", 3)), asyncio.async(factorial("C", 4))] loop.run_until_complete(asyncio.wait(tasks)) loop.close()
  23. 23. Async in Python • Frameworks o Tornado o Twisted o Gevent • Modules o Tulip o Asyncio
  24. 24. Tornado Async • Event Loop => tornado.ioloop • Coroutine => tornado.gen.coroutine • Future => tornado.concurrent.future • Transport/Protocol => tornado.iostream • Bridge the gap => tornado.platform.asyncio – Combines asyncio and tornado in same event loop
  25. 25. Tornado Async Http import tornado.ioloop from tornado.httpclient import AsyncHTTPClient def handle_request(response): '''callback needed when a response arrive''' if response.error: print("Error:", response.error) else: print(’Success') print(response.body) Before Event Loop Starts! Success b'{"ip": "117.192.252.80"}n' Callback http_client = AsyncHTTPClient() # initialize http client http_client.fetch(” http://ip.jsontest.com/", handle_request) print("Before Event Loop Starts!") tornado.ioloop.IOLoop.instance().start() # start the tornado ioloop
  26. 26. Tornado Coroutine import tornado.web import tornado.gen from tornado.httpclient import AsyncHTTPClient class GenAsyncHandler(tornado.web.RequestHandler): @tornado.gen.coroutine def get(self): http_client = AsyncHTTPClient() response = yield http_client.fetch("http://google.com") print(response) application = tornado.web.Application([ (r"/", GenAsyncHandler), ]) if __name__ == "__main__": application.listen(8888) tornado.ioloop.IOLoop.instance().start() gen.coroutine schedules the generator to be resumed when the Future is resolved ‘yield’ makes the function a generator The generator in turn returns a Future instance In this case, response, will resolve with response from fetch or an exception
  27. 27. Tornado Engine class MainHandlerAsync(tornado.web.RequestHandler): @tornado.web.asynchronous @tornado.gen.engine def get(self): req = tornado.httpclient.HTTPRequest("http://127.0.0.1:8888/",) client = tornado.httpclient.AsyncHTTPClient() response = yield tornado.gen.Task(client.fetch, req) self.finish() application = tornado.web.Application([ (r"/async", MainHandlerAsync), ]) if __name__ == "__main__": http_server = tornado.httpserver.HTTPServer(application) http_server.listen(8888) tornado.ioloop.IOLoop.instance().start()
  28. 28. Let’s create our Future! myfuture.py server.py import time import datetime from tornado.concurrent import return_future class AsyncHTTPClient(object): @return_future def fetch(self, url, callback=None): print("In my fetch") time.sleep(0.02) result = str(datetime.datetime.utcnow()) callback(result) import tornado.web import tornado.gen from myfuture import AsyncHTTPClient def test(arg): print('In test:' + arg) class GenAsync(tornado.web.RequestHandler): @tornado.gen.coroutine def get(self): http_client = AsyncHTTPClient() r = yield http_client.fetch(“http://google.com”,test) print(r) application = tornado.web.Application([ (r"/", GenAsync),]) if __name__ == "__main__": application.listen(8888) tornado.ioloop.IOLoop.instance().start()
  29. 29. Performance: Blocking vs Async
  30. 30. Work to be achieved
  31. 31. Performance Results ab -n 500 -c 10 http://localhost:8888/blocking ab -n 500 -c 10 http://localhost:8888/async 6000 5000 4000 3000 2000 1000 0 Time per request Async Blocking Async Blocking 200 150 100 50 0 Requests per second Async Blocking Async Blocking
  32. 32. Learnings • Async programming is an efficient, easy to understand design and code • Python asyncio module is comprehensive • Has generic use cases for vast variety of applications o Responsive web applications o Networking applications • Requires a new way to program and design
  33. 33. Recommendations • Async programming is not a holistic solution • It has its own pros and cons o Suitable for primarily I/O bound applications o Needs enough tasks available to run • asyncio module is only available for Python 3 applications • Also explore other methods of concurrency: o Eventlets o STM o Multiprocessing/threads o Special languages e.g. GO, Scala • Understand and use 
  34. 34. References • asyncio – http://python.org • Python asyncio – o http://www.buzzcapture.com o www.slideshare.net/saghul • Tornado – http://tornadoweb.org • Multithreading – www.drdobbs.com • Event loop: https://docs.python.org/3/library/asyncio-eventloop. html
  35. 35. Contact Us • Chetan Giridhar o www.technobeans.com o https://github.com/cjgiridhar • Vishal Kanaujia o www.freethreads.wordpress.com o https://github.com/vishalkanaujia
  36. 36. Backup
  37. 37. Asyncio: example import asyncio @asyncio.coroutine def create(): yield from asyncio.sleep(3.0) print("(1) create file") @asyncio.coroutine def write(): yield from asyncio.sleep(1.0) print("(2) write into file") @asyncio.coroutine def close(): print("(3) close file") @asyncio.coroutine def test(): asyncio.async(create()) asyncio.async(write()) asyncio.async(close()) yield from asyncio.sleep(2.0) loop.stop() loop = asyncio.get_event_loop() asyncio.async(test()) loop.run_forever() print("Pending tasks at exit: %s" % asyncio.Task.all_tasks(loop)) loop.close()
  38. 38. Python Async Modules • Asynccore • Asyncchat • Gevent • Twisted • Eventlets
  39. 39. Concurrency Techniques • Multithreading/processing • Green Threads • STM
  40. 40. Tornado + AsyncIO from tornado.platform.asyncio import AsyncIOMainLoop from tornado.httpclient import AsyncHTTPClient import asyncio AsyncIOMainLoop().install() -- # Tell Tornado to use the asyncio eventloop loop = asyncio.get_event_loop() -- # get the loop http_client = AsyncHTTPClient() -- # the Tornado HTTP client def aio_fetch(client, url, **kwargs): fut = asyncio.Future() client.fetch(url, callback=fut.set_result, **kwargs) return fut @asyncio.coroutine def main(): print("fetching my site") mysite = yield from aio_fetch(http_client, "http://google.com/") print("hello httpbin") httpbin = yield from aio_fetch(http_client, "http://httpbin.org?q=%d" % mysite.code) print(httpbin.body) loop.run_until_complete(main())
  41. 41. Co-routine v/s Callback import asyncio def just_print_messages(loop): print('Just print') loop.call_later(1, just_print_messages, loop) def main(): loop = asyncio.get_event_loop() try: loop.call_soon(just_print_messages, loop) loop.run_forever() finally: loop.close() if __name__ == '__main__': main() import asyncio @asyncio.coroutine def just_print_messages(): while True: print('Just print') yield from asyncio.sleep(1) def main(): loop = asyncio.get_event_loop() try: loop.run_until_complete(just_print_messages()) finally: loop.close() if __name__ == '__main__': main()
  42. 42. Async in NodeJS request is an var http = require(‘http’); event var server = http.createServer; server.on(‘request’, function(request,response) { response.writeHead(200); response.end(‘Hello World’); }).listen(8001); Callback console.log(‘Server running on port 8001’);

Editor's Notes

  • Event loops use cooperative scheduling: an event loop only runs one task at a time. Other tasks may run in parallel if other event loops are running in different threads. While a task waits for the completion of a future, the event loop executes a new task.
  • Event loop : Central execution device
    BaseEventLoop.add_reader(fd, callback, *args) Start watching the file descriptor for read availability and then call the callback with specified arguments.
    BaseEventLoop.remove_reader(fd) Stop watching the file descriptor for read availability.
    BaseEventLoop.add_writer(fd, callback, *args) Start watching the file descriptor for write availability and then call the callback with specified arguments.
    BaseEventLoop.remove_writer(fd) Stop watching the file descriptor for write availability.
  • Add an example
  • A Transport represents a connection
    – e.g. a socket, pipe, or SSL connection
    • typically implemented by the framework
    • A Protocol represents an application
    – e.g. an HTTP server or client
    • typically implemented by you!
  • ×