Introduction to asyncio


Published on

Slides from my "Introduction to asyncio" talk given at PyLadies Amsterdam, the 20th of March of 2014.

Published in: Technology
1 Comment
  • Very useful in trying to understand
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Introduction to asyncio

  1. 1. @saghul Introduction to asyncio Saúl Ibarra Corretgé PyLadies Amsterdam - 20th March 2014
  2. 2.
  3. 3. Sockets 101 import socket server = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) server.bind(('', 1234)) server.listen(128) print("Server listening on: {}".format(server.getsockname())) client, addr = server.accept() print("Client connected: {}".format(addr)) while True: data = client.recv(4096) if not data: print("Client has disconnected") break client.send(data) server.close()
  4. 4. Scaling Up! import socket import _thread def handle_client(client, addr): print("Client connected: {}".format(addr)) while True: data = client.recv(4096) if not data: print("Client has disconnected") break client.send(data.upper()) server = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) server.bind(('', 1234)) server.listen(128) print("Server listening on: {}".format(server.getsockname())) while True: client, addr = server.accept() _thread.start_new_thread(handle_client, (client, addr))
  5. 5. Scaling Up! (really?) Threads are too expensive Also, context switching Use an event loop instead!
  6. 6. The Async Way (TM) Single thread Block waiting for sockets to be ready to read or write Perform i/o operations and call the callbacks! Repeat (Windows is not like this)
  7. 7. Why asyncio? asyncore and asynchat are not enough Fresh new implementation of Asynchronous I/O Python >= 3.3 Trollius: backport for Python >= 2.6 Use new language features: yield from Designed to interoperate with other frameworks
  8. 8. Components Event loop, policy Coroutines, Futures, Tasks Transports, Protocols
  9. 9. asyncio 101 import asyncio loop = asyncio.get_event_loop() @asyncio.coroutine def infinity(): while True: print("hello!") yield from asyncio.sleep(1) loop.run_until_complete(infinity())
  10. 10. Coroutines, futures & tasks
  11. 11. Coroutines, futures & tasks Coroutine generator function, can receive values decorated with @coroutine Future promise of a result or an error Task Future which runs a coroutine
  12. 12. Futures Similar to Futures from PEP-3148 concurrent.futures.Future API (almost) identical: f.set_result(); r = f.result() f.set_exception(e); e = f.exception() f.done(); f.cancel(); f.cancelled() f.add_done_callback(x); f.remove_done_callback(x)
  13. 13. Futures + Coroutines yield from works with Futures! f = Future() Someone will set the result or exception r = yield from f Waits until done and returns f.result() Usually returned by functions
  14. 14. Undoing callbacks @asyncio.coroutine def sync_looking_function(*args): fut = asyncio.Future() def cb(result, error): if error is not None: fut.set_result(result) else: fut.set_exception(Exception(error)) async_function(cb, *args) return (yield from fut)
  15. 15. Tasks Unicorns covered in fairy dust It’s a coroutine wrapped in a Future WAT Inherits from Future Works with yield from! r = yield from Task(coro(...))
  16. 16. Tasks vs coroutines A coroutine doesn’t “advance” without a scheduling mechanism Tasks can advance on their own The event loop is the scheduler! Magic!
  17. 17. Echo Server import asyncio loop = asyncio.get_event_loop() class EchoProtocol(asyncio.Protocol): def connection_made(self, transport): print('Client connected') self.transport = transport def data_received(self, data): print('Received data:',data) self.transport.write(data) def connection_lost(self, exc): print('Connection closed', exc) f = loop.create_server(lambda: EchoProtocol(), 'localhost', 1234) server = loop.run_until_complete(f) print('Server started') loop.run_forever()
  18. 18. Echo Server Reloaded import asyncio loop = asyncio.get_event_loop() clients = {} # task -> (reader, writer) def accept_client(client_reader, client_writer): task = asyncio.Task(handle_client(client_reader, client_writer)) clients[task] = (client_reader, client_writer) def client_done(task): del clients[task] task.add_done_callback(client_done) @asyncio.coroutine def handle_client(client_reader, client_writer): while True: data = (yield from client_reader.readline()) client_writer.write(data) f = asyncio.start_server(accept_client, '', 1234) server = loop.run_until_complete(f) loop.run_forever()
  19. 19. HTTP Server import asyncio import aiohttp import aiohttp.server class HttpServer(aiohttp.server.ServerHttpProtocol): @asyncio.coroutine def handle_request(self, message, payload): print('method = {!r}; path = {!r}; version = {!r}'.format( message.method, message.path, message.version)) response = aiohttp.Response(self.transport, 200, close=True) response.add_header('Content-type', 'text/plain') response.send_headers() response.write(b'Hello world!rn') response.write_eof() loop = asyncio.get_event_loop() f = loop.create_server(lambda: HttpServer(), 'localhost', 1234) server = loop.run_until_complete(f) loop.run_forever()
  20. 20. Redis import asyncio import asyncio_redis @asyncio.coroutine def subscriber(channels): # Create connection connection = yield from asyncio_redis.Connection.create(host='localhost', port=6379) # Create subscriber. subscriber = yield from connection.start_subscribe() # Subscribe to channel. yield from subscriber.subscribe(channels) # Wait for incoming events. while True: reply = yield from subscriber.next_published() print('Received: ', repr(reply.value), 'on channel', loop = asyncio.get_event_loop() loop.run_until_complete(subscriber(['my-channel']))
  21. 21. More? We just scratched the surface! Read PEP-3156 (it’s an easy read, I promise!) Checkout the documentation Checkout the third-party libraries Go implement something cool! “I hear and I forget. I see and I remember. I do and I understand.” - Confucius
  22. 22. Questions? @saghul