SlideShare a Scribd company logo
1 of 33
Download to read offline
Syncing up with asyncio for (micro)service development
Joir-dan Gumbs - Senior Software Engineer
IBM San Francisco
twitter: @jagumbs
blog: tasteandtrace.co
what is asyncio?
what it isn’t
aioamqp
aiobotocore
aiocache
aiocassandra
aiocouchdb
aiodns
aiodocker
aioes
aioetcd
aiofiles
aiographite
aiohttp
aiokafka
aiomcache
aiomysql
aioodbc
aiopg
aioprocessing
aioredis
aioredlock
anoriak
aiosmtpd
aiotasks
aiozk
aiozmq
async_timeout
motor
prometheus_async
rethinkdb
sanic
uvloop
https://github.com/python/asyncio/wiki/ThirdParty
quite a few asyncio-ready packages exist already
building (micro)services
aiohttp
asyncio web framework
sanic
flask-like web server
async keyword is an explicit indicator of asynchronous construct
async def - Indicates coroutine (asynchronous function)
async with - Indicates asynchronous context manager
async for - Indicates asynchronous iteration
async + await + event loop = asyncio
await keyword is an explicit asynchronous “checkpoint”
Signal to swap to different coroutine
Will resume from checkpoint when task completed and has regained context
async + await + event loop = asyncio
event loop is the main execution device for asyncio
handles registration, execution, and cancelling of coroutines
expensive synchronous calls will block the event loop until completion
can delegate expensive (IO/CPU) calls to various Executors
async + await + event loop = asyncio
hello asyncio 1 - simple coroutine
import asyncio
import random
async def work_it(i, random_sleep):
await asyncio.sleep(random_sleep)
print("Task {0} completed after random sleep of {1} seconds".format(i, random_sleep))
if __name__ == "__main__":
print("Run basic async function.")
loop = asyncio.get_event_loop()
loop.run_until_complete(work_it(1, random.randrange(1, 5)))
print("Finished!")
hello sanic 1 - hello_sanic.py
import asyncio
import uvloop
from sanic import Sanic
from sanic.response import text
app = Sanic(“hello_sanic”)
@app.route(“/async”)
async def async_hello(request):
return text(“Asynchronous Hello Sanic!”)
@app.route(“/sync”)
def sync_hello(request):
return text(“Synchronous Hello Sanic!”)
if __name__ == "__main__":
# Use uvloop cause its fast!
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
app.run(host="0.0.0.0", port=9090)
coroutines return a Future object
using await unwraps Future for value(s), if available
we can await multiple Future objects using asyncio.gather
callbacks are optional with asyncio coroutines
created using <Future>.add_done_callback, if you want them
async + await + event loop = asyncio
hello asyncio 2 - coroutine collection
import asyncio
import random
async def work_it(i, random_sleep):
await asyncio.sleep(random_sleep)
print("Task {0} completed after random sleep of {1} seconds".format(i, random_sleep))
if __name__ == "__main__":
print("Run basic async function with grouping.")
loop = asyncio.get_event_loop()
futures = [work_it(i, random.randrange(1, 5)) for i in range(0, 10)]
group_future = asyncio.gather(*futures)
loop.run_until_complete(group_future)
print("Finished!")
hello sanic 2 - outbound.py
import asyncio
import aiohttp
import uvloop
from sanic import Sanic
from sanic.response import json
app = Sanic(“hello_sanic”)
@app.route(“/”)
async def hello(request):
site = “http://www.ibm.com"
response = await aiohttp.request(“GET”, site)
return json({“website”: site, “size”: len(await response.text())})
if __name__ == "__main__":
# Use uvloop cause its fast!
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
app.run(host="0.0.0.0", port=9090)
executors help us use synchronous functions asynchronously
<event_loop>.run_in_executor(executor, fn, *args) is our friend!
use functools.partial if you require **kwargs
works with concurrent.futures.[ThreadPoolExecutor, ProcessPoolExecutor]
defaults to ThreadPoolExecutor when None
asynci-oh no! - executors to the rescue
asynci-oh no! - a sync in my async
import asyncio
import requests
async def run(loop):
print("Accessing ibm.com”)
# requests is a synchronous http framework. . .
result = requests.get(“http://www.ibm.com”) # I’m a blocker!!
print("Status Code", result.status_code)
print("Body Length", len(result.text))
if __name__ == "__main__":
print("Run basic async function with executor.")
loop = asyncio.get_event_loop()
loop.run_until_complete(run(loop))
print("Finished!")
hello asyncio 3 - async a sync
import asyncio
import requests
async def run(loop):
print("Accessing ibm.com”)
# There are better ways of doing this though . . .
result = await loop.run_in_executor(None, requests.get, "http://www.ibm.com")
print("Status Code", result.status_code)
print("Body Length", len(result.text))
if __name__ == "__main__":
print("Run basic async function with executor.")
loop = asyncio.get_event_loop()
loop.run_until_complete(run(loop))
print("Finished!")
hello asyncio 3 - async a sync
import asyncio
import requests
async def run(loop):
print("Accessing ibm.com”)
# There are better ways of doing this though . . .
result = await loop.run_in_executor(None, requests.get, "http://www.ibm.com")
print("Status Code", result.status_code)
print("Body Length", len(result.text))
if __name__ == "__main__":
print("Run basic async function with executor.")
loop = asyncio.get_event_loop()
loop.run_until_complete(run(loop))
print("Finished!")
hello asyncio 3 - async a sync (aiohttp edition)
import aiohttp
import asyncio
async def run():
print("Accessing ibm.com")
result = await aiohttp.request("GET", "http://www.ibm.com")
print("Status Code", result.status)
print("Body Length", len(await result.text()))
if __name__ == "__main__":
print("Run basic async function with aiohttp.")
loop = asyncio.get_event_loop()
loop.run_until_complete(run())
print("Finished!")
hello asyncio 6 - amethyst
Go to code
https://github.com/SakuraSound/amethyst
Similar to standard python context manager
__aenter__ and __aexit__ vs. __enter__ and __exit__
can wait on I/O while creating/destroying “session”
async with - establishing “sessions”
hello asyncio 4 - PlayerState context manager definition
class PlayerState():
. . .
def __init__(self, player_id, region='US'):
self.loop = asyncio.get_event_loop()
self.player_id = player_id
self.region = region
async def __aenter__(self):
self.lock = await self.get_lock(self.player_id)
self.state = await self.loop.run_in_executor(None, self.get_state, self.player_id)
print("locking {0}".format(self.player_id))
return self
async def __aexit__(self, exc_type, exc, tb):
if hasattr(self, 'lock'):
if hasattr(self, 'state'):
await self.loop.run_in_executor(None, self.push_state)
await self.unlock()
else:
raise AssertionError("Did we have the lock?”)
. . .
hello asyncio 4 - PlayerState context manager usage
. . .
async def run(player):
async with PlayerState(player) as ps:
await ps.update("foo", "bar")
print(ps.state)
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(run("SakuraSound"))
. . .
Looping construct for handling iterators and generators
iterators must implement __aiter__ and __anext__
async for - iterate and generate
hello asyncio 5 - async_for_iterator.py
import asyncio
import random
class NextSweets(object):
__sprites__ = ["bonbon", "taffy", "mochi", "cookie", "cake"]
def __init__(self, to):
self.curr = 0
self.to = to
async def __anext__(self):
if self.curr >= self.to:
raise StopAsyncIteration
self.curr += 1
i = random.randint(0, len(NextSweets.__sprites__) - 1)
await asyncio.sleep(1)
return NextSweets.__sprites__[i]
async def __aiter__(self):
return self
. . .
hello asyncio 5 - async_for_iterator.py
. . .
@classmethod
async def get_next_k(cls, k):
async for sweet in NextSweets(k):
print(sweet)
loop = asyncio.get_event_loop()
loop.run_until_complete(NextSweets.get_next_k(30))
hello asyncio 6 - async_for_generator.py
import asyncio
import random
async def next_k_sweets(k, sweets):
for i in range(k):
# Simulating blocking call (DB? Service?)
await asyncio.sleep(1)
pos = random.randint(0, len(sweets) - 1)
yield sweets[pos]
async def run():
sweets = ["bonbon", "taffy", "mochi", "cookie", "cake", "ice_cream"]
async for sweet in next_k_sweets(50, sweets):
print(sweet)
hello asyncio 6 - starcraft_replay_processing
Go to code
https://github.com/SakuraSound/replay_parsing
not everything has to be a
coroutine!
try to use asyncio packages
when available...
...and use executors if they
aren’t.
questions?

More Related Content

What's hot

Fabric workshop(1) - (MOSG)
Fabric workshop(1) - (MOSG)Fabric workshop(1) - (MOSG)
Fabric workshop(1) - (MOSG)Soshi Nemoto
 
Minimal MVC in JavaScript
Minimal MVC in JavaScriptMinimal MVC in JavaScript
Minimal MVC in JavaScriptMosky Liu
 
WordPress Plugin Unit Tests (FR - WordCamp Paris 2015)
WordPress Plugin Unit Tests (FR - WordCamp Paris 2015)WordPress Plugin Unit Tests (FR - WordCamp Paris 2015)
WordPress Plugin Unit Tests (FR - WordCamp Paris 2015)Ozh
 
Php 7 hhvm and co
Php 7 hhvm and coPhp 7 hhvm and co
Php 7 hhvm and coPierre Joye
 
Making the most of your Test Suite
Making the most of your Test SuiteMaking the most of your Test Suite
Making the most of your Test Suiteericholscher
 
Testing your infrastructure with litmus
Testing your infrastructure with litmusTesting your infrastructure with litmus
Testing your infrastructure with litmusBram Vogelaar
 
Ansible roles done right
Ansible roles done rightAnsible roles done right
Ansible roles done rightDan Vaida
 
Ansible for beginners
Ansible for beginnersAnsible for beginners
Ansible for beginnersKuo-Le Mei
 
Py conkr 20150829_docker-python
Py conkr 20150829_docker-pythonPy conkr 20150829_docker-python
Py conkr 20150829_docker-pythonEric Ahn
 
Automation and Ansible
Automation and AnsibleAutomation and Ansible
Automation and Ansiblejtyr
 
Puppet Continuous Integration with PE and GitLab
Puppet Continuous Integration with PE and GitLabPuppet Continuous Integration with PE and GitLab
Puppet Continuous Integration with PE and GitLabAlessandro Franceschi
 
Ansible 實戰:top down 觀點
Ansible 實戰:top down 觀點Ansible 實戰:top down 觀點
Ansible 實戰:top down 觀點William Yeh
 
Bootstrap your Cloud Infrastructure using puppet and hashicorp stack
Bootstrap your Cloud Infrastructure using puppet and hashicorp stackBootstrap your Cloud Infrastructure using puppet and hashicorp stack
Bootstrap your Cloud Infrastructure using puppet and hashicorp stackBram Vogelaar
 
Easy contributable internationalization process with Sphinx @ pyconmy2015
Easy contributable internationalization process with Sphinx @ pyconmy2015Easy contributable internationalization process with Sphinx @ pyconmy2015
Easy contributable internationalization process with Sphinx @ pyconmy2015Takayuki Shimizukawa
 
wwc start-launched
wwc start-launchedwwc start-launched
wwc start-launchedMat Schaffer
 
Elegant concurrency
Elegant concurrencyElegant concurrency
Elegant concurrencyMosky Liu
 

What's hot (19)

Fabric workshop(1) - (MOSG)
Fabric workshop(1) - (MOSG)Fabric workshop(1) - (MOSG)
Fabric workshop(1) - (MOSG)
 
Minimal MVC in JavaScript
Minimal MVC in JavaScriptMinimal MVC in JavaScript
Minimal MVC in JavaScript
 
WordPress Plugin Unit Tests (FR - WordCamp Paris 2015)
WordPress Plugin Unit Tests (FR - WordCamp Paris 2015)WordPress Plugin Unit Tests (FR - WordCamp Paris 2015)
WordPress Plugin Unit Tests (FR - WordCamp Paris 2015)
 
Php 7 hhvm and co
Php 7 hhvm and coPhp 7 hhvm and co
Php 7 hhvm and co
 
Making the most of your Test Suite
Making the most of your Test SuiteMaking the most of your Test Suite
Making the most of your Test Suite
 
Testing your infrastructure with litmus
Testing your infrastructure with litmusTesting your infrastructure with litmus
Testing your infrastructure with litmus
 
DevOps with Fabric
DevOps with FabricDevOps with Fabric
DevOps with Fabric
 
Ansible roles done right
Ansible roles done rightAnsible roles done right
Ansible roles done right
 
はじめてのSymfony2
はじめてのSymfony2はじめてのSymfony2
はじめてのSymfony2
 
Ansible for beginners
Ansible for beginnersAnsible for beginners
Ansible for beginners
 
Py conkr 20150829_docker-python
Py conkr 20150829_docker-pythonPy conkr 20150829_docker-python
Py conkr 20150829_docker-python
 
Automation and Ansible
Automation and AnsibleAutomation and Ansible
Automation and Ansible
 
Puppet Continuous Integration with PE and GitLab
Puppet Continuous Integration with PE and GitLabPuppet Continuous Integration with PE and GitLab
Puppet Continuous Integration with PE and GitLab
 
Ansible 實戰:top down 觀點
Ansible 實戰:top down 觀點Ansible 實戰:top down 觀點
Ansible 實戰:top down 觀點
 
Bootstrap your Cloud Infrastructure using puppet and hashicorp stack
Bootstrap your Cloud Infrastructure using puppet and hashicorp stackBootstrap your Cloud Infrastructure using puppet and hashicorp stack
Bootstrap your Cloud Infrastructure using puppet and hashicorp stack
 
Easy contributable internationalization process with Sphinx @ pyconmy2015
Easy contributable internationalization process with Sphinx @ pyconmy2015Easy contributable internationalization process with Sphinx @ pyconmy2015
Easy contributable internationalization process with Sphinx @ pyconmy2015
 
wwc start-launched
wwc start-launchedwwc start-launched
wwc start-launched
 
Effective Benchmarks
Effective BenchmarksEffective Benchmarks
Effective Benchmarks
 
Elegant concurrency
Elegant concurrencyElegant concurrency
Elegant concurrency
 

Similar to Syncing up with Python’s asyncio for (micro) service development, Joir-dan Gumbs

Multithreading on iOS
Multithreading on iOSMultithreading on iOS
Multithreading on iOSMake School
 
Plack perl superglue for web frameworks and servers
Plack perl superglue for web frameworks and serversPlack perl superglue for web frameworks and servers
Plack perl superglue for web frameworks and serversTatsuhiko Miyagawa
 
The journey of asyncio adoption in instagram
The journey of asyncio adoption in instagramThe journey of asyncio adoption in instagram
The journey of asyncio adoption in instagramJimmy Lai
 
Node.js - iJS 2019
Node.js - iJS 2019Node.js - iJS 2019
Node.js - iJS 2019NilsMehlhorn
 
Original slides from Ryan Dahl's NodeJs intro talk
Original slides from Ryan Dahl's NodeJs intro talkOriginal slides from Ryan Dahl's NodeJs intro talk
Original slides from Ryan Dahl's NodeJs intro talkAarti Parikh
 
Burn down the silos! Helping dev and ops gel on high availability websites
Burn down the silos! Helping dev and ops gel on high availability websitesBurn down the silos! Helping dev and ops gel on high availability websites
Burn down the silos! Helping dev and ops gel on high availability websitesLindsay Holmwood
 
Denys Serhiienko "ASGI in depth"
Denys Serhiienko "ASGI in depth"Denys Serhiienko "ASGI in depth"
Denys Serhiienko "ASGI in depth"Fwdays
 
Threads, Queues, and More: Async Programming in iOS
Threads, Queues, and More: Async Programming in iOSThreads, Queues, and More: Async Programming in iOS
Threads, Queues, and More: Async Programming in iOSTechWell
 
Asynchronous Programming at Netflix
Asynchronous Programming at NetflixAsynchronous Programming at Netflix
Asynchronous Programming at NetflixC4Media
 
Servlet 3.1 Async I/O
Servlet 3.1 Async I/OServlet 3.1 Async I/O
Servlet 3.1 Async I/OSimone Bordet
 
Adopt DevOps philosophy on your Symfony projects (Symfony Live 2011)
Adopt DevOps philosophy on your Symfony projects (Symfony Live 2011)Adopt DevOps philosophy on your Symfony projects (Symfony Live 2011)
Adopt DevOps philosophy on your Symfony projects (Symfony Live 2011)Fabrice Bernhard
 
Intro to Selenium UI Tests with pytest & some useful pytest plugins
Intro to Selenium UI Tests with pytest & some useful pytest pluginsIntro to Selenium UI Tests with pytest & some useful pytest plugins
Intro to Selenium UI Tests with pytest & some useful pytest pluginsAsif Mohaimen
 
Building a js widget
Building a js widgetBuilding a js widget
Building a js widgetTudor Barbu
 

Similar to Syncing up with Python’s asyncio for (micro) service development, Joir-dan Gumbs (20)

aiohttp intro
aiohttp introaiohttp intro
aiohttp intro
 
Multithreading on iOS
Multithreading on iOSMultithreading on iOS
Multithreading on iOS
 
Plack perl superglue for web frameworks and servers
Plack perl superglue for web frameworks and serversPlack perl superglue for web frameworks and servers
Plack perl superglue for web frameworks and servers
 
The journey of asyncio adoption in instagram
The journey of asyncio adoption in instagramThe journey of asyncio adoption in instagram
The journey of asyncio adoption in instagram
 
Node.js - iJS 2019
Node.js - iJS 2019Node.js - iJS 2019
Node.js - iJS 2019
 
Original slides from Ryan Dahl's NodeJs intro talk
Original slides from Ryan Dahl's NodeJs intro talkOriginal slides from Ryan Dahl's NodeJs intro talk
Original slides from Ryan Dahl's NodeJs intro talk
 
Burn down the silos! Helping dev and ops gel on high availability websites
Burn down the silos! Helping dev and ops gel on high availability websitesBurn down the silos! Helping dev and ops gel on high availability websites
Burn down the silos! Helping dev and ops gel on high availability websites
 
Tech friday 22.01.2016
Tech friday 22.01.2016Tech friday 22.01.2016
Tech friday 22.01.2016
 
Denys Serhiienko "ASGI in depth"
Denys Serhiienko "ASGI in depth"Denys Serhiienko "ASGI in depth"
Denys Serhiienko "ASGI in depth"
 
Threads, Queues, and More: Async Programming in iOS
Threads, Queues, and More: Async Programming in iOSThreads, Queues, and More: Async Programming in iOS
Threads, Queues, and More: Async Programming in iOS
 
Python, do you even async?
Python, do you even async?Python, do you even async?
Python, do you even async?
 
Node.js
Node.jsNode.js
Node.js
 
Asynchronous Programming at Netflix
Asynchronous Programming at NetflixAsynchronous Programming at Netflix
Asynchronous Programming at Netflix
 
Servlet 3.1 Async I/O
Servlet 3.1 Async I/OServlet 3.1 Async I/O
Servlet 3.1 Async I/O
 
Webscraping with asyncio
Webscraping with asyncioWebscraping with asyncio
Webscraping with asyncio
 
Adopt DevOps philosophy on your Symfony projects (Symfony Live 2011)
Adopt DevOps philosophy on your Symfony projects (Symfony Live 2011)Adopt DevOps philosophy on your Symfony projects (Symfony Live 2011)
Adopt DevOps philosophy on your Symfony projects (Symfony Live 2011)
 
mobl
moblmobl
mobl
 
Intro to Selenium UI Tests with pytest & some useful pytest plugins
Intro to Selenium UI Tests with pytest & some useful pytest pluginsIntro to Selenium UI Tests with pytest & some useful pytest plugins
Intro to Selenium UI Tests with pytest & some useful pytest plugins
 
Node intro
Node introNode intro
Node intro
 
Building a js widget
Building a js widgetBuilding a js widget
Building a js widget
 

More from Pôle Systematic Paris-Region

OSIS19_IoT :Transparent remote connectivity to short-range IoT devices, by Na...
OSIS19_IoT :Transparent remote connectivity to short-range IoT devices, by Na...OSIS19_IoT :Transparent remote connectivity to short-range IoT devices, by Na...
OSIS19_IoT :Transparent remote connectivity to short-range IoT devices, by Na...Pôle Systematic Paris-Region
 
OSIS19_Cloud : SAFC: Scheduling and Allocation Framework for Containers in a ...
OSIS19_Cloud : SAFC: Scheduling and Allocation Framework for Containers in a ...OSIS19_Cloud : SAFC: Scheduling and Allocation Framework for Containers in a ...
OSIS19_Cloud : SAFC: Scheduling and Allocation Framework for Containers in a ...Pôle Systematic Paris-Region
 
OSIS19_Cloud : Qu’apporte l’observabilité à la gestion de configuration? par ...
OSIS19_Cloud : Qu’apporte l’observabilité à la gestion de configuration? par ...OSIS19_Cloud : Qu’apporte l’observabilité à la gestion de configuration? par ...
OSIS19_Cloud : Qu’apporte l’observabilité à la gestion de configuration? par ...Pôle Systematic Paris-Region
 
OSIS19_Cloud : Performance and power management in virtualized data centers, ...
OSIS19_Cloud : Performance and power management in virtualized data centers, ...OSIS19_Cloud : Performance and power management in virtualized data centers, ...
OSIS19_Cloud : Performance and power management in virtualized data centers, ...Pôle Systematic Paris-Region
 
OSIS19_Cloud : Des objets dans le cloud, et qui y restent -- L'expérience du ...
OSIS19_Cloud : Des objets dans le cloud, et qui y restent -- L'expérience du ...OSIS19_Cloud : Des objets dans le cloud, et qui y restent -- L'expérience du ...
OSIS19_Cloud : Des objets dans le cloud, et qui y restent -- L'expérience du ...Pôle Systematic Paris-Region
 
OSIS19_Cloud : Attribution automatique de ressources pour micro-services, Alt...
OSIS19_Cloud : Attribution automatique de ressources pour micro-services, Alt...OSIS19_Cloud : Attribution automatique de ressources pour micro-services, Alt...
OSIS19_Cloud : Attribution automatique de ressources pour micro-services, Alt...Pôle Systematic Paris-Region
 
OSIS19_IoT : State of the art in security for embedded systems and IoT, by Pi...
OSIS19_IoT : State of the art in security for embedded systems and IoT, by Pi...OSIS19_IoT : State of the art in security for embedded systems and IoT, by Pi...
OSIS19_IoT : State of the art in security for embedded systems and IoT, by Pi...Pôle Systematic Paris-Region
 
Osis19_IoT: Proof of Pointer Programs with Ownership in SPARK, by Yannick Moy
Osis19_IoT: Proof of Pointer Programs with Ownership in SPARK, by Yannick MoyOsis19_IoT: Proof of Pointer Programs with Ownership in SPARK, by Yannick Moy
Osis19_IoT: Proof of Pointer Programs with Ownership in SPARK, by Yannick MoyPôle Systematic Paris-Region
 
Osis18_Cloud : Virtualisation efficace d’architectures NUMA
Osis18_Cloud : Virtualisation efficace d’architectures NUMAOsis18_Cloud : Virtualisation efficace d’architectures NUMA
Osis18_Cloud : Virtualisation efficace d’architectures NUMAPôle Systematic Paris-Region
 
Osis18_Cloud : DeepTorrent Stockage distribué perenne basé sur Bittorrent
Osis18_Cloud : DeepTorrent Stockage distribué perenne basé sur BittorrentOsis18_Cloud : DeepTorrent Stockage distribué perenne basé sur Bittorrent
Osis18_Cloud : DeepTorrent Stockage distribué perenne basé sur BittorrentPôle Systematic Paris-Region
 
OSIS18_IoT: L'approche machine virtuelle pour les microcontrôleurs, le projet...
OSIS18_IoT: L'approche machine virtuelle pour les microcontrôleurs, le projet...OSIS18_IoT: L'approche machine virtuelle pour les microcontrôleurs, le projet...
OSIS18_IoT: L'approche machine virtuelle pour les microcontrôleurs, le projet...Pôle Systematic Paris-Region
 
OSIS18_IoT: La securite des objets connectes a bas cout avec l'os et riot
OSIS18_IoT: La securite des objets connectes a bas cout avec l'os et riotOSIS18_IoT: La securite des objets connectes a bas cout avec l'os et riot
OSIS18_IoT: La securite des objets connectes a bas cout avec l'os et riotPôle Systematic Paris-Region
 
OSIS18_IoT : Solution de mise au point pour les systemes embarques, par Julio...
OSIS18_IoT : Solution de mise au point pour les systemes embarques, par Julio...OSIS18_IoT : Solution de mise au point pour les systemes embarques, par Julio...
OSIS18_IoT : Solution de mise au point pour les systemes embarques, par Julio...Pôle Systematic Paris-Region
 
OSIS18_IoT : Securisation du reseau des objets connectes, par Nicolas LE SAUZ...
OSIS18_IoT : Securisation du reseau des objets connectes, par Nicolas LE SAUZ...OSIS18_IoT : Securisation du reseau des objets connectes, par Nicolas LE SAUZ...
OSIS18_IoT : Securisation du reseau des objets connectes, par Nicolas LE SAUZ...Pôle Systematic Paris-Region
 
OSIS18_IoT : Ada and SPARK - Defense in Depth for Safe Micro-controller Progr...
OSIS18_IoT : Ada and SPARK - Defense in Depth for Safe Micro-controller Progr...OSIS18_IoT : Ada and SPARK - Defense in Depth for Safe Micro-controller Progr...
OSIS18_IoT : Ada and SPARK - Defense in Depth for Safe Micro-controller Progr...Pôle Systematic Paris-Region
 
OSIS18_IoT : RTEMS pour l'IoT professionnel, par Pierre Ficheux (Smile ECS)
OSIS18_IoT : RTEMS pour l'IoT professionnel, par Pierre Ficheux (Smile ECS)OSIS18_IoT : RTEMS pour l'IoT professionnel, par Pierre Ficheux (Smile ECS)
OSIS18_IoT : RTEMS pour l'IoT professionnel, par Pierre Ficheux (Smile ECS)Pôle Systematic Paris-Region
 
PyParis 2017 / Un mooc python, by thierry parmentelat
PyParis 2017 / Un mooc python, by thierry parmentelatPyParis 2017 / Un mooc python, by thierry parmentelat
PyParis 2017 / Un mooc python, by thierry parmentelatPôle Systematic Paris-Region
 

More from Pôle Systematic Paris-Region (20)

OSIS19_IoT :Transparent remote connectivity to short-range IoT devices, by Na...
OSIS19_IoT :Transparent remote connectivity to short-range IoT devices, by Na...OSIS19_IoT :Transparent remote connectivity to short-range IoT devices, by Na...
OSIS19_IoT :Transparent remote connectivity to short-range IoT devices, by Na...
 
OSIS19_Cloud : SAFC: Scheduling and Allocation Framework for Containers in a ...
OSIS19_Cloud : SAFC: Scheduling and Allocation Framework for Containers in a ...OSIS19_Cloud : SAFC: Scheduling and Allocation Framework for Containers in a ...
OSIS19_Cloud : SAFC: Scheduling and Allocation Framework for Containers in a ...
 
OSIS19_Cloud : Qu’apporte l’observabilité à la gestion de configuration? par ...
OSIS19_Cloud : Qu’apporte l’observabilité à la gestion de configuration? par ...OSIS19_Cloud : Qu’apporte l’observabilité à la gestion de configuration? par ...
OSIS19_Cloud : Qu’apporte l’observabilité à la gestion de configuration? par ...
 
OSIS19_Cloud : Performance and power management in virtualized data centers, ...
OSIS19_Cloud : Performance and power management in virtualized data centers, ...OSIS19_Cloud : Performance and power management in virtualized data centers, ...
OSIS19_Cloud : Performance and power management in virtualized data centers, ...
 
OSIS19_Cloud : Des objets dans le cloud, et qui y restent -- L'expérience du ...
OSIS19_Cloud : Des objets dans le cloud, et qui y restent -- L'expérience du ...OSIS19_Cloud : Des objets dans le cloud, et qui y restent -- L'expérience du ...
OSIS19_Cloud : Des objets dans le cloud, et qui y restent -- L'expérience du ...
 
OSIS19_Cloud : Attribution automatique de ressources pour micro-services, Alt...
OSIS19_Cloud : Attribution automatique de ressources pour micro-services, Alt...OSIS19_Cloud : Attribution automatique de ressources pour micro-services, Alt...
OSIS19_Cloud : Attribution automatique de ressources pour micro-services, Alt...
 
OSIS19_IoT : State of the art in security for embedded systems and IoT, by Pi...
OSIS19_IoT : State of the art in security for embedded systems and IoT, by Pi...OSIS19_IoT : State of the art in security for embedded systems and IoT, by Pi...
OSIS19_IoT : State of the art in security for embedded systems and IoT, by Pi...
 
Osis19_IoT: Proof of Pointer Programs with Ownership in SPARK, by Yannick Moy
Osis19_IoT: Proof of Pointer Programs with Ownership in SPARK, by Yannick MoyOsis19_IoT: Proof of Pointer Programs with Ownership in SPARK, by Yannick Moy
Osis19_IoT: Proof of Pointer Programs with Ownership in SPARK, by Yannick Moy
 
Osis18_Cloud : Pas de commun sans communauté ?
Osis18_Cloud : Pas de commun sans communauté ?Osis18_Cloud : Pas de commun sans communauté ?
Osis18_Cloud : Pas de commun sans communauté ?
 
Osis18_Cloud : Projet Wolphin
Osis18_Cloud : Projet Wolphin Osis18_Cloud : Projet Wolphin
Osis18_Cloud : Projet Wolphin
 
Osis18_Cloud : Virtualisation efficace d’architectures NUMA
Osis18_Cloud : Virtualisation efficace d’architectures NUMAOsis18_Cloud : Virtualisation efficace d’architectures NUMA
Osis18_Cloud : Virtualisation efficace d’architectures NUMA
 
Osis18_Cloud : DeepTorrent Stockage distribué perenne basé sur Bittorrent
Osis18_Cloud : DeepTorrent Stockage distribué perenne basé sur BittorrentOsis18_Cloud : DeepTorrent Stockage distribué perenne basé sur Bittorrent
Osis18_Cloud : DeepTorrent Stockage distribué perenne basé sur Bittorrent
 
Osis18_Cloud : Software-heritage
Osis18_Cloud : Software-heritageOsis18_Cloud : Software-heritage
Osis18_Cloud : Software-heritage
 
OSIS18_IoT: L'approche machine virtuelle pour les microcontrôleurs, le projet...
OSIS18_IoT: L'approche machine virtuelle pour les microcontrôleurs, le projet...OSIS18_IoT: L'approche machine virtuelle pour les microcontrôleurs, le projet...
OSIS18_IoT: L'approche machine virtuelle pour les microcontrôleurs, le projet...
 
OSIS18_IoT: La securite des objets connectes a bas cout avec l'os et riot
OSIS18_IoT: La securite des objets connectes a bas cout avec l'os et riotOSIS18_IoT: La securite des objets connectes a bas cout avec l'os et riot
OSIS18_IoT: La securite des objets connectes a bas cout avec l'os et riot
 
OSIS18_IoT : Solution de mise au point pour les systemes embarques, par Julio...
OSIS18_IoT : Solution de mise au point pour les systemes embarques, par Julio...OSIS18_IoT : Solution de mise au point pour les systemes embarques, par Julio...
OSIS18_IoT : Solution de mise au point pour les systemes embarques, par Julio...
 
OSIS18_IoT : Securisation du reseau des objets connectes, par Nicolas LE SAUZ...
OSIS18_IoT : Securisation du reseau des objets connectes, par Nicolas LE SAUZ...OSIS18_IoT : Securisation du reseau des objets connectes, par Nicolas LE SAUZ...
OSIS18_IoT : Securisation du reseau des objets connectes, par Nicolas LE SAUZ...
 
OSIS18_IoT : Ada and SPARK - Defense in Depth for Safe Micro-controller Progr...
OSIS18_IoT : Ada and SPARK - Defense in Depth for Safe Micro-controller Progr...OSIS18_IoT : Ada and SPARK - Defense in Depth for Safe Micro-controller Progr...
OSIS18_IoT : Ada and SPARK - Defense in Depth for Safe Micro-controller Progr...
 
OSIS18_IoT : RTEMS pour l'IoT professionnel, par Pierre Ficheux (Smile ECS)
OSIS18_IoT : RTEMS pour l'IoT professionnel, par Pierre Ficheux (Smile ECS)OSIS18_IoT : RTEMS pour l'IoT professionnel, par Pierre Ficheux (Smile ECS)
OSIS18_IoT : RTEMS pour l'IoT professionnel, par Pierre Ficheux (Smile ECS)
 
PyParis 2017 / Un mooc python, by thierry parmentelat
PyParis 2017 / Un mooc python, by thierry parmentelatPyParis 2017 / Un mooc python, by thierry parmentelat
PyParis 2017 / Un mooc python, by thierry parmentelat
 

Recently uploaded

A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructureitnewsafrica
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...itnewsafrica
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationKnoldus Inc.
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI AgeCprime
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Nikki Chapple
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 

Recently uploaded (20)

A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog Presentation
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI Age
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 

Syncing up with Python’s asyncio for (micro) service development, Joir-dan Gumbs

  • 1. Syncing up with asyncio for (micro)service development Joir-dan Gumbs - Senior Software Engineer IBM San Francisco twitter: @jagumbs blog: tasteandtrace.co
  • 7. async keyword is an explicit indicator of asynchronous construct async def - Indicates coroutine (asynchronous function) async with - Indicates asynchronous context manager async for - Indicates asynchronous iteration async + await + event loop = asyncio
  • 8. await keyword is an explicit asynchronous “checkpoint” Signal to swap to different coroutine Will resume from checkpoint when task completed and has regained context async + await + event loop = asyncio
  • 9. event loop is the main execution device for asyncio handles registration, execution, and cancelling of coroutines expensive synchronous calls will block the event loop until completion can delegate expensive (IO/CPU) calls to various Executors async + await + event loop = asyncio
  • 10. hello asyncio 1 - simple coroutine import asyncio import random async def work_it(i, random_sleep): await asyncio.sleep(random_sleep) print("Task {0} completed after random sleep of {1} seconds".format(i, random_sleep)) if __name__ == "__main__": print("Run basic async function.") loop = asyncio.get_event_loop() loop.run_until_complete(work_it(1, random.randrange(1, 5))) print("Finished!")
  • 11. hello sanic 1 - hello_sanic.py import asyncio import uvloop from sanic import Sanic from sanic.response import text app = Sanic(“hello_sanic”) @app.route(“/async”) async def async_hello(request): return text(“Asynchronous Hello Sanic!”) @app.route(“/sync”) def sync_hello(request): return text(“Synchronous Hello Sanic!”) if __name__ == "__main__": # Use uvloop cause its fast! asyncio.set_event_loop_policy(uvloop.EventLoopPolicy()) app.run(host="0.0.0.0", port=9090)
  • 12. coroutines return a Future object using await unwraps Future for value(s), if available we can await multiple Future objects using asyncio.gather callbacks are optional with asyncio coroutines created using <Future>.add_done_callback, if you want them async + await + event loop = asyncio
  • 13. hello asyncio 2 - coroutine collection import asyncio import random async def work_it(i, random_sleep): await asyncio.sleep(random_sleep) print("Task {0} completed after random sleep of {1} seconds".format(i, random_sleep)) if __name__ == "__main__": print("Run basic async function with grouping.") loop = asyncio.get_event_loop() futures = [work_it(i, random.randrange(1, 5)) for i in range(0, 10)] group_future = asyncio.gather(*futures) loop.run_until_complete(group_future) print("Finished!")
  • 14. hello sanic 2 - outbound.py import asyncio import aiohttp import uvloop from sanic import Sanic from sanic.response import json app = Sanic(“hello_sanic”) @app.route(“/”) async def hello(request): site = “http://www.ibm.com" response = await aiohttp.request(“GET”, site) return json({“website”: site, “size”: len(await response.text())}) if __name__ == "__main__": # Use uvloop cause its fast! asyncio.set_event_loop_policy(uvloop.EventLoopPolicy()) app.run(host="0.0.0.0", port=9090)
  • 15. executors help us use synchronous functions asynchronously <event_loop>.run_in_executor(executor, fn, *args) is our friend! use functools.partial if you require **kwargs works with concurrent.futures.[ThreadPoolExecutor, ProcessPoolExecutor] defaults to ThreadPoolExecutor when None asynci-oh no! - executors to the rescue
  • 16. asynci-oh no! - a sync in my async import asyncio import requests async def run(loop): print("Accessing ibm.com”) # requests is a synchronous http framework. . . result = requests.get(“http://www.ibm.com”) # I’m a blocker!! print("Status Code", result.status_code) print("Body Length", len(result.text)) if __name__ == "__main__": print("Run basic async function with executor.") loop = asyncio.get_event_loop() loop.run_until_complete(run(loop)) print("Finished!")
  • 17. hello asyncio 3 - async a sync import asyncio import requests async def run(loop): print("Accessing ibm.com”) # There are better ways of doing this though . . . result = await loop.run_in_executor(None, requests.get, "http://www.ibm.com") print("Status Code", result.status_code) print("Body Length", len(result.text)) if __name__ == "__main__": print("Run basic async function with executor.") loop = asyncio.get_event_loop() loop.run_until_complete(run(loop)) print("Finished!")
  • 18. hello asyncio 3 - async a sync import asyncio import requests async def run(loop): print("Accessing ibm.com”) # There are better ways of doing this though . . . result = await loop.run_in_executor(None, requests.get, "http://www.ibm.com") print("Status Code", result.status_code) print("Body Length", len(result.text)) if __name__ == "__main__": print("Run basic async function with executor.") loop = asyncio.get_event_loop() loop.run_until_complete(run(loop)) print("Finished!")
  • 19. hello asyncio 3 - async a sync (aiohttp edition) import aiohttp import asyncio async def run(): print("Accessing ibm.com") result = await aiohttp.request("GET", "http://www.ibm.com") print("Status Code", result.status) print("Body Length", len(await result.text())) if __name__ == "__main__": print("Run basic async function with aiohttp.") loop = asyncio.get_event_loop() loop.run_until_complete(run()) print("Finished!")
  • 20. hello asyncio 6 - amethyst Go to code https://github.com/SakuraSound/amethyst
  • 21. Similar to standard python context manager __aenter__ and __aexit__ vs. __enter__ and __exit__ can wait on I/O while creating/destroying “session” async with - establishing “sessions”
  • 22. hello asyncio 4 - PlayerState context manager definition class PlayerState(): . . . def __init__(self, player_id, region='US'): self.loop = asyncio.get_event_loop() self.player_id = player_id self.region = region async def __aenter__(self): self.lock = await self.get_lock(self.player_id) self.state = await self.loop.run_in_executor(None, self.get_state, self.player_id) print("locking {0}".format(self.player_id)) return self async def __aexit__(self, exc_type, exc, tb): if hasattr(self, 'lock'): if hasattr(self, 'state'): await self.loop.run_in_executor(None, self.push_state) await self.unlock() else: raise AssertionError("Did we have the lock?”) . . .
  • 23. hello asyncio 4 - PlayerState context manager usage . . . async def run(player): async with PlayerState(player) as ps: await ps.update("foo", "bar") print(ps.state) if __name__ == "__main__": loop = asyncio.get_event_loop() loop.run_until_complete(run("SakuraSound")) . . .
  • 24. Looping construct for handling iterators and generators iterators must implement __aiter__ and __anext__ async for - iterate and generate
  • 25. hello asyncio 5 - async_for_iterator.py import asyncio import random class NextSweets(object): __sprites__ = ["bonbon", "taffy", "mochi", "cookie", "cake"] def __init__(self, to): self.curr = 0 self.to = to async def __anext__(self): if self.curr >= self.to: raise StopAsyncIteration self.curr += 1 i = random.randint(0, len(NextSweets.__sprites__) - 1) await asyncio.sleep(1) return NextSweets.__sprites__[i] async def __aiter__(self): return self . . .
  • 26. hello asyncio 5 - async_for_iterator.py . . . @classmethod async def get_next_k(cls, k): async for sweet in NextSweets(k): print(sweet) loop = asyncio.get_event_loop() loop.run_until_complete(NextSweets.get_next_k(30))
  • 27. hello asyncio 6 - async_for_generator.py import asyncio import random async def next_k_sweets(k, sweets): for i in range(k): # Simulating blocking call (DB? Service?) await asyncio.sleep(1) pos = random.randint(0, len(sweets) - 1) yield sweets[pos] async def run(): sweets = ["bonbon", "taffy", "mochi", "cookie", "cake", "ice_cream"] async for sweet in next_k_sweets(50, sweets): print(sweet)
  • 28. hello asyncio 6 - starcraft_replay_processing Go to code https://github.com/SakuraSound/replay_parsing
  • 29.
  • 30. not everything has to be a coroutine!
  • 31. try to use asyncio packages when available...
  • 32. ...and use executors if they aren’t.