SlideShare a Scribd company logo
1 of 32
Download to read offline
Introduction to
asynchronous programming
By Juti Noppornpitak
PYCON

CANADA

2019
Who am I?
Juti Noppornpitak
A senior software developer at DNAstack
and someone who accidentally become a
consultant on scientific application projects
@shiroyuki on GitHub and Twitter
www.shiroyuki.com
Let’s get started
Asynchronous programming
A way to run code in parallel with the main thread and notifies
the main thread when it finishes running, fails to complete, or
is still running.
Cooperative multitasking
Cooperative multitasking, also known as non-preemptive multitasking, is a style
of computer multitasking in which the operating system never initiates a context switch from
a running process to another process. Instead, processes voluntarily yield
control periodically or when idle or logically blocked in order to enable multiple applications
to be run concurrently. This type of multitasking is called "cooperative" because all programs
must cooperate for the entire scheduling scheme to work. In this scheme, the process
scheduler of an operating system is known as a cooperative scheduler, having its role
reduced down to starting the processes and letting them return control back to it voluntarily.
—Wikipedia
Let’s see how it works
Let’s start with a piece of
simple CPU-bound code.
You will notice that, without concurrent
programming, this code will spend at least
4 seconds to finish.
Now, we spiced things up
with threading.
Now, you notice that we have to pass a shared
dictionary around in order to retrieve the result
as func_001 and func_002 cannot return the
result back to main.
Now, simplified with
ThreadPoolExecutor
This looks a lot prettier and we don’t need to use

a shared dictionary to store the value as long as

we remember to call Future.result().
However, you still may need to check for errors

and completions manually.
Rewritten with coroutines
• The main function is now much simpler.
• The concurrency is managed by an event loop.
• Two CPU-bound code on line 15 and 22 are

handled with run_in_executor to achieve

the similar performance.
• We uses run_in_executor to avoid

CPU-bound code, like time.sleep, blocking

the event loop.
async def sum(*args) -> float:

…
result = await coro
task = asyncio.create_task(coro)
result = await task
← At this stage, sum is still a function.
coro = sum(1, 2)
However, unlike normal functions,

↓ sum(…) returns a coroutine instead.
↑ You can await a coroutine directly.

Internally, any given coroutine will

be wrapped as a task.
So, here is how we write cooperative multitasking code in Python with coroutines.
↑ This route is preferred.
↑ Create a task of the

coroutine and schedule

its execution
What do you need to watch out when you write
cooperative multitasking code?
•Avoiding CPU-bound code as it can block the corresponding event loop.
•In multithreading, the thread scheduling and context switching are handled by an
operating system and outside the control of Python.
•However, cooperative multitasking code manages its own task scheduling and
task switching with an event loop.
Multithreading vs Cooperative Multitasking
def foo() async def foo()
Unit of Work Thread Task
Scheduler Operating System Event Loop
High-level Usage Suppose you have a function:
Create a thread targeting foo();
then, start the thread;
then, wait for the thread to join back;
finally, good luck on fetching the result
back.
Suppose you have an async function:
Just await for the result from a task.
Directly call foo()… Whatever is given in return Coroutine of foo()
Now, why do you care about

cooperative multitasking?
Code Simplicity
• With PEP 492 (async and await), developers can write a cooperative
multitasking code as a simple sequence of instructions, but each instruction can
be executed in a much more complicated order based on I/O and upstream tasks’
completion.
• In comparison to conventional multithreading code, developers have to manage
threads on their own.
From our previous example
This is just a simple “lazy" code.
Now, after we parallelize

the code with just low-level

thread APIs…
Regardless to the logging messages, you start

to see the ceremonial steps to get target functions

run in their own thread.
We are also responsible to start the threads and

join them back manually.
Even with
ThreadPoolExecutor
While this looks a lot prettier, we still need to 

explicitly set up a thread pool.
With coroutine, asyncio,

async, and await
We gain the benefit of asynchronicity with

minimal ceremonial code.
Runaway Threads versus Cancellable Tasks
Can you cancel or stop an active thread?
• The short answer is NO.
• There are many suggestions on the internet on how to cancel/kill an active thread. One of a
popular suggestion is the magical-but-undocumented Thread._stop().
• Each thread in Python has a state lock, called _tstate_lock, which is only acquired at the
end of thread’s life.
• Thread._stop() only works when Thread._tstate_lock is acquired.
• Therefore, active threads cannot be stopped.
• Here is just an observation. Cancelling threads is generally discouraged as it could lead
your program to an undesirable state, such as memory management and deadlock.
demo/003-001-naive-thread-canceller.py
← This will cause _stop to raise AssertionError.
Can we cancel or stop a task?
• The short answer is kind of YES by using Task.cancel().
• When cancel() is called, the corresponding coroutine can catch CancelledError (an
exception from asyncio) so that the method can run a cleanup procedure.
• If the coroutine is cancelled before it starts running, the exception will not be raised.
• If the exception is not caught inside the coroutine, it will bubble up to the parent coroutine.
• So, as you can see, the cancellation is not guaranteed.
• Depending on implementation, a coroutine may suppress the cancellation (CancelledError)
and keep running like nothing even happens.
• Suppressing CancelledError is discouraged.
demo/003-002-naive-task-canceller.py
← 🤔 This line never get called.
← (3) We cancel the task here.
← (1) Create and schedule the coroutine.
← (2) Await the sleeping instruction (and trigger the event loop to start).
← 🙈 This line runs only once.
← (4) Get back to the sleepy brain.
← 🙈 Try to sleep for 6 seconds
What about cancel() from
concurrent.futures.Future?
• It is the same as Thread._stop().
Thread Safety (in asyncio)
What is thread safety?
Thread safety is a computer programming concept applicable to multi-threaded code.
Thread-safe code only manipulates shared data structures in a manner that ensures
that all threads behave properly and fulfill their design specifications without
unintended interaction. There are various strategies for making thread-safe data
structures.
— Wikipedia
Let's closely examine

BaseEventLoop.call_soon

from asyncio.base_events
• This method is a well-documented case of

what could go wrong. Can you guess where

the problem is?
• When this method is called simultaneously by

multiple threads, list operations may throw an

exception (IndexError).
• You can get away with the thread safety issue

by using call_soon_threadsafe.
Should you write asynchronous code?
• Generally, if your code requires speed, writing asynchronous code is usually a solution to speed things up.
• Multithreading is generally a good approach if your code is CPU-intensive.
• Cooperative multitasking is good for a few situations:
• Your code needs to be more responsive.
• Without context switching, your code does not have to sacrifice some CPU time to switch between tasks.
• Running the event loop in a single thread, your code tends to use less memory.*
• You can tolerate occasional blockages in the event loop by not-so-intense CPU-bound code.
• A cooperative multitasking app is slightly more difficult to design to have an application running as fast as a multithreading app.
• Your code is as good as a normal sequential code if some of your coroutines never yield the control back to the event loop.
• The placement of await is very important.
• In asyncio, when you create tasks of any coroutines, all tasks are scheduled right away.
• This means, as soon as your code start awaiting one of the tasks, the other tasks will be executed.
Thank you
• The source code in the demonstration is belong to the public domain, except the
code from CPython, which has its own license. The examples are available at
https://github.com/shiroyuki/2019-talk-demo-async. Please feel free to play
around.
Copyright Notices and Acknowledgements
• Any photos used in this presentation is copyrighted by Juti Noppornpitak.
Permitted to use in the video recording published by PyCon Canada.
• Definitions are derived from the documentation published by Microsoft and
Wikipedia.

More Related Content

What's hot

Async discussion 9_29_15
Async discussion 9_29_15Async discussion 9_29_15
Async discussion 9_29_15Cheryl Yaeger
 
Java concurrency in practice
Java concurrency in practiceJava concurrency in practice
Java concurrency in practiceMikalai Alimenkou
 
MERIMeeting du 27 mai 2014 - Parallel Programming
MERIMeeting du 27 mai 2014 - Parallel ProgrammingMERIMeeting du 27 mai 2014 - Parallel Programming
MERIMeeting du 27 mai 2014 - Parallel ProgrammingOlivier NAVARRE
 
Performance measurement methodology — Maksym Pugach | Elixir Evening Club 3
Performance measurement methodology — Maksym Pugach | Elixir Evening Club 3Performance measurement methodology — Maksym Pugach | Elixir Evening Club 3
Performance measurement methodology — Maksym Pugach | Elixir Evening Club 3Elixir Club
 
C# 5 deep drive into asynchronous programming
C# 5 deep drive into asynchronous programmingC# 5 deep drive into asynchronous programming
C# 5 deep drive into asynchronous programmingPraveen Prajapati
 
Introduction to Akka
Introduction to AkkaIntroduction to Akka
Introduction to AkkaKnoldus Inc.
 
CTU June 2011 - C# 5.0 - ASYNC & Await
CTU June 2011 - C# 5.0 - ASYNC & AwaitCTU June 2011 - C# 5.0 - ASYNC & Await
CTU June 2011 - C# 5.0 - ASYNC & AwaitSpiffy
 
Intro to Erlang
Intro to ErlangIntro to Erlang
Intro to ErlangKen Pratt
 
End to-end async and await
End to-end async and awaitEnd to-end async and await
End to-end async and awaitvfabro
 
Asynchronous programming in .net 4.5 with c#
Asynchronous programming in .net 4.5 with c#Asynchronous programming in .net 4.5 with c#
Asynchronous programming in .net 4.5 with c#Binu Bhasuran
 
Java Concurrency, Memory Model, and Trends
Java Concurrency, Memory Model, and TrendsJava Concurrency, Memory Model, and Trends
Java Concurrency, Memory Model, and TrendsCarol McDonald
 
Asynchronous Programming in C# - Part 1
Asynchronous Programming in C# - Part 1Asynchronous Programming in C# - Part 1
Asynchronous Programming in C# - Part 1Mindfire Solutions
 
Async in .NET
Async in .NETAsync in .NET
Async in .NETRTigger
 
Asynchronous I/O in Python 3
Asynchronous I/O in Python 3Asynchronous I/O in Python 3
Asynchronous I/O in Python 3Feihong Hsu
 
Sync, async and multithreading
Sync, async and multithreadingSync, async and multithreading
Sync, async and multithreadingTuan Chau
 
Async and Await on the Server
Async and Await on the ServerAsync and Await on the Server
Async and Await on the ServerDoug Jones
 
Evented Ruby VS Node.js
Evented Ruby VS Node.jsEvented Ruby VS Node.js
Evented Ruby VS Node.jsNitin Gupta
 

What's hot (20)

Async discussion 9_29_15
Async discussion 9_29_15Async discussion 9_29_15
Async discussion 9_29_15
 
Java concurrency in practice
Java concurrency in practiceJava concurrency in practice
Java concurrency in practice
 
MERIMeeting du 27 mai 2014 - Parallel Programming
MERIMeeting du 27 mai 2014 - Parallel ProgrammingMERIMeeting du 27 mai 2014 - Parallel Programming
MERIMeeting du 27 mai 2014 - Parallel Programming
 
Performance measurement methodology — Maksym Pugach | Elixir Evening Club 3
Performance measurement methodology — Maksym Pugach | Elixir Evening Club 3Performance measurement methodology — Maksym Pugach | Elixir Evening Club 3
Performance measurement methodology — Maksym Pugach | Elixir Evening Club 3
 
C# 5 deep drive into asynchronous programming
C# 5 deep drive into asynchronous programmingC# 5 deep drive into asynchronous programming
C# 5 deep drive into asynchronous programming
 
Async await...oh wait!
Async await...oh wait!Async await...oh wait!
Async await...oh wait!
 
Introduction to Akka
Introduction to AkkaIntroduction to Akka
Introduction to Akka
 
CTU June 2011 - C# 5.0 - ASYNC & Await
CTU June 2011 - C# 5.0 - ASYNC & AwaitCTU June 2011 - C# 5.0 - ASYNC & Await
CTU June 2011 - C# 5.0 - ASYNC & Await
 
Intro to Erlang
Intro to ErlangIntro to Erlang
Intro to Erlang
 
End to-end async and await
End to-end async and awaitEnd to-end async and await
End to-end async and await
 
Asynchronous programming in .net 4.5 with c#
Asynchronous programming in .net 4.5 with c#Asynchronous programming in .net 4.5 with c#
Asynchronous programming in .net 4.5 with c#
 
Java Concurrency, Memory Model, and Trends
Java Concurrency, Memory Model, and TrendsJava Concurrency, Memory Model, and Trends
Java Concurrency, Memory Model, and Trends
 
Asynchronous Programming in C# - Part 1
Asynchronous Programming in C# - Part 1Asynchronous Programming in C# - Part 1
Asynchronous Programming in C# - Part 1
 
Async in .NET
Async in .NETAsync in .NET
Async in .NET
 
Asynchronous I/O in Python 3
Asynchronous I/O in Python 3Asynchronous I/O in Python 3
Asynchronous I/O in Python 3
 
Sync, async and multithreading
Sync, async and multithreadingSync, async and multithreading
Sync, async and multithreading
 
Async and Await on the Server
Async and Await on the ServerAsync and Await on the Server
Async and Await on the Server
 
Async/Await
Async/AwaitAsync/Await
Async/Await
 
Evented Ruby VS Node.js
Evented Ruby VS Node.jsEvented Ruby VS Node.js
Evented Ruby VS Node.js
 
Redux
ReduxRedux
Redux
 

Similar to PyCon Canada 2019 - Introduction to Asynchronous Programming

Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareBeyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareMiro Samek
 
Multithreading in Scala
Multithreading in Scala Multithreading in Scala
Multithreading in Scala Knoldus Inc.
 
Concurrency Programming in Java - 01 - Introduction to Concurrency Programming
Concurrency Programming in Java - 01 - Introduction to Concurrency ProgrammingConcurrency Programming in Java - 01 - Introduction to Concurrency Programming
Concurrency Programming in Java - 01 - Introduction to Concurrency ProgrammingSachintha Gunasena
 
Multi threading
Multi threadingMulti threading
Multi threadinggndu
 
Asynchronous and parallel programming
Asynchronous and parallel programmingAsynchronous and parallel programming
Asynchronous and parallel programmingAnil Kumar
 
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareBeyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareQuantum Leaps, LLC
 
Async Web Frameworks in Python
Async Web Frameworks in PythonAsync Web Frameworks in Python
Async Web Frameworks in PythonRyan Johnson
 
Working With Concurrency In Java 8
Working With Concurrency In Java 8Working With Concurrency In Java 8
Working With Concurrency In Java 8Heartin Jacob
 
Asynchronous Python A Gentle Introduction
Asynchronous Python A Gentle IntroductionAsynchronous Python A Gentle Introduction
Asynchronous Python A Gentle IntroductionPyData
 
C# Async/Await Explained
C# Async/Await ExplainedC# Async/Await Explained
C# Async/Await ExplainedJeremy Likness
 
Yeahhhh the final requirement!!!
Yeahhhh the final requirement!!!Yeahhhh the final requirement!!!
Yeahhhh the final requirement!!!olracoatalub
 
5.7 Parallel Processing - Reactive Programming.pdf.pptx
5.7 Parallel Processing - Reactive Programming.pdf.pptx5.7 Parallel Processing - Reactive Programming.pdf.pptx
5.7 Parallel Processing - Reactive Programming.pdf.pptxMohamedBilal73
 
Coding For Cores - C# Way
Coding For Cores - C# WayCoding For Cores - C# Way
Coding For Cores - C# WayBishnu Rawal
 

Similar to PyCon Canada 2019 - Introduction to Asynchronous Programming (20)

Advanced python
Advanced pythonAdvanced python
Advanced python
 
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareBeyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
 
Multithreading in Scala
Multithreading in Scala Multithreading in Scala
Multithreading in Scala
 
Concurrency Programming in Java - 01 - Introduction to Concurrency Programming
Concurrency Programming in Java - 01 - Introduction to Concurrency ProgrammingConcurrency Programming in Java - 01 - Introduction to Concurrency Programming
Concurrency Programming in Java - 01 - Introduction to Concurrency Programming
 
Multi threading
Multi threadingMulti threading
Multi threading
 
Training – Going Async
Training – Going AsyncTraining – Going Async
Training – Going Async
 
Asynchronous and parallel programming
Asynchronous and parallel programmingAsynchronous and parallel programming
Asynchronous and parallel programming
 
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareBeyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
 
Async Web Frameworks in Python
Async Web Frameworks in PythonAsync Web Frameworks in Python
Async Web Frameworks in Python
 
Working With Concurrency In Java 8
Working With Concurrency In Java 8Working With Concurrency In Java 8
Working With Concurrency In Java 8
 
Asynchronous Python A Gentle Introduction
Asynchronous Python A Gentle IntroductionAsynchronous Python A Gentle Introduction
Asynchronous Python A Gentle Introduction
 
Concurrency
ConcurrencyConcurrency
Concurrency
 
Scheduling Thread
Scheduling  ThreadScheduling  Thread
Scheduling Thread
 
Asynchronyin net
Asynchronyin netAsynchronyin net
Asynchronyin net
 
Concurrency and parallel in .net
Concurrency and parallel in .netConcurrency and parallel in .net
Concurrency and parallel in .net
 
C# Async/Await Explained
C# Async/Await ExplainedC# Async/Await Explained
C# Async/Await Explained
 
Yeahhhh the final requirement!!!
Yeahhhh the final requirement!!!Yeahhhh the final requirement!!!
Yeahhhh the final requirement!!!
 
5.7 Parallel Processing - Reactive Programming.pdf.pptx
5.7 Parallel Processing - Reactive Programming.pdf.pptx5.7 Parallel Processing - Reactive Programming.pdf.pptx
5.7 Parallel Processing - Reactive Programming.pdf.pptx
 
Coding For Cores - C# Way
Coding For Cores - C# WayCoding For Cores - C# Way
Coding For Cores - C# Way
 
Concurrency in c#
Concurrency in c#Concurrency in c#
Concurrency in c#
 

Recently uploaded

A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DaySri Ambati
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demoHarshalMandlekar2
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI AgeCprime
 
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your Queries
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your QueriesExploring ChatGPT Prompt Hacks To Maximally Optimise Your Queries
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your QueriesSanjay Willie
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
Fact vs. Fiction: Autodetecting Hallucinations in LLMs
Fact vs. Fiction: Autodetecting Hallucinations in LLMsFact vs. Fiction: Autodetecting Hallucinations in LLMs
Fact vs. Fiction: Autodetecting Hallucinations in LLMsZilliz
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...Scott Andery
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 

Recently uploaded (20)

A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demo
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI Age
 
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your Queries
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your QueriesExploring ChatGPT Prompt Hacks To Maximally Optimise Your Queries
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your Queries
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
Fact vs. Fiction: Autodetecting Hallucinations in LLMs
Fact vs. Fiction: Autodetecting Hallucinations in LLMsFact vs. Fiction: Autodetecting Hallucinations in LLMs
Fact vs. Fiction: Autodetecting Hallucinations in LLMs
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 

PyCon Canada 2019 - Introduction to Asynchronous Programming

  • 1. Introduction to asynchronous programming By Juti Noppornpitak PYCON
 CANADA
 2019
  • 3. Juti Noppornpitak A senior software developer at DNAstack and someone who accidentally become a consultant on scientific application projects @shiroyuki on GitHub and Twitter www.shiroyuki.com
  • 5. Asynchronous programming A way to run code in parallel with the main thread and notifies the main thread when it finishes running, fails to complete, or is still running.
  • 6. Cooperative multitasking Cooperative multitasking, also known as non-preemptive multitasking, is a style of computer multitasking in which the operating system never initiates a context switch from a running process to another process. Instead, processes voluntarily yield control periodically or when idle or logically blocked in order to enable multiple applications to be run concurrently. This type of multitasking is called "cooperative" because all programs must cooperate for the entire scheduling scheme to work. In this scheme, the process scheduler of an operating system is known as a cooperative scheduler, having its role reduced down to starting the processes and letting them return control back to it voluntarily. —Wikipedia
  • 7. Let’s see how it works
  • 8. Let’s start with a piece of simple CPU-bound code. You will notice that, without concurrent programming, this code will spend at least 4 seconds to finish.
  • 9. Now, we spiced things up with threading. Now, you notice that we have to pass a shared dictionary around in order to retrieve the result as func_001 and func_002 cannot return the result back to main.
  • 10. Now, simplified with ThreadPoolExecutor This looks a lot prettier and we don’t need to use
 a shared dictionary to store the value as long as
 we remember to call Future.result(). However, you still may need to check for errors
 and completions manually.
  • 11. Rewritten with coroutines • The main function is now much simpler. • The concurrency is managed by an event loop. • Two CPU-bound code on line 15 and 22 are
 handled with run_in_executor to achieve
 the similar performance. • We uses run_in_executor to avoid
 CPU-bound code, like time.sleep, blocking
 the event loop.
  • 12. async def sum(*args) -> float:
 … result = await coro task = asyncio.create_task(coro) result = await task ← At this stage, sum is still a function. coro = sum(1, 2) However, unlike normal functions,
 ↓ sum(…) returns a coroutine instead. ↑ You can await a coroutine directly.
 Internally, any given coroutine will
 be wrapped as a task. So, here is how we write cooperative multitasking code in Python with coroutines. ↑ This route is preferred. ↑ Create a task of the
 coroutine and schedule
 its execution
  • 13. What do you need to watch out when you write cooperative multitasking code? •Avoiding CPU-bound code as it can block the corresponding event loop. •In multithreading, the thread scheduling and context switching are handled by an operating system and outside the control of Python. •However, cooperative multitasking code manages its own task scheduling and task switching with an event loop.
  • 14. Multithreading vs Cooperative Multitasking def foo() async def foo() Unit of Work Thread Task Scheduler Operating System Event Loop High-level Usage Suppose you have a function: Create a thread targeting foo(); then, start the thread; then, wait for the thread to join back; finally, good luck on fetching the result back. Suppose you have an async function: Just await for the result from a task. Directly call foo()… Whatever is given in return Coroutine of foo()
  • 15. Now, why do you care about
 cooperative multitasking?
  • 16. Code Simplicity • With PEP 492 (async and await), developers can write a cooperative multitasking code as a simple sequence of instructions, but each instruction can be executed in a much more complicated order based on I/O and upstream tasks’ completion. • In comparison to conventional multithreading code, developers have to manage threads on their own.
  • 17. From our previous example This is just a simple “lazy" code.
  • 18. Now, after we parallelize
 the code with just low-level
 thread APIs… Regardless to the logging messages, you start
 to see the ceremonial steps to get target functions
 run in their own thread. We are also responsible to start the threads and
 join them back manually.
  • 19. Even with ThreadPoolExecutor While this looks a lot prettier, we still need to 
 explicitly set up a thread pool.
  • 20. With coroutine, asyncio,
 async, and await We gain the benefit of asynchronicity with
 minimal ceremonial code.
  • 21. Runaway Threads versus Cancellable Tasks
  • 22. Can you cancel or stop an active thread? • The short answer is NO. • There are many suggestions on the internet on how to cancel/kill an active thread. One of a popular suggestion is the magical-but-undocumented Thread._stop(). • Each thread in Python has a state lock, called _tstate_lock, which is only acquired at the end of thread’s life. • Thread._stop() only works when Thread._tstate_lock is acquired. • Therefore, active threads cannot be stopped. • Here is just an observation. Cancelling threads is generally discouraged as it could lead your program to an undesirable state, such as memory management and deadlock.
  • 23. demo/003-001-naive-thread-canceller.py ← This will cause _stop to raise AssertionError.
  • 24. Can we cancel or stop a task? • The short answer is kind of YES by using Task.cancel(). • When cancel() is called, the corresponding coroutine can catch CancelledError (an exception from asyncio) so that the method can run a cleanup procedure. • If the coroutine is cancelled before it starts running, the exception will not be raised. • If the exception is not caught inside the coroutine, it will bubble up to the parent coroutine. • So, as you can see, the cancellation is not guaranteed. • Depending on implementation, a coroutine may suppress the cancellation (CancelledError) and keep running like nothing even happens. • Suppressing CancelledError is discouraged.
  • 25. demo/003-002-naive-task-canceller.py ← 🤔 This line never get called. ← (3) We cancel the task here. ← (1) Create and schedule the coroutine. ← (2) Await the sleeping instruction (and trigger the event loop to start). ← 🙈 This line runs only once. ← (4) Get back to the sleepy brain. ← 🙈 Try to sleep for 6 seconds
  • 26. What about cancel() from concurrent.futures.Future? • It is the same as Thread._stop().
  • 27. Thread Safety (in asyncio)
  • 28. What is thread safety? Thread safety is a computer programming concept applicable to multi-threaded code. Thread-safe code only manipulates shared data structures in a manner that ensures that all threads behave properly and fulfill their design specifications without unintended interaction. There are various strategies for making thread-safe data structures. — Wikipedia
  • 29. Let's closely examine
 BaseEventLoop.call_soon
 from asyncio.base_events • This method is a well-documented case of
 what could go wrong. Can you guess where
 the problem is? • When this method is called simultaneously by
 multiple threads, list operations may throw an
 exception (IndexError). • You can get away with the thread safety issue
 by using call_soon_threadsafe.
  • 30. Should you write asynchronous code? • Generally, if your code requires speed, writing asynchronous code is usually a solution to speed things up. • Multithreading is generally a good approach if your code is CPU-intensive. • Cooperative multitasking is good for a few situations: • Your code needs to be more responsive. • Without context switching, your code does not have to sacrifice some CPU time to switch between tasks. • Running the event loop in a single thread, your code tends to use less memory.* • You can tolerate occasional blockages in the event loop by not-so-intense CPU-bound code. • A cooperative multitasking app is slightly more difficult to design to have an application running as fast as a multithreading app. • Your code is as good as a normal sequential code if some of your coroutines never yield the control back to the event loop. • The placement of await is very important. • In asyncio, when you create tasks of any coroutines, all tasks are scheduled right away. • This means, as soon as your code start awaiting one of the tasks, the other tasks will be executed.
  • 31. Thank you • The source code in the demonstration is belong to the public domain, except the code from CPython, which has its own license. The examples are available at https://github.com/shiroyuki/2019-talk-demo-async. Please feel free to play around.
  • 32. Copyright Notices and Acknowledgements • Any photos used in this presentation is copyrighted by Juti Noppornpitak. Permitted to use in the video recording published by PyCon Canada. • Definitions are derived from the documentation published by Microsoft and Wikipedia.