Since Python 3.5 and PEP 492, we have been able to write asynchronous programs in an easy and Pythonic way without external libraries. Even so, it is still difficult to understand what asynchronous programming is all about and when we, Python developers, should consider using it. This talk will give you a gentle introduction to the world of asynchronous programming, focusing mostly on the core concept of async programming, how it works, and what its applications are, in order to provide a good foundation to Python developers on the topic. On top of that, we will explore a small code example (mostly involving the built-in asyncio) and briefly exam the source code of CPython to find out how it works. This talk will also give you some brief comparison of threading.Thread and ThreadPoolExecutor.
Every Java developer knows that multithreading is the root of all evil and it is quite hard to write correct code for concurrent environment. But what tasks do exist in real commercial development except running code in asynchronous way?
In this talk I will present several tasks from my real projects and solutions we designed for them. This talk is very application oriented and allows participants to extend their vision of concurrent programming.
Performance measurement methodology — Maksym Pugach | Elixir Evening Club 3Elixir Club
Доповідь Максима Пугача, Team Lead/Software Engineer at LITSLINK, на Elixir Evening Club 3, Kyiv, 13.12.2018
Наступна конференція - http://www.elixirkyiv.club/
A boss of mine once told me "Just see, my poorly written Vert.x app outperforms my poorly written Elixir app". Now it is time to take up the gauntlet.
Cлідкуйте за нами у соцмережах @ElixirClubUA та #ElixirClubUA
Анонси та матеріали конференцій - https://www.fb.me/ElixirClubUA
Новини - https://twitter.com/ElixirClubUA
Фото та невимушена атмосфера - https://www.instagram.com/ElixirClubUA
*Канал організаторів мітапа - https://t.me/incredevly
(Given to the Vancouver Erlang and Ruby/Rails Meetup groups on May 19, 2009.)
Erlang is an up-and-coming language on the web scene. New libraries and frameworks are sprouting up at a rampant rate, and web giants Facebook and Twitter are using it to develop highly-scalable web applications.
This talk will introduce Erlang as a language and platform, summarize its strengths and weaknesses, and cover how you can use Erlang and Ruby together to conquer the web frontier.
Speaker Bio:
Ken Pratt has been developing software for the web for over 10 years. He fell in love with Ruby four years ago, but is still passionate about learning other languages and platforms. He has developed scalable web services for Electronic Arts, built Rails-based web applications since pre-1.0, and been featured in interactive art installations.
All about asynchronous programming from .NET starting from initial framework to latest version. We will be looking into almost all features from .NET which are related to Asynchronous Programming. This seminar will cover up evolution of asynchronous programming and slowly moving to Task.
Feihong talks about PEP 3156 and basic usage of Tulip, the reference implementation.
Video: http://pyvideo.org/video/2194/asynchronous-io-in-python-3
Source code: https://github.com/feihong/tulip-talk/
In 5-2020, I and Minh have a sharing session about Sync, Async, and Multi-threading in C#.
We believe this is one of the most important things when working with .Net technology.
This sharing is aiming to Junior Dev, or even Sr Dev or anyone interested in Sync, Async, and Multi-threading.
I hope it will bring you some values.
This is the Async / Await feature added to .NET in .NET 4.5, specifically...Everything I Wish I Knew When I Started Using It! By avoiding the client side discussions around the UI and parallel processing, we can focus on the environment in which most of us live and have both an introduction and deeper dive into how it all works. This is about how we can all use the feature RIGHT NOW to write better performing code.
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareMiro Samek
Embedded software developers from different industries are independently re-discovering patterns for building concurrent software that is safer, more responsive and easier to understand than naked threads of a Real-Time Operating System (RTOS). These best practices universally favor event-driven, asynchronous, non-blocking, encapsulated active objects with state machines instead of naked, blocking RTOS threads. This presentation explains the concepts related to this increasingly popular "reactive approach", and specifically how they apply to real-time embedded systems.
Every Java developer knows that multithreading is the root of all evil and it is quite hard to write correct code for concurrent environment. But what tasks do exist in real commercial development except running code in asynchronous way?
In this talk I will present several tasks from my real projects and solutions we designed for them. This talk is very application oriented and allows participants to extend their vision of concurrent programming.
Performance measurement methodology — Maksym Pugach | Elixir Evening Club 3Elixir Club
Доповідь Максима Пугача, Team Lead/Software Engineer at LITSLINK, на Elixir Evening Club 3, Kyiv, 13.12.2018
Наступна конференція - http://www.elixirkyiv.club/
A boss of mine once told me "Just see, my poorly written Vert.x app outperforms my poorly written Elixir app". Now it is time to take up the gauntlet.
Cлідкуйте за нами у соцмережах @ElixirClubUA та #ElixirClubUA
Анонси та матеріали конференцій - https://www.fb.me/ElixirClubUA
Новини - https://twitter.com/ElixirClubUA
Фото та невимушена атмосфера - https://www.instagram.com/ElixirClubUA
*Канал організаторів мітапа - https://t.me/incredevly
(Given to the Vancouver Erlang and Ruby/Rails Meetup groups on May 19, 2009.)
Erlang is an up-and-coming language on the web scene. New libraries and frameworks are sprouting up at a rampant rate, and web giants Facebook and Twitter are using it to develop highly-scalable web applications.
This talk will introduce Erlang as a language and platform, summarize its strengths and weaknesses, and cover how you can use Erlang and Ruby together to conquer the web frontier.
Speaker Bio:
Ken Pratt has been developing software for the web for over 10 years. He fell in love with Ruby four years ago, but is still passionate about learning other languages and platforms. He has developed scalable web services for Electronic Arts, built Rails-based web applications since pre-1.0, and been featured in interactive art installations.
All about asynchronous programming from .NET starting from initial framework to latest version. We will be looking into almost all features from .NET which are related to Asynchronous Programming. This seminar will cover up evolution of asynchronous programming and slowly moving to Task.
Feihong talks about PEP 3156 and basic usage of Tulip, the reference implementation.
Video: http://pyvideo.org/video/2194/asynchronous-io-in-python-3
Source code: https://github.com/feihong/tulip-talk/
In 5-2020, I and Minh have a sharing session about Sync, Async, and Multi-threading in C#.
We believe this is one of the most important things when working with .Net technology.
This sharing is aiming to Junior Dev, or even Sr Dev or anyone interested in Sync, Async, and Multi-threading.
I hope it will bring you some values.
This is the Async / Await feature added to .NET in .NET 4.5, specifically...Everything I Wish I Knew When I Started Using It! By avoiding the client side discussions around the UI and parallel processing, we can focus on the environment in which most of us live and have both an introduction and deeper dive into how it all works. This is about how we can all use the feature RIGHT NOW to write better performing code.
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareMiro Samek
Embedded software developers from different industries are independently re-discovering patterns for building concurrent software that is safer, more responsive and easier to understand than naked threads of a Real-Time Operating System (RTOS). These best practices universally favor event-driven, asynchronous, non-blocking, encapsulated active objects with state machines instead of naked, blocking RTOS threads. This presentation explains the concepts related to this increasingly popular "reactive approach", and specifically how they apply to real-time embedded systems.
Concurrency Programming in Java - 01 - Introduction to Concurrency ProgrammingSachintha Gunasena
This session discusses a basic high-level introduction to concurrency programming with Java which include:
programming basics, OOP concepts, concurrency, concurrent programming, parallel computing, concurrent vs parallel, why concurrency, real world example, terms, Moore's Law, Amdahl's Law, types of parallel computation, MIMD Variants, shared memory model, distributed memory model, client server model, scoop mechanism, scoop preview - a sequential program, in a concurrent setting - using scoop, programming then & now, sequential programming, concurrent programming,
Since the introduction of C#, async/await concepts are still misunderstood by many developers.
Async programming tries to solve three problems (Offloading, Concurrency, Scalability) in a mean abstraction.
This presentation is a good starting point to asynchronous programming in .net. There are many links and references, so do not hesitate to go deeper.
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareQuantum Leaps, LLC
Embedded software developers from different industries are independently re-discovering patterns for building concurrent software that is safer, more responsive and easier to understand than naked threads of a Real-Time Operating System (RTOS). These best practices universally favor event-driven, asynchronous, non-blocking, encapsulated state machines instead of naked, blocking RTOS threads. This presentation explains the concepts related to this increasingly popular "reactive approach", and specifically how they apply to real-time embedded systems.
Aim of this presentation is not to make you masters in Java 8 Concurrency, but to help you guide towards that goal. Sometimes it helps just to know that there is some API that might be suitable for a particular situation. Make use of the pointers given to search more and learn more on those topics. Refer to books, Java API Documentation, Blogs etc. to learn more. Examples and demos for all cases discussed will be added to my blog www.javajee.com.
By James Kirk Cropcho
PyData New York City 2017
Want to start learning asynchronous programming techniques, but you’re feeling blocked? In this talk, I will explain asynchronous execution. Then, using assorted asynchronous libraries and frameworks, I’ll display and discuss different implementations of a realistic application.
Thread vs Process
scheduling
synchronization
The thread begins execution with the C/C run-time library startup code.
The startup code calls your main or WinMain and execution continues until the main function returns and the C/C library code calls ExitProcess.
Reflection is the ability of a managed code to read its own metadata for the purpose of finding assemblies, modules and type information at runtime. The classes that give access to the metadata of a running program are in System.Reflection.
System.Reflection namespace defines the following types to analyze the module's metadata of an assembly:
Assembly, Module, Enum, ParameterInfo, MemberInfo, Type, MethodInfo, ConstructorInfo, FieldInfo, EventInfo, and PropertyInfo
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
3. Juti Noppornpitak
A senior software developer at DNAstack
and someone who accidentally become a
consultant on scientific application projects
@shiroyuki on GitHub and Twitter
www.shiroyuki.com
5. Asynchronous programming
A way to run code in parallel with the main thread and notifies
the main thread when it finishes running, fails to complete, or
is still running.
6. Cooperative multitasking
Cooperative multitasking, also known as non-preemptive multitasking, is a style
of computer multitasking in which the operating system never initiates a context switch from
a running process to another process. Instead, processes voluntarily yield
control periodically or when idle or logically blocked in order to enable multiple applications
to be run concurrently. This type of multitasking is called "cooperative" because all programs
must cooperate for the entire scheduling scheme to work. In this scheme, the process
scheduler of an operating system is known as a cooperative scheduler, having its role
reduced down to starting the processes and letting them return control back to it voluntarily.
—Wikipedia
8. Let’s start with a piece of
simple CPU-bound code.
You will notice that, without concurrent
programming, this code will spend at least
4 seconds to finish.
9. Now, we spiced things up
with threading.
Now, you notice that we have to pass a shared
dictionary around in order to retrieve the result
as func_001 and func_002 cannot return the
result back to main.
10. Now, simplified with
ThreadPoolExecutor
This looks a lot prettier and we don’t need to use
a shared dictionary to store the value as long as
we remember to call Future.result().
However, you still may need to check for errors
and completions manually.
11. Rewritten with coroutines
• The main function is now much simpler.
• The concurrency is managed by an event loop.
• Two CPU-bound code on line 15 and 22 are
handled with run_in_executor to achieve
the similar performance.
• We uses run_in_executor to avoid
CPU-bound code, like time.sleep, blocking
the event loop.
12. async def sum(*args) -> float:
…
result = await coro
task = asyncio.create_task(coro)
result = await task
← At this stage, sum is still a function.
coro = sum(1, 2)
However, unlike normal functions,
↓ sum(…) returns a coroutine instead.
↑ You can await a coroutine directly.
Internally, any given coroutine will
be wrapped as a task.
So, here is how we write cooperative multitasking code in Python with coroutines.
↑ This route is preferred.
↑ Create a task of the
coroutine and schedule
its execution
13. What do you need to watch out when you write
cooperative multitasking code?
•Avoiding CPU-bound code as it can block the corresponding event loop.
•In multithreading, the thread scheduling and context switching are handled by an
operating system and outside the control of Python.
•However, cooperative multitasking code manages its own task scheduling and
task switching with an event loop.
14. Multithreading vs Cooperative Multitasking
def foo() async def foo()
Unit of Work Thread Task
Scheduler Operating System Event Loop
High-level Usage Suppose you have a function:
Create a thread targeting foo();
then, start the thread;
then, wait for the thread to join back;
finally, good luck on fetching the result
back.
Suppose you have an async function:
Just await for the result from a task.
Directly call foo()… Whatever is given in return Coroutine of foo()
15. Now, why do you care about
cooperative multitasking?
16. Code Simplicity
• With PEP 492 (async and await), developers can write a cooperative
multitasking code as a simple sequence of instructions, but each instruction can
be executed in a much more complicated order based on I/O and upstream tasks’
completion.
• In comparison to conventional multithreading code, developers have to manage
threads on their own.
18. Now, after we parallelize
the code with just low-level
thread APIs…
Regardless to the logging messages, you start
to see the ceremonial steps to get target functions
run in their own thread.
We are also responsible to start the threads and
join them back manually.
22. Can you cancel or stop an active thread?
• The short answer is NO.
• There are many suggestions on the internet on how to cancel/kill an active thread. One of a
popular suggestion is the magical-but-undocumented Thread._stop().
• Each thread in Python has a state lock, called _tstate_lock, which is only acquired at the
end of thread’s life.
• Thread._stop() only works when Thread._tstate_lock is acquired.
• Therefore, active threads cannot be stopped.
• Here is just an observation. Cancelling threads is generally discouraged as it could lead
your program to an undesirable state, such as memory management and deadlock.
24. Can we cancel or stop a task?
• The short answer is kind of YES by using Task.cancel().
• When cancel() is called, the corresponding coroutine can catch CancelledError (an
exception from asyncio) so that the method can run a cleanup procedure.
• If the coroutine is cancelled before it starts running, the exception will not be raised.
• If the exception is not caught inside the coroutine, it will bubble up to the parent coroutine.
• So, as you can see, the cancellation is not guaranteed.
• Depending on implementation, a coroutine may suppress the cancellation (CancelledError)
and keep running like nothing even happens.
• Suppressing CancelledError is discouraged.
25. demo/003-002-naive-task-canceller.py
← 🤔 This line never get called.
← (3) We cancel the task here.
← (1) Create and schedule the coroutine.
← (2) Await the sleeping instruction (and trigger the event loop to start).
← 🙈 This line runs only once.
← (4) Get back to the sleepy brain.
← 🙈 Try to sleep for 6 seconds
26. What about cancel() from
concurrent.futures.Future?
• It is the same as Thread._stop().
28. What is thread safety?
Thread safety is a computer programming concept applicable to multi-threaded code.
Thread-safe code only manipulates shared data structures in a manner that ensures
that all threads behave properly and fulfill their design specifications without
unintended interaction. There are various strategies for making thread-safe data
structures.
— Wikipedia
29. Let's closely examine
BaseEventLoop.call_soon
from asyncio.base_events
• This method is a well-documented case of
what could go wrong. Can you guess where
the problem is?
• When this method is called simultaneously by
multiple threads, list operations may throw an
exception (IndexError).
• You can get away with the thread safety issue
by using call_soon_threadsafe.
30. Should you write asynchronous code?
• Generally, if your code requires speed, writing asynchronous code is usually a solution to speed things up.
• Multithreading is generally a good approach if your code is CPU-intensive.
• Cooperative multitasking is good for a few situations:
• Your code needs to be more responsive.
• Without context switching, your code does not have to sacrifice some CPU time to switch between tasks.
• Running the event loop in a single thread, your code tends to use less memory.*
• You can tolerate occasional blockages in the event loop by not-so-intense CPU-bound code.
• A cooperative multitasking app is slightly more difficult to design to have an application running as fast as a multithreading app.
• Your code is as good as a normal sequential code if some of your coroutines never yield the control back to the event loop.
• The placement of await is very important.
• In asyncio, when you create tasks of any coroutines, all tasks are scheduled right away.
• This means, as soon as your code start awaiting one of the tasks, the other tasks will be executed.
31. Thank you
• The source code in the demonstration is belong to the public domain, except the
code from CPython, which has its own license. The examples are available at
https://github.com/shiroyuki/2019-talk-demo-async. Please feel free to play
around.
32. Copyright Notices and Acknowledgements
• Any photos used in this presentation is copyrighted by Juti Noppornpitak.
Permitted to use in the video recording published by PyCon Canada.
• Definitions are derived from the documentation published by Microsoft and
Wikipedia.