FastAPI is great, it helps many developers create REST APIs based on the OpenAPI standard and run them asynchronously. It has a thriving community and educational documentation.
FastAPI does a great job of getting people started with APIs quickly.
This talk will point out some obstacles and dark spots that I wish we had known about before. In this talk we want to highlight solutions.
2019 StartIT - Boosting your performance with BlackfireMarko Mitranić
A workshop held in StartIT as part of Catena Media learning sessions.
We aim to dispel the notion that large PHP applications tend to be sluggish, resource-intensive and slow compared to what the likes of Python, Erlang or even Node can do. The issue is not with optimising PHP internals - it's the lack of proper introspection tools and getting them into our every day workflow that counts! In this workshop we will talk about our struggles with whipping PHP Applications into shape, as well as work together on some of the more interesting examples of CPU or IO drain.
Big Data and OpenStack, a Love Story: Michael Still, RackspaceOpenStack
Big Data and OpenStack, a Love Story
Audience: Intermediate
Topic: Storage
Abstract: Increasingly we’re being asked to build out clusters of machines to solve big data problems. These clusters can become quite large, reaching up to thousands of machines. Of course, our operational budgets don’t scale linearly like our machine counts do, and we’re asked to do more and more with less. This talk will explore how organisations around the world are using OpenStack to automate the management of their big data implementations, harnessing interesting characteristics of big data workloads along the way.
Speaker Bio: Michael Still, Rackspace
OpenStack core developer and former Nova PTL, as well as experienced software and reliability engineer. Part of the team that grew Google Mobile to being a billion dollar business. Director of linux.conf.au 2013. Author of The Definitive Guide to ImageMagick (www.imagemagickbook.com) and Practical MythTV (www.mythtvbook.com) from Apress, as well as a bunch of articles.
OpenStack Australia Day Government - Canberra 2016
https://events.aptira.com/openstack-australia-day-canberra-2016/
All Change! How the new economics of Cloud will make you think differently ab...Steve Poole
Devoxxuk talk
http://cfp.devoxx.co.uk/2015/talk/AJY-8768/All_Change!_How_the_new_economics_of_Cloud_will_make_you_think_differently_about_Java
How far have you got with learning about Cloud? Got your head around Platform as a Service? Understand what IaaS means? Can spell Docker? Working in a DevOps mode? It's easy to focus on learning new technology but it's time to take a step back and look at what the technical implications are when an application is heading to the cloud. In the world of the cloud the benefits are high but the economics (financial and technical) can be radically different. Learn more about these new realities and how they can change application design, deployment and support The introduction of Cloud technologies and its rapid adoption creates new opportunities and challenges. Whether designer, developer or tester, this talk will help you to start thinking differently about Java and the Cloud
Use drupal 8 as a framework the romance recalibrationKevin Wenger
Ce talk s’inscrit dans une lignée de démystification de Drupal 8 et vous introduit comment utiliser pleinement son potentiel (sans uniquement cliquer dans l’interface).
La dernière monture de Drupal, le v8 remplaçant émérite d’un vieillissant – néanmoins valeureux – Drupal 7, permet d’ambitieux projet PHP allant d’une plateforme business à un ecommerce en passant par un site de contenu.
Kevin Wenger nous présente un rapide overview des nouvelles API de Drupal 8, partage ses retours d’expériences sur des développement de projets intranet, de plateformes communautaires et de business applications. Il illustre ensuite quelques contre exemple D7 vs D8 et termine par les bonnes pratiques appliquées au quotidien sur les projet Drupal chez Antistatique.
While everyone is talking about ‘stateless’ services as a way to achieve scalability and high availability, the truth is that they are about as real as the unicorns. Building applications and services that way simply pushes the problem further down the stack, which only makes it worse and more difficult to solve (although, on the upside, it might make it somebody else’s problem). This is painfully obvious when building microservices, where each service must truly own its state.
The reality is that you don’t need ‘stateless’ services to either scale out or be fault tolerant — what you really need is a scalable, fault tolerant state management solution that you can build your services around.
In this talk we will discuss how some of the popular microservices frameworks are tackling this problem, and will look at technologies available today that make it possible to build scalable, highly available systems without ‘stateless’ service layers, whether you are building microservices or good ol’ monoliths.
2019 StartIT - Boosting your performance with BlackfireMarko Mitranić
A workshop held in StartIT as part of Catena Media learning sessions.
We aim to dispel the notion that large PHP applications tend to be sluggish, resource-intensive and slow compared to what the likes of Python, Erlang or even Node can do. The issue is not with optimising PHP internals - it's the lack of proper introspection tools and getting them into our every day workflow that counts! In this workshop we will talk about our struggles with whipping PHP Applications into shape, as well as work together on some of the more interesting examples of CPU or IO drain.
Big Data and OpenStack, a Love Story: Michael Still, RackspaceOpenStack
Big Data and OpenStack, a Love Story
Audience: Intermediate
Topic: Storage
Abstract: Increasingly we’re being asked to build out clusters of machines to solve big data problems. These clusters can become quite large, reaching up to thousands of machines. Of course, our operational budgets don’t scale linearly like our machine counts do, and we’re asked to do more and more with less. This talk will explore how organisations around the world are using OpenStack to automate the management of their big data implementations, harnessing interesting characteristics of big data workloads along the way.
Speaker Bio: Michael Still, Rackspace
OpenStack core developer and former Nova PTL, as well as experienced software and reliability engineer. Part of the team that grew Google Mobile to being a billion dollar business. Director of linux.conf.au 2013. Author of The Definitive Guide to ImageMagick (www.imagemagickbook.com) and Practical MythTV (www.mythtvbook.com) from Apress, as well as a bunch of articles.
OpenStack Australia Day Government - Canberra 2016
https://events.aptira.com/openstack-australia-day-canberra-2016/
All Change! How the new economics of Cloud will make you think differently ab...Steve Poole
Devoxxuk talk
http://cfp.devoxx.co.uk/2015/talk/AJY-8768/All_Change!_How_the_new_economics_of_Cloud_will_make_you_think_differently_about_Java
How far have you got with learning about Cloud? Got your head around Platform as a Service? Understand what IaaS means? Can spell Docker? Working in a DevOps mode? It's easy to focus on learning new technology but it's time to take a step back and look at what the technical implications are when an application is heading to the cloud. In the world of the cloud the benefits are high but the economics (financial and technical) can be radically different. Learn more about these new realities and how they can change application design, deployment and support The introduction of Cloud technologies and its rapid adoption creates new opportunities and challenges. Whether designer, developer or tester, this talk will help you to start thinking differently about Java and the Cloud
Use drupal 8 as a framework the romance recalibrationKevin Wenger
Ce talk s’inscrit dans une lignée de démystification de Drupal 8 et vous introduit comment utiliser pleinement son potentiel (sans uniquement cliquer dans l’interface).
La dernière monture de Drupal, le v8 remplaçant émérite d’un vieillissant – néanmoins valeureux – Drupal 7, permet d’ambitieux projet PHP allant d’une plateforme business à un ecommerce en passant par un site de contenu.
Kevin Wenger nous présente un rapide overview des nouvelles API de Drupal 8, partage ses retours d’expériences sur des développement de projets intranet, de plateformes communautaires et de business applications. Il illustre ensuite quelques contre exemple D7 vs D8 et termine par les bonnes pratiques appliquées au quotidien sur les projet Drupal chez Antistatique.
While everyone is talking about ‘stateless’ services as a way to achieve scalability and high availability, the truth is that they are about as real as the unicorns. Building applications and services that way simply pushes the problem further down the stack, which only makes it worse and more difficult to solve (although, on the upside, it might make it somebody else’s problem). This is painfully obvious when building microservices, where each service must truly own its state.
The reality is that you don’t need ‘stateless’ services to either scale out or be fault tolerant — what you really need is a scalable, fault tolerant state management solution that you can build your services around.
In this talk we will discuss how some of the popular microservices frameworks are tackling this problem, and will look at technologies available today that make it possible to build scalable, highly available systems without ‘stateless’ service layers, whether you are building microservices or good ol’ monoliths.
Use Case Patterns for LLM Applications (1).pdfM Waleed Kadous
What are the "use case patterns" for deploying LLMs into production? Understanding these will allow you to spot "LLM-shaped" problems in your own industry.
Demystify Information Security & Threats for Data-Driven Platforms With Cheta...Chetan Khatri
Pragmatic presentation on Penetration testing for Data-Driven Platforms.
Agenda:
- Motivation
- Information Security - Ethics.
- Encryption
- Authentication
- Information Security & Potential threats with Open Source World.
- Find vulnerabilities.
- Checklist before using any Open Source library.
- Vulnerabilities report.
- Penetration Testing for Data-Driven Developments.
Data-Driven Transformation: Leveraging Big Data at Showtime with Apache SparkDatabricks
Interested in learning how Showtime is leveraging the power of Spark to transform a traditional premium cable network into a data-savvy analytical competitor? The growth in our over-the-top (OTT) streaming subscription business has led to an abundance of user-level data not previously available. To capitalize on this opportunity, we have been building and evolving our unified platform which allows data scientists and business analysts to tap into this rich behavioral data to support our business goals. We will share how our small team of data scientists is creating meaningful features which capture the nuanced relationships between users and content; productionizing machine learning models; and leveraging MLflow to optimize the runtime of our pipelines, track the accuracy of our models, and log the quality of our data over time. From data wrangling and exploration to machine learning and automation, we are augmenting our data supply chain by constantly rolling out new capabilities and analytical products to help the organization better understand our subscribers, our content, and our path forward to a data-driven future.
Authors: Josh McNutt, Keria Bermudez-Hernandez
Practical Chaos Engineering will show how to start running chaos experiments in your infrastructure and will try to guide your through the principles of chaos.
Presentation of my talk given in Phoenix Data Conference 2019. In this we will look at challenges with current Apache Hadoop ecosystem
Apache Hadoop is still relevant but way of doing Hadoop and enterprise data architecture has to be re-looked as we enter Cognitive and Cloud Native Era
We need
Architecture that is enabled by common run time layer across on premise and cloud
Architecture that can abstract away dependency and version conflicts with tons of open source machine learning out there. Yarn did not scale up in that aspect until one want to deal with multiple conda environment
Architecture that can enable real Hybrid Cloud and Multi Cloud portability
And many more challenges that one has to overcome to keep architecture simple, infrastructure agile and better utilized
To stay competitive in today's global economy, organizations need to harness data from multiple sources, extract information and then make real-time decisions. Depending on the industry and the organization, Big Data encompasses information from multiple internal and external sources. Capturing, integrating and preparing this data for downstream analysis is often time-consuming and presents a big challenge. Today, organizations are struggling to cobble together different open source software to build an effective data pipeline that can withstand the volume as well as the speed of data ingestion and analysis. Robin relieves customers from the pains of building and maintaining a data pipeline and helps enterprises to make the most of Big Data.
In this presentation, we will focus on how Robin’s containerization platform can be used to:
- Build an agile and elastic data pipeline
- Deploy, scale, and manage the most complex big data applications with just a single click of a button
- Deal with variety, velocity, and volume of enterprise Big Data
Let's say you're a data scientist, and you've been asked to build infrastructure. Here I've distilled some best practices as an introduction for people who are new to DevOps.
Importance of software-defined storage
Agile enterprises run on agile storage
What’s new in Red Hat Storage Server 3
• Petabyte scale capacity
• Enhanced data protection
• Advanced monitoring and logging
In this event, you'll learn how Red Hat Storage Server 3 enables:
• Unlimited scalability for maximum business agility.
• Data security where you need it – on-premise or in
the cloud.
• Elastic capacity and performance to best align to
shifting demands
• Business insights through in-place analytics for
Hadoop workloads.
Serverless in production, an experience report (FullStack 2018)Yan Cui
AWS Lambda has changed the way we deploy and run software, but this new serverless paradigm has created new challenges to old problems - how do you test a cloud-hosted function locally? How do you monitor them? What about logging and config management? And how do we start migrating from existing architectures?
In this talk Yan and Scott will discuss solutions to these challenges by drawing from real-world experience running Lambda in production and migrating from an existing monolithic architecture.
Cowboy dating with big data TechDays at Lohika-2020b0ris_1
The story about things that happen if data platforms are developed not by data engineers, what pitfalls and mistakes can be made.
This will help you to understand what data engineering is about.
Customer Applications Of Hadoop On Red Hat Storage ServerRed_Hat_Storage
"If analyzing data were easy, wouldn’t everybody do it? The fact is, several steps are necessary to effectively use Hadoop out of the box to process data for the information you need. From preprocessing relevant data to preparing it for consumption to copying large data sets to another location for analysis—the time it takes to get value from your data might be so long that the data is no longer relevant.
So why bring your data to the analytics engine when you can bring the analytics engine to your data? In this session, you’ll learn how Red Hat customers use Red Hat Storage Server to more quickly start mining their data by allowing the application that generates the data they need to write directly to where the analytics will run. And in the process of doing so, they can skip making an extra copy of the data."
Building on AWS Cloud One Codebase Aberdeen 25-apr-19Cloudsoft
How do you build a business and applications on AWS? What are the mindsets and frameworks that successful orgs use? What choices do they make, and what do they avoid? What can we learn from other orgs successes with case studies, and how do you get started?
Serverless in production, an experience report (IWOMM)Yan Cui
AWS Lambda has changed the way we deploy and run software, but this new serverless paradigm has created new challenges to old problems - how do you test a cloud-hosted function locally? How do you monitor them? What about logging and config management? And how do we start migrating from existing architectures?
In this talk Yan and Domas will discuss solutions to these challenges by drawing from real-world experience running Lambda in production and migrating from an existing monolithic architecture.
Use Case Patterns for LLM Applications (1).pdfM Waleed Kadous
What are the "use case patterns" for deploying LLMs into production? Understanding these will allow you to spot "LLM-shaped" problems in your own industry.
Demystify Information Security & Threats for Data-Driven Platforms With Cheta...Chetan Khatri
Pragmatic presentation on Penetration testing for Data-Driven Platforms.
Agenda:
- Motivation
- Information Security - Ethics.
- Encryption
- Authentication
- Information Security & Potential threats with Open Source World.
- Find vulnerabilities.
- Checklist before using any Open Source library.
- Vulnerabilities report.
- Penetration Testing for Data-Driven Developments.
Data-Driven Transformation: Leveraging Big Data at Showtime with Apache SparkDatabricks
Interested in learning how Showtime is leveraging the power of Spark to transform a traditional premium cable network into a data-savvy analytical competitor? The growth in our over-the-top (OTT) streaming subscription business has led to an abundance of user-level data not previously available. To capitalize on this opportunity, we have been building and evolving our unified platform which allows data scientists and business analysts to tap into this rich behavioral data to support our business goals. We will share how our small team of data scientists is creating meaningful features which capture the nuanced relationships between users and content; productionizing machine learning models; and leveraging MLflow to optimize the runtime of our pipelines, track the accuracy of our models, and log the quality of our data over time. From data wrangling and exploration to machine learning and automation, we are augmenting our data supply chain by constantly rolling out new capabilities and analytical products to help the organization better understand our subscribers, our content, and our path forward to a data-driven future.
Authors: Josh McNutt, Keria Bermudez-Hernandez
Practical Chaos Engineering will show how to start running chaos experiments in your infrastructure and will try to guide your through the principles of chaos.
Presentation of my talk given in Phoenix Data Conference 2019. In this we will look at challenges with current Apache Hadoop ecosystem
Apache Hadoop is still relevant but way of doing Hadoop and enterprise data architecture has to be re-looked as we enter Cognitive and Cloud Native Era
We need
Architecture that is enabled by common run time layer across on premise and cloud
Architecture that can abstract away dependency and version conflicts with tons of open source machine learning out there. Yarn did not scale up in that aspect until one want to deal with multiple conda environment
Architecture that can enable real Hybrid Cloud and Multi Cloud portability
And many more challenges that one has to overcome to keep architecture simple, infrastructure agile and better utilized
To stay competitive in today's global economy, organizations need to harness data from multiple sources, extract information and then make real-time decisions. Depending on the industry and the organization, Big Data encompasses information from multiple internal and external sources. Capturing, integrating and preparing this data for downstream analysis is often time-consuming and presents a big challenge. Today, organizations are struggling to cobble together different open source software to build an effective data pipeline that can withstand the volume as well as the speed of data ingestion and analysis. Robin relieves customers from the pains of building and maintaining a data pipeline and helps enterprises to make the most of Big Data.
In this presentation, we will focus on how Robin’s containerization platform can be used to:
- Build an agile and elastic data pipeline
- Deploy, scale, and manage the most complex big data applications with just a single click of a button
- Deal with variety, velocity, and volume of enterprise Big Data
Let's say you're a data scientist, and you've been asked to build infrastructure. Here I've distilled some best practices as an introduction for people who are new to DevOps.
Importance of software-defined storage
Agile enterprises run on agile storage
What’s new in Red Hat Storage Server 3
• Petabyte scale capacity
• Enhanced data protection
• Advanced monitoring and logging
In this event, you'll learn how Red Hat Storage Server 3 enables:
• Unlimited scalability for maximum business agility.
• Data security where you need it – on-premise or in
the cloud.
• Elastic capacity and performance to best align to
shifting demands
• Business insights through in-place analytics for
Hadoop workloads.
Serverless in production, an experience report (FullStack 2018)Yan Cui
AWS Lambda has changed the way we deploy and run software, but this new serverless paradigm has created new challenges to old problems - how do you test a cloud-hosted function locally? How do you monitor them? What about logging and config management? And how do we start migrating from existing architectures?
In this talk Yan and Scott will discuss solutions to these challenges by drawing from real-world experience running Lambda in production and migrating from an existing monolithic architecture.
Cowboy dating with big data TechDays at Lohika-2020b0ris_1
The story about things that happen if data platforms are developed not by data engineers, what pitfalls and mistakes can be made.
This will help you to understand what data engineering is about.
Customer Applications Of Hadoop On Red Hat Storage ServerRed_Hat_Storage
"If analyzing data were easy, wouldn’t everybody do it? The fact is, several steps are necessary to effectively use Hadoop out of the box to process data for the information you need. From preprocessing relevant data to preparing it for consumption to copying large data sets to another location for analysis—the time it takes to get value from your data might be so long that the data is no longer relevant.
So why bring your data to the analytics engine when you can bring the analytics engine to your data? In this session, you’ll learn how Red Hat customers use Red Hat Storage Server to more quickly start mining their data by allowing the application that generates the data they need to write directly to where the analytics will run. And in the process of doing so, they can skip making an extra copy of the data."
Building on AWS Cloud One Codebase Aberdeen 25-apr-19Cloudsoft
How do you build a business and applications on AWS? What are the mindsets and frameworks that successful orgs use? What choices do they make, and what do they avoid? What can we learn from other orgs successes with case studies, and how do you get started?
Serverless in production, an experience report (IWOMM)Yan Cui
AWS Lambda has changed the way we deploy and run software, but this new serverless paradigm has created new challenges to old problems - how do you test a cloud-hosted function locally? How do you monitor them? What about logging and config management? And how do we start migrating from existing architectures?
In this talk Yan and Domas will discuss solutions to these challenges by drawing from real-world experience running Lambda in production and migrating from an existing monolithic architecture.
Adjusting primitives for graph : SHORT REPORT / NOTESSubhajit Sahu
Graph algorithms, like PageRank Compressed Sparse Row (CSR) is an adjacency-list based graph representation that is
Multiply with different modes (map)
1. Performance of sequential execution based vs OpenMP based vector multiply.
2. Comparing various launch configs for CUDA based vector multiply.
Sum with different storage types (reduce)
1. Performance of vector element sum using float vs bfloat16 as the storage type.
Sum with different modes (reduce)
1. Performance of sequential execution based vs OpenMP based vector element sum.
2. Performance of memcpy vs in-place based CUDA based vector element sum.
3. Comparing various launch configs for CUDA based vector element sum (memcpy).
4. Comparing various launch configs for CUDA based vector element sum (in-place).
Sum with in-place strategies of CUDA mode (reduce)
1. Comparing various launch configs for CUDA based vector element sum (in-place).
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...pchutichetpong
M Capital Group (“MCG”) expects to see demand and the changing evolution of supply, facilitated through institutional investment rotation out of offices and into work from home (“WFH”), while the ever-expanding need for data storage as global internet usage expands, with experts predicting 5.3 billion users by 2023. These market factors will be underpinned by technological changes, such as progressing cloud services and edge sites, allowing the industry to see strong expected annual growth of 13% over the next 4 years.
Whilst competitive headwinds remain, represented through the recent second bankruptcy filing of Sungard, which blames “COVID-19 and other macroeconomic trends including delayed customer spending decisions, insourcing and reductions in IT spending, energy inflation and reduction in demand for certain services”, the industry has seen key adjustments, where MCG believes that engineering cost management and technological innovation will be paramount to success.
MCG reports that the more favorable market conditions expected over the next few years, helped by the winding down of pandemic restrictions and a hybrid working environment will be driving market momentum forward. The continuous injection of capital by alternative investment firms, as well as the growing infrastructural investment from cloud service providers and social media companies, whose revenues are expected to grow over 3.6x larger by value in 2026, will likely help propel center provision and innovation. These factors paint a promising picture for the industry players that offset rising input costs and adapt to new technologies.
According to M Capital Group: “Specifically, the long-term cost-saving opportunities available from the rise of remote managing will likely aid value growth for the industry. Through margin optimization and further availability of capital for reinvestment, strong players will maintain their competitive foothold, while weaker players exit the market to balance supply and demand.”
As Europe's leading economic powerhouse and the fourth-largest hashtag#economy globally, Germany stands at the forefront of innovation and industrial might. Renowned for its precision engineering and high-tech sectors, Germany's economic structure is heavily supported by a robust service industry, accounting for approximately 68% of its GDP. This economic clout and strategic geopolitical stance position Germany as a focal point in the global cyber threat landscape.
In the face of escalating global tensions, particularly those emanating from geopolitical disputes with nations like hashtag#Russia and hashtag#China, hashtag#Germany has witnessed a significant uptick in targeted cyber operations. Our analysis indicates a marked increase in hashtag#cyberattack sophistication aimed at critical infrastructure and key industrial sectors. These attacks range from ransomware campaigns to hashtag#AdvancedPersistentThreats (hashtag#APTs), threatening national security and business integrity.
🔑 Key findings include:
🔍 Increased frequency and complexity of cyber threats.
🔍 Escalation of state-sponsored and criminally motivated cyber operations.
🔍 Active dark web exchanges of malicious tools and tactics.
Our comprehensive report delves into these challenges, using a blend of open-source and proprietary data collection techniques. By monitoring activity on critical networks and analyzing attack patterns, our team provides a detailed overview of the threats facing German entities.
This report aims to equip stakeholders across public and private sectors with the knowledge to enhance their defensive strategies, reduce exposure to cyber risks, and reinforce Germany's resilience against cyber threats.
Techniques to optimize the pagerank algorithm usually fall in two categories. One is to try reducing the work per iteration, and the other is to try reducing the number of iterations. These goals are often at odds with one another. Skipping computation on vertices which have already converged has the potential to save iteration time. Skipping in-identical vertices, with the same in-links, helps reduce duplicate computations and thus could help reduce iteration time. Road networks often have chains which can be short-circuited before pagerank computation to improve performance. Final ranks of chain nodes can be easily calculated. This could reduce both the iteration time, and the number of iterations. If a graph has no dangling nodes, pagerank of each strongly connected component can be computed in topological order. This could help reduce the iteration time, no. of iterations, and also enable multi-iteration concurrency in pagerank computation. The combination of all of the above methods is the STICD algorithm. [sticd] For dynamic graphs, unchanged components whose ranks are unaffected can be skipped altogether.
Chatty Kathy - UNC Bootcamp Final Project Presentation - Final Version - 5.23...John Andrews
SlideShare Description for "Chatty Kathy - UNC Bootcamp Final Project Presentation"
Title: Chatty Kathy: Enhancing Physical Activity Among Older Adults
Description:
Discover how Chatty Kathy, an innovative project developed at the UNC Bootcamp, aims to tackle the challenge of low physical activity among older adults. Our AI-driven solution uses peer interaction to boost and sustain exercise levels, significantly improving health outcomes. This presentation covers our problem statement, the rationale behind Chatty Kathy, synthetic data and persona creation, model performance metrics, a visual demonstration of the project, and potential future developments. Join us for an insightful Q&A session to explore the potential of this groundbreaking project.
Project Team: Jay Requarth, Jana Avery, John Andrews, Dr. Dick Davis II, Nee Buntoum, Nam Yeongjin & Mat Nicholas
5 Things about fastAPI I wish we had known beforehand
1. 5 THINGS ABOUT FASTAPI.
I WISH WE HAD KNOWN BEFOREHAND
ALEXANDER CS HENDORF.
PYCON.IT 2023
2. THIS TALK IS BASED ON USE CASES.
- DATA MANAGEMENT VIA HUBS / SERVICES.
§ DOMAIN DRIVEN SERVICE CLIENTS.
§ ON PREMS DATA.
§ CLOUD DATA.
§ MIGRATION SZENARIOS.
§ INCLUSIVE ACCESS..
- MICROSERVICES.
3. WHY FASTAPI.
1. FAST AND HIGH-PERFORMANCE WEB FRAMEWORK
2. EASY TO USE AND LEARN WITH INTUITIVE API DOCUMENTATION
3. SCALABLE AND FLEXIBLE ARCHITECTURE FOR BUILDING APIS
4. AUTOMATIC VALIDATION OF REQUEST AND RESPONSE DATA
5. AUTOMATIC GENERATION OF OPENAPI & JSON SCHEMA
DOCUMENTATION
6. BUILT-IN TESTING FRAMEWORK
7. VERY ACTIVE COMMUNITY
4. ALEXANDER C. S. HENDORF
MANAGING PARTNER
DATA & AI AT KÖNIGSWEG
PYTHON SOFTWARE FOUNDATION FELLOW, PYTHON SOFTWAREVERBAND CHAIR,
EUROPYTHON BOARD, PYCONDE & PYDATA BERLIN CHAIR
PYDATA FRANKFURT & SÜDWEST
@HENDORF python@hendorf.com
6. A FASTAPI JOURNEY.
- REQUIREMENT
REST API UTILIZING THE OPENAPI STANDARD
- RESEARCH SUGGETS FASTAPI OR CONNEXION
(CONNEXION WAS GREAT BUT UNFORTUNATELY NEGLECTED…)
- PATH TAKEN:
FASTAPI IS THRIVING AND ALIVE: HELLO WORLD IN 5 MINUTES '
- HIGH TRANSFER FROM PREVIOUS REST FRAMEWORKS LIKE FLASK
- ( „FAST API IS ALL ONE NEEDS TO KNOW“
- BUILD STUFF ONE TUTORIAL AT A TIME )
7. SOME NEXT STEPS
ON THAT JOURNEY.
- ADD MORE ENDPOINTS
- ADD MORE REQUEST MODELS
- ADD MORE RESPONSE MODELS
- ADD COMPLEX MODELS
- ADD VALIDATORS
- BE CONISTENT IN NAMING
- ADD MIDDLEWARE LOGGING
- ADD MIDDLEWARE FOR CONTEXT
- …
8. SOME NEXT STEPS
ON THAT JOURNEY.
- ADD MORE AND EVEN MORE ENDPOINTS
- ADD MORE AND EVEN MORE REQUEST
MODELS
- ADD MORE AND EVEN MORE RESPONSE
MODELS
- ADD EVEN MORE COMPLEX MODELS
- ADD VALIDATORS
- BE CONISTENT IN NAMING
- ADD AND EVEN MORE MIDDLEWARE
- …
10. SOME CHALLENGES ON THAT JOURNEY.
- ONE NEEDS TO LEARN NOT ONLY FASTAPI BUT
§ PYDANTIC
§ STARLETTE
§ UVICORN
- DESIGN: EVERYTHING IS SOLVED BY ANOTHER LAYER OF
ABSTRACTION: UNCLEAR WHICH LIBRARY IS IN CHARGE *
11. SOME CHALLENGES ON THAT JOURNEY.
- NEED FOR SUSTAINABLE DESIGN OF ENDPOINTS
- PROBABLY TOO MANY WAYS TO SOLVE THINGS
- CALLS TO OTHER RESOURCES: `async - await`?
- ESTABLISH A ORGANISATION-WIDE STANDARD
- DRY PYDANTIC MODELS NEED A DATA ARCHTITECT
12. TODAY.
BUILT ON THE SHOULDERS OF GIANTS I..
BUILT ON THE SHOULDERS OF GIANTS II..
GREAT TUTORIALS, BUT IS THIS DOCUMENTATION?.
DON'T REPEAT YOURSELF WITH PYDANTIC.
OPENAPI IN SPHINX.
13. FASTAPI IS BUILT .
ON THE SHOULDERS OF GIANTS I..
PYDANTIC
STARLETTE
15. PYDANTIC.
- PYDANTIC PARSES THE INPUT DATA
- IF THE DATA CANNOT BE CASTED INTO THE TYPE PYDANTIC RAISES A
USER FRIEDLY EXCEPTION
- ["1", "2", "3"] is a valid List[int] -> [1, 2, 3]
h"ps://docs.pydan.c.dev
16. PYDANTIC MISUNDERSTOOD.
- VALIDATOR REQUIRES A RETURN VALUE +
- CAN ALSO BE USED TO MANGLE DATA OR DATA CLEANSING ,
- MORE ABOUT VALIDATORS LATER!
17. PYDANTIC DEFAULTS.
- USE CASE: FRONT END RETIEVES DATA FROM REST BACKEND -
- FE DESIGNER ADDS AN ADDITIONAL PARAMETER currency TO THE
REQUEST
- REQUEST AND RESPONSE MODELS IN PLACE!
- . WE ARE SAVE, AREN'T WE
18. !KNOW PYDANTIC DEFAULTS.
- FE DESIGNER: THINKS EVERYTHING WORKS, NO ERROR
- BACKEND: DOESN'T NOTICE, DEFAULT IS ` Extra.allow` : UNKNOW
PARAMETERS ARE JUST FILTERED OUT
- `Extra.forbid` ENFORCES CONTRACT
- DESIGN DECISION
19. FASTAPI IS BUILT .
ON THE SHOULDERS OF GIANTS II..
PYDANTIC
STARLETTE
20. ⭐ STARLETTE.
- "STARLETTE IS A LIGHTWEIGHT ASGI FRAMEWORK/TOOLKIT, WHICH IS
IDEAL FOR BUILDING ASYNC WEB SERVICES IN PYTHON."
§ HTTP WEB FRAMEWORK
§ WEBSOCKET SUPPORT
§ IN-PROCESS BACKGROUND TASKS
§ STARTUP AND SHUTDOWN EVENTS
§ CORS, GZIP, STATIC FILES, STREAMING RESPONSES
§ SESSION AND COOKIE SUPPORT
22. - CUSTOM PYTHON PACKAGE: INTERFACE TO THE DATA HUB
- DATA CLIENT PROVIDES CONVENIENT, SELF-EXPLANATORY INTERFACES
TO DATA SERVICES ENABLING DEVELOPERS AS WELL AS BUSINESS
USERS.
- client.portfolio.benchmark(id, benchmark=None)
- DATA HUB FASTAPI BE
- ENDPOINT: portfolio/benchmark
Request Model: id, benchmark
- HOW CAN WE GENERATE THE CLIENT AUTOMATICALLY?.
USE CASE: DOMAIN DRIVEN DATA CLIENT.
23. - CUSTOM PYTHON PACKAGE: INTERFACE TO THE DATA HUB
- DATA CLIENT PROVIDES CONVENIENT, SELF-EXPLANATORY INTERFACES
TO DATA SERVICES ENABLING DEVELOPERS AS WELL AS BUSINESS
USERS.
- client.portfolio.benchmark(id, benchmark=None)
- DATA HUB FASTAPI BE
- ENDPOINT: portfolio/benchmark
Request Model: id, benchmark
- HOW CAN WE GENERATE THE CLIENT AUTOMATICALLY?.
USE CASE: DOMAIN DRIVEN DATA CLIENT.
24.
25. USE CASE: DOMAIN DRIVEN DATA CLIENT.
- TLDR;
AUTO-GENERATORS BASED ON OPENAPI DID NOT DELIVER CODE
QUALITY EXPECTED
LOSS OF PYTHON OBJECTS OPENAPI IS JSON/YAML
- ✅ SOLUTION: RENDER CODE MEETING OUR STANDARDS WITH JINJA2.
- ISSUE: PATH PARAMETERS AND MODEL PARAMETERS ARE HARD TO
DISTISTINGUISH (AMBIGUTITIES 0)
Q: EG. CAN MIDDLEWARE FIX THIS??
- NEXT STEP: DIVE INTO MIDDLEWARE, WELCOME TO ⭐ STARLETTE !
26. USE CASE: ENABLING USERS BY SIMPIFING
INTERFACES WITH MIDDLEWARE.
- DON'T TRY THIS AT HOME
- SPIKE: MIDDLEWARE INSPECTS ALL GET REQUESTS ROUTES (INCL. THE
PYDANTIC MODEL) AND MOVES BODY PARAMETERS FROM QUERIES TO
THE REQUEST BODY.
- ✅ MODERATE / ALTER REQUESTS VIA THE MIDDLEWARE
- ✅ HANDLING IS COMPLEX. SOME (FIXABLE BUT UNFORESEEN)
COLLATERAL DAMAGES
- 2 THIS APPROACH FOR A SIMPLER INTERFACE FOR QUERIES WITH
LONGER PARAMETERS DID NOT MEET OUR EXPECTATIONS.
28. LET THE DOCSTRINGS OUT.
- FASTAPI 0.95.1 HAS 6022 LINES *
- 259 CLASSES
- 429 FUNCTION OR METHODS
- 11 DOCSTRINGS
- DUE TO DESIGN WITH MANY LAYERS OF ABSTRACTION,
IT'S DIFFICULT TO DEBUG 3
- MORE DOCSTRINGS WOULD BE HELPFUL.
* COUNT EVERY LINE IN FASTAPI: *.PY no dependencies, COUNT """ / 2, COUNT def, COUNT class
29. LET THE DOCSTRINGS OUT.
- STARLETTE 0.95.1 HAS 6022 LINES *
- 259 CLASSES
- 429 FUNCTION OR METHODS
- 11 DOCSTRINGS
- DUE TO DESIGN WITH MANY LAYERS OF ABSTRACTION,
IT'S DIFFICULT TO DEBUG 3
- MORE DOCSTRINGS WOULD BE HELPFUL.
* COUNT EVERY LINE IN FASTAPI: *.PY no dependencies, COUNT """ / 2, COUNT def, COUNT class
33. PYDANTIC INTERITANCE.
- DECOMPOSITIONS INTO SMALL UNITS
§ DRY: DEFINE ONCE INCL. EXAMPLES & VALIDATORS
§ CONSTENCY ACROSS PROJECT OR EVEN ORGANISATION EG. IF
RELEASED AS PACKAGE
- INTERITANCE IS PROBABLY COUNTER INTUITIVE (RIGHT TO LEFT)
35. USE CASE: DEFINING MODELS FOR AN
ENTIRE ORGANISATION.
- ⭐ INTRODUCE AND ENABLE WITH STANDARDIZARTION 4
- RELEASED AS INTERNAL PACKAGE 5
- 6 CLEAR AND CONCISE DESIGN RULES INCL. NAMING CONVENTIONS
- CENTRALISATION: UPDATES NEED TO BE COORDINATED 7
- LESSONS LEARNED:
§ FOCUS ON DEFINED, CROSS ORGANISATION ATTRIBUTES
§ HAVE SERVICES DESIGN CUSTOM MODELS
36. 5 THINGS ABOUT FASTAPI WE ❤.
ACCESSIBLILITY VIA INTROSPECTION IS GREAT.
DYNAMIC CREATION OF ROUTERS.
CUSTOM VALIDATON ERROR HANDLING .
MIDDLEWARE EASY TO ADD.
5 MORE THINGS !!.
37. OPENAPI IN SPHINX.
- sphinxcontrib-openapi EXTENSION 8
- DOCS: h?ps://sphinxcontrib-openapi.readthedocs.io/
- ADD TO SPHINX EXTENSIONS (JUST AS OTHERS)
- EXPORT OPENAPI DEF IN YAML
- ADD TO YOUR INDEX %
38.
39. TALK
What does Starlette really
do for FastAPI?
Saturday, May 27
16:00 - 16:30
LANGUAGE
English
AUDIENCE LEVEL
Intermediate
ELEVATOR PITCH
FastAPI has been growing a lot lately, but people aren’t really aware of the
packages that were the basis for its foundation.
On this talk, we are going to explore Starlette, the web framework on which
FastAPI is built on top of.
ABSTRACT
Starlette is the package that contains the core logic of FastAPI. It’s a timid, and less known
project in comparison to the latter.
We’ll understand what Starlette really is, and what it does for FastAPI. We are going to
explore both frameworks in a comparative way, to understand the things that are
responsibility from one package and the other.
After this talk, you are going to understand more about the internals of one of the most
popular web frameworks in Python.
TAGS
AsyncIO, APIs, Microservices, FastAPI, Starlette
Marcelo Trylesinski
16:00 - 16:30
LANGUAGE
English
AUDIENCE LEVEL
Intermediate
ELEVATOR PITCH
FastAPI has been growing a lot lately, but people aren’t really aware of the
packages that were the basis for its foundation.
On this talk, we are going to explore Starlette, the web framework on which
FastAPI is built on top of.
ABSTRACT
Starlette is the package that contains the core logic of FastAPI. It’s a timid, and less known
project in comparison to the latter.
We’ll understand what Starlette really is, and what it does for FastAPI. We are going to
explore both frameworks in a comparative way, to understand the things that are
responsibility from one package and the other.
After this talk, you are going to understand more about the internals of one of the most
popular web frameworks in Python.
TAGS
AsyncIO, APIs, Microservices, FastAPI, Starlette
Marcelo Trylesinski
Software Engineer
Uvicorn & Starlette maintainer
FastAPI Expert ⚡
40. THANK YOU!!
Q&A!
@HENDORF / LINKEDIN
THANKS TO MY COLLEAGUES AND PEERS FOR ALL INPUT AND DISCUSSIONS ON
RESULTS PRESENTED HERE: ALEX S. DIMITRI S. FLORIAN P. STEPHAN G.!
THANKS TO LAMBDALABS FOR SHARING STABLE DIFFUSION IMAGE
VARIATIONS $ THANKS TO MAX ERNST FOR PAINTING $