SlideShare a Scribd company logo
Python at Scale
Concurrency at Beeswax
Ron Rothman
Advertiser*
Ad Exchange
Advertiser*
Beeswax
Advertiser*
Advertiser*
ad
❕ 200 ms
❕ 2 million per second
❕ non-infinite $ budget
❕ 99.99% uptime
❕ ~optimal bids
ad request bid requests
bids
Event Collection
Ad
Exchange Bidder
Event
Collector
Event Log
Stream
BeeswaxInternet
Event Collection
Ad
Exchange Bidder
Event
Collector
Event Log
Stream
BeeswaxInternet
Event Collection
Ad
Exchange Bidder
Event
Collector
Event Log
Stream
Load O(10K/sec)
Response time < 250 ms p99
Availability ♾️ Nines
1. Durably record the event
2. Update counters (database)
1
2
def handle_request(req):
'''Process an incoming event.'''
unmarshall(req) # processing
validate(req) # processing
# some business logic # processing
record_event(req) # network i/o
update_db_counters(req) # network i/o
return_response()
work
work
work
wait
wait
10 requests/sec
(100 ms per request)
def handle_request(req):
'''Process an incoming event.'''
unmarshall(req) # processing
validate(req) # processing
# some business logic # processing
record_event(req) # network i/o
update_db_counters(req) # network i/o
return_response()
work
work
work
wait
wait
Layers of Concurrency
Many machines EC2; Autoscale groups
Many processes Preforking web servers
Many threads Greenlets; asyncio
Containers ECS tasks
Serverless Lambdas
Elastic Load Balancer
EC2 EC2 EC2 EC2 EC2
autoscale
group
event
notifications
(HTTP requests)
EC2 instance
event
notifications
(from LB)
web server
processes
web server
process
greenlets/threads
requests
Threads or Greenlets?
Threads Greenlets
Preemptive Cooperative
Requires extensive locking Requires no locking
Lightweight Very lightweight
Leverage multiple cores* Single core
*Sadly, untrue for CPython
Gevent
● Create & manipulate greenlets
● Allows you to do non-blocking i/o
def handle_request(req):
'''Process an incoming event.'''
unmarshall(req) # processing
validate(req) # processing
# some business logic # processing
record_event(req) # network i/o
update_db_counters(req) # network i/o
return_response()
work
work
work
wait
wait
Gevent
● Create & manipulate greenlets
● Allows you to do non-blocking i/o
● Makes (i/o) libraries that you use non-blocking!
○ "monkey patching"
def handle_request(req):
'''Process an incoming event.'''
unmarshall(req) # processing
validate(req) # processing
# some business logic # processing
record_event(req) # network i/o
update_db_counters(req) # network i/o
return_response()
work
work
work
yields
yields
Happily Ever After?
Reality def handle_request(req):
'''Process an incoming event.'''
unmarshall(req)
decrypt(req)
validate(req)
check_whether_duplicate(req) # BLOCKING i/o
# some business logic
# more business logic
# even more business logic
record_event(req) # nonblocking i/o
update_db_counters(req) # BLOCKING i/o
update_other_db_counters(req) # BLOCKING i/o
update_other_other_db_counters(req) # BLOCKING i/o
return_response()
Reality def handle_request(req):
'''Process an incoming event.'''
unmarshall(req)
decrypt(req)
validate(req)
check_whether_duplicate(req) # BLOCKING i/o
# some business logic
# more business logic
# even more business logic
record_event(req) # nonblocking i/o
update_db_counters(req) # BLOCKING i/o
update_other_db_counters(req) # BLOCKING i/o
update_other_other_db_counters(req) # BLOCKING i/o
return_response()
Reality def handle_request(req):
'''Process an incoming event.'''
unmarshall(req)
decrypt(req)
validate(req)
check_whether_duplicate(req) # BLOCKING i/o
# some business logic
# more business logic
# even more business logic
record_event(req) # nonblocking i/o
update_db_counters(req) # BLOCKING i/o
update_other_db_counters(req) # BLOCKING i/o
update_other_other_db_counters(req) # BLOCKING i/o
return_response()
Reality def handle_request(req):
'''Process an incoming event.'''
unmarshall(req)
decrypt(req)
validate(req)
check_whether_duplicate(req) # BLOCKING i/o
# some business logic
# more business logic
# even more business logic
record_event(req) # nonblocking i/o
update_db_counters(req) # BLOCKING i/o
update_other_db_counters(req) # BLOCKING i/o
update_other_other_db_counters(req) # BLOCKING i/o
return_response()
Redis client ✅ pure python
DynamoDB client ✅ pure python
Aerospike client ⛔ wrapped C code
Which DB Libraries Can Be Monkey Patched?
Too Much Processing? Yield Often.
def handle_request(req):
'''Process an incoming event.'''
unmarshall(req)
decrypt(req)
greenlet_yield()
validate(req)
check_whether_duplicate(req) # BLOCKING i/o
# some business logic
# more business logic
greenlet_yield()
# even more business logic
record_event(req) # nonblocking i/o
...
Blocking C Code? Batch & Timeouts.
def handle_request(req):
'''Process an incoming event.'''
unmarshall(req)
decrypt(req)
validate(req)
# BLOCKING i/o
check_whether_duplicate(req, timeout_ms=5, max_tries=3)
# business logic
record_event(req) # nonblocking i/o
update_counters_in_memory(req) # occasional i/o
return_response()
� Blocking I/O calls waste CPU
� You're not as I/O bound as you think hope
� C extensions play by different rules
● Blocking I/O calls waste CPU
■ Gevent + monkey patch
● You're not as I/O bound as you think hope
■ Buffer & batch
● C extensions play by different rules
■ Short timeouts w/retries
Thank You 🙏�
ron {at} beeswax.com
References
● Gevent
● Falcon
● Bottle
● The Sharp Corners of Gevent
● Beeswax

More Related Content

Similar to Concurrent Python at Beeswax - Ron Rothman - NYC Python Meetup 2020

Live Streaming & Server Sent Events
Live Streaming & Server Sent EventsLive Streaming & Server Sent Events
Live Streaming & Server Sent Events
tkramar
 
Client-Server-Kommunikation mit dem Command Pattern
Client-Server-Kommunikation mit dem Command PatternClient-Server-Kommunikation mit dem Command Pattern
Client-Server-Kommunikation mit dem Command Pattern
pgt technology scouting GmbH
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...
Tom Croucher
 
Agile without technical practices isn't agile
Agile without technical practices isn't agileAgile without technical practices isn't agile
Agile without technical practices isn't agile
kevin_yeung
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...
Tom Croucher
 
Venue Driver Technology Overview
Venue Driver Technology OverviewVenue Driver Technology Overview
Venue Driver Technology Overview
Ryan Porter
 
DASP Top10 for OWASP Thailand Chapter by s111s
DASP Top10 for OWASP Thailand Chapter by s111s DASP Top10 for OWASP Thailand Chapter by s111s
DASP Top10 for OWASP Thailand Chapter by s111s
s111s object
 
Otimizando seu projeto Rails
Otimizando seu projeto RailsOtimizando seu projeto Rails
Otimizando seu projeto Rails
Tiago Albineli Motta
 
Beyond php it's not (just) about the code
Beyond php   it's not (just) about the codeBeyond php   it's not (just) about the code
Beyond php it's not (just) about the code
Wim Godden
 
QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia
QCon 2019 - Opportunities and Pitfalls of Event-Driven UtopiaQCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia
QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia
Bernd Ruecker
 
Embrace NoSQL and Eventual Consistency with Ripple
Embrace NoSQL and Eventual Consistency with RippleEmbrace NoSQL and Eventual Consistency with Ripple
Embrace NoSQL and Eventual Consistency with Ripple
Sean Cribbs
 
Rhebok, High Performance Rack Handler / Rubykaigi 2015
Rhebok, High Performance Rack Handler / Rubykaigi 2015Rhebok, High Performance Rack Handler / Rubykaigi 2015
Rhebok, High Performance Rack Handler / Rubykaigi 2015
Masahiro Nagano
 
React native meetup 2019
React native meetup 2019React native meetup 2019
React native meetup 2019
Arjun Kava
 
Sane Sharding with Akka Cluster
Sane Sharding with Akka ClusterSane Sharding with Akka Cluster
Sane Sharding with Akka Cluster
miciek
 
JDD2015: Sharding with Akka Cluster: From Theory to Production - Krzysztof Ot...
JDD2015: Sharding with Akka Cluster: From Theory to Production - Krzysztof Ot...JDD2015: Sharding with Akka Cluster: From Theory to Production - Krzysztof Ot...
JDD2015: Sharding with Akka Cluster: From Theory to Production - Krzysztof Ot...
PROIDEA
 
Developing event-driven microservices with event sourcing and CQRS (svcc, sv...
Developing event-driven microservices with event sourcing and CQRS  (svcc, sv...Developing event-driven microservices with event sourcing and CQRS  (svcc, sv...
Developing event-driven microservices with event sourcing and CQRS (svcc, sv...
Chris Richardson
 
TDC São Paulo 2015 Ruby - Crescimento e performance em uma aplicação em Rails
TDC São Paulo 2015 Ruby - Crescimento e performance em uma aplicação em RailsTDC São Paulo 2015 Ruby - Crescimento e performance em uma aplicação em Rails
TDC São Paulo 2015 Ruby - Crescimento e performance em uma aplicação em Rails
andrehjr
 
Php assíncrono com_react_php
Php assíncrono com_react_phpPhp assíncrono com_react_php
Php assíncrono com_react_php
Renato Lucena
 
Apache Incubator Samza: Stream Processing at LinkedIn
Apache Incubator Samza: Stream Processing at LinkedInApache Incubator Samza: Stream Processing at LinkedIn
Apache Incubator Samza: Stream Processing at LinkedIn
Chris Riccomini
 
JCon Live 2023 - Lice coding some integration problems
JCon Live 2023 - Lice coding some integration problemsJCon Live 2023 - Lice coding some integration problems
JCon Live 2023 - Lice coding some integration problems
Bernd Ruecker
 

Similar to Concurrent Python at Beeswax - Ron Rothman - NYC Python Meetup 2020 (20)

Live Streaming & Server Sent Events
Live Streaming & Server Sent EventsLive Streaming & Server Sent Events
Live Streaming & Server Sent Events
 
Client-Server-Kommunikation mit dem Command Pattern
Client-Server-Kommunikation mit dem Command PatternClient-Server-Kommunikation mit dem Command Pattern
Client-Server-Kommunikation mit dem Command Pattern
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...
 
Agile without technical practices isn't agile
Agile without technical practices isn't agileAgile without technical practices isn't agile
Agile without technical practices isn't agile
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...
 
Venue Driver Technology Overview
Venue Driver Technology OverviewVenue Driver Technology Overview
Venue Driver Technology Overview
 
DASP Top10 for OWASP Thailand Chapter by s111s
DASP Top10 for OWASP Thailand Chapter by s111s DASP Top10 for OWASP Thailand Chapter by s111s
DASP Top10 for OWASP Thailand Chapter by s111s
 
Otimizando seu projeto Rails
Otimizando seu projeto RailsOtimizando seu projeto Rails
Otimizando seu projeto Rails
 
Beyond php it's not (just) about the code
Beyond php   it's not (just) about the codeBeyond php   it's not (just) about the code
Beyond php it's not (just) about the code
 
QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia
QCon 2019 - Opportunities and Pitfalls of Event-Driven UtopiaQCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia
QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia
 
Embrace NoSQL and Eventual Consistency with Ripple
Embrace NoSQL and Eventual Consistency with RippleEmbrace NoSQL and Eventual Consistency with Ripple
Embrace NoSQL and Eventual Consistency with Ripple
 
Rhebok, High Performance Rack Handler / Rubykaigi 2015
Rhebok, High Performance Rack Handler / Rubykaigi 2015Rhebok, High Performance Rack Handler / Rubykaigi 2015
Rhebok, High Performance Rack Handler / Rubykaigi 2015
 
React native meetup 2019
React native meetup 2019React native meetup 2019
React native meetup 2019
 
Sane Sharding with Akka Cluster
Sane Sharding with Akka ClusterSane Sharding with Akka Cluster
Sane Sharding with Akka Cluster
 
JDD2015: Sharding with Akka Cluster: From Theory to Production - Krzysztof Ot...
JDD2015: Sharding with Akka Cluster: From Theory to Production - Krzysztof Ot...JDD2015: Sharding with Akka Cluster: From Theory to Production - Krzysztof Ot...
JDD2015: Sharding with Akka Cluster: From Theory to Production - Krzysztof Ot...
 
Developing event-driven microservices with event sourcing and CQRS (svcc, sv...
Developing event-driven microservices with event sourcing and CQRS  (svcc, sv...Developing event-driven microservices with event sourcing and CQRS  (svcc, sv...
Developing event-driven microservices with event sourcing and CQRS (svcc, sv...
 
TDC São Paulo 2015 Ruby - Crescimento e performance em uma aplicação em Rails
TDC São Paulo 2015 Ruby - Crescimento e performance em uma aplicação em RailsTDC São Paulo 2015 Ruby - Crescimento e performance em uma aplicação em Rails
TDC São Paulo 2015 Ruby - Crescimento e performance em uma aplicação em Rails
 
Php assíncrono com_react_php
Php assíncrono com_react_phpPhp assíncrono com_react_php
Php assíncrono com_react_php
 
Apache Incubator Samza: Stream Processing at LinkedIn
Apache Incubator Samza: Stream Processing at LinkedInApache Incubator Samza: Stream Processing at LinkedIn
Apache Incubator Samza: Stream Processing at LinkedIn
 
JCon Live 2023 - Lice coding some integration problems
JCon Live 2023 - Lice coding some integration problemsJCon Live 2023 - Lice coding some integration problems
JCon Live 2023 - Lice coding some integration problems
 

Recently uploaded

Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptxOperational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
sandeepmenon62
 
Building the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Building the Ideal CI-CD Pipeline_ Achieving Visual PerfectionBuilding the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Building the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Applitools
 
Going AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applicationsGoing AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applications
Alina Yurenko
 
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
kalichargn70th171
 
Call Girls Bangalore🔥7023059433🔥Best Profile Escorts in Bangalore Available 24/7
Call Girls Bangalore🔥7023059433🔥Best Profile Escorts in Bangalore Available 24/7Call Girls Bangalore🔥7023059433🔥Best Profile Escorts in Bangalore Available 24/7
Call Girls Bangalore🔥7023059433🔥Best Profile Escorts in Bangalore Available 24/7
manji sharman06
 
TheFutureIsDynamic-BoxLang-CFCamp2024.pdf
TheFutureIsDynamic-BoxLang-CFCamp2024.pdfTheFutureIsDynamic-BoxLang-CFCamp2024.pdf
TheFutureIsDynamic-BoxLang-CFCamp2024.pdf
Ortus Solutions, Corp
 
Orca: Nocode Graphical Editor for Container Orchestration
Orca: Nocode Graphical Editor for Container OrchestrationOrca: Nocode Graphical Editor for Container Orchestration
Orca: Nocode Graphical Editor for Container Orchestration
Pedro J. Molina
 
What’s new in VictoriaMetrics - Q2 2024 Update
What’s new in VictoriaMetrics - Q2 2024 UpdateWhat’s new in VictoriaMetrics - Q2 2024 Update
What’s new in VictoriaMetrics - Q2 2024 Update
VictoriaMetrics
 
Secure-by-Design Using Hardware and Software Protection for FDA Compliance
Secure-by-Design Using Hardware and Software Protection for FDA ComplianceSecure-by-Design Using Hardware and Software Protection for FDA Compliance
Secure-by-Design Using Hardware and Software Protection for FDA Compliance
ICS
 
Cost-Effective Strategies For iOS App Development
Cost-Effective Strategies For iOS App DevelopmentCost-Effective Strategies For iOS App Development
Cost-Effective Strategies For iOS App Development
Softradix Technologies
 
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdfThe Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
kalichargn70th171
 
Folding Cheat Sheet #6 - sixth in a series
Folding Cheat Sheet #6 - sixth in a seriesFolding Cheat Sheet #6 - sixth in a series
Folding Cheat Sheet #6 - sixth in a series
Philip Schwarz
 
What is Continuous Testing in DevOps - A Definitive Guide.pdf
What is Continuous Testing in DevOps - A Definitive Guide.pdfWhat is Continuous Testing in DevOps - A Definitive Guide.pdf
What is Continuous Testing in DevOps - A Definitive Guide.pdf
kalichargn70th171
 
42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert
vaishalijagtap12
 
ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.
Maitrey Patel
 
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
Ortus Solutions, Corp
 
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSISDECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
Tier1 app
 
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in NashikUpturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies
 
Superpower Your Apache Kafka Applications Development with Complementary Open...
Superpower Your Apache Kafka Applications Development with Complementary Open...Superpower Your Apache Kafka Applications Development with Complementary Open...
Superpower Your Apache Kafka Applications Development with Complementary Open...
Paul Brebner
 
What’s New in VictoriaLogs - Q2 2024 Update
What’s New in VictoriaLogs - Q2 2024 UpdateWhat’s New in VictoriaLogs - Q2 2024 Update
What’s New in VictoriaLogs - Q2 2024 Update
VictoriaMetrics
 

Recently uploaded (20)

Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptxOperational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
 
Building the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Building the Ideal CI-CD Pipeline_ Achieving Visual PerfectionBuilding the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Building the Ideal CI-CD Pipeline_ Achieving Visual Perfection
 
Going AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applicationsGoing AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applications
 
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
 
Call Girls Bangalore🔥7023059433🔥Best Profile Escorts in Bangalore Available 24/7
Call Girls Bangalore🔥7023059433🔥Best Profile Escorts in Bangalore Available 24/7Call Girls Bangalore🔥7023059433🔥Best Profile Escorts in Bangalore Available 24/7
Call Girls Bangalore🔥7023059433🔥Best Profile Escorts in Bangalore Available 24/7
 
TheFutureIsDynamic-BoxLang-CFCamp2024.pdf
TheFutureIsDynamic-BoxLang-CFCamp2024.pdfTheFutureIsDynamic-BoxLang-CFCamp2024.pdf
TheFutureIsDynamic-BoxLang-CFCamp2024.pdf
 
Orca: Nocode Graphical Editor for Container Orchestration
Orca: Nocode Graphical Editor for Container OrchestrationOrca: Nocode Graphical Editor for Container Orchestration
Orca: Nocode Graphical Editor for Container Orchestration
 
What’s new in VictoriaMetrics - Q2 2024 Update
What’s new in VictoriaMetrics - Q2 2024 UpdateWhat’s new in VictoriaMetrics - Q2 2024 Update
What’s new in VictoriaMetrics - Q2 2024 Update
 
Secure-by-Design Using Hardware and Software Protection for FDA Compliance
Secure-by-Design Using Hardware and Software Protection for FDA ComplianceSecure-by-Design Using Hardware and Software Protection for FDA Compliance
Secure-by-Design Using Hardware and Software Protection for FDA Compliance
 
Cost-Effective Strategies For iOS App Development
Cost-Effective Strategies For iOS App DevelopmentCost-Effective Strategies For iOS App Development
Cost-Effective Strategies For iOS App Development
 
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdfThe Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
 
Folding Cheat Sheet #6 - sixth in a series
Folding Cheat Sheet #6 - sixth in a seriesFolding Cheat Sheet #6 - sixth in a series
Folding Cheat Sheet #6 - sixth in a series
 
What is Continuous Testing in DevOps - A Definitive Guide.pdf
What is Continuous Testing in DevOps - A Definitive Guide.pdfWhat is Continuous Testing in DevOps - A Definitive Guide.pdf
What is Continuous Testing in DevOps - A Definitive Guide.pdf
 
42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert
 
ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.
 
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
 
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSISDECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
 
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in NashikUpturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in Nashik
 
Superpower Your Apache Kafka Applications Development with Complementary Open...
Superpower Your Apache Kafka Applications Development with Complementary Open...Superpower Your Apache Kafka Applications Development with Complementary Open...
Superpower Your Apache Kafka Applications Development with Complementary Open...
 
What’s New in VictoriaLogs - Q2 2024 Update
What’s New in VictoriaLogs - Q2 2024 UpdateWhat’s New in VictoriaLogs - Q2 2024 Update
What’s New in VictoriaLogs - Q2 2024 Update
 

Concurrent Python at Beeswax - Ron Rothman - NYC Python Meetup 2020

  • 1. Python at Scale Concurrency at Beeswax Ron Rothman
  • 2.
  • 3. Advertiser* Ad Exchange Advertiser* Beeswax Advertiser* Advertiser* ad ❕ 200 ms ❕ 2 million per second ❕ non-infinite $ budget ❕ 99.99% uptime ❕ ~optimal bids ad request bid requests bids
  • 6. Event Collection Ad Exchange Bidder Event Collector Event Log Stream Load O(10K/sec) Response time < 250 ms p99 Availability ♾️ Nines 1. Durably record the event 2. Update counters (database) 1 2
  • 7. def handle_request(req): '''Process an incoming event.''' unmarshall(req) # processing validate(req) # processing # some business logic # processing record_event(req) # network i/o update_db_counters(req) # network i/o return_response() work work work wait wait
  • 8. 10 requests/sec (100 ms per request)
  • 9.
  • 10. def handle_request(req): '''Process an incoming event.''' unmarshall(req) # processing validate(req) # processing # some business logic # processing record_event(req) # network i/o update_db_counters(req) # network i/o return_response() work work work wait wait
  • 11. Layers of Concurrency Many machines EC2; Autoscale groups Many processes Preforking web servers Many threads Greenlets; asyncio Containers ECS tasks Serverless Lambdas
  • 12. Elastic Load Balancer EC2 EC2 EC2 EC2 EC2 autoscale group event notifications (HTTP requests)
  • 16. Threads Greenlets Preemptive Cooperative Requires extensive locking Requires no locking Lightweight Very lightweight Leverage multiple cores* Single core *Sadly, untrue for CPython
  • 17. Gevent ● Create & manipulate greenlets ● Allows you to do non-blocking i/o
  • 18. def handle_request(req): '''Process an incoming event.''' unmarshall(req) # processing validate(req) # processing # some business logic # processing record_event(req) # network i/o update_db_counters(req) # network i/o return_response() work work work wait wait
  • 19. Gevent ● Create & manipulate greenlets ● Allows you to do non-blocking i/o ● Makes (i/o) libraries that you use non-blocking! ○ "monkey patching"
  • 20. def handle_request(req): '''Process an incoming event.''' unmarshall(req) # processing validate(req) # processing # some business logic # processing record_event(req) # network i/o update_db_counters(req) # network i/o return_response() work work work yields yields
  • 22. Reality def handle_request(req): '''Process an incoming event.''' unmarshall(req) decrypt(req) validate(req) check_whether_duplicate(req) # BLOCKING i/o # some business logic # more business logic # even more business logic record_event(req) # nonblocking i/o update_db_counters(req) # BLOCKING i/o update_other_db_counters(req) # BLOCKING i/o update_other_other_db_counters(req) # BLOCKING i/o return_response()
  • 23. Reality def handle_request(req): '''Process an incoming event.''' unmarshall(req) decrypt(req) validate(req) check_whether_duplicate(req) # BLOCKING i/o # some business logic # more business logic # even more business logic record_event(req) # nonblocking i/o update_db_counters(req) # BLOCKING i/o update_other_db_counters(req) # BLOCKING i/o update_other_other_db_counters(req) # BLOCKING i/o return_response()
  • 24. Reality def handle_request(req): '''Process an incoming event.''' unmarshall(req) decrypt(req) validate(req) check_whether_duplicate(req) # BLOCKING i/o # some business logic # more business logic # even more business logic record_event(req) # nonblocking i/o update_db_counters(req) # BLOCKING i/o update_other_db_counters(req) # BLOCKING i/o update_other_other_db_counters(req) # BLOCKING i/o return_response()
  • 25. Reality def handle_request(req): '''Process an incoming event.''' unmarshall(req) decrypt(req) validate(req) check_whether_duplicate(req) # BLOCKING i/o # some business logic # more business logic # even more business logic record_event(req) # nonblocking i/o update_db_counters(req) # BLOCKING i/o update_other_db_counters(req) # BLOCKING i/o update_other_other_db_counters(req) # BLOCKING i/o return_response()
  • 26. Redis client ✅ pure python DynamoDB client ✅ pure python Aerospike client ⛔ wrapped C code Which DB Libraries Can Be Monkey Patched?
  • 27. Too Much Processing? Yield Often. def handle_request(req): '''Process an incoming event.''' unmarshall(req) decrypt(req) greenlet_yield() validate(req) check_whether_duplicate(req) # BLOCKING i/o # some business logic # more business logic greenlet_yield() # even more business logic record_event(req) # nonblocking i/o ...
  • 28. Blocking C Code? Batch & Timeouts. def handle_request(req): '''Process an incoming event.''' unmarshall(req) decrypt(req) validate(req) # BLOCKING i/o check_whether_duplicate(req, timeout_ms=5, max_tries=3) # business logic record_event(req) # nonblocking i/o update_counters_in_memory(req) # occasional i/o return_response()
  • 29. � Blocking I/O calls waste CPU � You're not as I/O bound as you think hope � C extensions play by different rules
  • 30. ● Blocking I/O calls waste CPU ■ Gevent + monkey patch ● You're not as I/O bound as you think hope ■ Buffer & batch ● C extensions play by different rules ■ Short timeouts w/retries
  • 31. Thank You 🙏� ron {at} beeswax.com
  • 32. References ● Gevent ● Falcon ● Bottle ● The Sharp Corners of Gevent ● Beeswax

Editor's Notes

  1. Design for framework flexibility Buffer blocking i/o Short timeouts w/retries
  2. Design for framework flexibility Buffer blocking i/o Short timeouts w/retries