SlideShare a Scribd company logo
1 of 15
Download to read offline
Handling massive traffic
with Python
Òscar Vilaplana, Paylogic
PyGrunn 2013
What’s the problem?
• High Traffic (>10k hits/s)
• Redirect low traffic to Paylogic
• Change redirected TPS
• Expect things to break
• Be fair, respect FIFO (within reason)
• Keep users informed
02
In more detail
• Open/hold/close sales
• Expect any server to go down
• Expect ALL servers to go down
• Expect users to disappear
• Display expected waiting time and other inf
• Keep it working
• Prevent attacks
03
How It Works
• A horde of customers appear!
• see a pretty page.
• get a position in the queue.
• page auto-refresh.
• your turn? to the Frontoffice!
• meanwhile info is shown.
• (waiting time, information from event managers…)
04
Data Storage
• Estimates
• Not much data, stored in the instances and synced.
• Tokens
• A LOT of data!
• way too much to store and sync
• use distributed storage
• (the browsers)
05
Architecture
• ELB
• Queue Instances
• Bouncer Process
• Syncer Process
• HTML/JS Queue Page in Cloudfront
06
ELB
• Auto-scales (but not fast enough).
• Many regions.
• Can boot/kill instances automatically.
• We don’t do it yet.
07
Queue Instances
• EC2 instances, which handle the traffic.
• All identical, sync eachother.
• They can be added or removed at will.
• If some (but not all) die, the users won’t notice.
• If all die, only the statistics will be affected.
• (Never happened).
08
Users Handler
• Give out and validate tokens.
• Determine if the user should:
• Keep waiting
• Go to the Frontoffice
• See the Sold Out page.
• Return the expected waiting time.
• Return the values configured by the Event Managers.
09
Synchronization of Statistics
• Keep the Queue Instances synced so they know:
• How many users are waiting.
• How to calculate the waiting time.
• How many users are being let through by the system
10
HTML/JS Queue Page in Cloudfront
• Uses Handlebars
• Served by Cloudfront so that the Queue keeps looking good even if all
our servers were down.
• Updated frequently.
• Calls the Load Balancer. Error? Retry.
• Errors are very rare.
11
Deployment
• Debs in private repos.
• Installed through tunnel.
• Custom python2deb tool (to be released).
12
Stresstest
• Custom client with human-like behaviour.
• Notify amazon!
13
What we learned
• Debugging distributed apps is hard.
• Last bugs are nasty.
• ELB doesn’t scale fast enough by itself.
14
Q&A
15

More Related Content

What's hot

Msdn Workflow Services And Windows Server App Fabric
Msdn Workflow Services And Windows Server App FabricMsdn Workflow Services And Windows Server App Fabric
Msdn Workflow Services And Windows Server App FabricJuan Pablo
 
Stored Procedure Superpowers: A Developer’s Guide
Stored Procedure Superpowers: A Developer’s GuideStored Procedure Superpowers: A Developer’s Guide
Stored Procedure Superpowers: A Developer’s GuideVoltDB
 
Achieving Zero Downtime for SQL
Achieving Zero Downtime for SQLAchieving Zero Downtime for SQL
Achieving Zero Downtime for SQLScaleArc
 
Magento Meetup New Delhi- Magento2 Speed Optimization
Magento Meetup New Delhi- Magento2 Speed OptimizationMagento Meetup New Delhi- Magento2 Speed Optimization
Magento Meetup New Delhi- Magento2 Speed OptimizationWebkul Software Pvt. Ltd.
 
Migrating to Cassandra
Migrating to CassandraMigrating to Cassandra
Migrating to CassandraInstaclustr
 
Metrics driven development with dedicated Observability Team
Metrics driven development with dedicated Observability TeamMetrics driven development with dedicated Observability Team
Metrics driven development with dedicated Observability TeamLINE Corporation
 
Meteor Revolution: From DDP to Blaze Reactive Rendering
Meteor Revolution: From DDP to Blaze Reactive Rendering Meteor Revolution: From DDP to Blaze Reactive Rendering
Meteor Revolution: From DDP to Blaze Reactive Rendering Massimo Sgrelli
 
Introduction to Apache Cassandra
Introduction to Apache CassandraIntroduction to Apache Cassandra
Introduction to Apache CassandraInstaclustr
 
GemStone/S @ Vienna University
GemStone/S @ Vienna University GemStone/S @ Vienna University
GemStone/S @ Vienna University ESUG
 
Building & Testing Scalable Rails Applications
Building & Testing Scalable Rails ApplicationsBuilding & Testing Scalable Rails Applications
Building & Testing Scalable Rails Applicationsevilmike
 
MongoDB and Machine Learning with Flowable
MongoDB and Machine Learning with FlowableMongoDB and Machine Learning with Flowable
MongoDB and Machine Learning with FlowableFlowable
 
Top 5 Challenges to Upgrading to SQL Server 2016
Top 5 Challenges to Upgrading to SQL Server 2016Top 5 Challenges to Upgrading to SQL Server 2016
Top 5 Challenges to Upgrading to SQL Server 2016ScaleArc
 

What's hot (18)

Msdn Workflow Services And Windows Server App Fabric
Msdn Workflow Services And Windows Server App FabricMsdn Workflow Services And Windows Server App Fabric
Msdn Workflow Services And Windows Server App Fabric
 
Stored Procedure Superpowers: A Developer’s Guide
Stored Procedure Superpowers: A Developer’s GuideStored Procedure Superpowers: A Developer’s Guide
Stored Procedure Superpowers: A Developer’s Guide
 
Achieving Zero Downtime for SQL
Achieving Zero Downtime for SQLAchieving Zero Downtime for SQL
Achieving Zero Downtime for SQL
 
Magento Meetup New Delhi- Magento2 Speed Optimization
Magento Meetup New Delhi- Magento2 Speed OptimizationMagento Meetup New Delhi- Magento2 Speed Optimization
Magento Meetup New Delhi- Magento2 Speed Optimization
 
Migrating to Cassandra
Migrating to CassandraMigrating to Cassandra
Migrating to Cassandra
 
Zone.js
Zone.jsZone.js
Zone.js
 
Metrics driven development with dedicated Observability Team
Metrics driven development with dedicated Observability TeamMetrics driven development with dedicated Observability Team
Metrics driven development with dedicated Observability Team
 
Meteor Revolution: From DDP to Blaze Reactive Rendering
Meteor Revolution: From DDP to Blaze Reactive Rendering Meteor Revolution: From DDP to Blaze Reactive Rendering
Meteor Revolution: From DDP to Blaze Reactive Rendering
 
Introduction to Apache Cassandra
Introduction to Apache CassandraIntroduction to Apache Cassandra
Introduction to Apache Cassandra
 
Flux and React.js
Flux and React.jsFlux and React.js
Flux and React.js
 
GemStone/S @ Vienna University
GemStone/S @ Vienna University GemStone/S @ Vienna University
GemStone/S @ Vienna University
 
Building & Testing Scalable Rails Applications
Building & Testing Scalable Rails ApplicationsBuilding & Testing Scalable Rails Applications
Building & Testing Scalable Rails Applications
 
Varnish by Aswin
Varnish by AswinVarnish by Aswin
Varnish by Aswin
 
Wicket Live on Stage
Wicket Live on StageWicket Live on Stage
Wicket Live on Stage
 
MongoDB and Machine Learning with Flowable
MongoDB and Machine Learning with FlowableMongoDB and Machine Learning with Flowable
MongoDB and Machine Learning with Flowable
 
Top 5 Challenges to Upgrading to SQL Server 2016
Top 5 Challenges to Upgrading to SQL Server 2016Top 5 Challenges to Upgrading to SQL Server 2016
Top 5 Challenges to Upgrading to SQL Server 2016
 
Cassandra
CassandraCassandra
Cassandra
 
Marketplace integration
Marketplace integrationMarketplace integration
Marketplace integration
 

Viewers also liked

Advanced task management with Celery
Advanced task management with CeleryAdvanced task management with Celery
Advanced task management with CeleryMahendra M
 
Lean Software Development: Validated Learning
Lean Software Development: Validated LearningLean Software Development: Validated Learning
Lean Software Development: Validated LearningÒscar Vilaplana
 
Software Architecture: How Much Design?
Software Architecture: How Much Design?Software Architecture: How Much Design?
Software Architecture: How Much Design?Òscar Vilaplana
 
Django deployment and rpm+yum
Django deployment and rpm+yumDjango deployment and rpm+yum
Django deployment and rpm+yumWalter Liu
 

Viewers also liked (8)

Advanced task management with Celery
Advanced task management with CeleryAdvanced task management with Celery
Advanced task management with Celery
 
Scrum
ScrumScrum
Scrum
 
Lean Software Development: Validated Learning
Lean Software Development: Validated LearningLean Software Development: Validated Learning
Lean Software Development: Validated Learning
 
Celery
CeleryCelery
Celery
 
Continuous deployment
Continuous deploymentContinuous deployment
Continuous deployment
 
Software Architecture: How Much Design?
Software Architecture: How Much Design?Software Architecture: How Much Design?
Software Architecture: How Much Design?
 
Tornado in Depth
Tornado in DepthTornado in Depth
Tornado in Depth
 
Django deployment and rpm+yum
Django deployment and rpm+yumDjango deployment and rpm+yum
Django deployment and rpm+yum
 

Similar to Massive traffic handling with Python

OpenNebulaConf 2013 - Monitoring of OpenNebula installations by Florian Heigl
OpenNebulaConf 2013 - Monitoring of OpenNebula installations by Florian Heigl OpenNebulaConf 2013 - Monitoring of OpenNebula installations by Florian Heigl
OpenNebulaConf 2013 - Monitoring of OpenNebula installations by Florian Heigl OpenNebula Project
 
Monitoring of OpenNebula installations
Monitoring of OpenNebula installationsMonitoring of OpenNebula installations
Monitoring of OpenNebula installationsNETWAYS
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...smallerror
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...xlight
 
Fixing twitter
Fixing twitterFixing twitter
Fixing twitterRoger Xia
 
John adams talk cloudy
John adams   talk cloudyJohn adams   talk cloudy
John adams talk cloudyJohn Adams
 
Tales Of The Black Knight - Keeping EverythingMe running
Tales Of The Black Knight - Keeping EverythingMe runningTales Of The Black Knight - Keeping EverythingMe running
Tales Of The Black Knight - Keeping EverythingMe runningDvir Volk
 
Microservices: The Best Practices
Microservices: The Best PracticesMicroservices: The Best Practices
Microservices: The Best PracticesPavel Mička
 
Suning OpenStack Cloud and Heat
Suning OpenStack Cloud and HeatSuning OpenStack Cloud and Heat
Suning OpenStack Cloud and HeatQiming Teng
 
Parallel and Asynchronous Programming - ITProDevConnections 2012 (Greek)
Parallel and Asynchronous Programming -  ITProDevConnections 2012 (Greek)Parallel and Asynchronous Programming -  ITProDevConnections 2012 (Greek)
Parallel and Asynchronous Programming - ITProDevConnections 2012 (Greek)Panagiotis Kanavos
 
Building a Database for the End of the World
Building a Database for the End of the WorldBuilding a Database for the End of the World
Building a Database for the End of the Worldjhugg
 
Design for Scale / Surge 2010
Design for Scale / Surge 2010Design for Scale / Surge 2010
Design for Scale / Surge 2010Christopher Brown
 
Parallel and Asynchronous Programming - ITProDevConnections 2012 (English)
Parallel and Asynchronous Programming -  ITProDevConnections 2012 (English)Parallel and Asynchronous Programming -  ITProDevConnections 2012 (English)
Parallel and Asynchronous Programming - ITProDevConnections 2012 (English)Panagiotis Kanavos
 
Dynamics CRM high volume systems - lessons from the field
Dynamics CRM high volume systems - lessons from the fieldDynamics CRM high volume systems - lessons from the field
Dynamics CRM high volume systems - lessons from the fieldStéphane Dorrekens
 
Tale of two streaming frameworks- Apace Storm & Apache Flink
Tale of two streaming frameworks- Apace Storm & Apache FlinkTale of two streaming frameworks- Apace Storm & Apache Flink
Tale of two streaming frameworks- Apace Storm & Apache FlinkKarthik Deivasigamani
 
Tale of two streaming frameworks (Karthik D - Walmart)
Tale of two streaming frameworks (Karthik D - Walmart)Tale of two streaming frameworks (Karthik D - Walmart)
Tale of two streaming frameworks (Karthik D - Walmart)KafkaZone
 
12-Step Program for Scaling Web Applications on PostgreSQL
12-Step Program for Scaling Web Applications on PostgreSQL12-Step Program for Scaling Web Applications on PostgreSQL
12-Step Program for Scaling Web Applications on PostgreSQLKonstantin Gredeskoul
 
Jeremy Edberg (MinOps ) - How to build a solid infrastructure for a startup t...
Jeremy Edberg (MinOps ) - How to build a solid infrastructure for a startup t...Jeremy Edberg (MinOps ) - How to build a solid infrastructure for a startup t...
Jeremy Edberg (MinOps ) - How to build a solid infrastructure for a startup t...Startupfest
 
Database failover from client perspective
Database failover from client perspectiveDatabase failover from client perspective
Database failover from client perspectivePriit Piipuu
 

Similar to Massive traffic handling with Python (20)

OpenNebulaConf 2013 - Monitoring of OpenNebula installations by Florian Heigl
OpenNebulaConf 2013 - Monitoring of OpenNebula installations by Florian Heigl OpenNebulaConf 2013 - Monitoring of OpenNebula installations by Florian Heigl
OpenNebulaConf 2013 - Monitoring of OpenNebula installations by Florian Heigl
 
Monitoring of OpenNebula installations
Monitoring of OpenNebula installationsMonitoring of OpenNebula installations
Monitoring of OpenNebula installations
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
 
Fixing twitter
Fixing twitterFixing twitter
Fixing twitter
 
Fixing_Twitter
Fixing_TwitterFixing_Twitter
Fixing_Twitter
 
John adams talk cloudy
John adams   talk cloudyJohn adams   talk cloudy
John adams talk cloudy
 
Tales Of The Black Knight - Keeping EverythingMe running
Tales Of The Black Knight - Keeping EverythingMe runningTales Of The Black Knight - Keeping EverythingMe running
Tales Of The Black Knight - Keeping EverythingMe running
 
Microservices: The Best Practices
Microservices: The Best PracticesMicroservices: The Best Practices
Microservices: The Best Practices
 
Suning OpenStack Cloud and Heat
Suning OpenStack Cloud and HeatSuning OpenStack Cloud and Heat
Suning OpenStack Cloud and Heat
 
Parallel and Asynchronous Programming - ITProDevConnections 2012 (Greek)
Parallel and Asynchronous Programming -  ITProDevConnections 2012 (Greek)Parallel and Asynchronous Programming -  ITProDevConnections 2012 (Greek)
Parallel and Asynchronous Programming - ITProDevConnections 2012 (Greek)
 
Building a Database for the End of the World
Building a Database for the End of the WorldBuilding a Database for the End of the World
Building a Database for the End of the World
 
Design for Scale / Surge 2010
Design for Scale / Surge 2010Design for Scale / Surge 2010
Design for Scale / Surge 2010
 
Parallel and Asynchronous Programming - ITProDevConnections 2012 (English)
Parallel and Asynchronous Programming -  ITProDevConnections 2012 (English)Parallel and Asynchronous Programming -  ITProDevConnections 2012 (English)
Parallel and Asynchronous Programming - ITProDevConnections 2012 (English)
 
Dynamics CRM high volume systems - lessons from the field
Dynamics CRM high volume systems - lessons from the fieldDynamics CRM high volume systems - lessons from the field
Dynamics CRM high volume systems - lessons from the field
 
Tale of two streaming frameworks- Apace Storm & Apache Flink
Tale of two streaming frameworks- Apace Storm & Apache FlinkTale of two streaming frameworks- Apace Storm & Apache Flink
Tale of two streaming frameworks- Apace Storm & Apache Flink
 
Tale of two streaming frameworks (Karthik D - Walmart)
Tale of two streaming frameworks (Karthik D - Walmart)Tale of two streaming frameworks (Karthik D - Walmart)
Tale of two streaming frameworks (Karthik D - Walmart)
 
12-Step Program for Scaling Web Applications on PostgreSQL
12-Step Program for Scaling Web Applications on PostgreSQL12-Step Program for Scaling Web Applications on PostgreSQL
12-Step Program for Scaling Web Applications on PostgreSQL
 
Jeremy Edberg (MinOps ) - How to build a solid infrastructure for a startup t...
Jeremy Edberg (MinOps ) - How to build a solid infrastructure for a startup t...Jeremy Edberg (MinOps ) - How to build a solid infrastructure for a startup t...
Jeremy Edberg (MinOps ) - How to build a solid infrastructure for a startup t...
 
Database failover from client perspective
Database failover from client perspectiveDatabase failover from client perspective
Database failover from client perspective
 

Recently uploaded

Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
Kuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialKuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialJoão Esperancinha
 
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...amber724300
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneUiPathCommunity
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentMahmoud Rabie
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI AgeCprime
 
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...Karmanjay Verma
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Nikki Chapple
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 
Français Patch Tuesday - Avril
Français Patch Tuesday - AvrilFrançais Patch Tuesday - Avril
Français Patch Tuesday - AvrilIvanti
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxfnnc6jmgwh
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 

Recently uploaded (20)

Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
Kuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialKuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorial
 
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyone
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career Development
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI Age
 
How Tech Giants Cut Corners to Harvest Data for A.I.
How Tech Giants Cut Corners to Harvest Data for A.I.How Tech Giants Cut Corners to Harvest Data for A.I.
How Tech Giants Cut Corners to Harvest Data for A.I.
 
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 
Français Patch Tuesday - Avril
Français Patch Tuesday - AvrilFrançais Patch Tuesday - Avril
Français Patch Tuesday - Avril
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 

Massive traffic handling with Python

  • 1. Handling massive traffic with Python Òscar Vilaplana, Paylogic PyGrunn 2013
  • 2. What’s the problem? • High Traffic (>10k hits/s) • Redirect low traffic to Paylogic • Change redirected TPS • Expect things to break • Be fair, respect FIFO (within reason) • Keep users informed 02
  • 3. In more detail • Open/hold/close sales • Expect any server to go down • Expect ALL servers to go down • Expect users to disappear • Display expected waiting time and other inf • Keep it working • Prevent attacks 03
  • 4. How It Works • A horde of customers appear! • see a pretty page. • get a position in the queue. • page auto-refresh. • your turn? to the Frontoffice! • meanwhile info is shown. • (waiting time, information from event managers…) 04
  • 5. Data Storage • Estimates • Not much data, stored in the instances and synced. • Tokens • A LOT of data! • way too much to store and sync • use distributed storage • (the browsers) 05
  • 6. Architecture • ELB • Queue Instances • Bouncer Process • Syncer Process • HTML/JS Queue Page in Cloudfront 06
  • 7. ELB • Auto-scales (but not fast enough). • Many regions. • Can boot/kill instances automatically. • We don’t do it yet. 07
  • 8. Queue Instances • EC2 instances, which handle the traffic. • All identical, sync eachother. • They can be added or removed at will. • If some (but not all) die, the users won’t notice. • If all die, only the statistics will be affected. • (Never happened). 08
  • 9. Users Handler • Give out and validate tokens. • Determine if the user should: • Keep waiting • Go to the Frontoffice • See the Sold Out page. • Return the expected waiting time. • Return the values configured by the Event Managers. 09
  • 10. Synchronization of Statistics • Keep the Queue Instances synced so they know: • How many users are waiting. • How to calculate the waiting time. • How many users are being let through by the system 10
  • 11. HTML/JS Queue Page in Cloudfront • Uses Handlebars • Served by Cloudfront so that the Queue keeps looking good even if all our servers were down. • Updated frequently. • Calls the Load Balancer. Error? Retry. • Errors are very rare. 11
  • 12. Deployment • Debs in private repos. • Installed through tunnel. • Custom python2deb tool (to be released). 12
  • 13. Stresstest • Custom client with human-like behaviour. • Notify amazon! 13
  • 14. What we learned • Debugging distributed apps is hard. • Last bugs are nasty. • ELB doesn’t scale fast enough by itself. 14