SlideShare a Scribd company logo

ConFoo Montreal - Approaches for application request throttling

Speaking from experience building a SaaS: users are insane. If you are lucky, they use your service, but in reality, they probably abuse. Crazy usage patterns resulting in more requests than expected, request bursts when users come back to the office after the weekend, and more! These all pose a potential threat to the health of our web application and may impact other users or the service as a whole. Ideally, we can apply some filtering at the front door: limit the number of requests over a given timespan, limiting bandwidth, ... In this talk, we’ll explore the simple yet complex realm of rate limiting. We’ll go over how to decide on which resources to limit, what the limits should be and where to enforce these limits – in our app, on the server, using a reverse proxy like Nginx or even an external service like CloudFlare or Azure API management. The takeaway? Know when and where to enforce rate limits so you can have both a happy application as well as happy customers.

1 of 49
Download to read offline
1
Approaches for application
request throttling
Maarten Balliauw
@maartenballiauw
3
Agenda
Users and traffic patterns
Rate limiting and considerations
Which resources?
Which limits?
Who to limit? Who not to limit?
What when a limit is reached?
Where to limit?
4
Users...
5
MyGet
Hosted private package repository – www.myget.org
NuGet, NPM, Bower, Maven, VSIX, PHP Composer, Symbols, ...
HTTP-based
Web UI for managing things
API for various package managers
PUT/POST – Upload package
DELETE – Delete package via API
GET – Fetch metadata or binary
6
Background workers for scale
Example: package upload
PUT/POST binary and metadata to front-end
PackageAddedEvent on queue with many handlers handled on back-end
ProcessSymbols
UpdateLatestVersion
Indexing
...
8
What could possibly go wrong...
Too many downloads!
Application logic has to check credentials, subscription, quota
404’s still need that application logic...
Package managers are crazy!
Total # requests Total # 404’s % 404’s
# of packages in solution 200 800 600
# on NuGet.org 190 200 10 5%
# on MyGet feed 1 5 200 195 97,5%
# on MyGet feed 2 4 200 196 98%
# on company-internal
TeamCity
1 200 199 99,5%

Recommended

ConFoo Montreal - Microservices for building an IDE - The innards of JetBrain...
ConFoo Montreal - Microservices for building an IDE - The innards of JetBrain...ConFoo Montreal - Microservices for building an IDE - The innards of JetBrain...
ConFoo Montreal - Microservices for building an IDE - The innards of JetBrain...Maarten Balliauw
 
How to really obfuscate your pdf malware
How to really obfuscate your pdf malwareHow to really obfuscate your pdf malware
How to really obfuscate your pdf malwarezynamics GmbH
 
Toward dynamic analysis of obfuscated android malware
Toward dynamic analysis of obfuscated android malwareToward dynamic analysis of obfuscated android malware
Toward dynamic analysis of obfuscated android malwareZongXian Shen
 
Java & low latency applications
Java & low latency applicationsJava & low latency applications
Java & low latency applicationsRuslan Shevchenko
 
Asynchronous I/O in NodeJS - new standard or challenges?
Asynchronous I/O in NodeJS - new standard or challenges?Asynchronous I/O in NodeJS - new standard or challenges?
Asynchronous I/O in NodeJS - new standard or challenges?Dinh Pham
 
Introduction to mobile reversing
Introduction to mobile reversingIntroduction to mobile reversing
Introduction to mobile reversingjduart
 
Node.js Enterprise Middleware
Node.js Enterprise MiddlewareNode.js Enterprise Middleware
Node.js Enterprise MiddlewareBehrad Zari
 
Os Worthington
Os WorthingtonOs Worthington
Os Worthingtonoscon2007
 

More Related Content

What's hot

How to implement a simple dalvik virtual machine
How to implement a simple dalvik virtual machineHow to implement a simple dalvik virtual machine
How to implement a simple dalvik virtual machineChun-Yu Wang
 
Low latency in java 8 by Peter Lawrey
Low latency in java 8 by Peter Lawrey Low latency in java 8 by Peter Lawrey
Low latency in java 8 by Peter Lawrey J On The Beach
 
Metaprogramming JavaScript
Metaprogramming  JavaScriptMetaprogramming  JavaScript
Metaprogramming JavaScriptdanwrong
 
Real-time Programming in Java
Real-time Programming in JavaReal-time Programming in Java
Real-time Programming in JavaAleš Plšek
 
How to reverse engineer Android applications
How to reverse engineer Android applicationsHow to reverse engineer Android applications
How to reverse engineer Android applicationshubx
 
Concurrent Programming with Ruby and Tuple Spaces
Concurrent Programming with Ruby and Tuple SpacesConcurrent Programming with Ruby and Tuple Spaces
Concurrent Programming with Ruby and Tuple Spacesluccastera
 
Reverse engineering android apps
Reverse engineering android appsReverse engineering android apps
Reverse engineering android appsPranay Airan
 
Perl6 DBDI YAPC::EU 201008
Perl6 DBDI YAPC::EU 201008Perl6 DBDI YAPC::EU 201008
Perl6 DBDI YAPC::EU 201008Tim Bunce
 
Steelcon 2015 Reverse-Engineering Obfuscated Android Applications
Steelcon 2015 Reverse-Engineering Obfuscated Android ApplicationsSteelcon 2015 Reverse-Engineering Obfuscated Android Applications
Steelcon 2015 Reverse-Engineering Obfuscated Android ApplicationsTom Keetch
 
Multithreading and concurrency in android
Multithreading and concurrency in androidMultithreading and concurrency in android
Multithreading and concurrency in androidRakesh Jha
 
Open Source Swift Under the Hood
Open Source Swift Under the HoodOpen Source Swift Under the Hood
Open Source Swift Under the HoodC4Media
 
Exploring .NET memory management - JetBrains webinar
Exploring .NET memory management - JetBrains webinarExploring .NET memory management - JetBrains webinar
Exploring .NET memory management - JetBrains webinarMaarten Balliauw
 
FOSDEM2016 - Ruby and OMR
FOSDEM2016 - Ruby and OMRFOSDEM2016 - Ruby and OMR
FOSDEM2016 - Ruby and OMRCharlie Gracie
 
OpenDDR and Jakarta MVC - JavaLand 2021
OpenDDR and Jakarta MVC - JavaLand 2021OpenDDR and Jakarta MVC - JavaLand 2021
OpenDDR and Jakarta MVC - JavaLand 2021Werner Keil
 
Workin' on the Rails Road
Workin' on the Rails RoadWorkin' on the Rails Road
Workin' on the Rails RoadVishnu Gopal
 
Swift - Under the Hood
Swift - Under the HoodSwift - Under the Hood
Swift - Under the HoodC4Media
 

What's hot (20)

How to implement a simple dalvik virtual machine
How to implement a simple dalvik virtual machineHow to implement a simple dalvik virtual machine
How to implement a simple dalvik virtual machine
 
Low latency in java 8 by Peter Lawrey
Low latency in java 8 by Peter Lawrey Low latency in java 8 by Peter Lawrey
Low latency in java 8 by Peter Lawrey
 
Metaprogramming JavaScript
Metaprogramming  JavaScriptMetaprogramming  JavaScript
Metaprogramming JavaScript
 
Real-time Programming in Java
Real-time Programming in JavaReal-time Programming in Java
Real-time Programming in Java
 
How to reverse engineer Android applications
How to reverse engineer Android applicationsHow to reverse engineer Android applications
How to reverse engineer Android applications
 
Android Binder: Deep Dive
Android Binder: Deep DiveAndroid Binder: Deep Dive
Android Binder: Deep Dive
 
Concurrent Programming with Ruby and Tuple Spaces
Concurrent Programming with Ruby and Tuple SpacesConcurrent Programming with Ruby and Tuple Spaces
Concurrent Programming with Ruby and Tuple Spaces
 
Binder: Android IPC
Binder: Android IPCBinder: Android IPC
Binder: Android IPC
 
Reverse engineering android apps
Reverse engineering android appsReverse engineering android apps
Reverse engineering android apps
 
Perl6 DBDI YAPC::EU 201008
Perl6 DBDI YAPC::EU 201008Perl6 DBDI YAPC::EU 201008
Perl6 DBDI YAPC::EU 201008
 
Steelcon 2015 Reverse-Engineering Obfuscated Android Applications
Steelcon 2015 Reverse-Engineering Obfuscated Android ApplicationsSteelcon 2015 Reverse-Engineering Obfuscated Android Applications
Steelcon 2015 Reverse-Engineering Obfuscated Android Applications
 
Multithreading and concurrency in android
Multithreading and concurrency in androidMultithreading and concurrency in android
Multithreading and concurrency in android
 
Open Source Swift Under the Hood
Open Source Swift Under the HoodOpen Source Swift Under the Hood
Open Source Swift Under the Hood
 
Ruby Insurgency
Ruby InsurgencyRuby Insurgency
Ruby Insurgency
 
Exploring .NET memory management - JetBrains webinar
Exploring .NET memory management - JetBrains webinarExploring .NET memory management - JetBrains webinar
Exploring .NET memory management - JetBrains webinar
 
FOSDEM2016 - Ruby and OMR
FOSDEM2016 - Ruby and OMRFOSDEM2016 - Ruby and OMR
FOSDEM2016 - Ruby and OMR
 
Lambdas puzzler - Peter Lawrey
Lambdas puzzler - Peter LawreyLambdas puzzler - Peter Lawrey
Lambdas puzzler - Peter Lawrey
 
OpenDDR and Jakarta MVC - JavaLand 2021
OpenDDR and Jakarta MVC - JavaLand 2021OpenDDR and Jakarta MVC - JavaLand 2021
OpenDDR and Jakarta MVC - JavaLand 2021
 
Workin' on the Rails Road
Workin' on the Rails RoadWorkin' on the Rails Road
Workin' on the Rails Road
 
Swift - Under the Hood
Swift - Under the HoodSwift - Under the Hood
Swift - Under the Hood
 

Similar to ConFoo Montreal - Approaches for application request throttling

Approaches for application request throttling - dotNetCologne
Approaches for application request throttling - dotNetCologneApproaches for application request throttling - dotNetCologne
Approaches for application request throttling - dotNetCologneMaarten Balliauw
 
Approaches for application request throttling - Cloud Developer Days Poland
Approaches for application request throttling - Cloud Developer Days PolandApproaches for application request throttling - Cloud Developer Days Poland
Approaches for application request throttling - Cloud Developer Days PolandMaarten Balliauw
 
Approaches to application request throttling
Approaches to application request throttlingApproaches to application request throttling
Approaches to application request throttlingMaarten Balliauw
 
VISUG - Approaches for application request throttling
VISUG - Approaches for application request throttlingVISUG - Approaches for application request throttling
VISUG - Approaches for application request throttlingMaarten Balliauw
 
How to get along with HATEOAS without letting the bad guys steal your lunch?
How to get along with HATEOAS without letting the bad guys steal your lunch?How to get along with HATEOAS without letting the bad guys steal your lunch?
How to get along with HATEOAS without letting the bad guys steal your lunch?Graham Charters
 
Expecto Performa! The Magic and Reality of Performance Tuning
Expecto Performa! The Magic and Reality of Performance TuningExpecto Performa! The Magic and Reality of Performance Tuning
Expecto Performa! The Magic and Reality of Performance TuningAtlassian
 
Monitoring your Python with Prometheus (Python Ireland April 2015)
Monitoring your Python with Prometheus (Python Ireland April 2015)Monitoring your Python with Prometheus (Python Ireland April 2015)
Monitoring your Python with Prometheus (Python Ireland April 2015)Brian Brazil
 
Monitoring as an entry point for collaboration
Monitoring as an entry point for collaborationMonitoring as an entry point for collaboration
Monitoring as an entry point for collaborationJulien Pivotto
 
DockerCon SF 2019 - Observability Workshop
DockerCon SF 2019 - Observability WorkshopDockerCon SF 2019 - Observability Workshop
DockerCon SF 2019 - Observability WorkshopKevin Crawley
 
Hadoop application architectures - Fraud detection tutorial
Hadoop application architectures - Fraud detection tutorialHadoop application architectures - Fraud detection tutorial
Hadoop application architectures - Fraud detection tutorialhadooparchbook
 
Scalable Apache for Beginners
Scalable Apache for BeginnersScalable Apache for Beginners
Scalable Apache for Beginnerswebhostingguy
 
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic System
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic SystemTimely Year Two: Lessons Learned Building a Scalable Metrics Analytic System
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic SystemAccumulo Summit
 
The Value of Reactive
The Value of ReactiveThe Value of Reactive
The Value of ReactiveVMware Tanzu
 
Tef con2016 (1)
Tef con2016 (1)Tef con2016 (1)
Tef con2016 (1)ggarber
 
Docker Logging and analysing with Elastic Stack - Jakub Hajek
Docker Logging and analysing with Elastic Stack - Jakub Hajek Docker Logging and analysing with Elastic Stack - Jakub Hajek
Docker Logging and analysing with Elastic Stack - Jakub Hajek PROIDEA
 
Docker Logging and analysing with Elastic Stack
Docker Logging and analysing with Elastic StackDocker Logging and analysing with Elastic Stack
Docker Logging and analysing with Elastic StackJakub Hajek
 
AWS re:Invent 2016: Amazon CloudFront Flash Talks: Best Practices on Configur...
AWS re:Invent 2016: Amazon CloudFront Flash Talks: Best Practices on Configur...AWS re:Invent 2016: Amazon CloudFront Flash Talks: Best Practices on Configur...
AWS re:Invent 2016: Amazon CloudFront Flash Talks: Best Practices on Configur...Amazon Web Services
 
2016 - 10 questions you should answer before building a new microservice
2016 - 10 questions you should answer before building a new microservice2016 - 10 questions you should answer before building a new microservice
2016 - 10 questions you should answer before building a new microservicedevopsdaysaustin
 
SnappyData Ad Analytics Use Case -- BDAM Meetup Sept 14th
SnappyData Ad Analytics Use Case -- BDAM Meetup Sept 14thSnappyData Ad Analytics Use Case -- BDAM Meetup Sept 14th
SnappyData Ad Analytics Use Case -- BDAM Meetup Sept 14thSnappyData
 

Similar to ConFoo Montreal - Approaches for application request throttling (20)

Approaches for application request throttling - dotNetCologne
Approaches for application request throttling - dotNetCologneApproaches for application request throttling - dotNetCologne
Approaches for application request throttling - dotNetCologne
 
Approaches for application request throttling - Cloud Developer Days Poland
Approaches for application request throttling - Cloud Developer Days PolandApproaches for application request throttling - Cloud Developer Days Poland
Approaches for application request throttling - Cloud Developer Days Poland
 
Approaches to application request throttling
Approaches to application request throttlingApproaches to application request throttling
Approaches to application request throttling
 
VISUG - Approaches for application request throttling
VISUG - Approaches for application request throttlingVISUG - Approaches for application request throttling
VISUG - Approaches for application request throttling
 
How to get along with HATEOAS without letting the bad guys steal your lunch?
How to get along with HATEOAS without letting the bad guys steal your lunch?How to get along with HATEOAS without letting the bad guys steal your lunch?
How to get along with HATEOAS without letting the bad guys steal your lunch?
 
Expecto Performa! The Magic and Reality of Performance Tuning
Expecto Performa! The Magic and Reality of Performance TuningExpecto Performa! The Magic and Reality of Performance Tuning
Expecto Performa! The Magic and Reality of Performance Tuning
 
Monitoring your Python with Prometheus (Python Ireland April 2015)
Monitoring your Python with Prometheus (Python Ireland April 2015)Monitoring your Python with Prometheus (Python Ireland April 2015)
Monitoring your Python with Prometheus (Python Ireland April 2015)
 
Monitoring as an entry point for collaboration
Monitoring as an entry point for collaborationMonitoring as an entry point for collaboration
Monitoring as an entry point for collaboration
 
DockerCon SF 2019 - Observability Workshop
DockerCon SF 2019 - Observability WorkshopDockerCon SF 2019 - Observability Workshop
DockerCon SF 2019 - Observability Workshop
 
Hadoop application architectures - Fraud detection tutorial
Hadoop application architectures - Fraud detection tutorialHadoop application architectures - Fraud detection tutorial
Hadoop application architectures - Fraud detection tutorial
 
Scalable Apache for Beginners
Scalable Apache for BeginnersScalable Apache for Beginners
Scalable Apache for Beginners
 
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic System
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic SystemTimely Year Two: Lessons Learned Building a Scalable Metrics Analytic System
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic System
 
The value of reactive
The value of reactiveThe value of reactive
The value of reactive
 
The Value of Reactive
The Value of ReactiveThe Value of Reactive
The Value of Reactive
 
Tef con2016 (1)
Tef con2016 (1)Tef con2016 (1)
Tef con2016 (1)
 
Docker Logging and analysing with Elastic Stack - Jakub Hajek
Docker Logging and analysing with Elastic Stack - Jakub Hajek Docker Logging and analysing with Elastic Stack - Jakub Hajek
Docker Logging and analysing with Elastic Stack - Jakub Hajek
 
Docker Logging and analysing with Elastic Stack
Docker Logging and analysing with Elastic StackDocker Logging and analysing with Elastic Stack
Docker Logging and analysing with Elastic Stack
 
AWS re:Invent 2016: Amazon CloudFront Flash Talks: Best Practices on Configur...
AWS re:Invent 2016: Amazon CloudFront Flash Talks: Best Practices on Configur...AWS re:Invent 2016: Amazon CloudFront Flash Talks: Best Practices on Configur...
AWS re:Invent 2016: Amazon CloudFront Flash Talks: Best Practices on Configur...
 
2016 - 10 questions you should answer before building a new microservice
2016 - 10 questions you should answer before building a new microservice2016 - 10 questions you should answer before building a new microservice
2016 - 10 questions you should answer before building a new microservice
 
SnappyData Ad Analytics Use Case -- BDAM Meetup Sept 14th
SnappyData Ad Analytics Use Case -- BDAM Meetup Sept 14thSnappyData Ad Analytics Use Case -- BDAM Meetup Sept 14th
SnappyData Ad Analytics Use Case -- BDAM Meetup Sept 14th
 

More from Maarten Balliauw

Bringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptxBringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptxMaarten Balliauw
 
Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...
Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...
Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...Maarten Balliauw
 
Building a friendly .NET SDK to connect to Space
Building a friendly .NET SDK to connect to SpaceBuilding a friendly .NET SDK to connect to Space
Building a friendly .NET SDK to connect to SpaceMaarten Balliauw
 
Microservices for building an IDE - The innards of JetBrains Rider - NDC Oslo...
Microservices for building an IDE - The innards of JetBrains Rider - NDC Oslo...Microservices for building an IDE - The innards of JetBrains Rider - NDC Oslo...
Microservices for building an IDE - The innards of JetBrains Rider - NDC Oslo...Maarten Balliauw
 
Indexing and searching NuGet.org with Azure Functions and Search - .NET fwday...
Indexing and searching NuGet.org with Azure Functions and Search - .NET fwday...Indexing and searching NuGet.org with Azure Functions and Search - .NET fwday...
Indexing and searching NuGet.org with Azure Functions and Search - .NET fwday...Maarten Balliauw
 
NDC Sydney 2019 - Microservices for building an IDE – The innards of JetBrain...
NDC Sydney 2019 - Microservices for building an IDE – The innards of JetBrain...NDC Sydney 2019 - Microservices for building an IDE – The innards of JetBrain...
NDC Sydney 2019 - Microservices for building an IDE – The innards of JetBrain...Maarten Balliauw
 
JetBrains Australia 2019 - Exploring .NET’s memory management – a trip down m...
JetBrains Australia 2019 - Exploring .NET’s memory management – a trip down m...JetBrains Australia 2019 - Exploring .NET’s memory management – a trip down m...
JetBrains Australia 2019 - Exploring .NET’s memory management – a trip down m...Maarten Balliauw
 
.NET Conf 2019 - Indexing and searching NuGet.org with Azure Functions and Se...
.NET Conf 2019 - Indexing and searching NuGet.org with Azure Functions and Se....NET Conf 2019 - Indexing and searching NuGet.org with Azure Functions and Se...
.NET Conf 2019 - Indexing and searching NuGet.org with Azure Functions and Se...Maarten Balliauw
 
CloudBurst 2019 - Indexing and searching NuGet.org with Azure Functions and S...
CloudBurst 2019 - Indexing and searching NuGet.org with Azure Functions and S...CloudBurst 2019 - Indexing and searching NuGet.org with Azure Functions and S...
CloudBurst 2019 - Indexing and searching NuGet.org with Azure Functions and S...Maarten Balliauw
 
NDC Oslo 2019 - Indexing and searching NuGet.org with Azure Functions and Search
NDC Oslo 2019 - Indexing and searching NuGet.org with Azure Functions and SearchNDC Oslo 2019 - Indexing and searching NuGet.org with Azure Functions and Search
NDC Oslo 2019 - Indexing and searching NuGet.org with Azure Functions and SearchMaarten Balliauw
 
Indexing and searching NuGet.org with Azure Functions and Search - Cloud Deve...
Indexing and searching NuGet.org with Azure Functions and Search - Cloud Deve...Indexing and searching NuGet.org with Azure Functions and Search - Cloud Deve...
Indexing and searching NuGet.org with Azure Functions and Search - Cloud Deve...Maarten Balliauw
 
CodeStock - Exploring .NET memory management - a trip down memory lane
CodeStock - Exploring .NET memory management - a trip down memory laneCodeStock - Exploring .NET memory management - a trip down memory lane
CodeStock - Exploring .NET memory management - a trip down memory laneMaarten Balliauw
 
Microservices for building an IDE – The innards of JetBrains Rider - TechDays...
Microservices for building an IDE – The innards of JetBrains Rider - TechDays...Microservices for building an IDE – The innards of JetBrains Rider - TechDays...
Microservices for building an IDE – The innards of JetBrains Rider - TechDays...Maarten Balliauw
 
JetBrains Day Seoul - Exploring .NET’s memory management – a trip down memory...
JetBrains Day Seoul - Exploring .NET’s memory management – a trip down memory...JetBrains Day Seoul - Exploring .NET’s memory management – a trip down memory...
JetBrains Day Seoul - Exploring .NET’s memory management – a trip down memory...Maarten Balliauw
 
DotNetFest - Let’s refresh our memory! Memory management in .NET
DotNetFest - Let’s refresh our memory! Memory management in .NETDotNetFest - Let’s refresh our memory! Memory management in .NET
DotNetFest - Let’s refresh our memory! Memory management in .NETMaarten Balliauw
 
What is going on - Application diagnostics on Azure - TechDays Finland
What is going on - Application diagnostics on Azure - TechDays FinlandWhat is going on - Application diagnostics on Azure - TechDays Finland
What is going on - Application diagnostics on Azure - TechDays FinlandMaarten Balliauw
 
ConFoo - Exploring .NET’s memory management – a trip down memory lane
ConFoo - Exploring .NET’s memory management – a trip down memory laneConFoo - Exploring .NET’s memory management – a trip down memory lane
ConFoo - Exploring .NET’s memory management – a trip down memory laneMaarten Balliauw
 
ConFoo - NuGet beyond Hello World
ConFoo - NuGet beyond Hello WorldConFoo - NuGet beyond Hello World
ConFoo - NuGet beyond Hello WorldMaarten Balliauw
 
NuGet beyond Hello World - DotNext Piter 2017
NuGet beyond Hello World - DotNext Piter 2017NuGet beyond Hello World - DotNext Piter 2017
NuGet beyond Hello World - DotNext Piter 2017Maarten Balliauw
 
Exploring .NET memory management (iSense)
Exploring .NET memory management (iSense)Exploring .NET memory management (iSense)
Exploring .NET memory management (iSense)Maarten Balliauw
 

More from Maarten Balliauw (20)

Bringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptxBringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptx
 
Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...
Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...
Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...
 
Building a friendly .NET SDK to connect to Space
Building a friendly .NET SDK to connect to SpaceBuilding a friendly .NET SDK to connect to Space
Building a friendly .NET SDK to connect to Space
 
Microservices for building an IDE - The innards of JetBrains Rider - NDC Oslo...
Microservices for building an IDE - The innards of JetBrains Rider - NDC Oslo...Microservices for building an IDE - The innards of JetBrains Rider - NDC Oslo...
Microservices for building an IDE - The innards of JetBrains Rider - NDC Oslo...
 
Indexing and searching NuGet.org with Azure Functions and Search - .NET fwday...
Indexing and searching NuGet.org with Azure Functions and Search - .NET fwday...Indexing and searching NuGet.org with Azure Functions and Search - .NET fwday...
Indexing and searching NuGet.org with Azure Functions and Search - .NET fwday...
 
NDC Sydney 2019 - Microservices for building an IDE – The innards of JetBrain...
NDC Sydney 2019 - Microservices for building an IDE – The innards of JetBrain...NDC Sydney 2019 - Microservices for building an IDE – The innards of JetBrain...
NDC Sydney 2019 - Microservices for building an IDE – The innards of JetBrain...
 
JetBrains Australia 2019 - Exploring .NET’s memory management – a trip down m...
JetBrains Australia 2019 - Exploring .NET’s memory management – a trip down m...JetBrains Australia 2019 - Exploring .NET’s memory management – a trip down m...
JetBrains Australia 2019 - Exploring .NET’s memory management – a trip down m...
 
.NET Conf 2019 - Indexing and searching NuGet.org with Azure Functions and Se...
.NET Conf 2019 - Indexing and searching NuGet.org with Azure Functions and Se....NET Conf 2019 - Indexing and searching NuGet.org with Azure Functions and Se...
.NET Conf 2019 - Indexing and searching NuGet.org with Azure Functions and Se...
 
CloudBurst 2019 - Indexing and searching NuGet.org with Azure Functions and S...
CloudBurst 2019 - Indexing and searching NuGet.org with Azure Functions and S...CloudBurst 2019 - Indexing and searching NuGet.org with Azure Functions and S...
CloudBurst 2019 - Indexing and searching NuGet.org with Azure Functions and S...
 
NDC Oslo 2019 - Indexing and searching NuGet.org with Azure Functions and Search
NDC Oslo 2019 - Indexing and searching NuGet.org with Azure Functions and SearchNDC Oslo 2019 - Indexing and searching NuGet.org with Azure Functions and Search
NDC Oslo 2019 - Indexing and searching NuGet.org with Azure Functions and Search
 
Indexing and searching NuGet.org with Azure Functions and Search - Cloud Deve...
Indexing and searching NuGet.org with Azure Functions and Search - Cloud Deve...Indexing and searching NuGet.org with Azure Functions and Search - Cloud Deve...
Indexing and searching NuGet.org with Azure Functions and Search - Cloud Deve...
 
CodeStock - Exploring .NET memory management - a trip down memory lane
CodeStock - Exploring .NET memory management - a trip down memory laneCodeStock - Exploring .NET memory management - a trip down memory lane
CodeStock - Exploring .NET memory management - a trip down memory lane
 
Microservices for building an IDE – The innards of JetBrains Rider - TechDays...
Microservices for building an IDE – The innards of JetBrains Rider - TechDays...Microservices for building an IDE – The innards of JetBrains Rider - TechDays...
Microservices for building an IDE – The innards of JetBrains Rider - TechDays...
 
JetBrains Day Seoul - Exploring .NET’s memory management – a trip down memory...
JetBrains Day Seoul - Exploring .NET’s memory management – a trip down memory...JetBrains Day Seoul - Exploring .NET’s memory management – a trip down memory...
JetBrains Day Seoul - Exploring .NET’s memory management – a trip down memory...
 
DotNetFest - Let’s refresh our memory! Memory management in .NET
DotNetFest - Let’s refresh our memory! Memory management in .NETDotNetFest - Let’s refresh our memory! Memory management in .NET
DotNetFest - Let’s refresh our memory! Memory management in .NET
 
What is going on - Application diagnostics on Azure - TechDays Finland
What is going on - Application diagnostics on Azure - TechDays FinlandWhat is going on - Application diagnostics on Azure - TechDays Finland
What is going on - Application diagnostics on Azure - TechDays Finland
 
ConFoo - Exploring .NET’s memory management – a trip down memory lane
ConFoo - Exploring .NET’s memory management – a trip down memory laneConFoo - Exploring .NET’s memory management – a trip down memory lane
ConFoo - Exploring .NET’s memory management – a trip down memory lane
 
ConFoo - NuGet beyond Hello World
ConFoo - NuGet beyond Hello WorldConFoo - NuGet beyond Hello World
ConFoo - NuGet beyond Hello World
 
NuGet beyond Hello World - DotNext Piter 2017
NuGet beyond Hello World - DotNext Piter 2017NuGet beyond Hello World - DotNext Piter 2017
NuGet beyond Hello World - DotNext Piter 2017
 
Exploring .NET memory management (iSense)
Exploring .NET memory management (iSense)Exploring .NET memory management (iSense)
Exploring .NET memory management (iSense)
 

Recently uploaded

Unleash the Solace Pub Sub connector | Banaglore MuleSoft Meetup #31
Unleash the Solace Pub Sub connector | Banaglore MuleSoft Meetup #31Unleash the Solace Pub Sub connector | Banaglore MuleSoft Meetup #31
Unleash the Solace Pub Sub connector | Banaglore MuleSoft Meetup #31shyamraj55
 
Utilising Energy Modelling for LCSF and PSDS Funding Applications
Utilising Energy Modelling for LCSF and PSDS Funding ApplicationsUtilising Energy Modelling for LCSF and PSDS Funding Applications
Utilising Energy Modelling for LCSF and PSDS Funding ApplicationsIES VE
 
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24Umar Saif
 
Python For Kids - Sách Lập trình cho trẻ em
Python For Kids - Sách Lập trình cho trẻ emPython For Kids - Sách Lập trình cho trẻ em
Python For Kids - Sách Lập trình cho trẻ emNho Vĩnh
 
Relationship Counselling: From Disjointed Features to Product-First Thinking ...
Relationship Counselling: From Disjointed Features to Product-First Thinking ...Relationship Counselling: From Disjointed Features to Product-First Thinking ...
Relationship Counselling: From Disjointed Features to Product-First Thinking ...Product School
 
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...Neo4j
 
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...Product School
 
Establishing data sharing standards to promote global industry development
Establishing data sharing standards to promote global industry developmentEstablishing data sharing standards to promote global industry development
Establishing data sharing standards to promote global industry developmentThorsten Huelsmann
 
Mastering Play Store App Listing and Optimization
Mastering Play Store App Listing and OptimizationMastering Play Store App Listing and Optimization
Mastering Play Store App Listing and OptimizationAppsthentic Technology
 
Building Bridges: Merging RPA Processes, UiPath Apps, and Data Service to bu...
Building Bridges:  Merging RPA Processes, UiPath Apps, and Data Service to bu...Building Bridges:  Merging RPA Processes, UiPath Apps, and Data Service to bu...
Building Bridges: Merging RPA Processes, UiPath Apps, and Data Service to bu...DianaGray10
 
Pragmatic UI testing with Compose Semantics.pdf
Pragmatic UI testing with Compose Semantics.pdfPragmatic UI testing with Compose Semantics.pdf
Pragmatic UI testing with Compose Semantics.pdfinfogdgmi
 
Centralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-ManagerCentralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-ManagerSaiLinnThu2
 
AGFM - Toyota Coaster 1HZ Install Guide.pdf
AGFM - Toyota Coaster 1HZ Install Guide.pdfAGFM - Toyota Coaster 1HZ Install Guide.pdf
AGFM - Toyota Coaster 1HZ Install Guide.pdfRodneyThomas28
 
Huntly presentation deck design for Behance
Huntly presentation deck design for BehanceHuntly presentation deck design for Behance
Huntly presentation deck design for Behancewhalesdesign
 
Introduction to Multimodal LLMs with LLaVA
Introduction to Multimodal LLMs with LLaVAIntroduction to Multimodal LLMs with LLaVA
Introduction to Multimodal LLMs with LLaVARobert McDermott
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfSafe Software
 
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...UiPathCommunity
 
Roundtable_-_API_Research__Testing_Tools.pdf
Roundtable_-_API_Research__Testing_Tools.pdfRoundtable_-_API_Research__Testing_Tools.pdf
Roundtable_-_API_Research__Testing_Tools.pdfMostafa Higazy
 
AI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficientAI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficientKari Kakkonen
 

Recently uploaded (20)

Unleash the Solace Pub Sub connector | Banaglore MuleSoft Meetup #31
Unleash the Solace Pub Sub connector | Banaglore MuleSoft Meetup #31Unleash the Solace Pub Sub connector | Banaglore MuleSoft Meetup #31
Unleash the Solace Pub Sub connector | Banaglore MuleSoft Meetup #31
 
Utilising Energy Modelling for LCSF and PSDS Funding Applications
Utilising Energy Modelling for LCSF and PSDS Funding ApplicationsUtilising Energy Modelling for LCSF and PSDS Funding Applications
Utilising Energy Modelling for LCSF and PSDS Funding Applications
 
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24
 
Python For Kids - Sách Lập trình cho trẻ em
Python For Kids - Sách Lập trình cho trẻ emPython For Kids - Sách Lập trình cho trẻ em
Python For Kids - Sách Lập trình cho trẻ em
 
Relationship Counselling: From Disjointed Features to Product-First Thinking ...
Relationship Counselling: From Disjointed Features to Product-First Thinking ...Relationship Counselling: From Disjointed Features to Product-First Thinking ...
Relationship Counselling: From Disjointed Features to Product-First Thinking ...
 
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
 
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...
 
Establishing data sharing standards to promote global industry development
Establishing data sharing standards to promote global industry developmentEstablishing data sharing standards to promote global industry development
Establishing data sharing standards to promote global industry development
 
Mastering Play Store App Listing and Optimization
Mastering Play Store App Listing and OptimizationMastering Play Store App Listing and Optimization
Mastering Play Store App Listing and Optimization
 
Building Bridges: Merging RPA Processes, UiPath Apps, and Data Service to bu...
Building Bridges:  Merging RPA Processes, UiPath Apps, and Data Service to bu...Building Bridges:  Merging RPA Processes, UiPath Apps, and Data Service to bu...
Building Bridges: Merging RPA Processes, UiPath Apps, and Data Service to bu...
 
Pragmatic UI testing with Compose Semantics.pdf
Pragmatic UI testing with Compose Semantics.pdfPragmatic UI testing with Compose Semantics.pdf
Pragmatic UI testing with Compose Semantics.pdf
 
In sharing we trust. Taking advantage of a diverse consortium to build a tran...
In sharing we trust. Taking advantage of a diverse consortium to build a tran...In sharing we trust. Taking advantage of a diverse consortium to build a tran...
In sharing we trust. Taking advantage of a diverse consortium to build a tran...
 
Centralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-ManagerCentralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-Manager
 
AGFM - Toyota Coaster 1HZ Install Guide.pdf
AGFM - Toyota Coaster 1HZ Install Guide.pdfAGFM - Toyota Coaster 1HZ Install Guide.pdf
AGFM - Toyota Coaster 1HZ Install Guide.pdf
 
Huntly presentation deck design for Behance
Huntly presentation deck design for BehanceHuntly presentation deck design for Behance
Huntly presentation deck design for Behance
 
Introduction to Multimodal LLMs with LLaVA
Introduction to Multimodal LLMs with LLaVAIntroduction to Multimodal LLMs with LLaVA
Introduction to Multimodal LLMs with LLaVA
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
 
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...
 
Roundtable_-_API_Research__Testing_Tools.pdf
Roundtable_-_API_Research__Testing_Tools.pdfRoundtable_-_API_Research__Testing_Tools.pdf
Roundtable_-_API_Research__Testing_Tools.pdf
 
AI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficientAI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficient
 

ConFoo Montreal - Approaches for application request throttling

  • 1. 1 Approaches for application request throttling Maarten Balliauw @maartenballiauw
  • 2. 3 Agenda Users and traffic patterns Rate limiting and considerations Which resources? Which limits? Who to limit? Who not to limit? What when a limit is reached? Where to limit?
  • 4. 5 MyGet Hosted private package repository – www.myget.org NuGet, NPM, Bower, Maven, VSIX, PHP Composer, Symbols, ... HTTP-based Web UI for managing things API for various package managers PUT/POST – Upload package DELETE – Delete package via API GET – Fetch metadata or binary
  • 5. 6 Background workers for scale Example: package upload PUT/POST binary and metadata to front-end PackageAddedEvent on queue with many handlers handled on back-end ProcessSymbols UpdateLatestVersion Indexing ...
  • 6. 8 What could possibly go wrong... Too many downloads! Application logic has to check credentials, subscription, quota 404’s still need that application logic... Package managers are crazy! Total # requests Total # 404’s % 404’s # of packages in solution 200 800 600 # on NuGet.org 190 200 10 5% # on MyGet feed 1 5 200 195 97,5% # on MyGet feed 2 4 200 196 98% # on company-internal TeamCity 1 200 199 99,5%
  • 7. 11 Other examples Web UI requests Trying to register spam accounts Trying to brute-force login/password reset Trying to validate credit card numbers via a form on your site ...cost money in the cloud (e.g. per serverless execution) Robots / Crawlers Imagine a spider adding 20k items to a shopping cart For us, usually fine (e.g. Googlebot by default up to 5 req/sec) Limiting is easy with rel=“nofollow” and robots.txt crawl-delay
  • 10. 14 Rate limiting – what? Limits # of requests in a given timeframe Or limits bandwidth, or another resource – up to you Helps eliminate: Unexpected traffic patterns Unwanted traffic patterns (e.g. script kiddie brute-force login) Potentiallly damaging traffic patterns (accidental and malicious)
  • 11. 15 Rate limit everything. - Maarten Balliauw
  • 12. 16 Rate limiting – everything??? Everything that could slow down or break your application Typically everything that depends on a scarce or external resource CPU Memory Disk I/O Database External API So yes, everything...
  • 13. 17 Let’s do this! Database with table Events UserIdentifier – who do we limit ActionIdentifier – what do we limit When – event timestamp so we can apply a query Filter attribute SELECT COUNT(*) FROM Events WHERE UserIdentifier = <user> AND ActionIdentifier = <action> AND When >= NOW() – X INSERT INTO Events (<user>, <action>, NOW()) DELETE FROM Events WHERE UserIdentifier = <user> AND ActionIdentifier = <action> AND When < NOW() – X
  • 16. 20 That database was a bad idea! Very flexible in defining various limits or doing combinations Very flexible in changing limits, e.g. changing the time period The database will suffer at scale... Every request is at least 2 – 3 queries Constant index churn We need to manually run DELETE to remove old events Database size!
  • 17. 21 That database was a bad idea! We created a denial of service opportunity! SELECT, INSERT, DELETE for every request Consider a simpler technique to limit # of operations Ideally just a simple counter “Buckets”
  • 18. 22 Quantized buckets Create “buckets” per <identifier> and <timespan> Use incr <bucket> on Redis and get back the current count per <timespan> public string GetBucketName(string operation, TimeSpan timespan) { var bucket = Math.Floor( DateTime.UtcNow.Ticks / timespan.TotalMilliseconds / 10000); return $"{operation}_{bucket}"; } Console.WriteLine(GetBucketName("someaction", TimeSpan.FromMinutes(10))); // someaction_106062120 <-- this will be the key for +/- 10 minutes
  • 19. 23 Quantized buckets Super easy and super cheap (atomic write and read on Redis, auto-expire LRU) Not accurate... (but that may be ok) (n-1)x2 / 10 sec Theoretically: max. 6 / 10 sec
  • 20. 24 Leaky bucket “Imagine a bucket where water is poured in at the top and leaks from the bottom. If the rate at which water is poured in exceeds the rate at which it leaks, the bucket overflows.“ Widely used in telecommunications to deal with bandwidth/bursts.
  • 21. 25 Leaky bucket Get <delta> tokens, with maximum <count> per <timespan> public int GetCallsLeft() { if (_tokens < _capacity) { var referenceTime = DateTime.UtcNow; var delta = (int)((referenceTime - _lastRefill).Ticks / _interval.Ticks); if (delta > 0) { _tokens = Math.Min(_capacity, _tokens + (delta * _capacity)); _lastRefill = referenceTime; } } return _tokens; }
  • 24. 28 Things to decide on Decide on the resources to limit Decide on a sensible limit Come up with an identifier to limit on Decide on exceptions to the rule
  • 25. 29 Which resources to limit? ...
  • 26. 30 Rate limit everything. - Maarten Balliauw
  • 27. 31 What are sensible limits? Approach 1 1. Figure out current # of requests for a certain resource 2. Set limits 3. Get angry phone calls from customers Approach 2 1. Figure out current # of requests for a certain resource 2. Set limits, but only log when a request would be limited 3. Analyze logs, set new limits, ... 4. Start rate limiting 5. Keep measuring
  • 28. 32 Will you allow bursts or not? Laddering! Different buckets per identifier and resource... 10 requests per second can be 36000 requests per hour. But 10 requests per second could also be 1000 requests per hour. Bucket Operation A Operation B Operation C Per second 10 10 100 Per minute 60 60 500 Per hour 3600 600 500 ... Steady flow of max. 10/sec Steady flow of max. 10/sec, but only 600/hour max. Bursts of up to 100/sec, but only 500/hour max.
  • 29. 33 What will be the identifier? Per IP address? But what with NAT/proxy? Per user? But how do you limit anonymous users? Per session? But what when the user starts a new session for every request? Or what if there is no such thing as a session? Per browser? But everyone uses Chrome! Probably a combination of these...
  • 30. 34 Decide on exceptions Do we rate limit all users? Do we have separate limits for certain users? Dynamic limiting Do we rate limit all IP addresses? What about ourselves? What about our monitoring tools? What about web crawlers? What about certain datacenter ranges? (https://github.com/client9/ipcat) “IP addresses that end web consumers should not be using"
  • 32. 36 What when the user hits the limit? Do we just “black hole” and close the connection? Do you tell the user? API: status code 429 Too Many Requests Web: error page stating rate limit exceeded / captcha (StackOverflow)
  • 33. 37 Try to always tell the user Format? Depends on Accept header (text/html vs. application/json) Tell them why they were throttled Can be a simple link to API documentation Tell them when to retry (e.g. GitHub does this even before rate limiting) Status: 200 OK X-RateLimit-Limit: 5000 X-RateLimit-Remaining: 4999 X-RateLimit-Reset: 1372700873
  • 34. 38 Where do we limit?
  • 35. 39 Rate limiting – where? MvcThrottle Runs as action filter Requests per timespan Per action, user, IP, ... (so knows about actions) Owin.Limits Runs as OWIN middleware Bandwidth, concurrent requests, ... No knowledge about application specifics Many, many others
  • 37. 41 How far do we allow traffic before saying no? KNOWLEDGE ABOUT THE OPERATION RESOURCES SPENT
  • 38. 42 How far do we allow traffic before saying no? KNOWLEDGE ABOUT THE OPERATION RESOURCES SPENT
  • 39. 43 What options are there? In our application ActionFilter / Middleware / HttpModule / ... Easy to add custom logic, based on request details On the server Outside of our server Outside of our datacenter
  • 40. 44 What options are there? In our application On the server IIS has dynamic IP restrictions, bit rate throttling, <limits /> Kestrel minimum speed throttle Found these less flexible in terms of configuraton... E.g. IIS dynamic IP restrictions returns 403 Forbidden, wth! Not a big fan, as these are usually HttpModules anyway (and thus hit our app) Outside of our server Outside of our datacenter
  • 41. 45 What options are there? In our application On the server Outside of our server Reverse proxy (IIS Application Request Routing, NGinx, HAProxy, Squid, ...) Traffic does not even hit our application server, yay! Outside of our datacenter
  • 42. 46 Rate limiting with NGinx Demo
  • 43. 47 What options are there? In our application On the server Outside of our server Outside of our datacenter Azure API management, CloudFlare, ... Filters traffic very early in the request, yay! Often also handle DDoS attacks Often more expensive
  • 45. 49 Imagine... Your marketing team decided to bridge the physical world with the virtual: “Use our in-store wifi to join this online contest and win!”
  • 46. 50 Monitor your rate limiting! Monitor what is happening in your application Who are we rate limiting, when, why Allow for circuit breakers (“exceptional exceptions”) “This flood of requests is fine for now”
  • 48. 52 Conclusion Users are crazy! (unintentional) We need rate limiting Decide on the resources to limit (everything!) Decide on a sensible limit (measure!) Come up with an identifier to limit on Decide on exceptions What when the user reaches a limit? Decide where in the request/response flow to limit Monitor your rate limiting

Editor's Notes

  1. https://pixabay.com
  2. https://pixabay.com/en/tires-used-tires-pfu-garbage-1846674/
  3. https://pixabay.com/en/tires-used-tires-pfu-garbage-1846674/
  4. Prerequisites: create database and make sure it works! Open demo 01 - DemoLetsDoThis.sln In Startup.cs explain adding EF context and show how EventsContext is built Next, show RateLimitFilter is applied to every request Implementation of RateLimitFilter Uses an identifier for the user (either User.Identity.Username or “anonymous” + IP address) Uses ControllerActionDescriptor to determine controller + action We then check if there are > 5 requests to this resource We always add an event to the DB – DANGEROUS!!! And drop older events Show in Fiddler, requesting: http://localhost:56983/api/hello/maarten
  5. Open MvcThrottle, in project MvcThrottle.Demo Show HomeController, show EnableThrottling attribute Run the application - http://localhost:53048/Home/About – see it in action after a few refreshes Mention we can respond to throttlign depending on the client type! Open MvcThrottleCustomFilter See filterContext.HttpContext.Request.AcceptTypes.Any(accept => accept.Contains("html")) -> custom view result Mention we can filter based on client IP In FilterConfig.cs, there is an IP whitelist of folks we never want to throttle Same goes with user agents Same goes with endpoints The REALLY nice thing: I can enable/disable per action in MVC Show BlogController REALLY NICE, throttling follows my logic The SAD thing: open 04-snapshot I did a load test – non scientific! This thing has low overhead (did a few thousand requests) but still my aplication spent 12% of its time rate limiting requests
  6. Run the nginx docker container from 05-nginx Show a few requests: http://localhost:8080/ proxies MyGet http://localhost:8080/F/googleanalyticstracker/api/v2 proxies a feed A few refreshes of http://localhost:8080/F/googleanalyticstracker/api/v2 get throttled So we proxy our app, and get to rate limit some calls, sweet! Open nginx.conf and go through some boiler-plate: Worker processes and worker connections (typically == to # cores) http section sets up a web server, we can add SSL etc here as well Under server, we define the different resources / just proxies www.myget.org and injects a header /Content proxies and caches (yay added bonus of NGinx) /F/ is where things get interesting – we limit requests to this one using “mylimit” Defines a key, names a zone, names the timespan, names the limit Can mix and match to create key: limit_req_zone $binary_remote_addr$http_authorization zone=mylimit:10m rate=2r/s;