SlideShare a Scribd company logo
Beyond Inbound:
Jan 2024
Unlocking the Secrets
of API Egress Traffic
Management
2024 All rights reserved | Lunar.Inc
2
HI!
The Management Platform Built for 3rd Party APIs at Scale
Head of DevRel
Sean Keegan
Former math teacher turned DevRel.
Big fan of ultimate frisbee, video
games, APIs, and houseplants.
Jan 2024
We’ve set out to build a new paradigm for management and
optimization of 3rd party API consumption!
2024 All rights reserved | Lunar.Inc
We’re Lunar.dev
3
Agenda
2024 All rights reserved | Lunar.Inc
Introduction
01
Jan 2024
API Producer Landscape (Ingress)
API Consumer Landscape (Egress)
Problems with API consumption
Live demo of lunar.dev
02
03
04
05
Q&A, closing, survey
06
4
Quick vocab
2024 All rights reserved | Lunar.Inc
Jan 2024
Ingress traffic Egress traffic
5
Imagine: Your Company's
API is Taking Off!
2024 All rights reserved | Lunar.Inc
Jan 2024
6
What are some considerations as
demand for your API grows?
2024 All rights reserved | Lunar.Inc
Jan 2024
Join at Slido.com
#163 0466
7
The issues for API Producers
2024 All rights reserved | Lunar.Inc
▪ Implement Rate Limiting
Jan 2024
▪ Utilize authentication/authorization
▪ Apply throttling techniques
▪ Monitor performance
▪ Introduce caching mechanisms
▪ Scale horizontally
8
Consumed APIs > Served APIs
2024 All rights reserved | Lunar.Inc
Jan 2024
9
What challenges are there around
consuming APIs?
2024 All rights reserved | Lunar.Inc
Jan 2024
Join at Slido.com
#5549 710
10
Let’s use GenAI in our App!
2024 All rights reserved | Lunar.Inc
Jan 2024
11
Let’s use GenAI in our App!
2024 All rights reserved | Lunar.Inc
Jan 2024
OpenAI Documentation
12
Prepare for API Rate Limits
2024 All rights reserved | Lunar.Inc
Jan 2024
• LLM APIs have hard rate limits based on your usage tiers
• Rate limits are usually based on both:
• RPM – Requests per Minute
• As your app scales, API rate limits can cause service outage
for your users
• TPM – Tokens per Minute
13
API Rate Limits – Account and
Region Load Balancing
Load Balancer
https://andre-openai-eastus.openai.azure.com/
Your Application
2024 All rights reserved | Lunar.Inc
Jan 2024
https://andre-openai-eastus-2.openai.azure.com/
https://andre-openai-
canadaeast.openai.azure.com/
https://andre-openai-francecentral.openai.azure.com/
…
Send API Call with Prompt
Write a story about the following
subject: {{user_input}}
15
Let’s use GenAI in our App!
2024 All rights reserved | Lunar.Inc
Jan 2024
16
The dark side of API Management
Producing APIs Consuming APIs
API Landscape
f
Cost
f
Reliability
f
Safety
f
Scalability
2024 All rights reserved | Lunar.Inc
Jan 2024
17
2024 All rights reserved | Lunar.Inc
Jan 2024
18
A mindset change - managing your Egress Traffic is a must
So…What can be done?
2024 All rights reserved | Lunar.Inc
Jan 2024
Smart Caching
Queue
Cost-controls
Alternating API Providers
Error handling
Logging API calls over time
Retry
API account orchestration
Fallbacks
Strategy – Based Throttling
PII removal
Authentication Mechanism
API Calls logging
Enforcing HTTPS
API response inspection
API call inspection
19
2024 All rights reserved | Lunar.Inc
Jan 2024
20
2024 All rights reserved | Lunar.Inc
Jan 2024
Questions?
21
2024 All rights reserved | Lunar.Inc
Jan 2024
Check us out!
22
Thank you for Listening!
Follow up at me@lunar.dev!
2024 All rights reserved | Lunar.Inc
Jan 2024

More Related Content

Similar to Beyond Inbound: Unlocking the Secrets of API Egress Traffic Management

apidays LIVE LONDON - API Lifecycle Management - Avoiding Breaches By Securin...
apidays LIVE LONDON - API Lifecycle Management - Avoiding Breaches By Securin...apidays LIVE LONDON - API Lifecycle Management - Avoiding Breaches By Securin...
apidays LIVE LONDON - API Lifecycle Management - Avoiding Breaches By Securin...
apidays
 
API Management - Practical Enterprise Implementation Experience
API Management - Practical Enterprise Implementation ExperienceAPI Management - Practical Enterprise Implementation Experience
API Management - Practical Enterprise Implementation Experience
Capgemini
 
IBM z Systems: At the heart of the Mobile, Digital & API Economy
IBM z Systems: At the heart of the Mobile, Digital & API EconomyIBM z Systems: At the heart of the Mobile, Digital & API Economy
IBM z Systems: At the heart of the Mobile, Digital & API Economy
Richard Gamblin
 
Transforming z Systems for the Mobile, Digital & API Economy
Transforming z Systems for the Mobile, Digital & API EconomyTransforming z Systems for the Mobile, Digital & API Economy
Transforming z Systems for the Mobile, Digital & API Economy
Ana Alves Sequeira
 
Serverless applications with AWS
Serverless applications with AWSServerless applications with AWS
Serverless applications with AWS
javier ramirez
 
Z api overview
Z api overviewZ api overview
Z api overview
bamadhu
 
Building Serverless Applications That Align with Twelve-Factor Methods - AWS ...
Building Serverless Applications That Align with Twelve-Factor Methods - AWS ...Building Serverless Applications That Align with Twelve-Factor Methods - AWS ...
Building Serverless Applications That Align with Twelve-Factor Methods - AWS ...
Amazon Web Services
 
Accelerating Mobile App Data Synchronization and Real-Time Data Development w...
Accelerating Mobile App Data Synchronization and Real-Time Data Development w...Accelerating Mobile App Data Synchronization and Real-Time Data Development w...
Accelerating Mobile App Data Synchronization and Real-Time Data Development w...
Amazon Web Services
 
APIs - the oil pumps of the 21st century
APIs - the oil pumps of the 21st centuryAPIs - the oil pumps of the 21st century
APIs - the oil pumps of the 21st century
Frank Schwab
 
API Management: Unlock the Value of Your Unique Assets with a Robust API
API Management: Unlock the Value of Your Unique Assets with a Robust APIAPI Management: Unlock the Value of Your Unique Assets with a Robust API
API Management: Unlock the Value of Your Unique Assets with a Robust API
Software AG
 
From Monolith to Microservices
From Monolith to MicroservicesFrom Monolith to Microservices
From Monolith to Microservices
Amazon Web Services
 
Disrupt or be disrupted – Using secure APIs to drive digital transformation
Disrupt or be disrupted – Using secure APIs to drive digital transformationDisrupt or be disrupted – Using secure APIs to drive digital transformation
Disrupt or be disrupted – Using secure APIs to drive digital transformation
Rogue Wave Software
 
Apidays Paris 2023 - The Federated Future, Tim Biedenkapp, adorsys
Apidays Paris 2023 - The Federated Future, Tim Biedenkapp, adorsysApidays Paris 2023 - The Federated Future, Tim Biedenkapp, adorsys
Apidays Paris 2023 - The Federated Future, Tim Biedenkapp, adorsys
apidays
 
Will you survive the API avalanche?
Will you survive the API avalanche?Will you survive the API avalanche?
Will you survive the API avalanche?
Rogue Wave Software
 
Api management customer
Api management customerApi management customer
Api management customer
nick_garrod
 
Open Banking APIs on AWS
Open Banking APIs on AWSOpen Banking APIs on AWS
Open Banking APIs on AWS
Amazon Web Services
 
The next generation of ap is luis weir.cwin18.telford
The next generation of ap is   luis weir.cwin18.telfordThe next generation of ap is   luis weir.cwin18.telford
The next generation of ap is luis weir.cwin18.telford
Capgemini
 
Wildrydes Serverless Workshop Tel Aviv
Wildrydes Serverless Workshop Tel AvivWildrydes Serverless Workshop Tel Aviv
Wildrydes Serverless Workshop Tel Aviv
Boaz Ziniman
 
App-First & Cloud-Native: How InterMiles Boosted CX with AWS & Infostretch
App-First & Cloud-Native: How InterMiles Boosted CX with AWS & InfostretchApp-First & Cloud-Native: How InterMiles Boosted CX with AWS & Infostretch
App-First & Cloud-Native: How InterMiles Boosted CX with AWS & Infostretch
Infostretch
 
Microservices for Startups
Microservices for StartupsMicroservices for Startups
Microservices for Startups
Amazon Web Services
 

Similar to Beyond Inbound: Unlocking the Secrets of API Egress Traffic Management (20)

apidays LIVE LONDON - API Lifecycle Management - Avoiding Breaches By Securin...
apidays LIVE LONDON - API Lifecycle Management - Avoiding Breaches By Securin...apidays LIVE LONDON - API Lifecycle Management - Avoiding Breaches By Securin...
apidays LIVE LONDON - API Lifecycle Management - Avoiding Breaches By Securin...
 
API Management - Practical Enterprise Implementation Experience
API Management - Practical Enterprise Implementation ExperienceAPI Management - Practical Enterprise Implementation Experience
API Management - Practical Enterprise Implementation Experience
 
IBM z Systems: At the heart of the Mobile, Digital & API Economy
IBM z Systems: At the heart of the Mobile, Digital & API EconomyIBM z Systems: At the heart of the Mobile, Digital & API Economy
IBM z Systems: At the heart of the Mobile, Digital & API Economy
 
Transforming z Systems for the Mobile, Digital & API Economy
Transforming z Systems for the Mobile, Digital & API EconomyTransforming z Systems for the Mobile, Digital & API Economy
Transforming z Systems for the Mobile, Digital & API Economy
 
Serverless applications with AWS
Serverless applications with AWSServerless applications with AWS
Serverless applications with AWS
 
Z api overview
Z api overviewZ api overview
Z api overview
 
Building Serverless Applications That Align with Twelve-Factor Methods - AWS ...
Building Serverless Applications That Align with Twelve-Factor Methods - AWS ...Building Serverless Applications That Align with Twelve-Factor Methods - AWS ...
Building Serverless Applications That Align with Twelve-Factor Methods - AWS ...
 
Accelerating Mobile App Data Synchronization and Real-Time Data Development w...
Accelerating Mobile App Data Synchronization and Real-Time Data Development w...Accelerating Mobile App Data Synchronization and Real-Time Data Development w...
Accelerating Mobile App Data Synchronization and Real-Time Data Development w...
 
APIs - the oil pumps of the 21st century
APIs - the oil pumps of the 21st centuryAPIs - the oil pumps of the 21st century
APIs - the oil pumps of the 21st century
 
API Management: Unlock the Value of Your Unique Assets with a Robust API
API Management: Unlock the Value of Your Unique Assets with a Robust APIAPI Management: Unlock the Value of Your Unique Assets with a Robust API
API Management: Unlock the Value of Your Unique Assets with a Robust API
 
From Monolith to Microservices
From Monolith to MicroservicesFrom Monolith to Microservices
From Monolith to Microservices
 
Disrupt or be disrupted – Using secure APIs to drive digital transformation
Disrupt or be disrupted – Using secure APIs to drive digital transformationDisrupt or be disrupted – Using secure APIs to drive digital transformation
Disrupt or be disrupted – Using secure APIs to drive digital transformation
 
Apidays Paris 2023 - The Federated Future, Tim Biedenkapp, adorsys
Apidays Paris 2023 - The Federated Future, Tim Biedenkapp, adorsysApidays Paris 2023 - The Federated Future, Tim Biedenkapp, adorsys
Apidays Paris 2023 - The Federated Future, Tim Biedenkapp, adorsys
 
Will you survive the API avalanche?
Will you survive the API avalanche?Will you survive the API avalanche?
Will you survive the API avalanche?
 
Api management customer
Api management customerApi management customer
Api management customer
 
Open Banking APIs on AWS
Open Banking APIs on AWSOpen Banking APIs on AWS
Open Banking APIs on AWS
 
The next generation of ap is luis weir.cwin18.telford
The next generation of ap is   luis weir.cwin18.telfordThe next generation of ap is   luis weir.cwin18.telford
The next generation of ap is luis weir.cwin18.telford
 
Wildrydes Serverless Workshop Tel Aviv
Wildrydes Serverless Workshop Tel AvivWildrydes Serverless Workshop Tel Aviv
Wildrydes Serverless Workshop Tel Aviv
 
App-First & Cloud-Native: How InterMiles Boosted CX with AWS & Infostretch
App-First & Cloud-Native: How InterMiles Boosted CX with AWS & InfostretchApp-First & Cloud-Native: How InterMiles Boosted CX with AWS & Infostretch
App-First & Cloud-Native: How InterMiles Boosted CX with AWS & Infostretch
 
Microservices for Startups
Microservices for StartupsMicroservices for Startups
Microservices for Startups
 

Recently uploaded

JAVIER LASA-EXPERIENCIA digital 1986-2024.pdf
JAVIER LASA-EXPERIENCIA digital 1986-2024.pdfJAVIER LASA-EXPERIENCIA digital 1986-2024.pdf
JAVIER LASA-EXPERIENCIA digital 1986-2024.pdf
Javier Lasa
 
This 7-second Brain Wave Ritual Attracts Money To You.!
This 7-second Brain Wave Ritual Attracts Money To You.!This 7-second Brain Wave Ritual Attracts Money To You.!
This 7-second Brain Wave Ritual Attracts Money To You.!
nirahealhty
 
How to Use Contact Form 7 Like a Pro.pptx
How to Use Contact Form 7 Like a Pro.pptxHow to Use Contact Form 7 Like a Pro.pptx
How to Use Contact Form 7 Like a Pro.pptx
Gal Baras
 
History+of+E-commerce+Development+in+China-www.cfye-commerce.shop
History+of+E-commerce+Development+in+China-www.cfye-commerce.shopHistory+of+E-commerce+Development+in+China-www.cfye-commerce.shop
History+of+E-commerce+Development+in+China-www.cfye-commerce.shop
laozhuseo02
 
一比一原版(SLU毕业证)圣路易斯大学毕业证成绩单专业办理
一比一原版(SLU毕业证)圣路易斯大学毕业证成绩单专业办理一比一原版(SLU毕业证)圣路易斯大学毕业证成绩单专业办理
一比一原版(SLU毕业证)圣路易斯大学毕业证成绩单专业办理
keoku
 
The+Prospects+of+E-Commerce+in+China.pptx
The+Prospects+of+E-Commerce+in+China.pptxThe+Prospects+of+E-Commerce+in+China.pptx
The+Prospects+of+E-Commerce+in+China.pptx
laozhuseo02
 
Internet-Security-Safeguarding-Your-Digital-World (1).pptx
Internet-Security-Safeguarding-Your-Digital-World (1).pptxInternet-Security-Safeguarding-Your-Digital-World (1).pptx
Internet-Security-Safeguarding-Your-Digital-World (1).pptx
VivekSinghShekhawat2
 
BASIC C++ lecture NOTE C++ lecture 3.pptx
BASIC C++ lecture NOTE C++ lecture 3.pptxBASIC C++ lecture NOTE C++ lecture 3.pptx
BASIC C++ lecture NOTE C++ lecture 3.pptx
natyesu
 
APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024
APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024
APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024
APNIC
 
1.Wireless Communication System_Wireless communication is a broad term that i...
1.Wireless Communication System_Wireless communication is a broad term that i...1.Wireless Communication System_Wireless communication is a broad term that i...
1.Wireless Communication System_Wireless communication is a broad term that i...
JeyaPerumal1
 
Comptia N+ Standard Networking lesson guide
Comptia N+ Standard Networking lesson guideComptia N+ Standard Networking lesson guide
Comptia N+ Standard Networking lesson guide
GTProductions1
 
guildmasters guide to ravnica Dungeons & Dragons 5...
guildmasters guide to ravnica Dungeons & Dragons 5...guildmasters guide to ravnica Dungeons & Dragons 5...
guildmasters guide to ravnica Dungeons & Dragons 5...
Rogerio Filho
 
Multi-cluster Kubernetes Networking- Patterns, Projects and Guidelines
Multi-cluster Kubernetes Networking- Patterns, Projects and GuidelinesMulti-cluster Kubernetes Networking- Patterns, Projects and Guidelines
Multi-cluster Kubernetes Networking- Patterns, Projects and Guidelines
Sanjeev Rampal
 
1比1复刻(bath毕业证书)英国巴斯大学毕业证学位证原版一模一样
1比1复刻(bath毕业证书)英国巴斯大学毕业证学位证原版一模一样1比1复刻(bath毕业证书)英国巴斯大学毕业证学位证原版一模一样
1比1复刻(bath毕业证书)英国巴斯大学毕业证学位证原版一模一样
3ipehhoa
 
原版仿制(uob毕业证书)英国伯明翰大学毕业证本科学历证书原版一模一样
原版仿制(uob毕业证书)英国伯明翰大学毕业证本科学历证书原版一模一样原版仿制(uob毕业证书)英国伯明翰大学毕业证本科学历证书原版一模一样
原版仿制(uob毕业证书)英国伯明翰大学毕业证本科学历证书原版一模一样
3ipehhoa
 
Bridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptx
Bridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptxBridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptx
Bridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptx
Brad Spiegel Macon GA
 
Latest trends in computer networking.pptx
Latest trends in computer networking.pptxLatest trends in computer networking.pptx
Latest trends in computer networking.pptx
JungkooksNonexistent
 
急速办(bedfordhire毕业证书)英国贝德福特大学毕业证成绩单原版一模一样
急速办(bedfordhire毕业证书)英国贝德福特大学毕业证成绩单原版一模一样急速办(bedfordhire毕业证书)英国贝德福特大学毕业证成绩单原版一模一样
急速办(bedfordhire毕业证书)英国贝德福特大学毕业证成绩单原版一模一样
3ipehhoa
 
test test test test testtest test testtest test testtest test testtest test ...
test test  test test testtest test testtest test testtest test testtest test ...test test  test test testtest test testtest test testtest test testtest test ...
test test test test testtest test testtest test testtest test testtest test ...
Arif0071
 
一比一原版(LBS毕业证)伦敦商学院毕业证成绩单专业办理
一比一原版(LBS毕业证)伦敦商学院毕业证成绩单专业办理一比一原版(LBS毕业证)伦敦商学院毕业证成绩单专业办理
一比一原版(LBS毕业证)伦敦商学院毕业证成绩单专业办理
eutxy
 

Recently uploaded (20)

JAVIER LASA-EXPERIENCIA digital 1986-2024.pdf
JAVIER LASA-EXPERIENCIA digital 1986-2024.pdfJAVIER LASA-EXPERIENCIA digital 1986-2024.pdf
JAVIER LASA-EXPERIENCIA digital 1986-2024.pdf
 
This 7-second Brain Wave Ritual Attracts Money To You.!
This 7-second Brain Wave Ritual Attracts Money To You.!This 7-second Brain Wave Ritual Attracts Money To You.!
This 7-second Brain Wave Ritual Attracts Money To You.!
 
How to Use Contact Form 7 Like a Pro.pptx
How to Use Contact Form 7 Like a Pro.pptxHow to Use Contact Form 7 Like a Pro.pptx
How to Use Contact Form 7 Like a Pro.pptx
 
History+of+E-commerce+Development+in+China-www.cfye-commerce.shop
History+of+E-commerce+Development+in+China-www.cfye-commerce.shopHistory+of+E-commerce+Development+in+China-www.cfye-commerce.shop
History+of+E-commerce+Development+in+China-www.cfye-commerce.shop
 
一比一原版(SLU毕业证)圣路易斯大学毕业证成绩单专业办理
一比一原版(SLU毕业证)圣路易斯大学毕业证成绩单专业办理一比一原版(SLU毕业证)圣路易斯大学毕业证成绩单专业办理
一比一原版(SLU毕业证)圣路易斯大学毕业证成绩单专业办理
 
The+Prospects+of+E-Commerce+in+China.pptx
The+Prospects+of+E-Commerce+in+China.pptxThe+Prospects+of+E-Commerce+in+China.pptx
The+Prospects+of+E-Commerce+in+China.pptx
 
Internet-Security-Safeguarding-Your-Digital-World (1).pptx
Internet-Security-Safeguarding-Your-Digital-World (1).pptxInternet-Security-Safeguarding-Your-Digital-World (1).pptx
Internet-Security-Safeguarding-Your-Digital-World (1).pptx
 
BASIC C++ lecture NOTE C++ lecture 3.pptx
BASIC C++ lecture NOTE C++ lecture 3.pptxBASIC C++ lecture NOTE C++ lecture 3.pptx
BASIC C++ lecture NOTE C++ lecture 3.pptx
 
APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024
APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024
APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024
 
1.Wireless Communication System_Wireless communication is a broad term that i...
1.Wireless Communication System_Wireless communication is a broad term that i...1.Wireless Communication System_Wireless communication is a broad term that i...
1.Wireless Communication System_Wireless communication is a broad term that i...
 
Comptia N+ Standard Networking lesson guide
Comptia N+ Standard Networking lesson guideComptia N+ Standard Networking lesson guide
Comptia N+ Standard Networking lesson guide
 
guildmasters guide to ravnica Dungeons & Dragons 5...
guildmasters guide to ravnica Dungeons & Dragons 5...guildmasters guide to ravnica Dungeons & Dragons 5...
guildmasters guide to ravnica Dungeons & Dragons 5...
 
Multi-cluster Kubernetes Networking- Patterns, Projects and Guidelines
Multi-cluster Kubernetes Networking- Patterns, Projects and GuidelinesMulti-cluster Kubernetes Networking- Patterns, Projects and Guidelines
Multi-cluster Kubernetes Networking- Patterns, Projects and Guidelines
 
1比1复刻(bath毕业证书)英国巴斯大学毕业证学位证原版一模一样
1比1复刻(bath毕业证书)英国巴斯大学毕业证学位证原版一模一样1比1复刻(bath毕业证书)英国巴斯大学毕业证学位证原版一模一样
1比1复刻(bath毕业证书)英国巴斯大学毕业证学位证原版一模一样
 
原版仿制(uob毕业证书)英国伯明翰大学毕业证本科学历证书原版一模一样
原版仿制(uob毕业证书)英国伯明翰大学毕业证本科学历证书原版一模一样原版仿制(uob毕业证书)英国伯明翰大学毕业证本科学历证书原版一模一样
原版仿制(uob毕业证书)英国伯明翰大学毕业证本科学历证书原版一模一样
 
Bridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptx
Bridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptxBridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptx
Bridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptx
 
Latest trends in computer networking.pptx
Latest trends in computer networking.pptxLatest trends in computer networking.pptx
Latest trends in computer networking.pptx
 
急速办(bedfordhire毕业证书)英国贝德福特大学毕业证成绩单原版一模一样
急速办(bedfordhire毕业证书)英国贝德福特大学毕业证成绩单原版一模一样急速办(bedfordhire毕业证书)英国贝德福特大学毕业证成绩单原版一模一样
急速办(bedfordhire毕业证书)英国贝德福特大学毕业证成绩单原版一模一样
 
test test test test testtest test testtest test testtest test testtest test ...
test test  test test testtest test testtest test testtest test testtest test ...test test  test test testtest test testtest test testtest test testtest test ...
test test test test testtest test testtest test testtest test testtest test ...
 
一比一原版(LBS毕业证)伦敦商学院毕业证成绩单专业办理
一比一原版(LBS毕业证)伦敦商学院毕业证成绩单专业办理一比一原版(LBS毕业证)伦敦商学院毕业证成绩单专业办理
一比一原版(LBS毕业证)伦敦商学院毕业证成绩单专业办理
 

Beyond Inbound: Unlocking the Secrets of API Egress Traffic Management

  • 1. Beyond Inbound: Jan 2024 Unlocking the Secrets of API Egress Traffic Management 2024 All rights reserved | Lunar.Inc
  • 2. 2 HI! The Management Platform Built for 3rd Party APIs at Scale Head of DevRel Sean Keegan Former math teacher turned DevRel. Big fan of ultimate frisbee, video games, APIs, and houseplants. Jan 2024 We’ve set out to build a new paradigm for management and optimization of 3rd party API consumption! 2024 All rights reserved | Lunar.Inc We’re Lunar.dev
  • 3. 3 Agenda 2024 All rights reserved | Lunar.Inc Introduction 01 Jan 2024 API Producer Landscape (Ingress) API Consumer Landscape (Egress) Problems with API consumption Live demo of lunar.dev 02 03 04 05 Q&A, closing, survey 06
  • 4. 4 Quick vocab 2024 All rights reserved | Lunar.Inc Jan 2024 Ingress traffic Egress traffic
  • 5. 5 Imagine: Your Company's API is Taking Off! 2024 All rights reserved | Lunar.Inc Jan 2024
  • 6. 6 What are some considerations as demand for your API grows? 2024 All rights reserved | Lunar.Inc Jan 2024 Join at Slido.com #163 0466
  • 7. 7 The issues for API Producers 2024 All rights reserved | Lunar.Inc ▪ Implement Rate Limiting Jan 2024 ▪ Utilize authentication/authorization ▪ Apply throttling techniques ▪ Monitor performance ▪ Introduce caching mechanisms ▪ Scale horizontally
  • 8. 8 Consumed APIs > Served APIs 2024 All rights reserved | Lunar.Inc Jan 2024
  • 9. 9 What challenges are there around consuming APIs? 2024 All rights reserved | Lunar.Inc Jan 2024 Join at Slido.com #5549 710
  • 10. 10 Let’s use GenAI in our App! 2024 All rights reserved | Lunar.Inc Jan 2024
  • 11. 11 Let’s use GenAI in our App! 2024 All rights reserved | Lunar.Inc Jan 2024 OpenAI Documentation
  • 12. 12 Prepare for API Rate Limits 2024 All rights reserved | Lunar.Inc Jan 2024 • LLM APIs have hard rate limits based on your usage tiers • Rate limits are usually based on both: • RPM – Requests per Minute • As your app scales, API rate limits can cause service outage for your users • TPM – Tokens per Minute
  • 13. 13 API Rate Limits – Account and Region Load Balancing Load Balancer https://andre-openai-eastus.openai.azure.com/ Your Application 2024 All rights reserved | Lunar.Inc Jan 2024 https://andre-openai-eastus-2.openai.azure.com/ https://andre-openai- canadaeast.openai.azure.com/ https://andre-openai-francecentral.openai.azure.com/ … Send API Call with Prompt Write a story about the following subject: {{user_input}}
  • 14. 15 Let’s use GenAI in our App! 2024 All rights reserved | Lunar.Inc Jan 2024
  • 15. 16 The dark side of API Management Producing APIs Consuming APIs API Landscape f Cost f Reliability f Safety f Scalability 2024 All rights reserved | Lunar.Inc Jan 2024
  • 16. 17 2024 All rights reserved | Lunar.Inc Jan 2024
  • 17. 18 A mindset change - managing your Egress Traffic is a must So…What can be done? 2024 All rights reserved | Lunar.Inc Jan 2024 Smart Caching Queue Cost-controls Alternating API Providers Error handling Logging API calls over time Retry API account orchestration Fallbacks Strategy – Based Throttling PII removal Authentication Mechanism API Calls logging Enforcing HTTPS API response inspection API call inspection
  • 18. 19 2024 All rights reserved | Lunar.Inc Jan 2024
  • 19. 20 2024 All rights reserved | Lunar.Inc Jan 2024 Questions?
  • 20. 21 2024 All rights reserved | Lunar.Inc Jan 2024 Check us out!
  • 21. 22 Thank you for Listening! Follow up at me@lunar.dev! 2024 All rights reserved | Lunar.Inc Jan 2024