SlideShare a Scribd company logo
Anurag Palsule
anurag@avinetworks.com
Ludicruous scaling of SSL Traffic
Increase Application Capacity, Reliability, and Scale
Why do we need ludicruous scale?
- Cashless transactions have gone to 25% from 5% in a matter of
few months!
- IRCTC bookings have grown from 29 tickets per day to 13L per
day !
- IHS forecasts that the IoT market will grow from an installed
base of 15.4 billion devices in 2015 to 30.7 billion devices in
2020 and 75.4 billion in 2025 => Huge scalability requirements
on IOT applications
Load Balancer Scalability – New Considerations
• SSL/TLS traffic seeing explosive growth
• Performance myth: Ultra expensive and inflexible
hardware appliances the only solution
• Moore’s law: advances in Intel x86 servers –
processors, memory, and networking
• Crypto advances: RSA 2K vs. ECC encryption keys
• Software-defined architectural advances enable
significant elasticity
Architectural Approaches to Scale Load Balancers
- Hierarchical load balancers
- DNS + Proxy load balancers
- Route injection/Anycast load balancers
Hierarchical Load Balancers
Concept:
• Chaining of load balancing services
• Tier 1 – Layer 4 (TCP/UDP) load balancing
• Tier 2 – Layer 7 load balancing
Pros:
• Simplest approach
• May suffice for small scale environments
Cons:
• Limited by performance of Tier-1 LB
Users
Tier 1
Load Balancer
Tier 2
Load Balancer
Application
Instances
DNS + Proxy Load Balancers
Concept:
• DNS redirections with server mirroring
• Dynamic mapping of hostname to IP
addresses
Pros:
• Easy to configure
• Scales well
Cons:
• DNS caches can become stale
Users DNS
Load Balancer
Application
Instances
IP1
IP2
IP3
IP4 IP1
IP2
IP3
IP4
Route injection/Anycast Load Balancers
Concept:
• DNS resolves to single IP
• Upstream router holds IP address
• Router performs flow-based ECMP to
next hop load balancers
Pros:
• Can scale significantly – most routers
support at least 64 next hops
Cons:
• Access to an upstream router is needed
Users
Router
Load Balancer
Application
Instances
Legacy 90s Arch,
Box approach
• Proprietary Hardware
• Manage Each Device
• No Automation
• No Telemetry
• Static Capacity
The State of Load Balancing/Application Delivery
WebScale computing is here but load balancing is still a bottleneck!
Takeaways from
AWS/FB/Microsoft
• Commodity x86
• Manage As One
• Highly Automated
• Built-In Telemetry
• Elastic
Flexible, Fluid CapacityRigid
Legacy
ADC/LBs WEB SCALE TECH
Load
Balancers
Virtualized Containers Public CloudCompute ComputeCompute
Modern Distributed Architecture
Separate Control and Dataplane
Manage as one, not many devices
Controller
Load Balancers
Management Plane: UI/CLI
Data Plane: LB
Virtualized Containers Public CloudCompute
Modern Distributed Architecture
Separate Control and Dataplane
Manage as one, not many devices
Controller
Load Balancers
Modern Distributed Architecture
Separate Control and Dataplane
Manage as one, not many devices
Load Balancers
Bare Metal Virtualized Containers Public Cloud
Controller
MESOS
Management & Orchestration
REST API
Multi-Cloud Fabric
Single solution, any environment
Automation
Highly programmable, Plug-n-Play
Built-In Visibility & Analytics
Actionable insights key to automation
Innovation
1 Million TPS on Google Compute Engine - Setup
Avi Networks – Elastic Application Services Fabric
320x Test
Clients
40x Avi Service Engines
(Load Balancers)
ab ab
ab
n1-highcpu-16
ab ab
ab
n1-highcpu-16
ab ab
ab
n1-highcpu-16
GCP
Router
Controller
ab ab
ab
n1-highcpu-16
Application
Instances
Key Stats
- Total cost for setup in Google Compute < $50
- SSL TPS – 0 to 1 million TPS in a few seconds
- Dataplane: 40 VM instances with 32 hyperthreaded cores each
- Traffic generators – 320 VM instances on 16 hyperthreaded cores
each
• Setup in Google Compute
• Bootstrap instance - 1 g1-small instance
• Avi Controller - 1 n1-standard-4 instance
• Avi Service Engines (load balancers) - 40 n1-highcpu-32 instances
• Pool server - 1 g1-small instance
• Test clients (load/traffic generators) - 320 n1-highcpu-16 instances
• Running the test
• https://github.com/avinetworks/avi-test-scripts : This public repo has all the scripts
required for anyone to perform the scalability test
Test setup and methodology
Avi Networks Proprietary and Confidential 2017
Scale Performance Up and Out
Managed as One Elastic Load Balancer Fabric
• 1 LB, 1 core
• 5 Gbps
• 2,500 SSL TPS
• 1 LB, 24 cores (2 Sockets)
• 20 Gbps (10 Gbps NICs)
• 60,000 SSL TPS
SCALE-UP
More cores & IO
LB performance scales with CPUs
(Moore’s Law) & IO (40 Gbps NICs)
• 1 LB, 2 core
• 10 Gbps
• 5,000 SSL TPS
Single App Perf
• 640 Gbps
• 1.9M SSL TPS
Performance
• 4 Tbps
• 12M SSL TPS
Scale to 200 LBs
• 2 LB, 1 core
• 10 Gbps
• 5,000 SSL TPS
SCALE-OUT
More LBs
Fabric performance scales
horizontally with LBs
Centralized
API
Management
Monitoring
Beyond Google Compute; Any Data Center or Public Cloud
Clients Load Balancers
Controller
Application
Instances
GCP
Router
DEMO
Real-time Insights for Elastic Application Services
The New Rules of Elastic, Cost-effective Load Balancing
1 Take advantage of WebScale architectures
2 Use analytics-driven decisions for on-demand elasticity
Automate L4 – L7 services with APIs3
Leverage load balancers for application intelligence4
Eliminate hardware overprovisioning5
Anurag Palsule
anurag@AviNetworks.com
Thank You!
Avi Networks (India) Pvt Ltd.
JB House, 110, 4th Cross,
5th Block, Koramangala Industrial Layout,
Bangalore 560 095, Karnataka.

More Related Content

Recently uploaded

GT degree offer diploma Transcript
GT degree offer diploma TranscriptGT degree offer diploma Transcript
GT degree offer diploma Transcript
attueb
 
Install Ruby on Rails Like a Pro: Effortless Guide
Install Ruby on Rails Like a Pro: Effortless GuideInstall Ruby on Rails Like a Pro: Effortless Guide
Install Ruby on Rails Like a Pro: Effortless Guide
rorbitssoftware
 
Attendance Tracking From Paper To Digital
Attendance Tracking From Paper To DigitalAttendance Tracking From Paper To Digital
Attendance Tracking From Paper To Digital
Task Tracker
 
當測試開始左移
當測試開始左移當測試開始左移
當測試開始左移
Jersey (CHE-PING) Su
 
Building infrastructure with code_ A deep dive into CDK for IaC in Java.pdf
Building infrastructure with code_ A deep dive into CDK for IaC in Java.pdfBuilding infrastructure with code_ A deep dive into CDK for IaC in Java.pdf
Building infrastructure with code_ A deep dive into CDK for IaC in Java.pdf
mohitd6
 
11 Top Cross Browser Testing Tools to Know About.pdf
11 Top Cross Browser Testing Tools to Know About.pdf11 Top Cross Browser Testing Tools to Know About.pdf
11 Top Cross Browser Testing Tools to Know About.pdf
kalichargn70th171
 
welcome to presentation on Google Apps
welcome to   presentation on Google Appswelcome to   presentation on Google Apps
welcome to presentation on Google Apps
AsifKarimJim
 
Il Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazioneIl Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazione
confluent
 
HIRE A HACKER FOR CHEATING HUSBAND/WIFE)
HIRE A HACKER FOR CHEATING HUSBAND/WIFE)HIRE A HACKER FOR CHEATING HUSBAND/WIFE)
HIRE A HACKER FOR CHEATING HUSBAND/WIFE)
josephinedrea942
 
React Native vs Flutter - SSTech System
React Native vs Flutter  - SSTech SystemReact Native vs Flutter  - SSTech System
React Native vs Flutter - SSTech System
SSTech System
 
NYGGS 360: A Complete ERP for Construction Innovation
NYGGS 360: A Complete ERP for Construction InnovationNYGGS 360: A Complete ERP for Construction Innovation
NYGGS 360: A Complete ERP for Construction Innovation
NYGGS Construction ERP Software
 
Vip Girls Call ServiCe Hyderabad 0000000000 Pooja Best High Class Hyderabad A...
Vip Girls Call ServiCe Hyderabad 0000000000 Pooja Best High Class Hyderabad A...Vip Girls Call ServiCe Hyderabad 0000000000 Pooja Best High Class Hyderabad A...
Vip Girls Call ServiCe Hyderabad 0000000000 Pooja Best High Class Hyderabad A...
ashiklo9823
 
Odoo E-commerce website development guides
Odoo E-commerce website development guidesOdoo E-commerce website development guides
Odoo E-commerce website development guides
jhkdigitalmarketing
 
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
singhlata50dh
 
Artificial intelligence in customer services or chatbots
Artificial intelligence  in customer services or chatbotsArtificial intelligence  in customer services or chatbots
Artificial intelligence in customer services or chatbots
kayash1656
 
Shivam Pandit working on Php Web Developer.
Shivam Pandit working on Php Web Developer.Shivam Pandit working on Php Web Developer.
Shivam Pandit working on Php Web Developer.
shivamt017
 
Wired_2.0_Create_AmsterdamJUG_09072024.pptx
Wired_2.0_Create_AmsterdamJUG_09072024.pptxWired_2.0_Create_AmsterdamJUG_09072024.pptx
Wired_2.0_Create_AmsterdamJUG_09072024.pptx
SimonedeGijt
 
Top Chinese Government-backed APT Groups
Top Chinese Government-backed APT GroupsTop Chinese Government-backed APT Groups
Top Chinese Government-backed APT Groups
SOCRadar
 
Independent Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class H...
Independent Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class H...Independent Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class H...
Independent Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class H...
aslasdfmkhan4750
 
Google ML-Kit - Understanding on-device machine learning
Google ML-Kit - Understanding on-device machine learningGoogle ML-Kit - Understanding on-device machine learning
Google ML-Kit - Understanding on-device machine learning
VishrutGoyani1
 

Recently uploaded (20)

GT degree offer diploma Transcript
GT degree offer diploma TranscriptGT degree offer diploma Transcript
GT degree offer diploma Transcript
 
Install Ruby on Rails Like a Pro: Effortless Guide
Install Ruby on Rails Like a Pro: Effortless GuideInstall Ruby on Rails Like a Pro: Effortless Guide
Install Ruby on Rails Like a Pro: Effortless Guide
 
Attendance Tracking From Paper To Digital
Attendance Tracking From Paper To DigitalAttendance Tracking From Paper To Digital
Attendance Tracking From Paper To Digital
 
當測試開始左移
當測試開始左移當測試開始左移
當測試開始左移
 
Building infrastructure with code_ A deep dive into CDK for IaC in Java.pdf
Building infrastructure with code_ A deep dive into CDK for IaC in Java.pdfBuilding infrastructure with code_ A deep dive into CDK for IaC in Java.pdf
Building infrastructure with code_ A deep dive into CDK for IaC in Java.pdf
 
11 Top Cross Browser Testing Tools to Know About.pdf
11 Top Cross Browser Testing Tools to Know About.pdf11 Top Cross Browser Testing Tools to Know About.pdf
11 Top Cross Browser Testing Tools to Know About.pdf
 
welcome to presentation on Google Apps
welcome to   presentation on Google Appswelcome to   presentation on Google Apps
welcome to presentation on Google Apps
 
Il Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazioneIl Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazione
 
HIRE A HACKER FOR CHEATING HUSBAND/WIFE)
HIRE A HACKER FOR CHEATING HUSBAND/WIFE)HIRE A HACKER FOR CHEATING HUSBAND/WIFE)
HIRE A HACKER FOR CHEATING HUSBAND/WIFE)
 
React Native vs Flutter - SSTech System
React Native vs Flutter  - SSTech SystemReact Native vs Flutter  - SSTech System
React Native vs Flutter - SSTech System
 
NYGGS 360: A Complete ERP for Construction Innovation
NYGGS 360: A Complete ERP for Construction InnovationNYGGS 360: A Complete ERP for Construction Innovation
NYGGS 360: A Complete ERP for Construction Innovation
 
Vip Girls Call ServiCe Hyderabad 0000000000 Pooja Best High Class Hyderabad A...
Vip Girls Call ServiCe Hyderabad 0000000000 Pooja Best High Class Hyderabad A...Vip Girls Call ServiCe Hyderabad 0000000000 Pooja Best High Class Hyderabad A...
Vip Girls Call ServiCe Hyderabad 0000000000 Pooja Best High Class Hyderabad A...
 
Odoo E-commerce website development guides
Odoo E-commerce website development guidesOdoo E-commerce website development guides
Odoo E-commerce website development guides
 
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
 
Artificial intelligence in customer services or chatbots
Artificial intelligence  in customer services or chatbotsArtificial intelligence  in customer services or chatbots
Artificial intelligence in customer services or chatbots
 
Shivam Pandit working on Php Web Developer.
Shivam Pandit working on Php Web Developer.Shivam Pandit working on Php Web Developer.
Shivam Pandit working on Php Web Developer.
 
Wired_2.0_Create_AmsterdamJUG_09072024.pptx
Wired_2.0_Create_AmsterdamJUG_09072024.pptxWired_2.0_Create_AmsterdamJUG_09072024.pptx
Wired_2.0_Create_AmsterdamJUG_09072024.pptx
 
Top Chinese Government-backed APT Groups
Top Chinese Government-backed APT GroupsTop Chinese Government-backed APT Groups
Top Chinese Government-backed APT Groups
 
Independent Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class H...
Independent Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class H...Independent Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class H...
Independent Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class H...
 
Google ML-Kit - Understanding on-device machine learning
Google ML-Kit - Understanding on-device machine learningGoogle ML-Kit - Understanding on-device machine learning
Google ML-Kit - Understanding on-device machine learning
 

Featured

AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
marketingartwork
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
Skeleton Technologies
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
Neil Kimberley
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
contently
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
Albert Qian
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
Search Engine Journal
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
SpeakerHub
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
Tessa Mero
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
Rajiv Jayarajah, MAppComm, ACC
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
Christy Abraham Joy
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best Practices
Vit Horky
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
MindGenius
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
RachelPearson36
 
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Applitools
 
12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work
GetSmarter
 
ChatGPT webinar slides
ChatGPT webinar slidesChatGPT webinar slides
ChatGPT webinar slides
Alireza Esmikhani
 
More than Just Lines on a Map: Best Practices for U.S Bike Routes
More than Just Lines on a Map: Best Practices for U.S Bike RoutesMore than Just Lines on a Map: Best Practices for U.S Bike Routes
More than Just Lines on a Map: Best Practices for U.S Bike Routes
Project for Public Spaces & National Center for Biking and Walking
 

Featured (20)

AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best Practices
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
 
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
 
12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work
 
ChatGPT webinar slides
ChatGPT webinar slidesChatGPT webinar slides
ChatGPT webinar slides
 
More than Just Lines on a Map: Best Practices for U.S Bike Routes
More than Just Lines on a Map: Best Practices for U.S Bike RoutesMore than Just Lines on a Map: Best Practices for U.S Bike Routes
More than Just Lines on a Map: Best Practices for U.S Bike Routes
 

Ludicrous scalewithloadbalancers

  • 1. Anurag Palsule anurag@avinetworks.com Ludicruous scaling of SSL Traffic Increase Application Capacity, Reliability, and Scale
  • 2. Why do we need ludicruous scale? - Cashless transactions have gone to 25% from 5% in a matter of few months! - IRCTC bookings have grown from 29 tickets per day to 13L per day ! - IHS forecasts that the IoT market will grow from an installed base of 15.4 billion devices in 2015 to 30.7 billion devices in 2020 and 75.4 billion in 2025 => Huge scalability requirements on IOT applications
  • 3. Load Balancer Scalability – New Considerations • SSL/TLS traffic seeing explosive growth • Performance myth: Ultra expensive and inflexible hardware appliances the only solution • Moore’s law: advances in Intel x86 servers – processors, memory, and networking • Crypto advances: RSA 2K vs. ECC encryption keys • Software-defined architectural advances enable significant elasticity
  • 4. Architectural Approaches to Scale Load Balancers - Hierarchical load balancers - DNS + Proxy load balancers - Route injection/Anycast load balancers
  • 5. Hierarchical Load Balancers Concept: • Chaining of load balancing services • Tier 1 – Layer 4 (TCP/UDP) load balancing • Tier 2 – Layer 7 load balancing Pros: • Simplest approach • May suffice for small scale environments Cons: • Limited by performance of Tier-1 LB Users Tier 1 Load Balancer Tier 2 Load Balancer Application Instances
  • 6. DNS + Proxy Load Balancers Concept: • DNS redirections with server mirroring • Dynamic mapping of hostname to IP addresses Pros: • Easy to configure • Scales well Cons: • DNS caches can become stale Users DNS Load Balancer Application Instances IP1 IP2 IP3 IP4 IP1 IP2 IP3 IP4
  • 7. Route injection/Anycast Load Balancers Concept: • DNS resolves to single IP • Upstream router holds IP address • Router performs flow-based ECMP to next hop load balancers Pros: • Can scale significantly – most routers support at least 64 next hops Cons: • Access to an upstream router is needed Users Router Load Balancer Application Instances
  • 8. Legacy 90s Arch, Box approach • Proprietary Hardware • Manage Each Device • No Automation • No Telemetry • Static Capacity The State of Load Balancing/Application Delivery WebScale computing is here but load balancing is still a bottleneck! Takeaways from AWS/FB/Microsoft • Commodity x86 • Manage As One • Highly Automated • Built-In Telemetry • Elastic Flexible, Fluid CapacityRigid Legacy ADC/LBs WEB SCALE TECH Load Balancers
  • 9. Virtualized Containers Public CloudCompute ComputeCompute Modern Distributed Architecture Separate Control and Dataplane Manage as one, not many devices Controller Load Balancers Management Plane: UI/CLI Data Plane: LB
  • 10. Virtualized Containers Public CloudCompute Modern Distributed Architecture Separate Control and Dataplane Manage as one, not many devices Controller Load Balancers
  • 11. Modern Distributed Architecture Separate Control and Dataplane Manage as one, not many devices Load Balancers Bare Metal Virtualized Containers Public Cloud Controller MESOS Management & Orchestration REST API Multi-Cloud Fabric Single solution, any environment Automation Highly programmable, Plug-n-Play Built-In Visibility & Analytics Actionable insights key to automation Innovation
  • 12. 1 Million TPS on Google Compute Engine - Setup Avi Networks – Elastic Application Services Fabric 320x Test Clients 40x Avi Service Engines (Load Balancers) ab ab ab n1-highcpu-16 ab ab ab n1-highcpu-16 ab ab ab n1-highcpu-16 GCP Router Controller ab ab ab n1-highcpu-16 Application Instances
  • 13. Key Stats - Total cost for setup in Google Compute < $50 - SSL TPS – 0 to 1 million TPS in a few seconds - Dataplane: 40 VM instances with 32 hyperthreaded cores each - Traffic generators – 320 VM instances on 16 hyperthreaded cores each
  • 14. • Setup in Google Compute • Bootstrap instance - 1 g1-small instance • Avi Controller - 1 n1-standard-4 instance • Avi Service Engines (load balancers) - 40 n1-highcpu-32 instances • Pool server - 1 g1-small instance • Test clients (load/traffic generators) - 320 n1-highcpu-16 instances • Running the test • https://github.com/avinetworks/avi-test-scripts : This public repo has all the scripts required for anyone to perform the scalability test Test setup and methodology
  • 15. Avi Networks Proprietary and Confidential 2017 Scale Performance Up and Out Managed as One Elastic Load Balancer Fabric • 1 LB, 1 core • 5 Gbps • 2,500 SSL TPS • 1 LB, 24 cores (2 Sockets) • 20 Gbps (10 Gbps NICs) • 60,000 SSL TPS SCALE-UP More cores & IO LB performance scales with CPUs (Moore’s Law) & IO (40 Gbps NICs) • 1 LB, 2 core • 10 Gbps • 5,000 SSL TPS Single App Perf • 640 Gbps • 1.9M SSL TPS Performance • 4 Tbps • 12M SSL TPS Scale to 200 LBs • 2 LB, 1 core • 10 Gbps • 5,000 SSL TPS SCALE-OUT More LBs Fabric performance scales horizontally with LBs Centralized API Management Monitoring
  • 16. Beyond Google Compute; Any Data Center or Public Cloud Clients Load Balancers Controller Application Instances GCP Router
  • 17. DEMO Real-time Insights for Elastic Application Services
  • 18. The New Rules of Elastic, Cost-effective Load Balancing 1 Take advantage of WebScale architectures 2 Use analytics-driven decisions for on-demand elasticity Automate L4 – L7 services with APIs3 Leverage load balancers for application intelligence4 Eliminate hardware overprovisioning5
  • 19. Anurag Palsule anurag@AviNetworks.com Thank You! Avi Networks (India) Pvt Ltd. JB House, 110, 4th Cross, 5th Block, Koramangala Industrial Layout, Bangalore 560 095, Karnataka.

Editor's Notes

  1. Customer: I have hundreds of servers in my data center. With Chef, Puppet, Ansible, I can turn a couple of racks in to webserver within 5 mins, and turn another 2 racks in to app servers.
  2. Customer: I have hundreds of servers in my data center. With Chef, Puppet, Ansible, I can turn a couple of racks in to webserver within 5 mins, and turn another 2 racks in to app servers.
  3. Customer: I have hundreds of servers in my data center. With Chef, Puppet, Ansible, I can turn a couple of racks in to webserver within 5 mins, and turn another 2 racks in to app servers.
  4. Customer: I have hundreds of servers in my data center. With Chef, Puppet, Ansible, I can turn a couple of racks in to webserver within 5 mins, and turn another 2 racks in to app servers.
  5. Customer: I have hundreds of servers in my data center. With Chef, Puppet, Ansible, I can turn a couple of racks in to webserver within 5 mins, and turn another 2 racks in to app servers.
  6. Customer: I have hundreds of servers in my data center. With Chef, Puppet, Ansible, I can turn a couple of racks in to webserver within 5 mins, and turn another 2 racks in to app servers.
  7. Customer: I have hundreds of servers in my data center. With Chef, Puppet, Ansible, I can turn a couple of racks in to webserver within 5 mins, and turn another 2 racks in to app servers.
  8. Load balancers sit in the network today, in front of every business critical application in your environment. These are largely standard x86 servers in a proprietary box, with each box containing it’s own management plane and data plane – so your teams manage each one by one as an independent appliance. Avi’s architecture combined the management plane into a centralized controller which allows you to manage the dataplane – what we call a service engine – as an elastic fabric that can grow and shrink based on capacity needs, without increasing the number of management points.
  9. As your infrastructure goes from bare metal to virtual to containers and public cloud – you are now able to spin up the service engines as bare metal appliances on standard x86 servers, or VMs, or containers, or in the public cloud depending on the application needs you are trying to meet. The bare metal deployments offer an easy transition from an existing hardware appliance based environment to a software-defined env while ensuring that future transition to virtual, container or public cloud environments is smooth. Across all of these environments, the controller offers a single point of management and monitoring.
  10. As your infrastructure goes from bare metal to virtual to containers and public cloud – you are now able to spin up the service engines as bare metal appliances on standard x86 servers, or VMs, or containers, or in the public cloud depending on the application needs you are trying to meet. The bare metal deployments offer an easy transition from an existing hardware appliance based environment to a software-defined env while ensuring that future transition to virtual, container or public cloud environments is smooth. Across all of these environments, the controller offers a single point of management and monitoring. The controller is pre-integrated with management and orchestration platforms like vCenter, SDN controllers, container cluster managers like Mesos and Kubernetes, as well as public clouds like AWS. This allows a completely automated experience where Service Engines can be spun up or down and connected to networks automatically as need. Finally, given the strategic location of load balancers, they are best positioned to provide visibility into application usage and performance. So we built 100s of virtual probes in the Service Engines which can send this real-time telemetry on app performance back to the controller. The controller has a real-time analytics engine which process billions of data points to provide insights on application performance, usage, end-usage experience, security posture, DDoS, etc. Now operations team can track this data on applications in real time for any application without needing monitoring fabrics, taps, or external network based app performance monitoring solutions.
  11. Customer: I have hundreds of servers in my data center. With Chef, Puppet, Ansible, I can turn a couple of racks in to webserver within 5 mins, and turn another 2 racks in to app servers.