SlideShare a Scribd company logo
1 of 21
SO YOU THINK YOU CAN
S C A L E
@lmacvittie @F5Networks
  
Why do we scale?
100 Milliseconds Slower
-1% SALES -0.2% SEARCHES -2% CONVERSION
$660M $45M $244M
H/T James Urquhart, SOASTA Data: Gartner, Walmart
1 Minute of Downtime
Data: Emerson Power
Costs an average of $7300
Average total cost of downtime per year across industries
PRODUCTIVITY IT PRODUCTIVITY LOST REVENUE
$53,608$140,543 $183,724
=
UP OUT
How do we scale today?
POLB
(Plain Old Load Balancing)
We throw POLB in front of everything
But architectures and apps are changing
THEN
MONOLITHIC MICROSERVICES & APIs
NOW
And so are “users”
THEN
Humans
NOW
Humans, Systems, Things
L7 LB
Layer 7 Load Balancing
Modern Apps Need Modern Scale
Scaling Modern Apps
THEN
CHOOSING AN
ALGORITHM
NOW
• Round Robin
• Least Connections
• Fastest Response
CHOOSING AN
ARCHITECTURE
• Horizontal
• Functional
• Partitioning

Growth and Innovation
Architectural Debt
Software has technical Debt. Networks have architectural debt.
L7 LB
Scalability architectures based on Scale Cube
Horizontal Duplication
Functional
Decomposition
Data Partitioning
(Sharding)
X-AXIS Y-AXIS Z-AXIS
• Like clustering
• Leverages cloning (horizontal duplication)
• Typically operates at layer 4 (TCP)
• This is POLB
X-AXIS
Y-AXIS
• Like routing / URL dispatch
• Uses identifiable variables (usually from HTTP headers)
• Operates at layer 7 (HTTP)
/login/
/checkout/
/search/
Z-AXIS
/car/[A-M]/
/car/[N-U]/
/car/[V-Z]/
• Like sharding
• Uses one or more identifiable variables (usually from HTTP headers)
• Operates at layer 7 (HTTP)
API
APIAPIAPI
q
API Dispatch
q
POLB
API Façade
L7
q
Device Driven
(Dispatch)
L7
L7
L7
APIAPIAPI
APIAPIAPIAPI
Purpose Driven
(functional decomposition)
/status
/update
/status
/update
/billing
/profile
/update
Capacity Driven
(POLB)
Variations on a Theme
• Data Partitioning (Sharding)
Architectures
• URL/HTTP dispatch
• Dynamic routing based on backend
data
• Scaling by Functional Decomposition
• API metering
• App versioning / migration
• API deprecation
L7 LB
Architecting
Scalability
Things to Consider Sooner Rather than Later
Is your API design well-suited to scaling in any way other than POLB?
How do you identify people, systems, and things?
How do you distinguish between API versions?
Multi-tenant or single tenant microservices?
The answers to these questions will impact your scaling architectural options*.
Emacs or vi?
*Maybe not that last one.
How are you monitoring (and what are you measuring with it)?
THANK YOU!
@lmacvittie @F5Networks
l.macvittie@f5.com

More Related Content

What's hot

Api gateway in microservices
Api gateway in microservicesApi gateway in microservices
Api gateway in microservices
Kunal Hire
 
Scaling from 1 to 10 million users - Hailo
Scaling from 1 to 10 million users - HailoScaling from 1 to 10 million users - Hailo
Scaling from 1 to 10 million users - Hailo
Boyan Dimitrov
 
Continuous Integration and Delivery at Shapeways (Matt Boyle)
Continuous Integration and Delivery at Shapeways (Matt Boyle)Continuous Integration and Delivery at Shapeways (Matt Boyle)
Continuous Integration and Delivery at Shapeways (Matt Boyle)
Nordic APIs
 

What's hot (20)

Api gateway in microservices
Api gateway in microservicesApi gateway in microservices
Api gateway in microservices
 
Magento Developer Talk. Microservice Architecture and Actor Model
Magento Developer Talk. Microservice Architecture and Actor ModelMagento Developer Talk. Microservice Architecture and Actor Model
Magento Developer Talk. Microservice Architecture and Actor Model
 
Scaling from 1 to 10 million users - Hailo
Scaling from 1 to 10 million users - HailoScaling from 1 to 10 million users - Hailo
Scaling from 1 to 10 million users - Hailo
 
Cultivating an eco system of success
Cultivating an eco system of success Cultivating an eco system of success
Cultivating an eco system of success
 
Microservices: next-steps
Microservices: next-stepsMicroservices: next-steps
Microservices: next-steps
 
Github Projects Overview and IBM Streams V4.1
Github Projects Overview and IBM Streams V4.1Github Projects Overview and IBM Streams V4.1
Github Projects Overview and IBM Streams V4.1
 
Moving to microservices – a technology and organisation transformational journey
Moving to microservices – a technology and organisation transformational journeyMoving to microservices – a technology and organisation transformational journey
Moving to microservices – a technology and organisation transformational journey
 
apidays LIVE Australia 2020 - Leveraging DevOps to visualize your digital eco...
apidays LIVE Australia 2020 - Leveraging DevOps to visualize your digital eco...apidays LIVE Australia 2020 - Leveraging DevOps to visualize your digital eco...
apidays LIVE Australia 2020 - Leveraging DevOps to visualize your digital eco...
 
The Most Common Errors That Aren’t Caught
The Most Common Errors That Aren’t CaughtThe Most Common Errors That Aren’t Caught
The Most Common Errors That Aren’t Caught
 
Introduction to the Spark MLLib Toolkit in IBM Streams V4.1
Introduction to the Spark MLLib Toolkit in IBM Streams V4.1Introduction to the Spark MLLib Toolkit in IBM Streams V4.1
Introduction to the Spark MLLib Toolkit in IBM Streams V4.1
 
Continuous Integration and Delivery at Shapeways (Matt Boyle)
Continuous Integration and Delivery at Shapeways (Matt Boyle)Continuous Integration and Delivery at Shapeways (Matt Boyle)
Continuous Integration and Delivery at Shapeways (Matt Boyle)
 
Will ServerLess kill containers and Operations
Will ServerLess kill containers and OperationsWill ServerLess kill containers and Operations
Will ServerLess kill containers and Operations
 
Apache Flink Online Training
Apache Flink Online TrainingApache Flink Online Training
Apache Flink Online Training
 
Reducing MTTR and False Escalations: Event Correlation at LinkedIn
Reducing MTTR and False Escalations: Event Correlation at LinkedInReducing MTTR and False Escalations: Event Correlation at LinkedIn
Reducing MTTR and False Escalations: Event Correlation at LinkedIn
 
How do async ap is survive in a rest world
How do async ap is survive in a rest world How do async ap is survive in a rest world
How do async ap is survive in a rest world
 
Apache flink
Apache flinkApache flink
Apache flink
 
REST vs. GraphQL: Critical Look
REST vs. GraphQL: Critical LookREST vs. GraphQL: Critical Look
REST vs. GraphQL: Critical Look
 
Let's Talk ProIV and Alexa
Let's Talk ProIV and AlexaLet's Talk ProIV and Alexa
Let's Talk ProIV and Alexa
 
Multi Team Architecture
Multi Team ArchitectureMulti Team Architecture
Multi Team Architecture
 
Apache flink
Apache flinkApache flink
Apache flink
 

Viewers also liked

LTM essentials
LTM essentialsLTM essentials
LTM essentials
bharadwajv
 
GeoMapFish, the Open Source WebGIS
GeoMapFish, the Open Source WebGISGeoMapFish, the Open Source WebGIS
GeoMapFish, the Open Source WebGIS
Camptocamp
 

Viewers also liked (15)

State of Application Delivery 2017 - DevOps Insights
State of Application Delivery 2017 - DevOps Insights State of Application Delivery 2017 - DevOps Insights
State of Application Delivery 2017 - DevOps Insights
 
Rancher 快速打造叢集的解決方案
Rancher 快速打造叢集的解決方案Rancher 快速打造叢集的解決方案
Rancher 快速打造叢集的解決方案
 
F5 Networks BIG-IP LTM Virtual Edition
F5 Networks BIG-IP LTM Virtual EditionF5 Networks BIG-IP LTM Virtual Edition
F5 Networks BIG-IP LTM Virtual Edition
 
HTTP by Hand: Exploring HTTP/1.0, 1.1 and 2.0
HTTP by Hand: Exploring HTTP/1.0, 1.1 and 2.0HTTP by Hand: Exploring HTTP/1.0, 1.1 and 2.0
HTTP by Hand: Exploring HTTP/1.0, 1.1 and 2.0
 
LTM essentials
LTM essentialsLTM essentials
LTM essentials
 
You are valuable
You are valuableYou are valuable
You are valuable
 
Csfpt ppcr mars 2017
Csfpt ppcr mars 2017Csfpt ppcr mars 2017
Csfpt ppcr mars 2017
 
C All 2008 7 22
C All 2008 7 22C All 2008 7 22
C All 2008 7 22
 
Missing vin (1)
Missing vin (1)Missing vin (1)
Missing vin (1)
 
Debugging Javascript - 0 to Heisenberg
Debugging Javascript - 0 to HeisenbergDebugging Javascript - 0 to Heisenberg
Debugging Javascript - 0 to Heisenberg
 
GeoMapFish, the Open Source WebGIS
GeoMapFish, the Open Source WebGISGeoMapFish, the Open Source WebGIS
GeoMapFish, the Open Source WebGIS
 
Dossier Pedagogique Festival Electrochoc 12
Dossier Pedagogique Festival Electrochoc 12 Dossier Pedagogique Festival Electrochoc 12
Dossier Pedagogique Festival Electrochoc 12
 
Maar wat is Sosiale Media?
Maar wat is Sosiale Media?Maar wat is Sosiale Media?
Maar wat is Sosiale Media?
 
Moving beyond Blackboard: The VLE journey at Dundee
Moving beyond Blackboard: The VLE journey at DundeeMoving beyond Blackboard: The VLE journey at Dundee
Moving beyond Blackboard: The VLE journey at Dundee
 
A Measurement Study of 4chan’s Politically Incorrect Forum and Its Effects on...
A Measurement Study of 4chan’s Politically Incorrect Forum and Its Effects on...A Measurement Study of 4chan’s Politically Incorrect Forum and Its Effects on...
A Measurement Study of 4chan’s Politically Incorrect Forum and Its Effects on...
 

Similar to So you think you can scale

REST - What's It All About? (SAP TechEd 2012, CD110)
REST - What's It All About? (SAP TechEd 2012, CD110)REST - What's It All About? (SAP TechEd 2012, CD110)
REST - What's It All About? (SAP TechEd 2012, CD110)
Sascha Wenninger
 
Mmc Lsmw Intro
Mmc Lsmw IntroMmc Lsmw Intro
Mmc Lsmw Intro
peach9613
 
Role of Rest vs. Web Services and EI
Role of Rest vs. Web Services and EIRole of Rest vs. Web Services and EI
Role of Rest vs. Web Services and EI
WSO2
 

Similar to So you think you can scale (20)

REST - What's It All About? (SAP TechEd 2012, CD110)
REST - What's It All About? (SAP TechEd 2012, CD110)REST - What's It All About? (SAP TechEd 2012, CD110)
REST - What's It All About? (SAP TechEd 2012, CD110)
 
Future of SOA & Modern APIs
Future of SOA & Modern APIsFuture of SOA & Modern APIs
Future of SOA & Modern APIs
 
Connect js nodejs_api_shubhra
Connect js nodejs_api_shubhraConnect js nodejs_api_shubhra
Connect js nodejs_api_shubhra
 
Building for, perceiving and measuring performance for mobile web
Building for, perceiving and measuring performance for mobile webBuilding for, perceiving and measuring performance for mobile web
Building for, perceiving and measuring performance for mobile web
 
Data Summer Conf 2018, “Building unified Batch and Stream processing pipeline...
Data Summer Conf 2018, “Building unified Batch and Stream processing pipeline...Data Summer Conf 2018, “Building unified Batch and Stream processing pipeline...
Data Summer Conf 2018, “Building unified Batch and Stream processing pipeline...
 
What is API - Understanding API Simplified
What is API - Understanding API SimplifiedWhat is API - Understanding API Simplified
What is API - Understanding API Simplified
 
Mmc Lsmw Intro
Mmc Lsmw IntroMmc Lsmw Intro
Mmc Lsmw Intro
 
Role of Rest vs. Web Services and EI
Role of Rest vs. Web Services and EIRole of Rest vs. Web Services and EI
Role of Rest vs. Web Services and EI
 
Data Pipelines - Big Data meets Salesforce
Data Pipelines - Big Data meets SalesforceData Pipelines - Big Data meets Salesforce
Data Pipelines - Big Data meets Salesforce
 
Moving existing apps to the cloud
 Moving existing apps to the cloud Moving existing apps to the cloud
Moving existing apps to the cloud
 
Dubbo and Weidian's practice on micro-service architecture
Dubbo and Weidian's practice on micro-service architectureDubbo and Weidian's practice on micro-service architecture
Dubbo and Weidian's practice on micro-service architecture
 
Node.js Frameworks & Design Patterns Webinar
Node.js Frameworks & Design Patterns WebinarNode.js Frameworks & Design Patterns Webinar
Node.js Frameworks & Design Patterns Webinar
 
Data Pipelines: Big Data Meets Salesforce
Data Pipelines: Big Data Meets SalesforceData Pipelines: Big Data Meets Salesforce
Data Pipelines: Big Data Meets Salesforce
 
Angular jS Introduction by Google
Angular jS Introduction by GoogleAngular jS Introduction by Google
Angular jS Introduction by Google
 
Picking the Right Node.js Framework for Your Use Case
Picking the Right Node.js Framework for Your Use CasePicking the Right Node.js Framework for Your Use Case
Picking the Right Node.js Framework for Your Use Case
 
(ATS6-DEV02) Web Application Strategies
(ATS6-DEV02) Web Application Strategies(ATS6-DEV02) Web Application Strategies
(ATS6-DEV02) Web Application Strategies
 
Data Pipelines -Big Data Meets Salesforce
Data Pipelines -Big Data Meets SalesforceData Pipelines -Big Data Meets Salesforce
Data Pipelines -Big Data Meets Salesforce
 
APEX Alpe Adria Mike Hichwa Keynote April 11th 2019- Zagreb
APEX Alpe Adria Mike Hichwa Keynote April 11th 2019- ZagrebAPEX Alpe Adria Mike Hichwa Keynote April 11th 2019- Zagreb
APEX Alpe Adria Mike Hichwa Keynote April 11th 2019- Zagreb
 
Microsoft Flow best practices European Collaboration Summit 2018
Microsoft Flow best practices European Collaboration Summit 2018Microsoft Flow best practices European Collaboration Summit 2018
Microsoft Flow best practices European Collaboration Summit 2018
 
[Luca/Van Campenhoudt] Microsoft Flow Beyone the Limits: Tips, Pitfalls, Patt...
[Luca/Van Campenhoudt] Microsoft Flow Beyone the Limits: Tips, Pitfalls, Patt...[Luca/Van Campenhoudt] Microsoft Flow Beyone the Limits: Tips, Pitfalls, Patt...
[Luca/Van Campenhoudt] Microsoft Flow Beyone the Limits: Tips, Pitfalls, Patt...
 

More from Lori MacVittie

Web 2 And Application Delivery Public
Web 2 And Application Delivery PublicWeb 2 And Application Delivery Public
Web 2 And Application Delivery Public
Lori MacVittie
 

More from Lori MacVittie (13)

So you think you can scale containers
So you think you can scale containersSo you think you can scale containers
So you think you can scale containers
 
State of Application Delivery 2017 - Cloud Insights
State of Application Delivery 2017 - Cloud Insights State of Application Delivery 2017 - Cloud Insights
State of Application Delivery 2017 - Cloud Insights
 
Devops is all greek
Devops is all greekDevops is all greek
Devops is all greek
 
Pushing the DevOps envelope into the network with microservices
Pushing the DevOps envelope into the network with microservicesPushing the DevOps envelope into the network with microservices
Pushing the DevOps envelope into the network with microservices
 
The Internet of Security Things (A Story about Change)
The Internet of Security Things (A Story about Change) The Internet of Security Things (A Story about Change)
The Internet of Security Things (A Story about Change)
 
HTTP/2 Changes Everything
HTTP/2 Changes EverythingHTTP/2 Changes Everything
HTTP/2 Changes Everything
 
5 ways to use node.js in the network
5 ways to use node.js in the network5 ways to use node.js in the network
5 ways to use node.js in the network
 
What are Software Defined Application Services
What are Software Defined Application ServicesWhat are Software Defined Application Services
What are Software Defined Application Services
 
Operationalize all the network things
Operationalize all the network thingsOperationalize all the network things
Operationalize all the network things
 
Programming proxies to do what we need so we don't have to talk to the networ...
Programming proxies to do what we need so we don't have to talk to the networ...Programming proxies to do what we need so we don't have to talk to the networ...
Programming proxies to do what we need so we don't have to talk to the networ...
 
Dynamic Infrastructure
Dynamic InfrastructureDynamic Infrastructure
Dynamic Infrastructure
 
Nine Ways to Use Network-Side Scripting
Nine Ways to Use Network-Side ScriptingNine Ways to Use Network-Side Scripting
Nine Ways to Use Network-Side Scripting
 
Web 2 And Application Delivery Public
Web 2 And Application Delivery PublicWeb 2 And Application Delivery Public
Web 2 And Application Delivery Public
 

Recently uploaded

Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
panagenda
 

Recently uploaded (20)

Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties ReimaginedEasier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & Ireland
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 
Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdf
 
Google I/O Extended 2024 Warsaw
Google I/O Extended 2024 WarsawGoogle I/O Extended 2024 Warsaw
Google I/O Extended 2024 Warsaw
 

So you think you can scale

Editor's Notes

  1. Performance and availability. Business. Engagement. If we don’t scale, the business doesn’t grow, we impede productivity. We lose. We don’t just scale today for technical reasons. Availability and its related metric of performance are tied to business metrics today like conversion rates, abandonment, bounce. There are real dollars tied to these operational metrics. And because in many cases – regardless of industry – apps are the business, there are real business metrics tied to them, as well, such as productivity and revenue growth.
  2. Three hundred million dollars was spent to lay a high-speed fiber optic cable from the futures market in Chicago to the exchanges in New Jersey to improve the speed of stock trading (particularly high- frequency trading — HFT) by 3 milliseconds. That’s 100 million dollars per millisecond. Most organizations work in bigger slices of time, like 100 milliseconds. A quick blink of your eye is 100 milliseconds.  Google considers page load latency in scoring ad relevance. https://theeconomiccontrarian.wordpress.com/2014/04/01/what-is-worth-100-million-per-millisecond/
  3. IT productivity is something we don’t often consider as a “thing” to be lost. But in the spring of 2000 I was a technical architect for a global transportation firm (they have big orange trucks ;-)) and we lost the primary application used to schedule and route shipments. For nearly 6 hours everyone not working to restore that system was diverted as runners between fax machines and the dispatch floor, trying to keep business running via sneaker net. That’s IT productivity lost right there, because I wasn’t coding – I was a messenger. http://www.emersonnetworkpower.com/documentation/en-us/brands/liebert/infographics/documents/ponemon-infographic-cost%20of%20downtime-r11-13-final.pdf
  4. We scale because
  5. How do we scale (everything) today? Up or out.
  6. How do we scale out? This is the monolithic approach.
  7. We need the ability to route, direct, and distribute load based on modern architectures that include highly decomposed applications (microservices), multiple versions of the same app and APIs and an increasingly diverse set of devices and things accessing those applications. That’s where L7 LB comes in. It has the visibility necessary and the location in an architecture (up front) to make the kinds of decisions required to scale modern, distributed architectures.
  8. And that’s exactly what we need – architectures. Scalability today is not just about throwing an LB in front of that API or service. It’s not even about how it fits operationally into the architecture. Every LB out there can be shoved into a VM or container and automated using APIs or scripts today. That’s the easy part. The harder part is about how to enable the deployment of scalable architectures – either by being flexible enough to route in a way that’s needed or by being a part of the architecture itself. You want to be able to scale like Pinterest: from one little MySQL server to 180 Web Engines, 240 API Engines, 88 MySQL DBs (cc2.8xlarge) + 1 slave each, 110 Redis Instances, and 200 Memcache Instances. The secret? Architecture. Architecting for scale doesn’t mean just choosing stateless over stateful services. It doesn’t just mean choosing a data sharding strategy or whether to decompose by function or object. It means considering how the scale fits into the architecture what it can do for you to help when it’s needed most.
  9. This is why you should think about architecting scale, not just deploying it. Complexity. Breaking up apps into microservices and APIs means we’re simplifying development but complexifying deployment. Operationally, we’re adding more and more components – like load balancing – to the mix. Guys from Pinterest: ““Sharding makes schema design harder · Waiting too long makes the transition harder” It’s important to think about the entire architecture – including the LB – before it’s deployed. How it scales and participates and even monitors is critical for keeping both technical and architectural debt down.
  10. At the application tier, eBay segments different functions into separate application pools. Selling functionality is served by one set of application servers, bidding by another, search by yet another. In total, they organize roughly 16,000 application servers into 220 different pools. This allows them to scale each pool independently of one another, according to the demands and resource consumption of its function. It further allows them to isolate and rationalize resource dependencies - the selling pool only needs to talk to a relatively small subset of backend resources, for example.
  11. API Façade: Interpolated in Network The façade provides the level of abstraction necessary for microservices-based apps today. Because each microservice are likely changing on different schedules, it’s highly disruptive to update the client for each and every one – especially early on. By virtualizing the API using a façade in the network, the API (and thus the client) can remain the same. Significant changes to any of the composite APIs can be reflected in the API façade without impacting everything else.
  12. Variations on a Theme Now we’ve got to consider the rise of things and how they impact scalability architectures. How do we efficiently and effectively design an architecture that scales while taking into consideration performance and security for a whole bunch of different “things” and “users” at the same time? Monitoring at the first tier can also provide insight into usage patterns of consumers and devices, which means better planning around upgrades, patches, etc… that may need to be accessed. Monitoring at the purpose-driven layer gives you insight into specific usage patterns of the API/application, which can help give a better picture of how to segment future versions or new applications. Monitoring at the last tier is all about speed and capacity; improving performance and ensuring availability. This is where algorithms and monitoring are most important, because both have a serious impact operationally and on the business.
  13. You’re architecting scale rather than bolting it on to the front.
  14. Well designed APIs are consistent and able to be parsed into their respective scalability domains. Clients should provide some indication (in headers, in data, in URL) of what they are to assist in security and performance and scaling. API versioning is a significant source of frustration. Are versions clearly identifiable such that upstream dispatching can assist in a more seamless upgrade/migration experience? Tenancy of microservices has a big impact on the network and scalability. Decide early which way you’re going to go – and whether you’re going to leverage the network to do it. Right answer is always vi.