SlideShare a Scribd company logo
Service Platform Architect
Brandon Kang
sangjinn@gmail.com
https://tech.brandonkang.net
May 2020
Scalability Strategies for
Cloud based System Architecture
Agenda
• Scalability & Availability for the Global Markets
• Global scaled Scalability, Availability and Security
• Architecture for 100, 1K, 100K, 500K, 1M and 10M global users
• Auto-Scaling
• Understand Cloud Services
• Cloud Demo(AWS, GCP, Azure and Cloudflare)
• Wrap-Up
- Scalability -
Scalability
• Scalability = capability of a system to handle a growing work
• Vertical : Scale Up or Down
ü Add or Remove Resources
ü CPU
ü Memory
ü Storage
• Horizontal: Scale Out or In
ü Add or Remove Systems
ü Instance Scale OutScale In
VM VM VM VMVM VMVM VM
Scale Down
Scale Up
VMVM
Scalability ≠ Availability
Scalability vs. Availability
• Need 4 * VMs to provide services
Scalability: (2* VMs in a region) + (2* VMs in another region)
Availability: (4* VMs in a region) + (4* VMs in another region) for HA
1 User
Network
Fixed IP
Application
Database
Users < 100
Network
Fixed IP
Application
Database
Users > 1,000
Master
Slave
Load Balancer
Zone A
Zone B
Write
Write
Read
Replication
Region
Users > 100,000 M
R
R
S
R
R
Active/
Write
Read
Replica
Read
Replica
Read
Replica
Read
Replica
Stand-by/
Write
Users > 100,000 M
R
R
S
R
R
Object
Storage
Object
Storage
www.example.com
api.example.com
…
CDN
static.example.com
image.example.com
…
Users > 500,000
M
R
R
S
R
R
Object
Storage
Object
Storage
CDN
static.example.com
image.example.com
www.example.com
… DB
Caching
DB
Caching
API
Gateway
Service
Micro-Services
Architecture
api.example.com
Users > 500,000
• MSA(Microservices Architecture)
ü Every functions move to Microservices
ü Independent and loosely coupled
• API Gateway
ü API Routing
ü API Security
ü Authentication
ü Authorization
ü API Caching
ü Hits Rate Limit
ü Hits Throttling
ü Traffic Monitoring
Users > 1M
M
R
R
S
R
R
Object
Storage
Object
Storage
CDN
static.example.com
image.example.com
www.example.com
… DB
Caching
DB
Caching
API
Gateway
Service
api.example.com
Global
Queue
No
SQL
No
SQL
Auto
Scaling
Actions for 1M Users
• What data should be move to NoSQL from RDBMS?
ü Temporary but massive data e.g.) click information, log data, session data
ü Hot tables (tables who have very frequent updates)
ü Meta data store and search e.g.) object information from storage
ü Dynamic schema
ü Time-series table e.g.) monitoring logs
Users > 10M
M
R
R
S
R
R
Object
Storag
e
Object
Storag
e
CDN
static.example.com
image.example.com
www.example.com
…
DB
Cachin
g
DB
Cachin
g
API
Gateway
Service
api.example.com
No
SQL
No
SQL
Auto
Scaling
DB
Federation
+
DB Sharding
M
M
M
User
Message
Forum
Id=1~3
Id=4~6
Id=7~9
Auto
Scaling
Auto
Scaling
Global
Queue
API
Gateway
API
Gateway
CDN
Serverless
Functions
y = f(x) + α
Actions for 10M Users
• Multi Zone architecture in every region
• Consider expansion for all components e.g.) Multi-Master DB
• Maximize caching utilization from architecture and network
• Make autonomous systems
• Build auto-scaled systems using Cloud or Kubernetes
ü Virtual machine
ü Load balancer
ü Object storage
ü Notification system
ü Queuing system,
ü Workflows like Emailing, Alarming, etc.
Actions for 10M Users
• Entire Architecture tuning
• Application level tuning
• Globally distributed systems
• Global HA architecture (Multi Zone -> Multi Region)
• DevOps based application deployment
• CI(Continuous Integration)& CD(Continuous Development)
- Auto Scaling -
Actions for 10M Users
• What is Autoscaling?
Actions for 10M Users
• Autoscaling in On-Premise
desiredReplicas = ceil[currentReplicas * ( currentMetricValue / desiredMetricValue )]
* RC
Replication Controller
* Pod
A Group of containers
Actions for 10M Users
• Autoscaling in Cloud
Actions for 10M Users
• Autoscaling in Cloud
- Cloud -
Cloud Pros
• Fully Managed Services
• Flexible Scale Up/Down and Scale Out/In
• Auto Scaling
• HA using Multi Availability Zone and Multi Regions
• Performance
• Unlimited Capacity
• Serverless Architecture
• KMS(Key Management System)
• Computing, Network Big Data, IoT, AI, Machine Learning, etc.
• Low CAPEX/OPEX
• Compliance and Regulation e.g.) GDPR, CCPA
Cloud Cons
• Not Cheap and Unexpected Cost
• Not easy to migrate from On-Premise
• Sensitive Security
• No regions in Vietnam yet
• Vendor Stickiness?
• 99.999999999% Availability e.g.)AWS S3
• Support from outside vendors
• Need Experience and Training
AWS(Amazon Web Services) Regions
AWS Edge Locations ­ CDN, Serverless
AWS Services
AWS(Amazon Web Services)
AWS Basic Networking
a Internet
Gateway
Virtual
Private
Gateway
Router
Route
Table
Route
Table
Network
ACL
Network
ACL
Public Subnet (10.0.1.0/24)
Security GroupInstance
Private Subnet (10.0.2.0/24)
Security GroupInstance
VPC (MY_VPC_NAME) ­ 10.0.0.0/16
REGION (us-east-1)
NAT
Gateway
GCP(Google Cloud Platform)
Microsoft Azure
Cloudflare (CDN, WAF, DDoS, Bot, VPN, DNS)
- Cloud Demo ­
Demo for 20minutes with these implementation
1. Video Conference(WebRTC) platform using GCP
2. Testing PostgreSQL Multi-Master using Azure
3. AWS Chatbot usages for CloudWatch and Serverless Lambda
4. CDN, DNS and WAF integration using Cloudflare
- Wrap Up -
Best Practices for Global Markets
• Multi Regional Data Center (or Cloud)
• CDN(Content Delivery Network)
• Prepare Attacks -> WAF, DDoS Scrubbing, Bot Managing
• Server -> Instance in Cloud -> Serverless/Kubernetes
• Maximize Queuing
• Authentication out of Server
• DB -> Master/Slave, Primary/Secondary
• N/W Caching, DB Caching, Client Caching for Performance!
• Manual and routine jobs -> Automatic Workflows
• Microservices Architecture, DevOps and CI/CD
• Global Regulations Awareness
• Smart small, Grow Bigger!
Best Practices for Global Markets
Auto-Scale Everything,
Cache Everything,
And Protect Everything!
- Thank You. -
Service Platform Architect
Brandon Kang
sangjinn@gmail.com
https://tech.brandonkang.net

More Related Content

What's hot

Storage Services
Storage ServicesStorage Services
Storage Services
Pavel Revenkov
 
HBaseCon 2015: Taming GC Pauses for Large Java Heap in HBase
HBaseCon 2015: Taming GC Pauses for Large Java Heap in HBaseHBaseCon 2015: Taming GC Pauses for Large Java Heap in HBase
HBaseCon 2015: Taming GC Pauses for Large Java Heap in HBase
HBaseCon
 
Service workers - Velocity 2016 Training
Service workers - Velocity 2016 TrainingService workers - Velocity 2016 Training
Service workers - Velocity 2016 Training
Patrick Meenan
 
HBase at Xiaomi
HBase at XiaomiHBase at Xiaomi
HBase at Xiaomi
HBaseCon
 
hbaseconasia2017: hbase-2.0.0
hbaseconasia2017: hbase-2.0.0hbaseconasia2017: hbase-2.0.0
hbaseconasia2017: hbase-2.0.0
HBaseCon
 
Microsoft Azure Media Services
Microsoft Azure Media ServicesMicrosoft Azure Media Services
Microsoft Azure Media Services
Pavel Revenkov
 
Let the Tiger Roar! - MongoDB 3.0 + WiredTiger
Let the Tiger Roar! - MongoDB 3.0 + WiredTigerLet the Tiger Roar! - MongoDB 3.0 + WiredTiger
Let the Tiger Roar! - MongoDB 3.0 + WiredTiger
Jon Rangel
 
Real-time, Exactly-once Data Ingestion from Kafka to ClickHouse at eBay
Real-time, Exactly-once Data Ingestion from Kafka to ClickHouse at eBayReal-time, Exactly-once Data Ingestion from Kafka to ClickHouse at eBay
Real-time, Exactly-once Data Ingestion from Kafka to ClickHouse at eBay
Altinity Ltd
 
Caching methodology and strategies
Caching methodology and strategiesCaching methodology and strategies
Caching methodology and strategies
Tiep Vu
 
hbaseconasia2017: Large scale data near-line loading method and architecture
hbaseconasia2017: Large scale data near-line loading method and architecturehbaseconasia2017: Large scale data near-line loading method and architecture
hbaseconasia2017: Large scale data near-line loading method and architecture
HBaseCon
 
Inside CynosDB: MariaDB optimized for the cloud at Tencent
Inside CynosDB: MariaDB optimized for the cloud at TencentInside CynosDB: MariaDB optimized for the cloud at Tencent
Inside CynosDB: MariaDB optimized for the cloud at Tencent
MariaDB plc
 
Elephants in the Cloud
Elephants in the CloudElephants in the Cloud
Elephants in the Cloud
Mike Fowler
 
캐시 분산처리 인프라
캐시 분산처리 인프라캐시 분산처리 인프라
캐시 분산처리 인프라
Park Chunduck
 
hbaseconasia2017: HBase Practice At XiaoMi
hbaseconasia2017: HBase Practice At XiaoMihbaseconasia2017: HBase Practice At XiaoMi
hbaseconasia2017: HBase Practice At XiaoMi
HBaseCon
 
Kafka Multi-Tenancy - 160 Billion Daily Messages on One Shared Cluster at LINE
Kafka Multi-Tenancy - 160 Billion Daily Messages on One Shared Cluster at LINEKafka Multi-Tenancy - 160 Billion Daily Messages on One Shared Cluster at LINE
Kafka Multi-Tenancy - 160 Billion Daily Messages on One Shared Cluster at LINE
kawamuray
 
WiredTiger MongoDB Integration
WiredTiger MongoDB Integration WiredTiger MongoDB Integration
WiredTiger MongoDB Integration
MongoDB
 
HBaseCon 2012 | Base Metrics: What They Mean to You - Cloudera
HBaseCon 2012 | Base Metrics: What They Mean to You - ClouderaHBaseCon 2012 | Base Metrics: What They Mean to You - Cloudera
HBaseCon 2012 | Base Metrics: What They Mean to You - Cloudera
Cloudera, Inc.
 
Webinar: Introduction to MongoDB 3.0
Webinar: Introduction to MongoDB 3.0Webinar: Introduction to MongoDB 3.0
Webinar: Introduction to MongoDB 3.0
MongoDB
 
hbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
hbaseconasia2017: Building online HBase cluster of Zhihu based on Kuberneteshbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
hbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
HBaseCon
 
HBaseCon 2013: A Developer’s Guide to Coprocessors
HBaseCon 2013: A Developer’s Guide to CoprocessorsHBaseCon 2013: A Developer’s Guide to Coprocessors
HBaseCon 2013: A Developer’s Guide to Coprocessors
Cloudera, Inc.
 

What's hot (20)

Storage Services
Storage ServicesStorage Services
Storage Services
 
HBaseCon 2015: Taming GC Pauses for Large Java Heap in HBase
HBaseCon 2015: Taming GC Pauses for Large Java Heap in HBaseHBaseCon 2015: Taming GC Pauses for Large Java Heap in HBase
HBaseCon 2015: Taming GC Pauses for Large Java Heap in HBase
 
Service workers - Velocity 2016 Training
Service workers - Velocity 2016 TrainingService workers - Velocity 2016 Training
Service workers - Velocity 2016 Training
 
HBase at Xiaomi
HBase at XiaomiHBase at Xiaomi
HBase at Xiaomi
 
hbaseconasia2017: hbase-2.0.0
hbaseconasia2017: hbase-2.0.0hbaseconasia2017: hbase-2.0.0
hbaseconasia2017: hbase-2.0.0
 
Microsoft Azure Media Services
Microsoft Azure Media ServicesMicrosoft Azure Media Services
Microsoft Azure Media Services
 
Let the Tiger Roar! - MongoDB 3.0 + WiredTiger
Let the Tiger Roar! - MongoDB 3.0 + WiredTigerLet the Tiger Roar! - MongoDB 3.0 + WiredTiger
Let the Tiger Roar! - MongoDB 3.0 + WiredTiger
 
Real-time, Exactly-once Data Ingestion from Kafka to ClickHouse at eBay
Real-time, Exactly-once Data Ingestion from Kafka to ClickHouse at eBayReal-time, Exactly-once Data Ingestion from Kafka to ClickHouse at eBay
Real-time, Exactly-once Data Ingestion from Kafka to ClickHouse at eBay
 
Caching methodology and strategies
Caching methodology and strategiesCaching methodology and strategies
Caching methodology and strategies
 
hbaseconasia2017: Large scale data near-line loading method and architecture
hbaseconasia2017: Large scale data near-line loading method and architecturehbaseconasia2017: Large scale data near-line loading method and architecture
hbaseconasia2017: Large scale data near-line loading method and architecture
 
Inside CynosDB: MariaDB optimized for the cloud at Tencent
Inside CynosDB: MariaDB optimized for the cloud at TencentInside CynosDB: MariaDB optimized for the cloud at Tencent
Inside CynosDB: MariaDB optimized for the cloud at Tencent
 
Elephants in the Cloud
Elephants in the CloudElephants in the Cloud
Elephants in the Cloud
 
캐시 분산처리 인프라
캐시 분산처리 인프라캐시 분산처리 인프라
캐시 분산처리 인프라
 
hbaseconasia2017: HBase Practice At XiaoMi
hbaseconasia2017: HBase Practice At XiaoMihbaseconasia2017: HBase Practice At XiaoMi
hbaseconasia2017: HBase Practice At XiaoMi
 
Kafka Multi-Tenancy - 160 Billion Daily Messages on One Shared Cluster at LINE
Kafka Multi-Tenancy - 160 Billion Daily Messages on One Shared Cluster at LINEKafka Multi-Tenancy - 160 Billion Daily Messages on One Shared Cluster at LINE
Kafka Multi-Tenancy - 160 Billion Daily Messages on One Shared Cluster at LINE
 
WiredTiger MongoDB Integration
WiredTiger MongoDB Integration WiredTiger MongoDB Integration
WiredTiger MongoDB Integration
 
HBaseCon 2012 | Base Metrics: What They Mean to You - Cloudera
HBaseCon 2012 | Base Metrics: What They Mean to You - ClouderaHBaseCon 2012 | Base Metrics: What They Mean to You - Cloudera
HBaseCon 2012 | Base Metrics: What They Mean to You - Cloudera
 
Webinar: Introduction to MongoDB 3.0
Webinar: Introduction to MongoDB 3.0Webinar: Introduction to MongoDB 3.0
Webinar: Introduction to MongoDB 3.0
 
hbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
hbaseconasia2017: Building online HBase cluster of Zhihu based on Kuberneteshbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
hbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
 
HBaseCon 2013: A Developer’s Guide to Coprocessors
HBaseCon 2013: A Developer’s Guide to CoprocessorsHBaseCon 2013: A Developer’s Guide to Coprocessors
HBaseCon 2013: A Developer’s Guide to Coprocessors
 

Similar to Scalability strategies for cloud based system architecture

Building a Just-in-Time Application Stack for Analysts
Building a Just-in-Time Application Stack for AnalystsBuilding a Just-in-Time Application Stack for Analysts
Building a Just-in-Time Application Stack for Analysts
Avere Systems
 
HPC in AWS - Technical Workshop
HPC in AWS - Technical WorkshopHPC in AWS - Technical Workshop
HPC in AWS - Technical Workshop
Alex Barbosa Coqueiro
 
Azure vs AWS Best Practices: What You Need to Know
Azure vs AWS Best Practices: What You Need to KnowAzure vs AWS Best Practices: What You Need to Know
Azure vs AWS Best Practices: What You Need to Know
RightScale
 
Introduction of AWS Cloud Computing and its future for Biometric Department
Introduction of AWS Cloud Computing and its future for Biometric DepartmentIntroduction of AWS Cloud Computing and its future for Biometric Department
Introduction of AWS Cloud Computing and its future for Biometric Department
Kevin Lee
 
Microsoft cloud stack
Microsoft cloud stackMicrosoft cloud stack
Microsoft cloud stack
Michael Rüefli
 
AWS Meetup Fort Lauderdale Re:invent Recap
AWS Meetup Fort Lauderdale Re:invent RecapAWS Meetup Fort Lauderdale Re:invent Recap
AWS Meetup Fort Lauderdale Re:invent Recap
Anthony Palmer
 
The Best of re:invent 2016
The Best of re:invent 2016The Best of re:invent 2016
The Best of re:invent 2016
Amazon Web Services
 
re:Invent Recap-AWSMeetup
re:Invent Recap-AWSMeetupre:Invent Recap-AWSMeetup
re:Invent Recap-AWSMeetup
CloudHesive
 
Cloud computing and its job opportunities
Cloud computing and its job opportunities Cloud computing and its job opportunities
Cloud computing and its job opportunities
Ramya SK
 
Best of re:Invent
Best of re:InventBest of re:Invent
Best of re:Invent
Amazon Web Services
 
AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...
AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...
AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...
Amazon Web Services
 
AWS 101, London - September 2014
AWS 101, London - September 2014AWS 101, London - September 2014
AWS 101, London - September 2014
Ian Massingham
 
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
Adrian Cockcroft
 
AWS Cloud Computing for Startups Werner Vogels -part i
AWS Cloud Computing for Startups   Werner Vogels -part iAWS Cloud Computing for Startups   Werner Vogels -part i
AWS Cloud Computing for Startups Werner Vogels -part iAmazon Web Services
 
Building a Big Data & Analytics Platform using AWS
Building a Big Data & Analytics Platform using AWS Building a Big Data & Analytics Platform using AWS
Building a Big Data & Analytics Platform using AWS
Amazon Web Services
 
Achieve big data analytic platform with lambda architecture on cloud
Achieve big data analytic platform with lambda architecture on cloudAchieve big data analytic platform with lambda architecture on cloud
Achieve big data analytic platform with lambda architecture on cloud
Scott Miao
 
Introduction to Amazon EC2
Introduction to Amazon EC2Introduction to Amazon EC2
Introduction to Amazon EC2
Amazon Web Services
 
Getting Started with AWS Compute Services
Getting Started with AWS Compute ServicesGetting Started with AWS Compute Services
Getting Started with AWS Compute Services
Amazon Web Services
 
AWS Webcast - Website Hosting in the Cloud
AWS Webcast - Website Hosting in the CloudAWS Webcast - Website Hosting in the Cloud
AWS Webcast - Website Hosting in the Cloud
Amazon Web Services
 

Similar to Scalability strategies for cloud based system architecture (20)

Building a Just-in-Time Application Stack for Analysts
Building a Just-in-Time Application Stack for AnalystsBuilding a Just-in-Time Application Stack for Analysts
Building a Just-in-Time Application Stack for Analysts
 
HPC in AWS - Technical Workshop
HPC in AWS - Technical WorkshopHPC in AWS - Technical Workshop
HPC in AWS - Technical Workshop
 
Azure vs AWS Best Practices: What You Need to Know
Azure vs AWS Best Practices: What You Need to KnowAzure vs AWS Best Practices: What You Need to Know
Azure vs AWS Best Practices: What You Need to Know
 
Introduction of AWS Cloud Computing and its future for Biometric Department
Introduction of AWS Cloud Computing and its future for Biometric DepartmentIntroduction of AWS Cloud Computing and its future for Biometric Department
Introduction of AWS Cloud Computing and its future for Biometric Department
 
Microsoft cloud stack
Microsoft cloud stackMicrosoft cloud stack
Microsoft cloud stack
 
AWS 101 December 2014
AWS 101 December 2014AWS 101 December 2014
AWS 101 December 2014
 
AWS Meetup Fort Lauderdale Re:invent Recap
AWS Meetup Fort Lauderdale Re:invent RecapAWS Meetup Fort Lauderdale Re:invent Recap
AWS Meetup Fort Lauderdale Re:invent Recap
 
The Best of re:invent 2016
The Best of re:invent 2016The Best of re:invent 2016
The Best of re:invent 2016
 
re:Invent Recap-AWSMeetup
re:Invent Recap-AWSMeetupre:Invent Recap-AWSMeetup
re:Invent Recap-AWSMeetup
 
Cloud computing and its job opportunities
Cloud computing and its job opportunities Cloud computing and its job opportunities
Cloud computing and its job opportunities
 
Best of re:Invent
Best of re:InventBest of re:Invent
Best of re:Invent
 
AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...
AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...
AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...
 
AWS 101, London - September 2014
AWS 101, London - September 2014AWS 101, London - September 2014
AWS 101, London - September 2014
 
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
 
AWS Cloud Computing for Startups Werner Vogels -part i
AWS Cloud Computing for Startups   Werner Vogels -part iAWS Cloud Computing for Startups   Werner Vogels -part i
AWS Cloud Computing for Startups Werner Vogels -part i
 
Building a Big Data & Analytics Platform using AWS
Building a Big Data & Analytics Platform using AWS Building a Big Data & Analytics Platform using AWS
Building a Big Data & Analytics Platform using AWS
 
Achieve big data analytic platform with lambda architecture on cloud
Achieve big data analytic platform with lambda architecture on cloudAchieve big data analytic platform with lambda architecture on cloud
Achieve big data analytic platform with lambda architecture on cloud
 
Introduction to Amazon EC2
Introduction to Amazon EC2Introduction to Amazon EC2
Introduction to Amazon EC2
 
Getting Started with AWS Compute Services
Getting Started with AWS Compute ServicesGetting Started with AWS Compute Services
Getting Started with AWS Compute Services
 
AWS Webcast - Website Hosting in the Cloud
AWS Webcast - Website Hosting in the CloudAWS Webcast - Website Hosting in the Cloud
AWS Webcast - Website Hosting in the Cloud
 

More from SangJin Kang

웹에 빠른 날개를 달아주는 웹 성능 향상 이야기
웹에 빠른 날개를 달아주는 웹 성능 향상 이야기웹에 빠른 날개를 달아주는 웹 성능 향상 이야기
웹에 빠른 날개를 달아주는 웹 성능 향상 이야기
SangJin Kang
 
How to Replicate PostgreSQL Database
How to Replicate PostgreSQL DatabaseHow to Replicate PostgreSQL Database
How to Replicate PostgreSQL Database
SangJin Kang
 
HTTP/3 시대의 웹 성능 최적화 기술 이해하기
HTTP/3 시대의 웹 성능 최적화 기술 이해하기HTTP/3 시대의 웹 성능 최적화 기술 이해하기
HTTP/3 시대의 웹 성능 최적화 기술 이해하기
SangJin Kang
 
수요자 중심의 클라우드 운영 및 전략 (CIO Summit 2019)
수요자 중심의 클라우드 운영 및 전략 (CIO Summit 2019)수요자 중심의 클라우드 운영 및 전략 (CIO Summit 2019)
수요자 중심의 클라우드 운영 및 전략 (CIO Summit 2019)
SangJin Kang
 
How to develop and localize Xbox 360 titles
How to develop and localize Xbox 360 titlesHow to develop and localize Xbox 360 titles
How to develop and localize Xbox 360 titles
SangJin Kang
 
Akamai 서비스 트러블 슈팅 및 테스트 방법과 도구
Akamai 서비스 트러블 슈팅 및 테스트 방법과 도구Akamai 서비스 트러블 슈팅 및 테스트 방법과 도구
Akamai 서비스 트러블 슈팅 및 테스트 방법과 도구
SangJin Kang
 
HTTP/2와 웹 성능 최적화 방안
HTTP/2와 웹 성능 최적화 방안HTTP/2와 웹 성능 최적화 방안
HTTP/2와 웹 성능 최적화 방안
SangJin Kang
 
Akamai Korea - Tech Day (2015/03/11) DNS
Akamai Korea - Tech Day (2015/03/11) DNSAkamai Korea - Tech Day (2015/03/11) DNS
Akamai Korea - Tech Day (2015/03/11) DNS
SangJin Kang
 
Akamai Korea - Tech Day (2015/03/11) HTTP/2
Akamai Korea - Tech Day (2015/03/11) HTTP/2Akamai Korea - Tech Day (2015/03/11) HTTP/2
Akamai Korea - Tech Day (2015/03/11) HTTP/2
SangJin Kang
 
HTML5 for web app. development
HTML5 for web app. developmentHTML5 for web app. development
HTML5 for web app. development
SangJin Kang
 
Agile - SCRUM을 통한 개발관리
Agile - SCRUM을 통한 개발관리Agile - SCRUM을 통한 개발관리
Agile - SCRUM을 통한 개발관리
SangJin Kang
 
XNA2.0 Network Programming
XNA2.0 Network ProgrammingXNA2.0 Network Programming
XNA2.0 Network Programming
SangJin Kang
 
XNA Introduction
XNA IntroductionXNA Introduction
XNA Introduction
SangJin Kang
 

More from SangJin Kang (13)

웹에 빠른 날개를 달아주는 웹 성능 향상 이야기
웹에 빠른 날개를 달아주는 웹 성능 향상 이야기웹에 빠른 날개를 달아주는 웹 성능 향상 이야기
웹에 빠른 날개를 달아주는 웹 성능 향상 이야기
 
How to Replicate PostgreSQL Database
How to Replicate PostgreSQL DatabaseHow to Replicate PostgreSQL Database
How to Replicate PostgreSQL Database
 
HTTP/3 시대의 웹 성능 최적화 기술 이해하기
HTTP/3 시대의 웹 성능 최적화 기술 이해하기HTTP/3 시대의 웹 성능 최적화 기술 이해하기
HTTP/3 시대의 웹 성능 최적화 기술 이해하기
 
수요자 중심의 클라우드 운영 및 전략 (CIO Summit 2019)
수요자 중심의 클라우드 운영 및 전략 (CIO Summit 2019)수요자 중심의 클라우드 운영 및 전략 (CIO Summit 2019)
수요자 중심의 클라우드 운영 및 전략 (CIO Summit 2019)
 
How to develop and localize Xbox 360 titles
How to develop and localize Xbox 360 titlesHow to develop and localize Xbox 360 titles
How to develop and localize Xbox 360 titles
 
Akamai 서비스 트러블 슈팅 및 테스트 방법과 도구
Akamai 서비스 트러블 슈팅 및 테스트 방법과 도구Akamai 서비스 트러블 슈팅 및 테스트 방법과 도구
Akamai 서비스 트러블 슈팅 및 테스트 방법과 도구
 
HTTP/2와 웹 성능 최적화 방안
HTTP/2와 웹 성능 최적화 방안HTTP/2와 웹 성능 최적화 방안
HTTP/2와 웹 성능 최적화 방안
 
Akamai Korea - Tech Day (2015/03/11) DNS
Akamai Korea - Tech Day (2015/03/11) DNSAkamai Korea - Tech Day (2015/03/11) DNS
Akamai Korea - Tech Day (2015/03/11) DNS
 
Akamai Korea - Tech Day (2015/03/11) HTTP/2
Akamai Korea - Tech Day (2015/03/11) HTTP/2Akamai Korea - Tech Day (2015/03/11) HTTP/2
Akamai Korea - Tech Day (2015/03/11) HTTP/2
 
HTML5 for web app. development
HTML5 for web app. developmentHTML5 for web app. development
HTML5 for web app. development
 
Agile - SCRUM을 통한 개발관리
Agile - SCRUM을 통한 개발관리Agile - SCRUM을 통한 개발관리
Agile - SCRUM을 통한 개발관리
 
XNA2.0 Network Programming
XNA2.0 Network ProgrammingXNA2.0 Network Programming
XNA2.0 Network Programming
 
XNA Introduction
XNA IntroductionXNA Introduction
XNA Introduction
 

Recently uploaded

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
UiPathCommunity
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
Dorra BARTAGUIZ
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
Elena Simperl
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
Cheryl Hung
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
Prayukth K V
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
Thijs Feryn
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Inflectra
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
Elena Simperl
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Jeffrey Haguewood
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Product School
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
Frank van Harmelen
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
91mobiles
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
DanBrown980551
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
Paul Groth
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 

Recently uploaded (20)

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 

Scalability strategies for cloud based system architecture

  • 1. Service Platform Architect Brandon Kang sangjinn@gmail.com https://tech.brandonkang.net May 2020 Scalability Strategies for Cloud based System Architecture
  • 2. Agenda • Scalability & Availability for the Global Markets • Global scaled Scalability, Availability and Security • Architecture for 100, 1K, 100K, 500K, 1M and 10M global users • Auto-Scaling • Understand Cloud Services • Cloud Demo(AWS, GCP, Azure and Cloudflare) • Wrap-Up
  • 4. Scalability • Scalability = capability of a system to handle a growing work • Vertical : Scale Up or Down ü Add or Remove Resources ü CPU ü Memory ü Storage • Horizontal: Scale Out or In ü Add or Remove Systems ü Instance Scale OutScale In VM VM VM VMVM VMVM VM Scale Down Scale Up VMVM
  • 6. Scalability vs. Availability • Need 4 * VMs to provide services Scalability: (2* VMs in a region) + (2* VMs in another region) Availability: (4* VMs in a region) + (4* VMs in another region) for HA
  • 8. Users < 100 Network Fixed IP Application Database
  • 9. Users > 1,000 Master Slave Load Balancer Zone A Zone B Write Write Read Replication Region
  • 10. Users > 100,000 M R R S R R Active/ Write Read Replica Read Replica Read Replica Read Replica Stand-by/ Write
  • 11. Users > 100,000 M R R S R R Object Storage Object Storage www.example.com api.example.com … CDN static.example.com image.example.com …
  • 12. Users > 500,000 M R R S R R Object Storage Object Storage CDN static.example.com image.example.com www.example.com … DB Caching DB Caching API Gateway Service Micro-Services Architecture api.example.com
  • 13. Users > 500,000 • MSA(Microservices Architecture) ü Every functions move to Microservices ü Independent and loosely coupled • API Gateway ü API Routing ü API Security ü Authentication ü Authorization ü API Caching ü Hits Rate Limit ü Hits Throttling ü Traffic Monitoring
  • 14. Users > 1M M R R S R R Object Storage Object Storage CDN static.example.com image.example.com www.example.com … DB Caching DB Caching API Gateway Service api.example.com Global Queue No SQL No SQL Auto Scaling
  • 15. Actions for 1M Users • What data should be move to NoSQL from RDBMS? ü Temporary but massive data e.g.) click information, log data, session data ü Hot tables (tables who have very frequent updates) ü Meta data store and search e.g.) object information from storage ü Dynamic schema ü Time-series table e.g.) monitoring logs
  • 16. Users > 10M M R R S R R Object Storag e Object Storag e CDN static.example.com image.example.com www.example.com … DB Cachin g DB Cachin g API Gateway Service api.example.com No SQL No SQL Auto Scaling DB Federation + DB Sharding M M M User Message Forum Id=1~3 Id=4~6 Id=7~9 Auto Scaling Auto Scaling Global Queue API Gateway API Gateway CDN Serverless Functions y = f(x) + α
  • 17. Actions for 10M Users • Multi Zone architecture in every region • Consider expansion for all components e.g.) Multi-Master DB • Maximize caching utilization from architecture and network • Make autonomous systems • Build auto-scaled systems using Cloud or Kubernetes ü Virtual machine ü Load balancer ü Object storage ü Notification system ü Queuing system, ü Workflows like Emailing, Alarming, etc.
  • 18. Actions for 10M Users • Entire Architecture tuning • Application level tuning • Globally distributed systems • Global HA architecture (Multi Zone -> Multi Region) • DevOps based application deployment • CI(Continuous Integration)& CD(Continuous Development)
  • 20. Actions for 10M Users • What is Autoscaling?
  • 21. Actions for 10M Users • Autoscaling in On-Premise desiredReplicas = ceil[currentReplicas * ( currentMetricValue / desiredMetricValue )] * RC Replication Controller * Pod A Group of containers
  • 22. Actions for 10M Users • Autoscaling in Cloud
  • 23. Actions for 10M Users • Autoscaling in Cloud
  • 25. Cloud Pros • Fully Managed Services • Flexible Scale Up/Down and Scale Out/In • Auto Scaling • HA using Multi Availability Zone and Multi Regions • Performance • Unlimited Capacity • Serverless Architecture • KMS(Key Management System) • Computing, Network Big Data, IoT, AI, Machine Learning, etc. • Low CAPEX/OPEX • Compliance and Regulation e.g.) GDPR, CCPA
  • 26. Cloud Cons • Not Cheap and Unexpected Cost • Not easy to migrate from On-Premise • Sensitive Security • No regions in Vietnam yet • Vendor Stickiness? • 99.999999999% Availability e.g.)AWS S3 • Support from outside vendors • Need Experience and Training
  • 28. AWS Edge Locations ­ CDN, Serverless
  • 31. AWS Basic Networking a Internet Gateway Virtual Private Gateway Router Route Table Route Table Network ACL Network ACL Public Subnet (10.0.1.0/24) Security GroupInstance Private Subnet (10.0.2.0/24) Security GroupInstance VPC (MY_VPC_NAME) ­ 10.0.0.0/16 REGION (us-east-1) NAT Gateway
  • 34. Cloudflare (CDN, WAF, DDoS, Bot, VPN, DNS)
  • 35. - Cloud Demo ­ Demo for 20minutes with these implementation 1. Video Conference(WebRTC) platform using GCP 2. Testing PostgreSQL Multi-Master using Azure 3. AWS Chatbot usages for CloudWatch and Serverless Lambda 4. CDN, DNS and WAF integration using Cloudflare
  • 37. Best Practices for Global Markets • Multi Regional Data Center (or Cloud) • CDN(Content Delivery Network) • Prepare Attacks -> WAF, DDoS Scrubbing, Bot Managing • Server -> Instance in Cloud -> Serverless/Kubernetes • Maximize Queuing • Authentication out of Server • DB -> Master/Slave, Primary/Secondary • N/W Caching, DB Caching, Client Caching for Performance! • Manual and routine jobs -> Automatic Workflows • Microservices Architecture, DevOps and CI/CD • Global Regulations Awareness • Smart small, Grow Bigger!
  • 38. Best Practices for Global Markets Auto-Scale Everything, Cache Everything, And Protect Everything!
  • 39. - Thank You. - Service Platform Architect Brandon Kang sangjinn@gmail.com https://tech.brandonkang.net