Ian Massingham gave a presentation on scaling applications on AWS from initial launch to over 1 million users. He began by discussing foundational AWS services and database options. He then walked through examples of scaling an application from 1 user to over 500,000 users by leveraging services like EC2, RDS, DynamoDB, ElastiCache, S3, CloudFront, and Auto Scaling. Key strategies included separating components across instances, adding redundancy, implementing caching, and leveraging auto scaling to dynamically scale resources based on demand. Massingham concluded by discussing strategies for scaling beyond 500,000 users such as service-oriented architectures and workload distribution across availability zones.

1. WEB SUMMIT
DUBLIN 2014
#AWS #WEBSUMMIT
Scaling on AWS for the
First 10 Million Users
Ian Massingham — Technical Evangelist
ianmas@amazon.com
@IanMmmm
STARTING AT 11:00

2. An Introduction to Scaling on AWS

3. An awful lot of things to read
Not where we want to start
EC2 Auto Scaling is one tool, not
the single thing that ‘fixes’
everything or enables you to scale

4. SO WHAT DO WE NEED FIRST?

5. SOME BASICS

6. 2004 2014
$7B+ retail business
8,000+ employees
A whole lot of servers
Every day, AWS adds
enough server
capacity to power that
entire $7B enterprise

7. Foundation
Services
Infrastructure
Collaboration and Sharing
Storage
(Object, Block and Archive) Security & Networking
Regions Availability Zones
Access Control
Platform
Services
Databases
Relational
NoSQL
Caching
Analytics
Hadoop
Real-time
Data
warehouse
App Services
Queuing
Orchestration
App streaming
Transcoding
Email
Search
Deployment & Management
Containers
Dev/ops Tools
Resource
Templates
Mobile Services
Identity
Sync
Mobile
Analytics
Notifications
Enterprise
Applications Virtual Desktops
Data
Workflows
Usage
Tracking
Monitoring
and Logs
Compute
(VMs, Auto-scaling and Load Balancing)
CDN and Points of Presence

8. Startups on AWS
http://aws.amazon.com/solutions/case-studies/

9. DAY ONE, USER ONE. PROBABLY YOU!
• A single EC2 instance
• With a full stack on this host
• Web app
• Database
• Management
• A single Elastic IP address
• Amazon Route 53 for DNS
Elastic IP
address
EC2
instance
Amazon
Route 53
User

10. WE’RE GOING TO NEED A BIGGER BOX!
• Simplest approach to scaling
• You can now leverage PIOPs
• High I/O instances
• High memory instances
• High CPU instances
• High storage instances
• Easy to change instance sizes
• Will hit an endpoint eventually

11. WE’RE GOING TO NEED A BIGGER BOX!
• We could potentially get to a few
hundred to a few thousand
depending on application
complexity and traffic
• No failover
• No redundancy
• Too many eggs in one basket
Elastic IP
address
EC2
instance
Amazon
Route 53
User

12. DAY TWO, USER >1
Web
instance
Database
instance
Elastic IP
address
Amazon
Route 53
User
• Separate out the functions on our
single host onto more than one:
• Web
• Database
• Make use of a database service

13. Microsoft SQL, Oracle,
MySQL or PostgresSQL
managed service
Flexible licensing: BYOL
or Included
Self-Managed Fully-Managed
Massively parallel,
petabyte scale, data
warehouse service
Fast, powerful &
easy to scale
Managed NoSQL database
service using SSD storage
Seamless scalability & zero
administration
Database server
on Amazon EC2
Amazon RDS
Amazon DynamoDB
Amazon Redshift
Your chosen database
running on Amazon EC2
Database Options

14. Building Blocks for Scaling on AWS

15. HOW DO I DETERMINE WHICH DB
TECHNOLOGY I NEED?

16. Why start with SQL?
• Established and well-worn technology
• Lots of existing code, communities, books & tools
• You aren’t going to break SQL DBs in your first 10 million users. No
really, you won’t*
• Clear patterns to scalability
* Unless you are manipulating data at MASSIVE scale; even then, SQL will have a place in your stack

17. Why might you need to use a NoSQL DB?
• Super low latency applications
• Metadata driven datasets
• Highly non-relational data
• Need schema-less data constructs*
• Massive amounts of data (again, in the TB range)
• Rapid ingest of data ( thousands of records/sec )
*Need != “it is easier to do dev without schemas”

18. IF YOU DO NEED NOSQL,
CONSIDER AMAZON DYNAMODB
• Managed, provisioned throughput NoSQL database
• Fast, predictable performance
• Fully distributed, fault tolerant architecture

19. Elastic IP
address
Web
instance
Amazon
Route 53
RDS DB
instance
User
USER >100
• First, let’s separate out our single
host into more than one:
• Web
• Database
• Use Amazon RDS to make your
life easier

20. USER >1000
Web
instance
Amazon RDS DB instance
Web
instance
Amazon RDS DB instance
Active (Multi-AZ)
Availability Zone Availability Zone
Standby (Multi-AZ)
Elastic Load
Balancing
Amazon
Route 53
User
• Next, let’s address the lack of
failover and redundancy issues:
• Elastic Load Balancing
• Another web instance
• In another Availability Zone
• Enable Amazon RDS Multi-AZ

21. USER 10,000 to 100,000s
RDS DB Instance
Active (Multi-AZ)
RDS DB Instance
Standby (Multi-AZ)
Elastic Load
Balancing
RDS DB Instance
Read Replica
Availability Zone Availability Zone
RDS DB Instance
Read Replica
RDS DB Instance
Read Replica
RDS DB Instance
Read Replica
Web
instance
Web
instance
Web
instance
Web
instance
Web
instance
Web
instance
Web
instance
Web
instance
Amazon
Route 53
User

22. Shift Some Load Around
Elastic Load
Balancing
Web
instance
RDS DB Instance
Active (Multi-AZ)
Availability Zone
Amazon
CloudFront
Amazon S3
Amazon
Route 53
User
ElastiCache
Amazon
Dynamo DB
• Think components and services:
• Move static content from the
web instance to Amazon S3
and Amazon CloudFront
• Move session/state and DB
caching to Amazon ElastiCache
or Amazon Dynamo DB

23. NOW THAT OUR WEB TIER IS
MUCH MORE LIGHTWEIGHT:
AUTOSCALING!

24. AMAZON EC2 AUTO SCALING
Automatic resizing of compute
clusters based on demand metrics
Trigger auto-scaling policy
Amazon
CloudWatch
Feature Details
Control Define minimum and maximum
instance pool sizes and when scaling
and cool down occurs
Integration Use metrics gathered by CloudWatch
to drive scaling (including custom
metrics)
Instance Types Use Auto Scaling with On-Demand
and Spot Instances

25. NOVEMBER TRAFFIC TO AMAZON.COM
AUTO SCALING LETS YOU DO THIS
76% of traditionally provisioned capacity
Traditional provisioning level
24% of traditionally provisioned capacity

26. Web
instance
Availability Zone
Amazon
Route 53
User
Amazon S3
Amazon
CloudFront
Web
instance
Availability Zone
Elastic Load
Balancing
Amazon
RDS DB Instance DynamoDB
Read Replica
Web
instance
Web
instance
ElastiCache RDS DB Instance
Read Replica
Web
instance
Web
instance
RDS DB Instance ElastiCache
Standby (Multi-AZ)
RDS DB Instance
Active (Multi-AZ)
USER 500,000+

27. GOING BEYOND 500,000 USERS

28. GOING BEYOND 500,000 USERS
• Monitoring, metrics and logging
• Listen and respond to customer feedback as well as your metrics
• Optimise each service/component
• Implement a Service Orientated Architecture with loose coupling
• Load balance between tiers

29. RDS DB Instance
Active (Multi-AZ)
Availability Zone
Elastic Load
Balancing
RDS DB Instance
Read Replica
RDS DB Instance
Read Replica
Web
instance
Web
instance
Web
instance
Web
instance
Amazon
Route 53
User
Amazon S3
Amazon
CloudFront
Amazon SQS
Amazon
DynamoDB
ElastiCache
Worker
instance
Worker
instance
Amazon
CloudWatch
Internal app
instance
Internal app
instance
Amazon SES
USER 1 MILLION+

30. WANT TO LEARN MORE?
COME TO OUR ARCHITECTURE
ASK US ANYTHING
TODAY AT 16:00

31. WEB SUMMIT
DUBLIN 2014
#AWS #WEBSUMMIT
DON’T FORGET YOUR CREDITS
Ian Massingham — Technical Evangelist
@IanMmmm
AWS Global News and Service Launches @AWScloud
AWS UK & Ireland News, Events & Webinars @AWS_UKI