Building powerful web applications in the AWS Cloud : A Love Story, Design patterns in web-based cloud architecture, Jinesh Varia gave this talk at Cloud Connect and several other places
http://aws.typepad.com/aws/2011/03/building-powerful-web-applications-in-the-aws-cloud-a-love-story.html
5. ThursDate.com
Magical Elastic Dating Every Thursday
An ephemeral website on On-Demand Cloud Infrastructure
Goes Live: Thursday 4:00 PM
Shuts Down: Thursday 7:00 PM
“Black Friday Effect”
Repeats every Thursday of the Week
13. media.myjavawebsite.com
(Static and Streaming data)
www.myjavawebsite.com
(Dynamic data)
Elastic IP
Route53
Hosted Zone
Amazon
CloudFront
Distribution
Apache
Logs
Static Data
Tomcat
MySQL
Amazon EC2
Instance
Root
Amazon S3
Bucket
Backups
Data
Snapshots
Amazon EBS Volume
Availability Zone #1
14.
15. media.myjavawebsite.com
(Static and Streaming data)
www.myjavawebsite.com
(Dynamic data)
Elastic IP
Route53
Hosted Zone
Amazon
CloudFront
Distribution
Apache
Logs
Static Data
Tomcat
MySQL
Amazon EC2
Instance
Root
Amazon S3
Bucket
Backups
Data
Snapshots
Amazon EBS Volume
Availability Zone #1
23. media.myjavawebsite.com
(Static and Streaming data)
www.myjavawebsite.com
(Dynamic data)
Route53
Hosted Zone
Amazon
CloudFront
Distribution
Amazon ELB
Apache
Apache
Tomcat
Tomcat
Logs
Static Data
Amazon S3
Bucket
Auto Scaling Group : Web Tier
Amazon EC2
Instances
MySQL
Amazon RDS DB Instance
Availability Zone #1
Backups
37. www.myjavawebsite.com
(Dynamic data)
Route53
Hosted Zone
media.myjavawebsite.com
(Static and Streaming data)
Amazon
CloudFront
Distribution
Amazon ELB
Apache
Tomcat
Amazon CloudSearch
(Index all your data)
Apache
Tomcat
Logs
Static Data
Amazon S3
Bucket
Auto Scaling Group : Web Tier
Amazon SNS
(notifications)
Amazon CloudWatch
(Monitoring)
ElastiCache
Async
Amazon Simple Email Service
(Send email)
RDS Read
Replica
Availability Zone #1
Backups
Availability Zone #2
RDS Standby
Slave
38. Tip: Loose coupling for other parts of the application
(For example, Image Processing)
39. Loose coupling sets you free
Use Amazon SQS as Buffers
Tight Coupling
Controller A
Q
Loose Coupling
using Queues
Controller B
Q
Controller A
Controller C
Q
Controller B
Controller C
40. CloudFront
Download
Distribution
RRS S3
Bucket to
Serve
content to
CloudFront
S3 Bucket
For Ingest
Instances
User
SQS Queue
Size for Thumbnail
Auto scaling
Group
Instances
SNS Topic
Tip:
Use Queues as
Buffer
SQS Queue
Size Image for
Mobile
SQS Queue
Size Image for Web
Auto scaling
Group
Instances
Auto scaling
Group
S3 Bucket
For originals
41. CloudFront
Download
Distribution
RRS S3
Bucket to
Serve
content to
CloudFront
S3 Bucket
For Ingest
Instances
User
Auto scaling
Group
Instances
Tip:
Automate by
using
workflows
Auto scaling
Group
SWF
Instances
Instance running
decider
Auto scaling
Group
S3 Bucket
For originals
42.
43. Pattern #8: Automate Your software development
lifecycle (Continuous Integration + Deployment)
44. Tip:
Bootstrap your
EC2 Instances
www.myjavawebsite.com
(Dynamic data)
Route53
Hosted Zone
media.myjavawebsite.com
(Static and Streaming data)
Amazon
CloudFront
Distribution
Amazon ELB
Apache
Amazon Machine
Image
Tomcat
Amazon
EC2
Instance
Logs
Static Data
Amazon S3
Bucket
Auto Scaling Group
MySQL
Backups
Amazon RDS DB Instance
CloudFormation
Template
Git
Build
Instance
Bootstrap
Scripts
Artifact
Bucket
Continuous Integration
Availability Zone #1
45. Apache
Your Code
Your Code
Fetch on boot
Fetch on boot
Tomcat
Apache
Struts
Your Code
Struts
Log4J
Spring
Tomcat
Struts
Fetch on boot
Amazon S3
Apache
Struts
Tomcat
Log4J
Hibernate
Spring
Amazon S3
scripts
Chef/puppet
Your Code
Log4J
Spring
Apache
A
p
a
c
h
e
Log4J
A
p
a
c
h
e
T
o
m
c
a
tS
t
r
u
t
s
Y
o
u
r
Spring
Hibernate
Hibernate
C
L
o
d
g
e
4
J
S
p
r
i
n
g
H
i
b
e
r
n
J
a
E
t
E
e
L
i
n
u
x
A
p
a
c
h
e
T
o
m
c
a
tS
t
r
u
t
s
Y
o
u
r
C
L
o
d
g
e
4
J
S
p
r
i
n
g
H
i
b
e
r
n
J
a
E
t
E
e
L
i
n
u
x
T
o
m
c
a
t
Tomcat
A
p
a
c
h
e
T
o
m
c
a
tS
t
r
u
t
s
Y
o
u
r
C
L
o
d
g
e
4
J
S
p
r
i
n
g
H
i
b
e
r
n
J
a
E
t
E
e
L
i
n
u
x
JEE
A
p
a
c
h
e
T
o
m
c
a
tS
t
r
u
t
s
Y
o
u
r
C
L
o
d
g
e
4
J
S
p
r
i
n
g
H
i
b
e
r
n
J
a
E
t
E
e
H
i
b
e
r
n
a
J
t
E
e
E
L
i
n
u
x
Hibernate
A
p
a
c
h
e
T
o
m
c
a
t
H
i
b
e
r
n
a
J
t
E
e
E
L
i
n
u
x
A
p
a
c
h
e
T
o
m
c
a
t
H
i
b
e
r
n
a
J
t
E
e
E
L
i
n
u
x
CHEF
Puppet
JEE
JEE
Linux
JEE
Linux
JEE
L
i
n
u
x
Linux
Linux
JEE
Linux
Java App Stack
Linux
Amazon EC2
Amazon EC2
Java AMI
Java AMI
Inventory of AMIs
Frozen Pizza Model
Golden AMI and
Fetch binaries on boot
Take N Bake Pizza Model
Amazon EC2
JeOS AMI
JeOS AMI and library of recipes
(install scripts)
Made to order Pizza Model
46. Cloud-Powered Software Development Lifecycle
Dev
All in one instance
Test/QA
2-tier app
with small DB
Staging
2-tier app with
production data
AWS CloudFormation + Chef (or Puppet)
Prod
2-tier app with
production data
Multi-AZ and HA
47. "80% of time we are wrong what
the customers wants"
51. High Error Rate
Monitoring
Load Balancing
(CloudWatch)
(ELB)
v1.1
v1.1
v1.2
v1.2
v1.1
v1.1
v1.2
v1.2
Web Server Fleet
(Amazon EC2)
Database Fleet
(RDS or DB on EC2)
52. Auto Scaling
Auto Scaling Group (Min, Max # of instances, Availability Zones .. )
Health Check (Maintain Min # active…)
Launch Configuration (AMIID, Instance type, UserData, Security Groups..)
Scaling Trigger (Metric, Upper Threshold, Lower Threshold, Time interval …)
Types of Scaling (Scale by Schedule, Scale by Policy)
Alarm (Notification Email, SMS, SQS, HTTP)
Availability Zones and Regions
68. Amazon S3
Via Flume/Fluentd
(Log Aggregator)
Logs from
EC2
Instances
Amazon S3
Task
Node
Amazon Elastic
MapReduce
Code/
Scripts
Mapper
Reducer
HiveQL
Pig Latin
Cascading
Amazon DynamoDB/RDS
Name
Node
Task
Node
Runs multiple
JobFlow Steps
Core
Node
HiveQL
Pig Latin
Query
Tip:
Log analysis
EMR Clusters
Core
Node
HDFS
Amazon Elastic MapReduce
Hadoop Cluster
JDBC/ODBC
BI Apps
69. Tip:
Pick the right “Big Data Stack” for your data
Client
Ingest
Mobile Client
•Kinesis
•Flume
•Kafka
Store
•S3
•DynamoDB
•RDS
Process
•Hadoop/EMR
•Redshift
•Spark/EMR
Visualize
•Tableau
•Datameer
•D3
(“Thing”)
Sensor
Questions:
Hot data vs. Cold data
Size of data & Speed of data ingest
Adhoc query (reports) vs. frequent queries (dashboard)
Batch vs. Stream
Low latency responses (ms) vs. high latency responses (Hours)
Cost tradeoffs
Insights
Reports
71. Pattern #9:
Build cost-aware architectures;
Optimize for cost and see savings as early as your next month’s bill
72. On-demand
Instances
• Pay as you go
• Zero commitment
Reserved
Instances
• One time low
upfront fee +
discounted hourly
costs
• Upto 71% savings
over On-Demand
Spot
Instances
• Requested Bid
Price and Pay as
you go
• Price change
every hour based
on unused EC2
capacity
Dedicated Instances
• Standard and
Reserved
• Multi-Tenant
Single Customer
• Ideal for
compliance and
regulatory
workloads
Billing Options
73. Mix and Match Instances
12
10
On-Demand
8
6
Light RI
Light RI
Light RI
Light RI
4
2
Heavy Utilization Reserved Instances
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
74. Upload large
datasets or log
files directly
Data
Source
Amazon S3
Amazon S3
Task
Node
Amazon Elastic
MapReduce
Code/
Scripts
Mapper
Reducer
HiveQL
Pig Latin
Cascading
Amazon DynamoDB/RDS
Name
Node
Task
Node
Runs multiple
JobFlow Steps
Core
Node
HiveQL
Pig Latin
Query
Adhoc EMR
Clusters
+
Task Nodes
on Spot
Core
Node
HDFS
Amazon Elastic MapReduce
Hadoop Cluster
JDBC/ODBC
BI Apps
78. www.myjavawebsite.com
(Dynamic data)
Route53
Hosted Zone
media.myjavawebsite.com
(Static and Streaming data)
Amazon ELB
Amazon
CloudFront
Distribution
Auto Scaling Group
Web Tier
Auto Scaling Group
Web Tier
Auto Scaling Group
App Tier
Auto Scaling Group
App Tier
Logs
Static Data
RDS Standby
Slave
Availability Zone #1
Availability Zone #2
Availability Zone #3
Amazon S3
Bucket
Backups
81. www.myjavawebsite.com
(Dynamic data)
Route53
Hosted Zone
#Permit HTTP(S) access to Web Layer from
the Entire Internet
ec2auth Web -p 80,443 -s 0.0.0.0/0
Amazon ELB
Amazon
CloudFront
Distribution
Auto Scaling Group
#Permit Web Layer access to App Layer
ec2auth App -p 8000 –o Web
# Permit App Layer access to DB
ec2auth DB -p 3209 –o App
# Permit admin access SSH to all three layers
# First allow connection from office to Web
tier, and from there to the other layers
ec2auth Web -p 22 -s <for example, network
block of your office>
ec2auth App -p 22 -o Web
ec2auth DB -p 22 -o Web
media.myjavawebsite.com
(Static and Streaming data)
Web Security Group
Web Tier
Logs
Static Data
Amazon S3
Bucket
Auto Scaling Group
App Tier
App Security Group
Backups
Availability Zone #2
DB Security Group
RDS Standby
Slave
Availability Zone #3
Amazon VPC Subnet
82. Implement security best practices at
every layer
Protect Your AWS Account
Control Internal Access to AWS Resources
Limit External Access to Your AWS Cloud
Protect Data in Transit and at Rest
Secure your data assets
Secure your compute assets (OS, Instance, App)
Backup and easily recover
Keep Track of Your Cloud Resources (Monitoring)
http://bit.ly/aws-security-best-practices-new
85. US West Traffic
Web
Web
App
Web
App
App
Web
Web
App
Web
App
App
US East Traffic
Web
Web
App
Web
App
App
Europe Traffic
Web
Web
App
Web
App
App
Web
Web
App
Web
App
App
Web
Web
App
Web
App
App
Asia Traffic
Web
Web
App
Web
App
App
Web
Web
App
Web
App
App
Auto Scaling group : Web
App Tier
Auto Scaling group : Web
App Tier
Auto Scaling group : Web
App Tier
Auto Scaling group : Web
App Tier
RDS
Master
RDS
Master
RDS
Master
RDS
Master
US-West
US-West-1b
RDS
Multi-AZ
US-East
US-East-1b
RDS
Multi-AZ
EU-West
EU-West-1b
Software-based Data Replicator
RDS
Multi-AZ
AP-SOUTHEAST
AP-SOUTHEAST-1b
RDS
Multi-AZ
86. Pattern #1: Design for failure and nothing will fail
Pattern #2: Edge cache static content
Pattern #3: Implement Elasticity
Pattern #4: Leverage Multiple Availability Zones
Pattern #5: Isolate read and write traffic; Isolate static and dynamic traffic
Pattern #6: Cache as much as possible; Cache is King
Pattern #7: Leverage app services to increase your productivity and scale your app
Pattern #8: Automate Your software development lifecycle
Pattern #9: Optimize for Cost and See Savings as early as your Next Month’s Bill
Pattern #10: Harden security at every stage
Pattern #11: Go global quickly (with single API)
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Using Elastic Ips for upgrade process
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Goal is to get instant feedback from your users and how your application behaves when real load. A/B testing is great. Control and Treatment http://20bits.com/articles/an-introduction-to-ab-testing/Prove success of interface changesProve interest in new features
So how can you do deployments and quickly deploy without any downtime
One simple technique is to do blue green deloyments. Green is your current deployment, you de
BG + Autoscaling = Cost savings + Scaling up
BG + Darklaunches in which you deploy both the versions side by side but customers does not know about it. Two separate code paths but only one activated. The other one is activated by a feature flag. This can be your beta test where you explicitly turn
Goal is to get instant feedback from your users and how your application behaves when real load. A/B testing is great. Control and Treatment http://20bits.com/articles/an-introduction-to-ab-testing/Prove success of interface changesProve interest in new features
What if you have 1000s of nodes. The blue green deployments works well but you may not want to spin up additional 100% capacity. So then you can gradually spin up in small increments of 10% so you only pay for additional 10% of your nodes. Plus you learn more about your customer behavior too.By routing only a small of users to your newly code deployed code base. Start small with 1% - A/B testing is randomizes the traffic and collect feedback on what’s working and what is not working. With a simple configuration change Autoscaling on the other hand scales. You don’t want to send all your traffic to new nodes just yet.Note: you have tested all your application, brought additional capacity before you route real traffic.
Slowly ramp of traffic by simply changing the configuration file in a/b testing and see autoscaling kicking off additional instances as needed. Note here you are not disrupting user behavior because they see only that version that they hit.
Slowly rampup traffic you are know whether the feature is working for your customer and you are learning more about customer.
Pager beeping. Something went wrong. Time to do rollback. Rollbacks are easy…
Change the config file route all the traffic back to your previous hosts….Test what went wrong…fix….deploy new instances using the same process
Start ramping up again…..
You are gradually deploying new software, using the dynamism of the cloud with zero downtime. And that’s cool.
Autoscaling handles all the capacity planning for you.
Batch Processing Architecture using Amazon Elastic MapReduce
puts focus on the date rather than the individual. Users propose fun activities. Other users can send them messages if they like their ideas.
Batch Processing Architecture using Amazon Elastic MapReduce
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ
Single InstanceWeb Application Architecture in 1 AZ