Highly Available Web Properties in Aws

Building HighlyAvailable, Scalable Web Properties with AWS Joe Ziegler │Technical Evangelist @jiyosub

1. ELASTICITY2. DESIGN FOR FAILURE3. LOOSE COUPLING4. SECURITY5. PERFORMANCE

# 1ELASTICITY ●○○○○

AMAZON EC2ELASTIC COMPUTE CLOUD

AMAZON CLOUDWATCHMONITORING FOR AWS RESOURCES

AUTO SCALINGSCALE UP/DOWN EC2 CAPACITY

ELASTIC LOAD BALANCINGNETWORK TRAFFIC DISTRIBUTION

without elasticity you cant accelerate

what you really need is… …adjustable capacity

# 2DESIGN FOR FAILURE ●●○○○

« Everything fails all the time » Werner Vogels CTO of Amazon

YOUR GOALApplications should continue to function even if the underlying physical hardware fails or is removed or replaced

Avoid single points of failure.Assume everything fails, and designbackwards.

AMAZON RDSRELATIONAL DATABASE SERVICE

AMAZONROUTE 53DOMAIN NAME SERVICE

AMAZON RDSMULTI-AZDEPLOYMENT

AMAZON CLOUDWATCH ALARMS

AWS BUILDING BLOCKSInherently Fault-Tolerant Services Fault-Tolerant with the right architecture Amazon S3  Elastic Load Balancing  Amazon EC2 Amazon SimpleDB  AWS IAM  Amazon EBS Amazon DynamoDB  AWS Elastic  Amazon RDS Amazon CloudFront Beanstalk Amazon SWF  Amazon VPC  Amazon Amazon SQS ElastiCache Amazon SNS  Amazon EMR Amazon SES  Amazon CloudSearch Amazon Route53

NETFLIXCHAOS MONKEY

# 3 LOOSECOUPLING ●●●○○

BUILD LOOSELYCOUPLED SYSTEMS The looser the are coupled, the bigger they scale

Create independent components

Create independent componentsDesign everything as a Black Box

Create independent componentsDesign everything as a Black BoxThink in terms of services

TRANSCODERECEIVE & PUBLISH

TRANSCODERECEIVE & PUBLISH QUEUE

AMAZON SQSSIMPLE QUEUE SERVICE

START CHECK REJECT STOP VIDEO YES NO TOO SPAM PUBLISH SPAM? LONG? CHECK & NOTIFYYES NO SHORTEN TRANSCODE VIDEO

MAINTENANCEAUDITABILITYFLEXIBILITYLOW-LEVEL

WHAT ARE WETRYING TO DO?

START CHECK REJECT STOP VIDEO YES NO TOO SPAM PUBLISH SPAM? LONG? CHECK & NOTIFYYES NO SHORTEN TRANSCODE VIDEO

TASKSDECISIONS STATELESS ! HISTORY

STATELESS SCALES HORIZONTALLY

AMAZON SWF WORKFLOW SERVICE FORSCALABLE, RESILIENT APPLICATIONS

AMAZON SWF Keeps track of :  State  Executed tasks  Timeouts  Errors

WORKFLOWACTORS

DECIDERS COORDINATION LOGIC1. Poll for work on a decision list Long polling: 60 seconds2. Evaluate workflow execution history SWF sends full history in JSON format3. Return decision to Amazon SWF Usually scheduling another task

WORKERS COORDINATION LOGIC1. Poll for work on a specific task list Long polling: 60 seconds2. Execute works, send heartbeats SWF sends input data from deciders3. Return success / failure Detailed data can be provided to deciders

ALLHORIZONTAL SCALING PATTERNS APPLY

NO NEW LANGUAGE TO LEARNYOUR CODE IS YOUR WORKFLOW LANGUAGE SWF MAINTAINS STATE

AWS FLOW FRAMEWORKJava Library • Entire workflow can be expressed in sequential code • Integrated with Java Utils API

CHAINED TASKS WITHOUT DECISIONS?RECEIVE TRANSCODE NOTIFY use AMAZON SQS

TASK GRAPH WITH DECISIONS? GOOD OKRECEIVE CHECK SPAM TRANSCODE VIDEO LENGTH CHECK LONG SPAM SHORTEN PUBLISH REJECT VIDEO & NOTIFY use AMAZON SWF

# 4SECURITY ●●●●○

AWS IAMIDENTITY AND ACCESS MANAGEMENT

MULTI-MULTI-FACTORAUTHENTICATION

SECURITY GROUPS LAYERED SECURITY

CERTIFICATIONS& COMPLIANCE SAS 70 Type II audit ISO 27001 Certification Sarbanes-Oxley SOX HIPAA healthcare FISMA US Federal Government DIACAP MAC III Sensitive IATO

 PCI DSS Compliant LEVEL 1 SERVICE PROVIDER

# 5PERFORMANCE ●●●●●

VERTICALSCALING

CLUSTER COMPUTEEIGHT EXTRA LARGE 2x Intel Xeon E5-2670, eight-core Sandy Bridge architecture 60.5 GB of memory – 3.3 TB of storage

CLUSTER GPUQUADRUPLE EXTRA LARGE Intel Xeon X5570, quad-core 2x Nehalem architecture NVIDIA Tesla Fermi 2x M2050 GPUs 22 GB of memory – 1.7 TB of storage

AMAZONELASTICACHEMEMCACHED COMPATIBLE IN-MEMORY CACHE

SEARCHENGINES

Relevance & Ranking

Faceting

RangeSearching

AMAZONCLOUDSEARCHFULLY-MANAGED SEARCH SERVICE

DEMOCRATIZE SEARCH Amazon CloudSearch delivers a fully-managed search service in the cloudthat can be set up and running in less than 1 hour, with automatic scaling for data & traffic, at a price starting at less than $100 per month.

MEDIA SHARINGREFERENCE ARCHITECTURE

1. ELASTICITY2. DESIGN FOR FAILURE3. LOOSE COUPLING4. SECURITY5. PERFORMANCE

« If youre not embarrassed when you ship your first version you waited too long » Matt Mullenweg CEO & Founder of WordPress

aws-apac-marketing@amazon.com

AWS ANZ Customer Appreciation DayIs Coming to Sydney November 13 th #AWSCADAU

http://www.newvem.com	169
http://www.scoop.it	21
http://www.linkedin.com	2
https://twitter.com	1
http://info.zooppa.com	1

Highly Available Web Properties in Aws

by Amazon Web Services on Aug 20, 2012

Accessibility

Categories

Upload Details

Usage Rights

5 Embeds 194

Statistics