http://idcee.org/p/steffen-krause/
Steffen Krause is a Technology Evangelist for Amazon Web Services. After working for many years in the implementation of complex information management, BI and database solutions for customers of all sizes, he now works on highly scalable, cost effective IT solutions in the cloud. Steffen is a regular speaker at international conferences on databases and cloud services.
Pic's are here: http://www.flickr.com/photos/idcee/sets/
More @ http://idcee.org
Follow us on:
YouTube: http://www.youtube.com/user/OfficialIDCEEChannel
Facebook: https://www.facebook.com/IDCEE
Linkedin: http://www.linkedin.com/groups/IDCEE-3940138
Twitter: https://twitter.com/idcee_eu
Google+: http://gplus.to/idcee
Flickr: http://www.flickr.com/photos/idcee/collections/
9. Rule 1: Service all web requests
Rule 2: Service requests as fast as possible
Rule 3: Handle requests at any scale
Rule 4: Simplify architecture with services
Rule 5: Automate operational management
Rule 6: Leverage unique cloud properties
11. Rule 1: Service all web requests
a) Make sure requests get to your ‘front door’
Request
DNS
Application
Data
12. Rule 1: Service all web requests
a) Make sure requests get to your ‘front door’
Request
DNS
Clients can’t resolve
you?
Application
Data
…then this is
irrelevant
13. Rule 1: Service all web requests
a) Make sure requests get to your ‘front door’
Request
DNS
Application
Feature
Global
“100%
Available”
SLA
Route53
Scalable
Latency based routing
Integrated
http://aws.amazon.com/route53/sla
Secure
Data
Details
Supported from AWS global edge locations for fast and reliable domain
name resolution
Automatically scales based upon query volumes
Supports resolution of endpoints based upon latency, enabling multiregion application delivery
Integrates with other AWS services allowing Route 53 to front load
balancers, S3 and EC2
Integrates with IAM giving fine grained control over DNS record access
14. Rule 1: Service all web requests
a) Make sure requests get to your ‘front door’
b) Make sure you open the door when they arrive
Request
Data
Application
DNS
Region
Availability Zone
Elastic load balancing
Multi-availability zone
Multi-region
Availability Zone
Route53
Availability Zone
Elastic
Load
Balancer
Availability Zone
Region
15. Rule 1: Service all web requests
a) Make sure requests get to your ‘front door’
b) Make sure you open the door when they arrive
c) Have the data to form a response
Request
Application
DNS
Data
Region
RDS
Availability Zone
Multi-AZ
Master-slave
Availability Zone
Route53
Read-replicas
Availability Zone
Elastic
Load
Balancer
Availability Zone
Region
16. Rule 1: Service all web requests
Rule 2: Service requests as fast as possible
Rule 3: Handle requests at any scale
Rule 4: Simplify architecture with services
Rule 5: Automate operational management
Rule 6: Leverage unique cloud properties
17. Rule 2: Service requests as fast as possible
a) Choose the fastest route
Request
Region
A
Route53
Region B
18. Rule 2: Service requests as fast as possible
a) Choose the fastest route
Request
16ms
Region
A
Route53
92ms
Region B
19. Rule 2: Service requests as fast as possible
a) Choose the fastest route
Request
Region A DNS entry
Route53
16ms
Region
A
Region B
20. Rule 2: Service requests as fast as possible
a) Choose the fastest route
b) Offload your application servers
CloudFront
3
Served from S3
World-wide content distribution network
/images/*
Easily distribute content to end users with low
latency, high data transfer speeds, and no
commitments.
2
London
Served from EC2
*.php
Paris
1
Single CNAME
www.mysite.com
NY
21. Rule 2: Service requests as fast as possible
a) Choose the fastest route
b) Offload your application servers
c) Cache it if you can
ElastiCache
Memcached compatible caching
Serve frequently requested & slow
changing data from scalable cache
clusters
Reduce load on database and other
servers
22. Rule 2: Service requests as fast as possible
Database Query Performance
a)
b)
c)
d)
Choose the fastest route
Offload your application servers
Cache it if you can
Single digit latencies where it matters
Desired consistency, predictability
Actual
degraded
performance
with scale
Scale
23. Rule 2: Service requests as fast as possible
Database Query Performance
a)
b)
c)
d)
Choose the fastest route
Offload your application servers
Cache it if you can
Single digit latencies where it matters
Desired consistency, predictability
Management problems
Data sharding
Data caching
Provisioning
Cluster management
Fault management
Actual
degraded
performance
with scale
Scale
24. Rule 2: Service requests as fast as possible
a)
b)
c)
d)
Choose the fastest route
Offload your application servers
Cache it if you can
Single digit latencies where it matters
Database Query Performance
DynamoDB
Low latency
Large scale
Zero admin
Predictable performance
Dynamo DB Query Performance
Average single-digit milliseconds server side
latencies
Runs on solid state drives, and is built to
maintain consistent, fast latencies at any scale
Scale
25. Rule 1: Service all web requests
Rule 2: Service requests as fast as possible
Rule 3: Handle requests at any scale
Rule 4: Simplify architecture with services
Rule 5: Automate operational management
Rule 6: Leverage unique cloud properties
26. Rule 3: Handle requests at any scale
a) Scale up
Vertical Scaling
From $0.02/hr
Scale up with Elastic Compute Cloud (EC2)
Basic unit of compute capacity
Range of CPU, memory & local disk options
18 Instance types available, from micro through cluster
compute to SSD backed
27. Rule 3: Handle requests at any scale
a) Scale up
b) Scale out
as-create-auto-scaling-group MyGroup
--launch-configuration MyConfig
--availability-zones eu-west-1a
--min-size 4
--max-size 200
Trigger
auto-scaling
policy
Auto-scaling
Automatic re-sizing of compute clusters based upon demand
28. Rule 3: Handle requests at any scale
a) Scale up
b) Scale out
Manually
By Schedule
Send an API call use CLI to
Preemptiveormanual
launch/terminate instances – Only need
toscaling of capacity
specify capacity change (+/-)
Scale up/downscaling date and time
Regular based on up and
e.g. before a marketing event add 10 more
instances
e.g. scale from 0 to 2 for batch processing
every night or double capacity on Fridays
By Policy
Auto-Rebalance
Scale in response to based
Dynamic scalechanging
conditions, based on user configured
upon custom metrics
real-time monitoring and alerts
Instances are automatically
Maintain capacity across
launched/terminated to ensure the
availability across multiple
application is balanced zones
e.g. SQS queue depth, Average CPU load,
ELB latency
Azs
e.g. Instance availability maintained in
event of AZ becoming unavailable
down of instances
29. Rule 3: Handle requests at any scale
a) Scale up
b) Scale out
c) Dial it up
Elastic Block Store
DynamoDB
Provisioned IOPS up to 4000 per EBS
Provisioned read/write performance per
volume
table
Predictable performance for
Predictable high performance scaled via
demanding workloads such as
console or API
databases
30. “AWS gave us the flexibility to bring a massive
amount of capacity online in a short period of
time and allowed us to do so in an operationally
straightforward way.
DynamoDB:
over 500,000 writes per
second
Amazon EMR:
more than 1 million writes
per second
AWS is now Shazam’s cloud provider of choice,”
Jason Titus,
CTO
31. Rule 1: Service all web requests
Rule 2: Service requests as fast as possible
Rule 3: Handle requests at any scale
Rule 4: Simplify architecture with services
Rule 5: Automate operational management
Rule 6: Leverage unique cloud properties
32. Rule 4: Simplify architecture with services
30%
On-Premise
Infrastructure
70%
Your
Business
Managing All of the
“Undifferentiated Heavy Lifting”
33. Rule 4: Simplify architecture with services
30%
On-Premise
Infrastructure
AWS
Cloud-Based
Infrastructure
70%
Your
Business
Managing All of the
“Undifferentiated Heavy Lifting”
More Time to Focus on
Your Business
70%
Configuring Your
Cloud Assets
30%
34. Rule 4: Simplify architecture with services
We take care of it…
Data Centers
Power
Cooling
Cabling
Networking
Racks
Servers
Storage
Labor
So you don’t have to …
Buy and install new hardware
Setup and configure new software
build or upgrade data centers
36. Rule 4: Simplify architecture with services
Relational Database Service
Use RDS for databases
Database-as-a-Service
No need to install or manage database instances
Scalable and fault tolerant configurations
DynamoDB
Provisioned throughput NoSQL database
Fast, predictable performance
Fully distributed, fault tolerant architecture
Use DynamoDB for
high performance keyvalue DB
37. Rule 4: Simplify architecture with services
Amazon SQS
Reliable message
queuing without
additional software
Reliable, highly scalable, queue service
Processing results
for storing messages as they travel
Amazon SQS
between instances
1
Processing
task/processing
2
trigger
Push inter-process
workflows into the
cloud with SWF
Simple Workflow
Task A
Reliably coordinate processing steps
across applications
Task B
3
(Auto-scaling)
Integrate AWS and non-AWS resources
Manage distributed state in complex
systems
Task C
38. Rule 4: Simplify architecture with services
Cloud Search
Don’t install search
software, use
CloudSearch
Document
Server
Elastic search engine based upon
Amazon A9 search engine
Fully managed service with
Search
Server
sophisticated feature set
Scales automatically
Results
Elastic MapReduce
Elastic Hadoop cluster
Integrates with S3 & DynamoDB
Leverage Hive & Pig analytics scripts
Integrates with instance types such as
spot
Process large volumes
of data cost effectively
with EMR
39. “Amazon CloudSearch is a game-changing
product that has allowed us to deliver powerful
new search capabilities. Our customers can now
find what they are looking for faster and more
easily than ever before…
….We saved many months of re-architecture
and development time by going with Amazon
CloudSearch”
Don MacAskill
CEO & Chief Geek
SmugMug
40. Rule 1: Service all web requests
Rule 2: Service requests as fast as possible
Rule 3: Handle requests at any scale
Rule 4: Simplify architecture with services
Rule 5: Automate operational management
Rule 6: Leverage unique cloud properties
41. Rule 5: Automate operational management
a) Everything is programmable
Access everything
via CLI, API or
Console
Compute
Security Scaling
CDN Backup
DNS Database
Storage Load Balancing
Workflow Monitoring
Networking
Messaging
Achieve the highest levels
of automation
sophistication with ease
42. Rule 5: Automate operational management
a) Everything is programmable
b) Think disposable, one click deployments
AWS
OpsWorks
AWS
CloudFormation
AWS Elastic
Beanstalk
Dev-Ops framework for
application lifecycle
management
Templates to deploy &
manage
Automate resource
management
Web App
Enterprise
App
Database
43. Rule 5: Automate operational management
a) Everything is programmable
b) Think disposable, one click deployments
c) Design for failure, implement self healing
Bootstrapping
Auto-scaling
Cloud Watch
Customize instance
startup
Maintain capacity of
instances
Know what’s going on,
take automated actions
Get instances to ask ‘who am
I?’ question on startup and be
configured dynamically upon
being answered
Using a minimum pool
size will maintain
capacity in the event of
instance failures
Use CloudWatch standard and
custom metrics to create
alarms.
Respond with automated
administration actions
44. Rule 5: Automate operational management
a) Everything is programmable
b) Think disposable, one click deployments
c) Design for failure, implement self healing
45. Rule 1: Service all web requests
Rule 2: Service requests as fast as possible
Rule 3: Handle requests at any scale
Rule 4: Simplify architecture with services
Rule 5: Automate operational management
Rule 6: Leverage unique cloud properties
47. Rule 6: Leverage unique cloud properties
a) Optimize costs with instance types
On-demand instances
Reserved instances
Spot instances
Unix/Linux instances start at
$0.02/hour
1- or 3-year terms
Bid on unused EC2 capacity
Pay as you go for compute power
Pay low up-front fee, receive significant hourly
discount
Spot Price based on
supply/demand, determined automatically
Low cost and flexibility
Low Cost / Predictability
Cost / Large Scale, dynamic workload handling
Pay only for what you use, no up-front
commitments or long-term contracts
Helps ensure compute capacity is available
when needed
Use Cases:
Applications with short term, spiky, or
unpredictable workloads;
Application development or testing
Use Cases:
Use Cases:
Applications with flexible start and end times
Applications with steady state or predictable
usage
Applications only feasible at very low compute
prices
Applications that require reserved
capacity, including disaster recovery
48. Rule 6: Leverage unique cloud properties
a) Optimize costs with instance types
b) Get insight fast with Elastic MapReduce
Elastic MapReduce
Feature
Details
Managed, elastic Hadoop cluster
Scalable
Use as many or as few compute instances running
Hadoop as you want. Modify the number of
instances while your job flow is running
Integrates with S3 & DynamoDB
Leverage Hive & Pig analytics scripts
Integrates with instance types such as spot
Integrated with
other services
Works seamlessly with S3 as origin and output.
Integrates with DynamoDB
Comprehensive
Supports languages such as Hive and Pig for
defining analytics, and allows complex definitions
in Cascading, Java, Ruby, Perl, Python, PHP, R, or
C++
Cost effective
Monitoring
Works with Spot instance types
Monitor job flows from with the management
console
49. Rule 6: Leverage unique cloud properties
a) Optimize costs with instance types
b) Get insight fast with Elastic MapReduce
c) Create a supercomputer backend when you need it
Cluster compute instances
Network placement groups
Implement HVM process execution
Cluster instances deployed in a ‘Placement Group’ enjoy low
Intel® Xeon® E5-2670 processors
latency, full bisection 10 Gbps bandwidth
10 Gigabit Ethernet
80 EC2
Compute Units
60GB RAM
3TB Local
Disk
Cluster Compute
10Gbps
50. Rule 1: Service all web requests
Rule 2: Service requests as fast as possible
Rule 3: Handle requests at any scale
Rule 4: Simplify architecture with services
Rule 5: Automate operational management
Rule 6: Leverage unique cloud properties
56. Testen Sie - kostenlos
•
•
•
•
http://aws.amazon.com/de/free/
Für neue AWS Kunden
Bis zu 12 Monate kostenlose Nutzung für viele AWS Dienste
Kreditkarte für Anmeldung erforderlich
– Aber keine Abbuchungen, so lange Sie im „Free Tier“ bleiben
Our goal, and what our customers tell us they see, is that this ratio is inverted after moving to AWS. When you move your infrastructure to the cloud, this changes things drastically. Only 30% of your time should be spent architecting for the cloud and configuring your assets. This gives you 70% of your time to focus on your business. Project teams are free to add value to the business and it's customers, to innovate more quickly, and to deliver products to market quickly as well.
There’s a shared responsibility to accomplish security and compliance objectives in AWS cloud. There are some elements that AWS takes responsibility for, and others that the customer must address. The outcome of the collaborative approach is positive results seen by customers around the world.