Cloud computing with AWS

Cloud Computing With AWS
An Overview

Tim Bixler
Federal Solutions Architecture Manager & Principal Solutions Architect
Worldwide Public Sector
October 11, 2012

How did Amazon…

?

…get into cloud computing?

Consumer Business Seller IT Infrastructure
Business Business

Tens of millions of Sell on Amazon Cloud computing
active customer websites infrastructure for
accounts hosting web-scale
Use Amazon
solutions
technology for your
own retail website
Eight countries: Hundreds of
US, UK, Germany, Leverage Amazon’s thousands of
Japan, France, Canada, massive fulfillment registered customers
China, Italy center network in over 190 countries

Over 10 years in the making

Enablement of sellers on Amazon

Internal need for scalable deployment environment

Early forays proved developers were hungry for more

AWS Mission

Enable businesses and developers to
use web services* to build scalable,
sophisticated applications.

*What people now call “the cloud”

Utility computing

On demand Pay as you go

Uniform Available

Utility computing

On demand Pay as you go
Compute
Scaling
Security
CDN Backup
DNS
Database
Storage Load Balancing
Workflow Monitoring
Networking
Uniform Messaging Available

No Up-Front Capital Expense

On-Premise
Up-Front On-Premise Costs VariableCloud Computing Costs
Cloud Computing

Physical Space
Cabling
Power
Cooling
Networking
Racks
Servers
$0
to Get Started
Storage no long-term contracts
Certification
Labor

Elastic capacity Traditional IT
capacity

Capacity

Time
Your IT needs

Elastic capacity

On and Off Fast Growth

Variable peaks Predictable peaks

Elastic capacity
WASTE

On and Off Fast Growth

Variable peaks Predictable peaks

CUSTOMER DISSATISFACTION

40 servers to 5000 in 3 days
EC2 scaled to peak of 5000
instances

Number of EC2 Instances

“Techcrunched”

Launch of Facebook
modification

Steady state of ~40
instances

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 Day 8 Day 9

Each day AWS adds the equivalent server capacity
to power Amazon when it was a global, $2.76B
enterprise

(circa 2000)

Relational Database Service
Virtual Private Cloud Simple Notification Service
Elastic Map Reduce Route 53
Auto Scaling RDS Multi-AZ
Reserved Instances 2009 Singapore Region
Elastic Load Balancer 2010 Identity Access Management

48 61
Cluster Instances

Elastic Beanstalk
Simple Email Service
CloudFormation
2008 RDS for Oracle
ElastiCache

24
SimpleDB
CloudFront 2012 2011
EBS
Availability Zones
Elastic IPs 63 82
2007
Amazon FPS
9 Red Hat EC2 DynamoDB
Simple Workflow
CloudSearch
Storage Gateway
Route 53 Latency Based Routing

number of released features, sample services described

Large Partner Ecosystem

System Integrators Independent Software Vendors

AWS is Open and Flexible

Operating Systems Languages & Libraries Certified Applications

AWS Platform

Your Applications
Management & Administration
Identity & Access Deployment & Automation
AWS IAM Web Interface Monitoring AWS Elastic Beanstalk
Identity Federation Management Console Amazon CloudWatch
AWS CloudFormation
Consolidated Billing

Application Platform Services
Application Svcs Libraries & SDKs
Content Distribution Simple Workflow Service Parallel Processing
Java, PHP, Python,
Amazon CloudFront CloudSearch Elastic MapReduce
Ruby, .NET
Amazon SNS, SQS, SES

Foundation Services
Storage Database Networking
Compute Amazon RDS Amazon VPC
Amazon S3
Amazon EC2 Amazon SimpleDB Elastic Load Balancing
Amazon EBS
Auto Scale Amazon ElastiCache Amazon Route 53
Amazon StorageGateway
Amazon DynamoDB AWS Direct Connect

Availability Zones
AWS Global Infrastructure Edge Locations
Regions

AWS Global Infrastructure
GovCloud US West US West US East South EU Asia Asia
(US ITAR (Northern (Oregon) (Northern America (Ireland) Pacific Pacific
Region) California) Virginia) (Sao Paulo) (Singapore) (Tokyo)

AWS Regions (8)
AWS Edge Locations (33)

AWS Regions & Availability Zones

Customer Decides Where Applications and Data Reside
Note: Conceptual drawing only. The number of Availability Zones may vary.

Built to Enterprise & Gov Standards

Physical Certifications and Accreditations
• ISO 27001
• Datacenters in nondescript facilities
• SSAE 16 / ISAE 3402 / SOC1 (formerly U.S.
• Physical access strictly controlled standard SAS-70 Type II)
• FISMA Moderate & DIACAP Controls; ITAR region
• Must pass two-factor authentication at least
twice for floor access • HIPAA applications certified on AWS
• Payment Card Industry (PCI) Data Security
• Physical access logged and audited
Standard (DSS) Level 1

Hardware, Software & Network Security & Compliance Resources
• Systematic change management • Security & Compliance Center:
• Phased updates deployment http://aws.amazon.com/security

• Safe storage decommission • Security Overview & Best Practices
• Automated monitoring and self-audit
• AWS Risk & Compliance Whitepaper
• Advanced network protection systems
• Creating HIPAA Compliant Applications

Foundation Services

Your Applications
AWS CloudFormation

Java, PHP, Python,
Ruby, .NET

Foundation Services
Amazon S3
Amazon EBS

Availability Zones
Regions

Compute

Virtual Servers

Auto Scaling

Compute

Amazon Elastic Compute Cloud (Amazon EC2)
EC2 Instances = Virtual Servers

• Resizable compute capacity in 14 instance types
• Reduces the time required to obtain and boot new server instances to minutes or seconds
• Scale capacity as your computing requirements change
• Pay only for capacity that you actually use
• Choose Linux or Windows
• Deploy across Regions and Availability Zones for reliability
• Flexible networking (NAT/classic, VPC, Elastic IPs)
• Support for virtual network interfaces that can be attached to EC2 instances in your VPC

Compute
Hi-Mem 4XL 68.4 GB High I/O 4XL 60.5 GB
26 EC2 Compute Units 35 EC2 Compute Units

128 Amazon Elastic Compute Cloud (Amazon EC2) 8 virtual cores
$1.80/2.28
$3.10/3.58

Cluster Compute 8XL 60.5 GB

64
Hi-Mem 2XL 34.2 GB 88 EC2 Compute Units
13 EC2 Compute Units $2.40/2.97
4 virtual cores
$0.90/1.14

Cluster Compute 4XL 23 GB

32 33.5 EC2 Compute Units
$1.30/1.61
Memory (GB)

Hi-Mem XL 17.1 GB
Cluster GPU 4XL 22 GB
6.5 EC2 Compute Units
33.5 EC2 Compute Units,

16
2 virtual cores
2 x NVIDIA Tesla “Fermi” M2050
$0.45/0.57
Extra Large 15 GB GPUs
8 EC2 Compute Units $2.10/2.60
4 virtual cores
Medium 3.75 GB $0.64/0.92
2 EC2 Compute Units

8 1 virtual cores
$0.16/0.23
Large 7.5 GB
4 EC2 Compute Units
2 virtual cores
$0.32/0.46
Small 1.7 GB, 32-Bit High-CPU XL 7 GB

4 1 EC2 Compute Unit
1 virtual core
$0.08/0.115 High-CPU Med 1.7 GB
5 EC2 Compute Units
20 EC2 Compute Units
8 virtual cores
$0.66/1.14

2 virtual cores

2
$0.165/0.285

Micro 613 MB
Up to 2 ECUs (for short
bursts)
$0.02/0.03

1
1 2 4 8 16 32 64 128
EC2 Compute Units (HP)

Compute

Auto Scaling
• Client Defined Business Rules
• Scale your Amazon EC2 capacity automatically once you define the conditions (may be 1000’s of
servers)
• Can scale up just a little…doesn’t need to be massive number of servers (may be simply 2 servers)
• Well suited for applications that experience variability in usage
• Set minimum and maximum scaling policies
• Alternate Use is for Fault Tolerance

Storage

S3
Import/Export

EBS

Storage Gateway

Glacier
so new we don’t have an icon!

Storage

Simple Storage Service (S3)
Web-scale Internet Storage
• A “Bucket” is equivalent to a “folder”
• Able to store unlimited number of Objects in a Bucket
• Objects from 1B-5 TB; no bucket size limit
• Highly available storage for the Internet (object store)
• HTTP/S endpoint to store and retrieve any amount of data, at any time, from anywhere on the web
• Highly scalable, reliable, fast, and inexpensive
• Over 1 trillion objects stored
• Peak requests 750,000+ per second
• Ideal Use Cases:
• Static web content – often used with CloudFront CDN
• Source and output storage for large-scale “Big Data” analytics
• Backup, archival, and DR storage that is always “live”

Objects in S3
1 Trillion
1000.000

750.000

500.000

250.000

0.000

750,000+ peak transactions per second

Storage

Elastic Block Store (EBS)
EBS Volumes = Virtual Disks
• Use for persistent storage
• Can use to create RAID configuration for a server
• Off-instance block storage that persists independently
• Storage volumes for use with Amazon EC2 instances – create, attach, backup, restore and delete
• Can be attached to a running Amazon EC2 instance and exposed as a block device for raw or
formatted (filesystem) access
• Volumes behave like unformatted block devices for Linux or Windows instances
• Ideas use cases:
• OS Boot device / root file system; secondary volumes/filesystems
• Typical basis for database storage
• Raw block devices for RAID, some databases

Storage

AWS Glacier
• A low-cost storage service for data archiving and backup
• $0.01 per GB / Month
• Optimized for data that is infrequently accessed
• Retrieval times measured in hours not days or weeks
• Annual durability of 99.999999999% for an archive
• AES 256 data at rest encryption
• Data stored as archives within a vault. Vaults are located within a specific AWS region
• Archives can be up-to 40 TB in size

Storage

AWS Import/Export
• Accelerates moving large amounts of data into and out of S3 or EBS
• Transfers your data directly onto and off of USB or SATA storage devices shipped to AWS with
manifest file
• Final copy uses high-speed datacenter network

Storage

AWS Storage Gateway
• Storage gateway service connects an on-premise software appliance with cloud-based storage
• On-premises software appliance solution to store data on Amazon S3’s storage infrastructure
• Exposes standard iSCSI interface to on-premises applications, while maintaining low-latency data
access
• Data in Amazon S3 stored as Amazon EBS snapshots for local & EC2-based recovery
• Use Cases
• Backup/Restore on-premise data
• Set up a test/dev environment with production data
• Migrating applications to the cloud
• On-premise DR/COOP to AWS

Database

DynamoDB ElastiCache

RDS RDS SimpleDB

Database

DynamoDB
• Fully managed NoSQL database.
• Eliminates the administrative burden of data modeling, index maintenance, and performance
tuning.
• Durability and high-availability - stores data on Solid State Drives (SSDs) and replicates it
synchronously across multiple AWS Availability Zones in an AWS Region.
• Scalability - With AWS Console, you can grow your DynamoDB table from 10 to 100,000 writes per
sec.
• See video: http://www.youtube.com/watch?v=oz-7wJJ9HZ0

Database

Amazon Relational Database Service (RDS)
RDS • Fully-managed, tuned MySQL, Oracle 11g, or MS SQL databases
• Cost-efficient and resizable capacity
• Manages time-consuming database admin tasks
• Code, applications, and tools you already use today work seamlessly
• Automatically patches the database software and backs up your database
• Flexible Licensing: BYOL or License Include

Database

Amazon ElastiCache
• Fully-managed, distributed, in-memory cache
• Memcached compliant cache cluster on-demand
• Manages patching, cache node failure detection and recovery
• Simple APIs calls to grow and shrink the cache cluster
• Seamlessly caches in front of SimpleDB or RDS instances
• Integrated with CloudWatch and SNS for monitoring and alerts

Database

Amazon SimpleDB
• Core database functions of data indexing and querying of text data
• No schema, automatic indexing
• Eliminates the administrative burden of data modeling, index maintenance, and performance tuning
• Real-time lookup and simple querying of structured data
• Use cases:
• Metadata storage -- often used in conjunction with S3
• Structured, fine-grained data needing query
• Data needing flexible schema

Networking

ELB Route 53 VPC

Networking

Amazon Elastic Load Balancing
• Supports the routing and load balancing of HTTP, HTTPS and generic TCP traffic to EC2 instances
• Supports health checks to ensure detect and remove failing instances
• Dynamically grows and shrinks required resources based on traffic
• Seamlessly integrates with Auto-scaling to add and remove instances based on scaling activities
• Single CNAME provides stable entry point for DNS configuration

Networking

Amazon Route 53
• Route end users to Internet applications
• Answers DNS queries with low latency by using a global network of DNS servers
• Latency based routing to closest AWS endpoint (e.g. EC2 instances, Elastic IPs or ELBs)
• Deep integration with other AWS services (ELB,
EC2 NAT/EIP, etc.)

Networking

Amazon Virtual Private Cloud (VPC)
• Secure and seamless bridge between a company’s existing private network and the AWS cloud
• Connect existing infrastructure to a set of isolated AWS compute resources via a Virtual Private
Network (VPN) connection
• Bring your own address space and extend existing management capabilities


Your Applications
AWS CloudFormation

Java, PHP, Python,
Ruby, .NET

Foundation Services
Amazon S3
Amazon EBS

Availability Zones
Regions

Content Delivery

Amazon CloudFront
• Web service for content delivery
• Distribute content to end users with low latency, high data transfer speeds, and no commitments
• Delivers your content using a global network of 33 edge locations
• Supports download, streaming, live streaming, and dynamic content
• Key features: RTMP Streaming, HTTPS Delivery, Private Content for HTTP &
Streaming, Programmatic Invalidation, Detailed Logs for HTTP & Streaming, Default Root
Object
• Use Cases: Video and Rich Media, Online Gaming, Interactive Agencies, Software Downloads, Static
Websites
• Static web content that must be delivered to global user base at Highest bandwidth /
Lowest latency / Lowest cost

Application Services

SNS
SWF

SQS
CloudSearch
SES


Amazon Simple Notification Service (SNS)
• Set up, operate, and send notifications
• Publish messages from an application and immediately deliver them to subscribers or other
applications


Amazon Simple Queue Service (SQS)
• Hosted queue for storing messages as they travel between computers
• Move data between distributed components of their applications


Amazon Simple Email Service (SES, beta)
• Bulk and transactional email-sending service
• Eliminates the hassle of email server management, network configuration, and meeting rigorous
Internet Service Provider (ISP) standards
• Provides a built-in feedback loop, which includes notifications of bounce backs, failed and successful
delivery attempts, and spam complaints


Amazon Simple Workflow Service (SWF)
• Easily manage workflows, including state, decisions, executions, tasks and logging
• Coordinate processing steps across distributed systems
• Ensure tasks are executed reliably, in order, and without duplication
• Simple API calls that can be executed from code written in any language and run on your EC2
instances, or any of your machines located anywhere in the world that can access the Internet


Amazon CloudSearch (beta)
• Fully-managed search service
• Integrate fast and highly scalable search functionality into applications
• Scales automatically: with increases in searchable data or as query rate changes
• AWS manages hardware provisioning, data partitioning, and software patches

Parallel Processing

Amazon Elastic MapReduce (EMR)
• Managed Hadoop 0.20.205 infrastructure
• Reduces complexity of Hadoop management
• Handles node provisioning, customization, and shutdown
• Tunes Hadoop to your hardware and network
• Provides tools to debug and monitor your Hadoop clusters
• Provides tight integration with AWS services
• Optimized for Amazon Simple Storage Service (S3)
• EC2 integration with automatic re-provisioning on node failure
• Cluster monitoring/alarming through CloudWatch
• Leverages significant operational experience
• Monitor thousands of clusters per day
• Use cases span from University students to Fortune 50

Libraries & SDKs

• Your choice of programming language (Java, PHP, Python, Ruby, .NET) and mobile platform
(Android, iOS)

• The Developer Centers contains sample code, documentation, tools, and additional resources to
help you build applications on Amazon Web Services.
• http://aws.amazon.com/java/
• http://aws.amazon.com/mobile/
• http://aws.amazon.com/php/
• http://aws.amazon.com/python/
• http://aws.amazon.com/ruby/
• http://aws.amazon.com/net/


Your Applications
AWS CloudFormation

Java, PHP, Python,
Ruby, .NET

Foundation Services
Amazon S3
Amazon EBS

Availability Zones
Regions

Web Console

On-demand, Self Service
Management Access

Identity & Access Management

• IAM enables customers to create and manage users in AWS’s identity
system
• Identity Federation with local directory is an option for
enterprises
• Very familiar security model
• Users, groups, permissions
• Allows customers to
• Create users
• Assign individual passwords, access keys, multi-factor
authentication devices
• Grant fine-grained permissions
• Optionally grant them access to the AWS Console
• Organize users in groups

Consolidated Billing with IAM

• Allows you to get one bill for multiple accounts
• You can easily track each account's costs and download the cost data in
CSV format
• You may be able to reduce costs by combining usage from all the
accounts to qualify for volume pricing discounts

Deployment and Management

AWS Elastic Beanstalk (beta)
• Simply upload your application (Java, NET, and PHP)
• Automatically handles the deployment details of capacity provisioning, load balancing, auto-
scaling, and application health monitoring
• Retain full control over the AWS resources powering your application


AWS CloudFormation
• Create templates of stack of resources
• Deploy stack from template with runtime parameters
• Templates are simple JSON formatted text files
• CloudFormer supports generating templates from running environments

"Resources" : {
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
"Tags" : [{
"Key" : "MyTag",
"Value" : "TagValue"
}]
}
},


Amazon CloudWatch
• Visibility into resource utilization, operational performance, and overall demand patterns
• Metrics such as CPU utilization, disk reads and writes, and network traffic
• Accessible via the AWS Management Console, web service APIs or Command Line Tools
• Add custom metrics of your own
• Alarms (which tie into auto-scaling, SNS, SQS, etc.)
• Billing Alerts to help manage charges on AWS bill

Your Applications

Your Applications
AWS CloudFormation

Java, PHP, Python,
Ruby, .NET

Foundation Services
Amazon S3
Amazon EBS

Availability Zones
Regions

A scalable compute platform

• Researchers and scientists want:
– A platform that can scale
– Offers choice at run time
– Can be automated to run complex workflows
– Don’t want to be bothered about the muck of
managing infrastructure
• AWS provides Just-in-Time infrastructure

So you can go from one instance…

…and choose the EC2 instance type…

Hi-Mem 4XL 68.4 GB
26 ECUs
8 virtual cores Cluster Compute 8XL 60.5 GB
88 ECUs
Hi-Mem 2XL 34.2 GB 8 core 2 x Intel Xeon
13 ECUs
4 virtual cores
33.5 ECUs
Hi-Mem XL 17.1 GB 8 Nehalem virtual cores
6.5 ECUs
2 virtual cores
Extra Large 15 GB Cluster GPU 4XL 22 GB
8 ECUs 33.5 ECUs
4 virtual cores 8 Nehalem virtual cores
2 x NVIDIA Tesla
“Fermi” M2050 GPUs
Large 7.5 GB
4 ECUs High-CPU XL 7 GB
2 virtual cores 20 ECUs
8 virtual cores
Small 1.7 GB, Medium 3.75 GB
1 ECU 2 ECUs
1 virtual core 1 virtual cores
High-CPU Med 1.7 GB
Micro 613 MB
5 ECUs
Up to 2 ECUs (for
2 virtual cores
short bursts)

Parallel
…and choose the EC2 instance type… workloads, high
performance
Memory intensive computing
applications, in- Hi-Mem 4XL 68.4 GB
26 ECUs
memory computations 8 virtual cores Cluster Compute 8XL 60.5 GB
88 ECUs
Hi-Mem 2XL 34.2 GB 8 core 2 x Intel Xeon
13 ECUs
4 virtual cores
33.5 ECUs
Hi-Mem XL 17.1 GB 8 Nehalem virtual cores
6.5 ECUs
2 virtual cores
Cluster GPU 4XL 22 GB
Average applications Extra Large 15 GB
8 ECUs 33.5 ECUs
4 virtual cores 8 Nehalem virtual cores
and workloads 2 x NVIDIA Tesla
“Fermi” M2050 GPUs
Large 7.5 GB
4 ECUs High-CPU XL 7 GB
2 virtual cores 20 ECUs
8 virtual cores
Small 1.7 GB, Medium 3.75 GB
1 ECU
1 virtual core
2 ECUs
1 virtual cores
Processor intensive
High-CPU Med 1.7 GB
Micro 613 MB
Up to 2 ECUs (for
5 ECUs workloads, encoding,
2 virtual cores
short bursts)
modeling
Low resource
requirement
applications

GPUs for Molecular Dynamics

GPU compute instances
Intel® Xeon® X5570 processors
2 x NVIDIA Tesla “Fermi” M2050 GPUs
I/O Performance: Very High (10 Gigabit Ethernet)

33.5 EC2
Compute Units

20GB RAM
2x NVIDIA GPU
@ >400 Cores
Each
Cluster GPU

CC2 Instance Cluster

240 TFLOPS
Making it the 72nd fastest
supercomputer in the world

Yours for $2554/hr – on demand

A cluster that you can automate, control, auto-scale…
CLI, API and Console
Scripted configurations as-create-auto-scaling-group MyGroup
--launch-configuration MyConfig
--availability-zones eu-west-1a
--min-size 2
--max-size 200

ec2-run-instances ami-b232d0db
--instance-count 3
--availability-zone eu-west-1a
--instance-type m1.small

…and coordinate workloads and task clusters in
Handle long running processes across many nodes and task steps
with Simple Workflow

1

2
Task A

3
Task B
(Auto-
scaling)
Task C

Optimize cost with spot prices

Bid on unused Amazon EC2 capacity

Harvard Medical School
The Laboratory of Personal Medicine
Run EC2 clusters to analyze entire genomes

“The AWS solution is stable, robust, flexible, and low cost. It
has everything to recommend it.”
Dr. Peter Tonellato, LPM, Center for Biomedical Informatics, Harvard Medical School

Leverage Spot instances in workflows
1 days worth of effort
resulted in
50% savings in cost

Computation drug analysis

Estimated computation time 12.55 years

51,132 Core AWS Cluster
Completed in 3 hours
Costing $4828.85 / hr

Data Ingestion

• AWS Import/Export
AWS
– Move large amounts of data into and outside AWS
Import/Export – Data Migration, Content Distribution, DR, etc.
• AWS Direct Connect
AWS Direct – Secure private link to AWS
Connect
– 1Gbps, 10Gbps connectivity
– You can also co-locate hardware in AWS DX locations
• Bandwidth Optimization Solutions
– Commercial providers – Aspera, Riverbed, Attunity, etc.
– Open Source – Tsunami UDP, Globus Online

Data Collection

Fully managed SQL, NoSQL and object storage

Relational Database Service DynamoDB S3
Fully managed database NoSQL, Schemaless, Prov Object datastore up to 5TB
(MySQL, Oracle, MSSQL) isioned throughput per object
database 99.999999999% durability

Data Archival
• Announcing Amazon Glacier
– Meet your regulatory requirements
– Long term archival
– 11 9’s of durability as S3 standard
– All data encrypted using Server Side Encryption
– Starting at $0.01/GB/month
“Every day our genome sequencers produce terabytes of data. As our company
moves into the clinical space, we face a legal requirement to archive patient data
for years that would drastically raise the cost of storage. Thanks to Amazon
Glacier’s secure and scalable solution, we will be able to provide cost-
effective, long-term storage and thereby eliminate a barrier to providing whole
genome sequencing for medical treatment of cancer and other genetic diseases.”
- Keith Raffel, Senior Vice President and Chief Commercial Officer, Complete
Genomics

Share your data
• Share Amazon Machine Images (AMIs)
– Share installations of your software packages and tools with collaborators
so that they can duplicate your set up using EBS snapshots
– Collaborate by sharing your images with partners and customers
• Share architecture templates
– Share the collection of resources required to run your pipeline with
collaborators by using CloudFormation templates
• Share data
– Decouple your compute from data and share storage buckets with
collaborators
– Create Requester Pays buckets so the charges associated with accessing
data are paid by the requesters

AWS Public Data Sets
• A centralized repository of public datasets
• Seamless integration with cloud based applications
• No charge to the community
• Some of the datasets available today:
– 1000 Genomes Project
– Ensembl
– GenBank
– Illumina – Jay Flateley Human Genome Dataset
– YRI Trio Dataset
– The Cannabis Sativa Genome
– UniGene
– Influenza Virrus
– PubChem
• Tell us what else you’d like for us to host …

Putting Foundation Services
Together

deployment model:
availability

1. Use multiple
availability zones

2. Use RDS with replicas
and standby

4. Use Elastic Load
Balancing

5. Use Route53 to host
DNS zones

Three Services: Better Together

CloudWatch
Auto Scaling
Elastic Load Balancer

Architect to use cloud strengths

Elastic Load Balancing Route 53 RDS Auto-scaling

Use at regional level Leverage SLA Scale databases without Dynamically scale resources &
Combined with autoscaling will Improve application reliability with admin overhead control costs
balance requests and resource Route 53’s SLA on requests served Choose instance size for databases Only provision the resources that
capacity across availability zones and scale up over time are required with scale up and cool
Weighted routing down policies that match demand
Within VPC Perform A/B analysis, and staged Add high availability from
Use to loadbalance between application roll-outs by moving a management console
application tiers within an portion of traffic to new Create master-slave configurations
availability zone infrastructure and read-replicas. AWS takes care of
the failover and recreation of a new
Instance migrations Control TTLs and updates slave in event of master DB loss
Easily move instances from dev Take absolute control of DNS
environments to test environments updates for more decisive system
by moving between ELBs updates

Services not software

Use AWS services + Your technology skills

=
Less time managing and installing software
More time focused on mission applications

let AWS do the heavy lifting


Relational Database Service
Use RDS for databases Database-as-a-Service
No need to install or manage database instances
Scalable and fault tolerant configurations

DynamoDB Use DynamoDB for
Provisioned throughput NoSQL database high performance key-
Fast, predictable performance
value DB
Fully distributed, fault tolerant architecture

Processing results
Amazon SQS Reliable message
Reliable, highly scalable, queue service
Amazon SQS
queuing without
for storing messages as they travel
between instances
additional software

Processing
task/processing
trigger 1

2

Push inter-process Simple Workflow Task A

workflows into the Reliably coordinate processing steps
Task B 3
across applications
cloud with SWF (Auto-scaling)

Integrate AWS and non-AWS resources
Manage distributed state in complex
systems Task C

Document
Cloud Search Server
Don’t install search
Elastic search engine based upon
software, use
Amazon A9 search engine
CloudSearch Fully managed service with sophisticated
feature set
Search
Scales automatically
Server

Results

Elastic MapReduce
Elastic Hadoop cluster
Process large volumes
Integrates with S3 & DynamoDB of data cost effectively
Leverage Hive & Pig analytics scripts with EMR
Integrates with instance types such as
spot

NASA - Mission Data Processing
Challenge
Because of the latency of data transmission from and to
Mars, during a 2 hour window, it took mission planners 90 minutes
to process telemetry data from the Mars Rover, 20 mins to decide
where to move the Rover to, and 10 mins to up load the data.

Solution
NASA-JPL, loading their custom software application on EC2, was
able to horizontally scale the number of virtual machines
supporting the data processing.

Benefit
Reduced data processing time from 90 minutes to 15 minutes using
parallel processing
Increased mission planning time, resulting in high quality scientific
observations

NASA - Mission Data Processing

Daily Mars Rover Data Processing Window
Pre-cloud:
Process Plan Upload

Cloud:
Process Plan Upload

Increase available mission planning time from
15 minutes to 105 minutes!

“We were able to reduce our DNS
costs by ninety-three
percent, which in tandem allowed
us to shorten our time-to-live
(TTLs) for easier, timelier
management of DNS records.”

Nathan Butler
The Newsweek/Daily Beast Company

RECOVERY.GOV – Website/App Hosting
Challenge
Recovery and Transparency Board needed a platform for their
website that was scalable, secure, could be quickly deployed, and
saved tax payer money

Solution “By migrating to the public cloud, the Recovery
Board is in position to leverage many advantages
RATB chose a FISMA-compliant cloud computing solution based on
including the ability keep the site up as millions of
Amazon Web Services Americans help report potential fraud, waste, and
deployed applications: abuse. The Board expects savings of about
Microsoft SharePoint for web Content Management $750,000 during its current budget cycle and
Business Objection SAP for BI significantly more savings in the long-term.”

Benefit - Vivek Kundra, CIO, United States
• Avoided Capital expense, and added capacity to scale up and
down based on demand
• Saved $750k per year in first year and additional dollars from
existing solution

Security and Information:
A Reprise

Shared responsibility

Customer Data
Customer

Platform, Applications, Identity & Access Management

Operating System, Network & Firewall Configuration

Client-side Data Encryption & Data Server-side Encryption Network Traffic Protection
Integrity Authentication (File System and/or Data) (Encryption/Integrity/Identity)

Foundation Services • SAS-70 Type II
• ISO 27001/ 2 Certification
Amazon

Compute Storage Database Networking • Payment Card Industry (PCI)
Data Security Standard (DSS)
• FISMA Compliant Controls
Availability Zones • DIACAP Compliant Controls
Edge Locations • FedRAMP Compliant Controls
AWS Global
Regions • HIPAA and ITAR Compliant
Infrastructure

Shared responsibility

AWS Customer
• Facilities • Operating System
• Physical Security • Application
• Physical Infrastructure • Security Groups
• Network Infrastructure • Network ACLs
• Virtualization • Network Configuration
Infrastructure • Account Management

Examples of Customer Responsibilities

• Apply Your Information Management Program - • Use Encryption – for data in transit, for data at
that integrates Information Assurance rest, filesystem
• Standardize Machine Images – create gold copy • Key Management – rotate keys used to access
images for production deployment/to launch new your resources (AWS does not hold these…you do)
instances • Setup Monitoring/Alerting – collect metrics and
• Build and test in a sandbox environment – work enable alerting for when events occur
out the bugs, figure out how to break it, architect • Vulnerability Scans – allowed via a permission
to be resilient process (else we’ll kill/block the source of scans)
• Do the same stuff you do in-house – quarterly • Prepare for Failure – create backups, store data
patch in more than one location, test backups, have a
management, IDS/IPS, logging, tripwire, etc. contingency system ready
• Conduct a Risk Assessment - to determine level of
security controls you require
• Role Based Access Controls – restrict access to
system components based upon need to know

Build upon AWS features

Dedicated Instances Security Groups VPC Direct Connect & VPN

Single Tenant Instance firewalls Subnet control Private connections to VPC
Physical Nodes Firewall control on instances via Create low level networking Secured access to resources in AWS
Run your virtualized operating Security Groups constraints for resource access, such over software or hardware VPN and
systems and apps in a “single tenant as public and private dedicated network links
per physical node” model within the CLIs and APIs subnets, internet gateways and
AWS infrastructure NATs
Instantly audit your entire AWS
infrastructure from scriptable APIs –
generate an on-demand IT inventory
Bastion hosts
enabled by programmatic nature of Only allow access for management
AWS of production resources from a
bastion host. Turn off when not
needed

AWS Multi-Factor Authentication

Groups Account Roles
• Helps prevent access based on unauthorized
knowledge of your e-mail address and password
• Additional protection for account information Administrators Developers Applications
• Works with master account and IAM users
• Integrated into Jim Brad Reporting
• AWS Management Console
• Key pages on the AWS Portal Bob Mark Console
• S3 (Secure Delete)
Susan Tomcat
• Virtual MFA (using OATH standard)
Kevin
Multi-factor authentication AWS system entitlements

Account Management/Isolation

Payor Account
Linked Account
Linked Account Linked Account Linked Account
Reseller Internal
Customer 1 Customer 2 Customer 3 Use
Identity & Access Management

End User 1 End User 1 End User Group Reseller User 1

End User 2 End User 2 End User 1 Reseller User 2

End User 3 End User 3 End User 2 Reseller User 3

End User 4 End User 3 Reseller User 4

End User 5 End User 4

AWS GovCloud – Who can Use?

• US Government/State/Local Clients & organizations conducting
work on their behalf
• AWS will screen customers prior to providing access to the AWS
GovCloud (US). Customers must be:
• U.S. Persons
• Not subject to export restrictions
• Agree to comply with U.S. export control laws and
regulations, including the International Traffic In Arms
Regulations

Useful Resources & Links

• Architecture Center: http://aws.amazon.com/architecture
• Security Center: http://aws.amazon.com/security
• Whitepapers: http://aws.amazon.com/whitepapers
• Resources: http://aws.amazon.com/resources
• Case Studies: http://aws.amazon.com/solutions/case-studies
• Solution Providers: http://aws.amazon.com/solutions/global-solution-providers/

Cloud computing with AWS

More Related Content

What's hot

Viewers also liked

Similar to Cloud computing with AWS

More from ikanow

Cloud computing with AWS

Editor's Notes