SlideShare a Scribd company logo
Scientific Computing with 
Amazon Web Services 
Jamie Kinney 
Director of Scientific Computing Amazon Web Services 
jkinney@amazon.com 
@jamiekinney
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
Amazon Global Impact Initiatives 
Scientific Computing 
- Global “Big Science” Projects 
- Enabling the “long tail of science” 
- Collaborative research 
- Accelerating the transition to “Networked Science” 
- Initial focus on Life Sciences, Earth Sciences, Astronomy/Astrophysics and High 
Energy Physics 
! 
Open Data 
- AWS as the platform for open-access data portals and archives 
- Amazon Public Data Sets 
- Public/Private Data partnerships 
! 
Economic Development 
- The Amazon Job Accelerators program
Why are we focusing on the Scientific Community? 
•In order to meaningfully change our world for the better by accelerating 
the pace of scientific discovery 
! 
•Scientific computing is a profitable business for AWS 
! 
•To develop new capabilities which will benefit all AWS customers 
- Streaming data processing & analytics 
- Exabyte scale data management solutions 
- Collaborative research tools and techniques 
- New AWS regions 
- Significant advances in low-power compute, storage and data centers 
- Identify efficiencies which will lower our costs and pricing for customers 
- Push our existing services to support exabyte/exaflop scale workloads
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
Self Hosting 
Waste 
Actual demand 
Customer 
Dissatisfaction 
Predicted 
Demand 
Rigid 
Actual 
demand 
Elastic
2003 
$5.2B retail business 
7,800 employees 
A whole lot of servers
2003 2013 
$5.2B retail business 
7,800 employees 
A whole lot of servers
2003 2013 
$5.2B retail business 
7,800 employees 
A whole lot of servers
2003 2013 
$5.2B retail business 
7,800 employees 
A whole lot of servers 
Every day, AWS adds enough 
server capacity to power this 
$5B enterprise
Said another way… 
AWS is deploying the equivalent of a 
top-20 supercomputer every few days
Support Professional Services Training Certification 
Technology Partners Consulting Partners Ecosystem AWS Marketplace 
Elastic Beanstalk for Java, Node.js, 
Python, Ruby, PHP and .Net OpsWorks CloudFormation Containers & Deployment 
(PaaS) 
Management & 
IAM CloudTrail Cloud HSM CloudWatch Administration Management Console APIs and SDKs Command Line Interface 
Analytics 
Application Services 
EMR Redshift Kinesis Data Pipeline CloudFront SNS SQS SES SWF WorkSpaces 
AppStream CloudSearch 
Networking VPC Direct Connect Route 53 
Compute Storage Databases 
MySQL, PostgreSQL 
Oracle, SQL Server 
S3 EBS Glacier Storage Gateway Import/Export RDS 
DynamoDB ElastiCache 
EC2 Elastic Load Balancer Auto Scaling 
Regions Availability Zones Content Delivery POPs
Broad Features and Functionality for Each Service 
IAM 
Redshift 
Elastic Map 
Reduce 
Data Pipeline 
CloudFront Route 53 
CloudFormation 
Elastic Load 
EC2 Balancer 
EBS 
S3 
DynamoDB 
AppStream 
Kinesis
Broad Features and Functionality for Each Service 
56 new features 
since Feb 2013 
Regional expansion to US West (Oregon) 
Support for temporary credentials when loading data from Amazon S3 
Regional expansion to EU West (Dublin) 
SOC1/2/3 Compliance certification 
Ability to UNLOAD encrypted files in parallel to Amazon S3 
Regional expansion to Asia Pacific (Tokyo) 
Support for JDBC fetch size to enable extraction of large data sets over JDBC/ODBC 
Enable logging of UNLOAD statements 
New built-in function to compute the SHA1 hash of a value 
Added support for UTF-8 characters up to 4 bytes in size 
Ability to share snapshots between accounts to simplify manageability. 
Support for statement timeouts to automatically terminate queries that exceeded allotted execution time 
Added support for timezone conversion in SQL 
Added support for datetime values expressed in milliseconds since EPOCH to simplify ingestion 
Simplified ingestion by automatically detecting date and time formats. 
Added support for automatic query timeouts to workload management queues. 
Enabled the use of wildcards when assigning queries to workload management queues. 
New built-in function to enable customers to calculate the CRC32 checksum of a value 
Console improvements to show progress bars for backup and restore operations. 
Added the ability to support IAM at the resource level allowing tight control of who can take what actions on which resources. 
Obtained PCI compliance 
Added the ability to substitute a customer chosen character for invalid UTF-8 characters to simplify ingestion 
Allowed customers to store JSON data in VARCHAR columns and added built-in functions to enable data extraction 
Added support for POSIX regex expressions when using SIMILAR to in SQL queries 
Added Cursor support to enable extraction of large data sets over ODBC connections 
Built-in function to enable splitting a string using a supplied delimiter to make parsing values easier 
Added system tables to enable logging of database activity for auditing 
Regional expansion to Asia Pacific (Singapore, Sydney) 
Enable customers to control cluster encryption keys by using an on premises hardware security module (HSM) or Amazon CloudHSM 
Enable customers to receive alerts via SNS for informational or error-related events for cluster monitoring, management, configuration and 
security. 
Integration with Canal to enable streaming data ingestion 
Copy from an arbitrary SSH connection enabling direct copy from Amazon EMR, HDFS, or any other database that supports SSH access and 
script execution 
Enable distributing tables to all compute nodes to speed up queries, especially those involving star or snowflake schemas 
Logging of database logins, failed logins, SQL execution and data loads to S3 and integration with CloudTrails for control plane events 
Enabled caching of database blocks to speed up access to frequently queried data 
Increase cluster concurrency limits from 15 to 50 to enable higher concurrent query execution 
Optimizations to resize code that lead to 2-4x improvement in resize performance 
Approximate COUNT DISTINCT using HyperLogLog giving 10-20x performance improvements with less than 1% error 
Enable customers to continuously, automatically and incrementally back up data to a second AWS region for DR 
On track to obtain Fedramp certification 
Deliver Redshift on SSD instances enabling a lower-cost, high performance entry point 
IAM 
Redshift 
Elastic Map 
Reduce 
Data Pipeline 
CloudFront Route 53 
CloudFormation 
Elastic Load 
EC2 Balancer 
EBS 
S3 
DynamoDB 
AppStream 
Kinesis
AWS Big Data Technologies 
Amazon RDS Amazon 
DBA$ 
DynamoDB 
Amazon 
Elasticache 
Amazon Elastic 
Map Reduce 
Amazon Data 
Pipeline 
Amazon 
Redshift 
Hosted 
Relational 
Databases 
Managed 
NoSQL 
Database 
In-memory 
Caching Service 
Hosted Hadoop 
framework 
Move data 
among AWS 
services and on-premise 
data 
sources 
Petabyte-scale 
columnar 
relational data 
warehouse 
service 
http://aws.amazon.com/big-data/
The AWS Big Data Stack Also Includes… 
•HTCondor 
•SNAP 
•Spark/Shark 
•BlinkDB 
•Cassandra 
•Galaxy 
•MongoDB 
•Lustre 
•Hadoop 
•Gluster 
•Pegasus 
•Galaxy 
•SLURM 
•BOINC 
•MIT StarCluster 
•… and many more
Amazon S3
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
Amazon S3: Over 2 Trillion Total Objects 
> 2,000,000 requests/second 
1,100,000 Million 
peak requests/sec
Amazon EC2
EC2 Instance Type History: 
Increasing Customer Choice 
m2.2xlarge 
m2.4xlarge 
c1.medium 
c1.xlarge 
m1.xlarge 
m1.large 
m1.small 
! 
2006 2007 2008 2009 2010 2011 2012-­‐2013 April, 
2014 
m1.small 
m1.xlarge 
m1.large 
m1.small 
cc2.8xlarge 
cc1.4xlarge 
cg1.4xlarge 
t1.micro 
m2.xlarge 
m2.2xlarge 
m2.4xlarge 
c1.medium 
c1.xlarge 
m1.xlarge 
m1.large 
m1.small 
cr1.8xlarge 
hs1.8xlarge 
m3.xlarge 
m3.2xlarge 
hi1.4xlarge 
m1.medium 
cc2.8xlarge 
cg1.4xlarge 
t1.micro 
m2.xlarge 
m2.2xlarge 
m2.4xlarge 
c1.medium 
c1.xlarge 
m1.xlarge 
m1.large 
m1.small 
cc1.4xlarge 
cg1.4xlarge 
t1.micro 
m2.xlarge 
m2.2xlarge 
m2.4xlarge 
c1.medium 
c1.xlarge 
m1.xlarge 
m1.large 
m1.small 
hs1.8xlarge 
m3.xlarge 
m3.2xlarge 
hi1.4xlarge 
m1.medium 
cc2.8xlarge 
cg1.4xlarge 
t1.micro 
m2.xlarge 
m2.2xlarge 
m2.4xlarge 
c1.medium 
c1.xlarge 
m1.xlarge 
m1.large 
m1.small 
c1.medium 
c1.xlarge 
m1.xlarge 
m1.large 
m1.small 
existing 
g2.2xlarge 
hs1.xlarge 
hs1.2xlarge 
hs1.4xlarge 
c3.large 
c3.xlarge 
c3.2xlarge 
c3.4xlarge 
c3.8xlarge 
m3.medium 
m3.large 
i2.large 
i2.xlarge 
i2.4xlarge 
i2.8xlarge 
r3.large 
r3.xlarge 
r3.2xlarge 
r3.4xlarge 
r3.8xlarge 
new
Integrated Architectures 
EC2 EC2 
EC2 EC2 
Los Angeles 
Singapore 
Japan 
London 
São Paolo 
New York 
Sydney 
AWS Direct Amazon VPC 
Connect
Customer’s 
Network 
Customer’s 
isolated 
AWS 
resources 
Amazon 
Web 
Services 
Secure 
VPN 
Cloud 
Connection 
over 
the 
Internet 
Subnets 
Router 
VPN 
Gateway 
Internet NAT 
Private 
Private 
Private 
AWS Virtual Private Cloud Networking 
TIC
VPC Deployment Models 
VPC With a 
Single Subnet 
Simple web hosting, 
collaborative research 
environments, on-demand 
HPC/HTC 
clusters 
VPC With Private 
and Public Subnets 
Multi-tier web hosting 
VPC With Private and 
Public Subnets & 
Hardware VPN access 
Multi-tier web hosting 
with access to internal 
infrastructure 
VPC With a 
Private Subnet Only & 
Hardware VPN Access 
Seamless private 
expansion of on-premise 
infrastructure. “Burst 
capacity” or dedicated 
cloud environments with 
connectivity to on-premise 
resources
Who is using AWS?
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
MSL Distributed Operations 
JPL 
Pasadena, CA 
CDSCC 
Canberra Deep Space 
Communication Complex 
MDSCC 
Madrid Deep Space 
Communication Complex 
GDSCC 
Goldstone Deep Space 
Communication Complex 
ARC 
CheMin 
Moffett Field, CA 
MSSS 
MARDI, MAHLI, 
MastCam 
San Diego, CA 
KSC 
IKI 
DAN 
Moscow, Russia 
INTA 
REMS 
Madrid, 
Spain 
LANL 
ChemCam 
Los Alamos, NM 
UofGuelph 
APXS 
SwRI Guelph, Ontario 
RAD 
Boulder, CO 
GSFC 
SAM 
Greenbelt, MD 
Plus hundreds of other 
sites around the world for 
Co-Is and Colleagues
Collaboration is More Secure in the Cloud 
Bring the users to the data, don’t send the 
data to the users
Collaboration is More Secure in the Cloud 
Bring the users to the data, don’t send the 
data to the users
Baylor College of Medicine - CHARGE 
A platform built by Baylor College of Medicine Human 
Genome Sequencing Center and DNANexus using the 
Mercury Pipeline for the Cohorts for Heart and Aging 
Research in Genomic Epidemiology (CHARGE) Consortium 
! 
Supports 300+ researchers around the world 
! 
Analyzed the genomes of over 14,000 individuals, 
encompassing 3,751 whole genomes and 10,940 whole 
exomes (~1PB of data) 
! 
Used 3.3 million core hours over 4 weeks to complete the 
job 5.7x faster than what could have been accomplished 
on-premise 
! 
The outcomes? 
1. Easier collaboration 
2. Faster time to science 
3. Cost-effective: On-premise was prohibitively expensive 
4. No longer constrained by on-premise capacity 
5. Scientists focusing on Science as opposed to 
infrastructure 
! 
https://aws.amazon.com/solutions/case-studies/baylor/ 
!
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
Globally Distributed Compute for LHC on Amazon EC2 Spot 
http://www.hep.wisc.edu/~dan/talks/EC2SpotForCMS.pdf
Globally Distributed Compute for LHC on Amazon EC2 Spot 
http://www.hep.wisc.edu/~dan/talks/EC2SpotForCMS.pdf
Belle & Belle II
Hybrid Computation Model 
<- Belle on AWS Price/Performance Benchmark 
• 170M events (3.6 TB) produced in 6 days 
• Amazon Spot Instances -> $0.20 / 10k events 
(May, 2010 pricing)
High Throughput Computing at Cloud Scale
The Square Kilometer Array
The Square Kilometer Array 
SKA Mid and Survey Dishes SKA-low Dipoles
The Square Kilometer Array 
SKA Mid and Survey Dishes SKA-low Dipoles
SCALING MWA/ASKAP TO SKA1 
MWA$ ASKAP$ SKA1)SURVEY$ SKA1)LOW$ SKA1)MID$ 
Digi$ser(Output( 0.12(Tbit/s( 
( 
1x( 
69(Tbit/s( 
( 
575x( 
( 
184(Tbit/s( 
( 
1533x( 
10(Tbit/s([3]( 
( 
83x( 
17.1(Tbit/s([1]( 
( 
143x( 
( 
Input(to( 
Science(Data( 
Processor( 
3.2(Gbit/s( 
( 
1x( 
20(Gbit/s( 
( 
6.3x( 
37360(Gbit/s([2]( 
( 
11675x( 
6736(Gbit/s([2]( 
( 
2105x( 
26000(Gbit/s([2]( 
( 
8125x( 
( 
Archived( 
Science(Data( 
Products( 
3(PB/year( 
(25%(duty( 
cycle)([4]( 
5(PB/year( 
( 
Exabytes(per(year( 
( 
Exabytes(per(year( 
( 
Exabytes(per(year( 
( 
[1] 
From 
SADT 
ConsorAum 
Technical 
Development 
Plan 
(SKA-­‐TEL.SADT-­‐PROP_TECH-­‐RED-­‐001) 
[2] 
From 
SKA1 
System 
Baseline 
Design 
(SKA-­‐TEL-­‐SKO-­‐DD-­‐001) 
[3] 
From 
LFAA 
Technical 
DescripAon 
(AADC-­‐TEL.LFAA.SE.MGT-­‐AADC-­‐PL-­‐002) 
[4] 
MWA 
is 
archiving 
correlator 
output 
data, 
not 
science 
data 
processor 
output 
data
What the SKA is now saying about AWS 
“No one’s ever built anything this big before, and we really 
don’t understand the ins and outs of operating it…Cloud 
systems — which provide on-demand, ‘elastic’ access to 
shared, remote computing resources — would provide an 
amount of flexibility for the project that buying dedicated 
hardware might not.” 
- SKA architect Tim Cornwell, Nature May 27, 2014 
Professor Peter Quinn, Director of the International 
Center for Radio Astronomy Research
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
SkyNet - ICRAR EPO System for the SKA 
ICRAR SkyNet Architecture 
! 
http://aws.amazon.com/solutions/case-studies/icrar/
National Database for Autism Research 
http://ndar.nih.gov/cloud_overview.html 
All autism research funded by NIMH 
must be publicly accessible 
! 
NDAR provides a web interface to query 
the aggregate data set
Download and Copy 
RDS 
images 
videos 
files 
binaries 
snapshots
Download and Copy 
RDS 
images 
videos 
files 
binaries 
snapshots
Download and Copy 
RDS 
images 
videos 
files 
binaries 
snapshots
Download and Copy 
RDS 
images 
videos 
files 
binaries 
snapshots
Access in the Cloud 
RDS 
images 
videos 
files 
binaries 
snapshots
Access in the Cloud 
RDS 
images 
videos 
files 
binaries 
snapshots 
RDS
Access in the Cloud 
RDS 
images 
videos 
files 
binaries 
snapshots 
RDS 
RDS
Access in the Cloud 
RDS 
images 
videos 
files 
binaries 
snapshots 
RDS 
RDS 
RDS 
RDS RDS 
RDS 
RDS
Computation in the Cloud 
RDS 
images 
videos 
files 
binaries 
snapshots
Where is this Going? 
1. Researcher conducts experiment
Where is this Going? 
1. Researcher conducts experiment 
2. Experimental data and results uploaded to the cloud along with 
reproducible machine images 
images 
videos 
files 
binaries 
snapshots 
EC2 
Amazon 
Elastic MapReduce 
EC2 Machine Amazon S3 
Images
Where is this Going? 
1. Researcher conducts experiment 
2. Experimental data and results uploaded to the cloud along with 
reproducible machine images 
3. Reviewers leverage cloud resources to reproduce and validate results. 
images 
videos 
files 
binaries 
snapshots 
EC2 
Amazon 
Elastic MapReduce 
EC2 Machine Amazon S3 
Images 
aws.amazon.com/managementconsole 
Instance Type M3 Extra Large 
Number of Instances 1,000 
Availability Zone US-West-2b 
Launch
Where is this Going? 
1. Researcher conducts experiment 
2. Experimental data and results uploaded to the cloud along with 
reproducible machine images 
3. Reviewers leverage cloud resources to reproduce and validate results. 
4. Results published in a peer-reviewed journal, including 
references (e.g. DOIs) to cloud data and AMIs 
images 
videos 
files 
binaries 
snapshots 
EC2 
Amazon 
Elastic MapReduce 
EC2 Machine Amazon S3 
Images 
Scientific 
Journal
Where is this Going? 
1. Researcher conducts experiment 
2. Experimental data and results uploaded to the cloud along with 
reproducible machine images 
3. Reviewers leverage cloud resources to reproduce and validate results. 
4. Results published in a peer-reviewed journal, including 
references (e.g. DOIs) to cloud data and AMIs 
5. Other researchers use these resources as a jumping off point for 
further research, also publishing their results in the cloud.
Where is this Going? 
1. Researcher conducts experiment 
2. Experimental data and results uploaded to the cloud along with 
reproducible machine images 
3. Reviewers leverage cloud resources to reproduce and validate results. 
4. Results published in a peer-reviewed journal, including 
references (e.g. DOIs) to cloud data and AMIs 
5. Other researchers use these resources as a jumping off point for 
further research, also publishing their results in the cloud. 
6. Automated workflows re-run the original researcher’s experiments 
on the new data using the original machine images. Interesting 
results trigger notifications and further review.
AWS Research Grants 
AWS.amazon.com/grants
AWS Public Data Sets 
AWS.amazon.com/datasets
Multi-Spectrum Atlas of the Galactic Plane 
•Collaboration between AWS, Caltech/IPAC and USC/ISI 
•All images are publicly accessible via direct download and VAO APIs 
•16 wavelength infrared atlas spanning 1μm to 70μm 
•Datasets from GLIMPSE and MIPSGAL, 2MASS, MSX, WISE 
•Spatial sampling of 1 arcsec with ±180° longitude and ±20° latitude 
•Mosaics generated by Caltech’s Montage (http://montage.ipac.caltech.edu) 
•Compute resources coordinated by USC’s Pegasus (http://pegasus.isi.edu/)
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
AWS and the NASA Earth eXchange (NEX) 
images 
videos 
files 
binaries 
snapshots 
! •Data and Software now available to those without @nasa.gov email addresses 
! •Enables crowd-sourced citizen science applications like those found on the Zooniverse 
EC2 
Amazon 
Elastic MapReduce 
EC2 Machine Amazon S3 
Images 
•National Climate Assesment datasets hosted on AWS 
! •Machine images, tutorials and hosted workshops provided by NASA 
Climate Researchers
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
EC2 
Ames 
On-demand access to effectively 
Specialized 
supercomputing 
limitless resources 
resources NASA 
Researcher 
Small-scale 
shared 
compute
Mission-Critical Computing
One server running for 1000 hours 
has the same cost as 1000 servers 
running for one hour.
Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services
Cloud for Scalability 
! 
Cloud for Global Collaboration 
! 
Cloud for Big Data
Thank 
You

More Related Content

What's hot

Deep Dive on Amazon Aurora
Deep Dive on Amazon AuroraDeep Dive on Amazon Aurora
Deep Dive on Amazon Aurora
Amazon Web Services
 
Module 2 - Datalake
Module 2 - DatalakeModule 2 - Datalake
Module 2 - Datalake
Lam Le
 
AWS re:Invent 2016: Simplified Data Center Migration—Lessons Learned by Live ...
AWS re:Invent 2016: Simplified Data Center Migration—Lessons Learned by Live ...AWS re:Invent 2016: Simplified Data Center Migration—Lessons Learned by Live ...
AWS re:Invent 2016: Simplified Data Center Migration—Lessons Learned by Live ...
Amazon Web Services
 
Building a Server-less Data Lake on AWS - Technical 301
Building a Server-less Data Lake on AWS - Technical 301Building a Server-less Data Lake on AWS - Technical 301
Building a Server-less Data Lake on AWS - Technical 301
Amazon Web Services
 
BigData: AWS RedShift with S3, EC2
BigData: AWS RedShift with S3, EC2BigData: AWS RedShift with S3, EC2
BigData: AWS RedShift with S3, EC2
Paulraj Pappaiah
 
New Database Migration Services & RDS Updates
New Database Migration Services & RDS UpdatesNew Database Migration Services & RDS Updates
New Database Migration Services & RDS Updates
Amazon Web Services
 
AWS re:Invent 2016: Large-scale AWS Migrations (ENT204)
AWS re:Invent 2016: Large-scale AWS Migrations (ENT204)AWS re:Invent 2016: Large-scale AWS Migrations (ENT204)
AWS re:Invent 2016: Large-scale AWS Migrations (ENT204)
Amazon Web Services
 
Databases - State of the Union
Databases - State of the UnionDatabases - State of the Union
Databases - State of the Union
Amazon Web Services
 
Amazon Aurora: Amazon’s New Relational Database Engine
Amazon Aurora: Amazon’s New Relational Database EngineAmazon Aurora: Amazon’s New Relational Database Engine
Amazon Aurora: Amazon’s New Relational Database Engine
Danilo Poccia
 
AWS re:Invent 2016: Automating Workflows for Analytics Pipelines (DEV401)
AWS re:Invent 2016: Automating Workflows for Analytics Pipelines (DEV401)AWS re:Invent 2016: Automating Workflows for Analytics Pipelines (DEV401)
AWS re:Invent 2016: Automating Workflows for Analytics Pipelines (DEV401)
Amazon Web Services
 
Modern Data Architectures for Business Insights at Scale
Modern Data Architectures for Business Insights at ScaleModern Data Architectures for Business Insights at Scale
Modern Data Architectures for Business Insights at Scale
Amazon Web Services
 
AWS March 2016 Webinar Series - Building Big Data Solutions with Amazon EMR a...
AWS March 2016 Webinar Series - Building Big Data Solutions with Amazon EMR a...AWS March 2016 Webinar Series - Building Big Data Solutions with Amazon EMR a...
AWS March 2016 Webinar Series - Building Big Data Solutions with Amazon EMR a...
Amazon Web Services
 
Data Analytics on AWS
Data Analytics on AWSData Analytics on AWS
Data Analytics on AWS
Danilo Poccia
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
Amazon Web Services
 
Building Serverless Web Applications - DevDay Los Angeles 2017
Building Serverless Web Applications - DevDay Los Angeles 2017Building Serverless Web Applications - DevDay Los Angeles 2017
Building Serverless Web Applications - DevDay Los Angeles 2017
Amazon Web Services
 
Big Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWSBig Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWS
Amazon Web Services
 
Migrating On-Premises Databases to Cloud
Migrating On-Premises Databases to CloudMigrating On-Premises Databases to Cloud
Migrating On-Premises Databases to Cloud
Amazon Web Services
 
Modern Data Architectures for Business Outcomes
Modern Data Architectures for Business OutcomesModern Data Architectures for Business Outcomes
Modern Data Architectures for Business Outcomes
Amazon Web Services
 
Big data on aws
Big data on awsBig data on aws
Big data on aws
Serkan Özal
 
AWS Database Migration Service
AWS Database Migration ServiceAWS Database Migration Service
AWS Database Migration Service
techugo
 

What's hot (20)

Deep Dive on Amazon Aurora
Deep Dive on Amazon AuroraDeep Dive on Amazon Aurora
Deep Dive on Amazon Aurora
 
Module 2 - Datalake
Module 2 - DatalakeModule 2 - Datalake
Module 2 - Datalake
 
AWS re:Invent 2016: Simplified Data Center Migration—Lessons Learned by Live ...
AWS re:Invent 2016: Simplified Data Center Migration—Lessons Learned by Live ...AWS re:Invent 2016: Simplified Data Center Migration—Lessons Learned by Live ...
AWS re:Invent 2016: Simplified Data Center Migration—Lessons Learned by Live ...
 
Building a Server-less Data Lake on AWS - Technical 301
Building a Server-less Data Lake on AWS - Technical 301Building a Server-less Data Lake on AWS - Technical 301
Building a Server-less Data Lake on AWS - Technical 301
 
BigData: AWS RedShift with S3, EC2
BigData: AWS RedShift with S3, EC2BigData: AWS RedShift with S3, EC2
BigData: AWS RedShift with S3, EC2
 
New Database Migration Services & RDS Updates
New Database Migration Services & RDS UpdatesNew Database Migration Services & RDS Updates
New Database Migration Services & RDS Updates
 
AWS re:Invent 2016: Large-scale AWS Migrations (ENT204)
AWS re:Invent 2016: Large-scale AWS Migrations (ENT204)AWS re:Invent 2016: Large-scale AWS Migrations (ENT204)
AWS re:Invent 2016: Large-scale AWS Migrations (ENT204)
 
Databases - State of the Union
Databases - State of the UnionDatabases - State of the Union
Databases - State of the Union
 
Amazon Aurora: Amazon’s New Relational Database Engine
Amazon Aurora: Amazon’s New Relational Database EngineAmazon Aurora: Amazon’s New Relational Database Engine
Amazon Aurora: Amazon’s New Relational Database Engine
 
AWS re:Invent 2016: Automating Workflows for Analytics Pipelines (DEV401)
AWS re:Invent 2016: Automating Workflows for Analytics Pipelines (DEV401)AWS re:Invent 2016: Automating Workflows for Analytics Pipelines (DEV401)
AWS re:Invent 2016: Automating Workflows for Analytics Pipelines (DEV401)
 
Modern Data Architectures for Business Insights at Scale
Modern Data Architectures for Business Insights at ScaleModern Data Architectures for Business Insights at Scale
Modern Data Architectures for Business Insights at Scale
 
AWS March 2016 Webinar Series - Building Big Data Solutions with Amazon EMR a...
AWS March 2016 Webinar Series - Building Big Data Solutions with Amazon EMR a...AWS March 2016 Webinar Series - Building Big Data Solutions with Amazon EMR a...
AWS March 2016 Webinar Series - Building Big Data Solutions with Amazon EMR a...
 
Data Analytics on AWS
Data Analytics on AWSData Analytics on AWS
Data Analytics on AWS
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
Building Serverless Web Applications - DevDay Los Angeles 2017
Building Serverless Web Applications - DevDay Los Angeles 2017Building Serverless Web Applications - DevDay Los Angeles 2017
Building Serverless Web Applications - DevDay Los Angeles 2017
 
Big Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWSBig Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWS
 
Migrating On-Premises Databases to Cloud
Migrating On-Premises Databases to CloudMigrating On-Premises Databases to Cloud
Migrating On-Premises Databases to Cloud
 
Modern Data Architectures for Business Outcomes
Modern Data Architectures for Business OutcomesModern Data Architectures for Business Outcomes
Modern Data Architectures for Business Outcomes
 
Big data on aws
Big data on awsBig data on aws
Big data on aws
 
AWS Database Migration Service
AWS Database Migration ServiceAWS Database Migration Service
AWS Database Migration Service
 

Viewers also liked

Scientific Computing With Amazon Web Services
Scientific Computing With Amazon Web ServicesScientific Computing With Amazon Web Services
Scientific Computing With Amazon Web Services
Jamie Kinney
 
Beyond tang
Beyond tangBeyond tang
Beyond tang
Jamie Kinney
 
Announcing Amazon EC2 F1 Instances with Custom FPGAs
Announcing Amazon EC2 F1 Instances with Custom FPGAsAnnouncing Amazon EC2 F1 Instances with Custom FPGAs
Announcing Amazon EC2 F1 Instances with Custom FPGAs
Amazon Web Services
 
NEW LAUNCH! Delivering Powerful Graphics-Intensive Applications from the AWS ...
NEW LAUNCH! Delivering Powerful Graphics-Intensive Applications from the AWS ...NEW LAUNCH! Delivering Powerful Graphics-Intensive Applications from the AWS ...
NEW LAUNCH! Delivering Powerful Graphics-Intensive Applications from the AWS ...
Amazon Web Services
 
Day 1 - Introduction to Cloud Computing with Amazon Web Services
Day 1 - Introduction to Cloud Computing with Amazon Web ServicesDay 1 - Introduction to Cloud Computing with Amazon Web Services
Day 1 - Introduction to Cloud Computing with Amazon Web Services
Amazon Web Services
 
E-commerce hardware and software - Welcome to DePaul University
E-commerce hardware and software - Welcome to DePaul UniversityE-commerce hardware and software - Welcome to DePaul University
E-commerce hardware and software - Welcome to DePaul University
webhostingguy
 
AWS re:Invent 2016: High Performance Computing on AWS (CMP207)
AWS re:Invent 2016: High Performance Computing on AWS (CMP207)AWS re:Invent 2016: High Performance Computing on AWS (CMP207)
AWS re:Invent 2016: High Performance Computing on AWS (CMP207)
Amazon Web Services
 
Introduction to AWS Batch
Introduction to AWS BatchIntroduction to AWS Batch
Introduction to AWS Batch
Amazon Web Services
 
E commerce infrastructure
E commerce infrastructureE commerce infrastructure
E commerce infrastructure
Sovan Kundu
 

Viewers also liked (9)

Scientific Computing With Amazon Web Services
Scientific Computing With Amazon Web ServicesScientific Computing With Amazon Web Services
Scientific Computing With Amazon Web Services
 
Beyond tang
Beyond tangBeyond tang
Beyond tang
 
Announcing Amazon EC2 F1 Instances with Custom FPGAs
Announcing Amazon EC2 F1 Instances with Custom FPGAsAnnouncing Amazon EC2 F1 Instances with Custom FPGAs
Announcing Amazon EC2 F1 Instances with Custom FPGAs
 
NEW LAUNCH! Delivering Powerful Graphics-Intensive Applications from the AWS ...
NEW LAUNCH! Delivering Powerful Graphics-Intensive Applications from the AWS ...NEW LAUNCH! Delivering Powerful Graphics-Intensive Applications from the AWS ...
NEW LAUNCH! Delivering Powerful Graphics-Intensive Applications from the AWS ...
 
Day 1 - Introduction to Cloud Computing with Amazon Web Services
Day 1 - Introduction to Cloud Computing with Amazon Web ServicesDay 1 - Introduction to Cloud Computing with Amazon Web Services
Day 1 - Introduction to Cloud Computing with Amazon Web Services
 
E-commerce hardware and software - Welcome to DePaul University
E-commerce hardware and software - Welcome to DePaul UniversityE-commerce hardware and software - Welcome to DePaul University
E-commerce hardware and software - Welcome to DePaul University
 
AWS re:Invent 2016: High Performance Computing on AWS (CMP207)
AWS re:Invent 2016: High Performance Computing on AWS (CMP207)AWS re:Invent 2016: High Performance Computing on AWS (CMP207)
AWS re:Invent 2016: High Performance Computing on AWS (CMP207)
 
Introduction to AWS Batch
Introduction to AWS BatchIntroduction to AWS Batch
Introduction to AWS Batch
 
E commerce infrastructure
E commerce infrastructureE commerce infrastructure
E commerce infrastructure
 

Similar to Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services

Keynote sp summit 2014 final
Keynote sp summit 2014  finalKeynote sp summit 2014  final
Keynote sp summit 2014 final
Amazon Web Services LATAM
 
Effective and Efficient Computing for the Government
Effective and Efficient Computing for the GovernmentEffective and Efficient Computing for the Government
Effective and Efficient Computing for the Government
Amazon Web Services
 
AWS reinvent 2019 recap - Riyadh - Database and Analytics - Assif Abbasi
AWS reinvent 2019 recap - Riyadh - Database and Analytics - Assif AbbasiAWS reinvent 2019 recap - Riyadh - Database and Analytics - Assif Abbasi
AWS reinvent 2019 recap - Riyadh - Database and Analytics - Assif Abbasi
AWS Riyadh User Group
 
AWS Services Overview and Quarterly Update - April 2017 AWS Online Tech Talks
AWS Services Overview and Quarterly Update - April 2017 AWS Online Tech TalksAWS Services Overview and Quarterly Update - April 2017 AWS Online Tech Talks
AWS Services Overview and Quarterly Update - April 2017 AWS Online Tech Talks
Amazon Web Services
 
AWS Services Overview and Quarterly Update - April 2017 AWS Online Tech Talks
AWS Services Overview and Quarterly Update - April 2017 AWS Online Tech TalksAWS Services Overview and Quarterly Update - April 2017 AWS Online Tech Talks
AWS Services Overview and Quarterly Update - April 2017 AWS Online Tech Talks
Amazon Web Services
 
透過 Amazon Redshift 打造數據分析服務及 Amazon Redshift 新功能案例介紹
透過 Amazon Redshift 打造數據分析服務及 Amazon Redshift 新功能案例介紹透過 Amazon Redshift 打造數據分析服務及 Amazon Redshift 新功能案例介紹
透過 Amazon Redshift 打造數據分析服務及 Amazon Redshift 新功能案例介紹
Amazon Web Services
 
Vancouver keynote - AWS Innovate - Sam Elmalak
Vancouver keynote - AWS Innovate - Sam ElmalakVancouver keynote - AWS Innovate - Sam Elmalak
Vancouver keynote - AWS Innovate - Sam Elmalak
Amazon Web Services
 
Introduction to Cloud Computing with Amazon Web Services
Introduction to Cloud Computing with Amazon Web ServicesIntroduction to Cloud Computing with Amazon Web Services
Introduction to Cloud Computing with Amazon Web Services
Amazon Web Services
 
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Amazon Web Services
 
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Amazon Web Services
 
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Amazon Web Services
 
AWS Overview - Cloud for the Enterprise - AWS Enterprise Tour - SF - 2010, D...
AWS Overview  - Cloud for the Enterprise - AWS Enterprise Tour - SF - 2010, D...AWS Overview  - Cloud for the Enterprise - AWS Enterprise Tour - SF - 2010, D...
AWS Overview - Cloud for the Enterprise - AWS Enterprise Tour - SF - 2010, D...
Amazon Web Services
 
Solution architecture Amazon web services
Solution architecture Amazon web servicesSolution architecture Amazon web services
Solution architecture Amazon web services
Sandeep Sharma IIMK Smart City,IoT,Bigdata,Cloud,BI,DW
 
AWS Big Data Solution Days
AWS Big Data Solution DaysAWS Big Data Solution Days
AWS Big Data Solution Days
Amazon Web Services
 
Werner Vogels
Werner Vogels Werner Vogels
Werner Vogels
Amazon Web Services
 
AWS Webcast - Attunity Couchsurfing
AWS Webcast - Attunity CouchsurfingAWS Webcast - Attunity Couchsurfing
AWS Webcast - Attunity Couchsurfing
Amazon Web Services
 
Journey Through the Cloud - Data Analysis
Journey Through the Cloud - Data AnalysisJourney Through the Cloud - Data Analysis
Journey Through the Cloud - Data Analysis
Amazon Web Services
 
Data Analysis - Journey Through the Cloud
Data Analysis - Journey Through the CloudData Analysis - Journey Through the Cloud
Data Analysis - Journey Through the Cloud
Ian Massingham
 
Serverless Kafka on AWS as Part of a Cloud-native Data Lake Architecture
Serverless Kafka on AWS as Part of a Cloud-native Data Lake ArchitectureServerless Kafka on AWS as Part of a Cloud-native Data Lake Architecture
Serverless Kafka on AWS as Part of a Cloud-native Data Lake Architecture
Kai Wähner
 
A1 keynote oracle_infrastructure_as_a_service_move_any_workload_to_the_cloud
A1 keynote oracle_infrastructure_as_a_service_move_any_workload_to_the_cloudA1 keynote oracle_infrastructure_as_a_service_move_any_workload_to_the_cloud
A1 keynote oracle_infrastructure_as_a_service_move_any_workload_to_the_cloud
Dr. Wilfred Lin (Ph.D.)
 

Similar to Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services (20)

Keynote sp summit 2014 final
Keynote sp summit 2014  finalKeynote sp summit 2014  final
Keynote sp summit 2014 final
 
Effective and Efficient Computing for the Government
Effective and Efficient Computing for the GovernmentEffective and Efficient Computing for the Government
Effective and Efficient Computing for the Government
 
AWS reinvent 2019 recap - Riyadh - Database and Analytics - Assif Abbasi
AWS reinvent 2019 recap - Riyadh - Database and Analytics - Assif AbbasiAWS reinvent 2019 recap - Riyadh - Database and Analytics - Assif Abbasi
AWS reinvent 2019 recap - Riyadh - Database and Analytics - Assif Abbasi
 
AWS Services Overview and Quarterly Update - April 2017 AWS Online Tech Talks
AWS Services Overview and Quarterly Update - April 2017 AWS Online Tech TalksAWS Services Overview and Quarterly Update - April 2017 AWS Online Tech Talks
AWS Services Overview and Quarterly Update - April 2017 AWS Online Tech Talks
 
AWS Services Overview and Quarterly Update - April 2017 AWS Online Tech Talks
AWS Services Overview and Quarterly Update - April 2017 AWS Online Tech TalksAWS Services Overview and Quarterly Update - April 2017 AWS Online Tech Talks
AWS Services Overview and Quarterly Update - April 2017 AWS Online Tech Talks
 
透過 Amazon Redshift 打造數據分析服務及 Amazon Redshift 新功能案例介紹
透過 Amazon Redshift 打造數據分析服務及 Amazon Redshift 新功能案例介紹透過 Amazon Redshift 打造數據分析服務及 Amazon Redshift 新功能案例介紹
透過 Amazon Redshift 打造數據分析服務及 Amazon Redshift 新功能案例介紹
 
Vancouver keynote - AWS Innovate - Sam Elmalak
Vancouver keynote - AWS Innovate - Sam ElmalakVancouver keynote - AWS Innovate - Sam Elmalak
Vancouver keynote - AWS Innovate - Sam Elmalak
 
Introduction to Cloud Computing with Amazon Web Services
Introduction to Cloud Computing with Amazon Web ServicesIntroduction to Cloud Computing with Amazon Web Services
Introduction to Cloud Computing with Amazon Web Services
 
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
 
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
 
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
 
AWS Overview - Cloud for the Enterprise - AWS Enterprise Tour - SF - 2010, D...
AWS Overview  - Cloud for the Enterprise - AWS Enterprise Tour - SF - 2010, D...AWS Overview  - Cloud for the Enterprise - AWS Enterprise Tour - SF - 2010, D...
AWS Overview - Cloud for the Enterprise - AWS Enterprise Tour - SF - 2010, D...
 
Solution architecture Amazon web services
Solution architecture Amazon web servicesSolution architecture Amazon web services
Solution architecture Amazon web services
 
AWS Big Data Solution Days
AWS Big Data Solution DaysAWS Big Data Solution Days
AWS Big Data Solution Days
 
Werner Vogels
Werner Vogels Werner Vogels
Werner Vogels
 
AWS Webcast - Attunity Couchsurfing
AWS Webcast - Attunity CouchsurfingAWS Webcast - Attunity Couchsurfing
AWS Webcast - Attunity Couchsurfing
 
Journey Through the Cloud - Data Analysis
Journey Through the Cloud - Data AnalysisJourney Through the Cloud - Data Analysis
Journey Through the Cloud - Data Analysis
 
Data Analysis - Journey Through the Cloud
Data Analysis - Journey Through the CloudData Analysis - Journey Through the Cloud
Data Analysis - Journey Through the Cloud
 
Serverless Kafka on AWS as Part of a Cloud-native Data Lake Architecture
Serverless Kafka on AWS as Part of a Cloud-native Data Lake ArchitectureServerless Kafka on AWS as Part of a Cloud-native Data Lake Architecture
Serverless Kafka on AWS as Part of a Cloud-native Data Lake Architecture
 
A1 keynote oracle_infrastructure_as_a_service_move_any_workload_to_the_cloud
A1 keynote oracle_infrastructure_as_a_service_move_any_workload_to_the_cloudA1 keynote oracle_infrastructure_as_a_service_move_any_workload_to_the_cloud
A1 keynote oracle_infrastructure_as_a_service_move_any_workload_to_the_cloud
 

Recently uploaded

Anatomy, and reproduction of Gnetum.pptx
Anatomy, and reproduction of Gnetum.pptxAnatomy, and reproduction of Gnetum.pptx
Anatomy, and reproduction of Gnetum.pptx
karthiksaran8
 
Properties of virus(Ultrastructure and types of virus)
Properties of virus(Ultrastructure and types of virus)Properties of virus(Ultrastructure and types of virus)
Properties of virus(Ultrastructure and types of virus)
saloniswain225
 
El Nuevo Cohete Ariane de la Agencia Espacial Europea-6_Media-Kit_english.pdf
El Nuevo Cohete Ariane de la Agencia Espacial Europea-6_Media-Kit_english.pdfEl Nuevo Cohete Ariane de la Agencia Espacial Europea-6_Media-Kit_english.pdf
El Nuevo Cohete Ariane de la Agencia Espacial Europea-6_Media-Kit_english.pdf
Champs Elysee Roldan
 
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpectiveGasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
Recupera
 
smallintestinedisorders-causessymptoms-240626051934-b669b27d.pptx
smallintestinedisorders-causessymptoms-240626051934-b669b27d.pptxsmallintestinedisorders-causessymptoms-240626051934-b669b27d.pptx
smallintestinedisorders-causessymptoms-240626051934-b669b27d.pptx
muralinath2
 
2. Osmotic pressure, osmotic potential, turgor pressure, wall pressure, water...
2. Osmotic pressure, osmotic potential, turgor pressure, wall pressure, water...2. Osmotic pressure, osmotic potential, turgor pressure, wall pressure, water...
2. Osmotic pressure, osmotic potential, turgor pressure, wall pressure, water...
khadija07kubra
 
Adjusted NuGOweek 2024 Ghent programme flyer
Adjusted NuGOweek 2024 Ghent programme flyerAdjusted NuGOweek 2024 Ghent programme flyer
Adjusted NuGOweek 2024 Ghent programme flyer
pablovgd
 
Lunar Mobility Drivers and Needs - Artemis
Lunar Mobility Drivers and Needs - ArtemisLunar Mobility Drivers and Needs - Artemis
Lunar Mobility Drivers and Needs - Artemis
Sérgio Sacani
 
A slightly oblate dark matter halo revealed by a retrograde precessing Galact...
A slightly oblate dark matter halo revealed by a retrograde precessing Galact...A slightly oblate dark matter halo revealed by a retrograde precessing Galact...
A slightly oblate dark matter halo revealed by a retrograde precessing Galact...
Sérgio Sacani
 
degree Certificate of Aston University
degree Certificate of Aston Universitydegree Certificate of Aston University
degree Certificate of Aston University
ebgyz
 
The cryptoterrestrial hypothesis: A case for scientific openness to a conceal...
The cryptoterrestrial hypothesis: A case for scientific openness to a conceal...The cryptoterrestrial hypothesis: A case for scientific openness to a conceal...
The cryptoterrestrial hypothesis: A case for scientific openness to a conceal...
Sérgio Sacani
 
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
James AH Campbell
 
LOB LOD LOQ for method validation in laboratory
LOB LOD LOQ for method validation in laboratoryLOB LOD LOQ for method validation in laboratory
LOB LOD LOQ for method validation in laboratory
JCKH
 
Dalghren, Thorne and Stebbins System of Classification of Angiosperms
Dalghren, Thorne and Stebbins System of Classification of AngiospermsDalghren, Thorne and Stebbins System of Classification of Angiosperms
Dalghren, Thorne and Stebbins System of Classification of Angiosperms
Gurjant Singh
 
Bragg Brentano Alignment for D4 with LynxEye Rev3.pptx
Bragg Brentano Alignment for D4 with LynxEye Rev3.pptxBragg Brentano Alignment for D4 with LynxEye Rev3.pptx
Bragg Brentano Alignment for D4 with LynxEye Rev3.pptx
Lisandro Cunci
 
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Hossein Fani
 
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopyA mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
Sérgio Sacani
 
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptxlargeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
muralinath2
 
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physicsTHE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
Dr. sreeremya S
 
Modelling, Simulation, and Computer-aided Design in Computational, Evolutiona...
Modelling, Simulation, and Computer-aided Design in Computational, Evolutiona...Modelling, Simulation, and Computer-aided Design in Computational, Evolutiona...
Modelling, Simulation, and Computer-aided Design in Computational, Evolutiona...
University of Maribor
 

Recently uploaded (20)

Anatomy, and reproduction of Gnetum.pptx
Anatomy, and reproduction of Gnetum.pptxAnatomy, and reproduction of Gnetum.pptx
Anatomy, and reproduction of Gnetum.pptx
 
Properties of virus(Ultrastructure and types of virus)
Properties of virus(Ultrastructure and types of virus)Properties of virus(Ultrastructure and types of virus)
Properties of virus(Ultrastructure and types of virus)
 
El Nuevo Cohete Ariane de la Agencia Espacial Europea-6_Media-Kit_english.pdf
El Nuevo Cohete Ariane de la Agencia Espacial Europea-6_Media-Kit_english.pdfEl Nuevo Cohete Ariane de la Agencia Espacial Europea-6_Media-Kit_english.pdf
El Nuevo Cohete Ariane de la Agencia Espacial Europea-6_Media-Kit_english.pdf
 
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpectiveGasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
 
smallintestinedisorders-causessymptoms-240626051934-b669b27d.pptx
smallintestinedisorders-causessymptoms-240626051934-b669b27d.pptxsmallintestinedisorders-causessymptoms-240626051934-b669b27d.pptx
smallintestinedisorders-causessymptoms-240626051934-b669b27d.pptx
 
2. Osmotic pressure, osmotic potential, turgor pressure, wall pressure, water...
2. Osmotic pressure, osmotic potential, turgor pressure, wall pressure, water...2. Osmotic pressure, osmotic potential, turgor pressure, wall pressure, water...
2. Osmotic pressure, osmotic potential, turgor pressure, wall pressure, water...
 
Adjusted NuGOweek 2024 Ghent programme flyer
Adjusted NuGOweek 2024 Ghent programme flyerAdjusted NuGOweek 2024 Ghent programme flyer
Adjusted NuGOweek 2024 Ghent programme flyer
 
Lunar Mobility Drivers and Needs - Artemis
Lunar Mobility Drivers and Needs - ArtemisLunar Mobility Drivers and Needs - Artemis
Lunar Mobility Drivers and Needs - Artemis
 
A slightly oblate dark matter halo revealed by a retrograde precessing Galact...
A slightly oblate dark matter halo revealed by a retrograde precessing Galact...A slightly oblate dark matter halo revealed by a retrograde precessing Galact...
A slightly oblate dark matter halo revealed by a retrograde precessing Galact...
 
degree Certificate of Aston University
degree Certificate of Aston Universitydegree Certificate of Aston University
degree Certificate of Aston University
 
The cryptoterrestrial hypothesis: A case for scientific openness to a conceal...
The cryptoterrestrial hypothesis: A case for scientific openness to a conceal...The cryptoterrestrial hypothesis: A case for scientific openness to a conceal...
The cryptoterrestrial hypothesis: A case for scientific openness to a conceal...
 
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
 
LOB LOD LOQ for method validation in laboratory
LOB LOD LOQ for method validation in laboratoryLOB LOD LOQ for method validation in laboratory
LOB LOD LOQ for method validation in laboratory
 
Dalghren, Thorne and Stebbins System of Classification of Angiosperms
Dalghren, Thorne and Stebbins System of Classification of AngiospermsDalghren, Thorne and Stebbins System of Classification of Angiosperms
Dalghren, Thorne and Stebbins System of Classification of Angiosperms
 
Bragg Brentano Alignment for D4 with LynxEye Rev3.pptx
Bragg Brentano Alignment for D4 with LynxEye Rev3.pptxBragg Brentano Alignment for D4 with LynxEye Rev3.pptx
Bragg Brentano Alignment for D4 with LynxEye Rev3.pptx
 
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
 
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopyA mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
 
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptxlargeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
 
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physicsTHE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
 
Modelling, Simulation, and Computer-aided Design in Computational, Evolutiona...
Modelling, Simulation, and Computer-aided Design in Computational, Evolutiona...Modelling, Simulation, and Computer-aided Design in Computational, Evolutiona...
Modelling, Simulation, and Computer-aided Design in Computational, Evolutiona...
 

Astroinformatics 2014: Scientific Computing on the Cloud with Amazon Web Services

  • 1. Scientific Computing with Amazon Web Services Jamie Kinney Director of Scientific Computing Amazon Web Services jkinney@amazon.com @jamiekinney
  • 6. Amazon Global Impact Initiatives Scientific Computing - Global “Big Science” Projects - Enabling the “long tail of science” - Collaborative research - Accelerating the transition to “Networked Science” - Initial focus on Life Sciences, Earth Sciences, Astronomy/Astrophysics and High Energy Physics ! Open Data - AWS as the platform for open-access data portals and archives - Amazon Public Data Sets - Public/Private Data partnerships ! Economic Development - The Amazon Job Accelerators program
  • 7. Why are we focusing on the Scientific Community? •In order to meaningfully change our world for the better by accelerating the pace of scientific discovery ! •Scientific computing is a profitable business for AWS ! •To develop new capabilities which will benefit all AWS customers - Streaming data processing & analytics - Exabyte scale data management solutions - Collaborative research tools and techniques - New AWS regions - Significant advances in low-power compute, storage and data centers - Identify efficiencies which will lower our costs and pricing for customers - Push our existing services to support exabyte/exaflop scale workloads
  • 9. Self Hosting Waste Actual demand Customer Dissatisfaction Predicted Demand Rigid Actual demand Elastic
  • 10. 2003 $5.2B retail business 7,800 employees A whole lot of servers
  • 11. 2003 2013 $5.2B retail business 7,800 employees A whole lot of servers
  • 12. 2003 2013 $5.2B retail business 7,800 employees A whole lot of servers
  • 13. 2003 2013 $5.2B retail business 7,800 employees A whole lot of servers Every day, AWS adds enough server capacity to power this $5B enterprise
  • 14. Said another way… AWS is deploying the equivalent of a top-20 supercomputer every few days
  • 15. Support Professional Services Training Certification Technology Partners Consulting Partners Ecosystem AWS Marketplace Elastic Beanstalk for Java, Node.js, Python, Ruby, PHP and .Net OpsWorks CloudFormation Containers & Deployment (PaaS) Management & IAM CloudTrail Cloud HSM CloudWatch Administration Management Console APIs and SDKs Command Line Interface Analytics Application Services EMR Redshift Kinesis Data Pipeline CloudFront SNS SQS SES SWF WorkSpaces AppStream CloudSearch Networking VPC Direct Connect Route 53 Compute Storage Databases MySQL, PostgreSQL Oracle, SQL Server S3 EBS Glacier Storage Gateway Import/Export RDS DynamoDB ElastiCache EC2 Elastic Load Balancer Auto Scaling Regions Availability Zones Content Delivery POPs
  • 16. Broad Features and Functionality for Each Service IAM Redshift Elastic Map Reduce Data Pipeline CloudFront Route 53 CloudFormation Elastic Load EC2 Balancer EBS S3 DynamoDB AppStream Kinesis
  • 17. Broad Features and Functionality for Each Service 56 new features since Feb 2013 Regional expansion to US West (Oregon) Support for temporary credentials when loading data from Amazon S3 Regional expansion to EU West (Dublin) SOC1/2/3 Compliance certification Ability to UNLOAD encrypted files in parallel to Amazon S3 Regional expansion to Asia Pacific (Tokyo) Support for JDBC fetch size to enable extraction of large data sets over JDBC/ODBC Enable logging of UNLOAD statements New built-in function to compute the SHA1 hash of a value Added support for UTF-8 characters up to 4 bytes in size Ability to share snapshots between accounts to simplify manageability. Support for statement timeouts to automatically terminate queries that exceeded allotted execution time Added support for timezone conversion in SQL Added support for datetime values expressed in milliseconds since EPOCH to simplify ingestion Simplified ingestion by automatically detecting date and time formats. Added support for automatic query timeouts to workload management queues. Enabled the use of wildcards when assigning queries to workload management queues. New built-in function to enable customers to calculate the CRC32 checksum of a value Console improvements to show progress bars for backup and restore operations. Added the ability to support IAM at the resource level allowing tight control of who can take what actions on which resources. Obtained PCI compliance Added the ability to substitute a customer chosen character for invalid UTF-8 characters to simplify ingestion Allowed customers to store JSON data in VARCHAR columns and added built-in functions to enable data extraction Added support for POSIX regex expressions when using SIMILAR to in SQL queries Added Cursor support to enable extraction of large data sets over ODBC connections Built-in function to enable splitting a string using a supplied delimiter to make parsing values easier Added system tables to enable logging of database activity for auditing Regional expansion to Asia Pacific (Singapore, Sydney) Enable customers to control cluster encryption keys by using an on premises hardware security module (HSM) or Amazon CloudHSM Enable customers to receive alerts via SNS for informational or error-related events for cluster monitoring, management, configuration and security. Integration with Canal to enable streaming data ingestion Copy from an arbitrary SSH connection enabling direct copy from Amazon EMR, HDFS, or any other database that supports SSH access and script execution Enable distributing tables to all compute nodes to speed up queries, especially those involving star or snowflake schemas Logging of database logins, failed logins, SQL execution and data loads to S3 and integration with CloudTrails for control plane events Enabled caching of database blocks to speed up access to frequently queried data Increase cluster concurrency limits from 15 to 50 to enable higher concurrent query execution Optimizations to resize code that lead to 2-4x improvement in resize performance Approximate COUNT DISTINCT using HyperLogLog giving 10-20x performance improvements with less than 1% error Enable customers to continuously, automatically and incrementally back up data to a second AWS region for DR On track to obtain Fedramp certification Deliver Redshift on SSD instances enabling a lower-cost, high performance entry point IAM Redshift Elastic Map Reduce Data Pipeline CloudFront Route 53 CloudFormation Elastic Load EC2 Balancer EBS S3 DynamoDB AppStream Kinesis
  • 18. AWS Big Data Technologies Amazon RDS Amazon DBA$ DynamoDB Amazon Elasticache Amazon Elastic Map Reduce Amazon Data Pipeline Amazon Redshift Hosted Relational Databases Managed NoSQL Database In-memory Caching Service Hosted Hadoop framework Move data among AWS services and on-premise data sources Petabyte-scale columnar relational data warehouse service http://aws.amazon.com/big-data/
  • 19. The AWS Big Data Stack Also Includes… •HTCondor •SNAP •Spark/Shark •BlinkDB •Cassandra •Galaxy •MongoDB •Lustre •Hadoop •Gluster •Pegasus •Galaxy •SLURM •BOINC •MIT StarCluster •… and many more
  • 25. Amazon S3: Over 2 Trillion Total Objects > 2,000,000 requests/second 1,100,000 Million peak requests/sec
  • 27. EC2 Instance Type History: Increasing Customer Choice m2.2xlarge m2.4xlarge c1.medium c1.xlarge m1.xlarge m1.large m1.small ! 2006 2007 2008 2009 2010 2011 2012-­‐2013 April, 2014 m1.small m1.xlarge m1.large m1.small cc2.8xlarge cc1.4xlarge cg1.4xlarge t1.micro m2.xlarge m2.2xlarge m2.4xlarge c1.medium c1.xlarge m1.xlarge m1.large m1.small cr1.8xlarge hs1.8xlarge m3.xlarge m3.2xlarge hi1.4xlarge m1.medium cc2.8xlarge cg1.4xlarge t1.micro m2.xlarge m2.2xlarge m2.4xlarge c1.medium c1.xlarge m1.xlarge m1.large m1.small cc1.4xlarge cg1.4xlarge t1.micro m2.xlarge m2.2xlarge m2.4xlarge c1.medium c1.xlarge m1.xlarge m1.large m1.small hs1.8xlarge m3.xlarge m3.2xlarge hi1.4xlarge m1.medium cc2.8xlarge cg1.4xlarge t1.micro m2.xlarge m2.2xlarge m2.4xlarge c1.medium c1.xlarge m1.xlarge m1.large m1.small c1.medium c1.xlarge m1.xlarge m1.large m1.small existing g2.2xlarge hs1.xlarge hs1.2xlarge hs1.4xlarge c3.large c3.xlarge c3.2xlarge c3.4xlarge c3.8xlarge m3.medium m3.large i2.large i2.xlarge i2.4xlarge i2.8xlarge r3.large r3.xlarge r3.2xlarge r3.4xlarge r3.8xlarge new
  • 28. Integrated Architectures EC2 EC2 EC2 EC2 Los Angeles Singapore Japan London São Paolo New York Sydney AWS Direct Amazon VPC Connect
  • 29. Customer’s Network Customer’s isolated AWS resources Amazon Web Services Secure VPN Cloud Connection over the Internet Subnets Router VPN Gateway Internet NAT Private Private Private AWS Virtual Private Cloud Networking TIC
  • 30. VPC Deployment Models VPC With a Single Subnet Simple web hosting, collaborative research environments, on-demand HPC/HTC clusters VPC With Private and Public Subnets Multi-tier web hosting VPC With Private and Public Subnets & Hardware VPN access Multi-tier web hosting with access to internal infrastructure VPC With a Private Subnet Only & Hardware VPN Access Seamless private expansion of on-premise infrastructure. “Burst capacity” or dedicated cloud environments with connectivity to on-premise resources
  • 31. Who is using AWS?
  • 33. MSL Distributed Operations JPL Pasadena, CA CDSCC Canberra Deep Space Communication Complex MDSCC Madrid Deep Space Communication Complex GDSCC Goldstone Deep Space Communication Complex ARC CheMin Moffett Field, CA MSSS MARDI, MAHLI, MastCam San Diego, CA KSC IKI DAN Moscow, Russia INTA REMS Madrid, Spain LANL ChemCam Los Alamos, NM UofGuelph APXS SwRI Guelph, Ontario RAD Boulder, CO GSFC SAM Greenbelt, MD Plus hundreds of other sites around the world for Co-Is and Colleagues
  • 34. Collaboration is More Secure in the Cloud Bring the users to the data, don’t send the data to the users
  • 35. Collaboration is More Secure in the Cloud Bring the users to the data, don’t send the data to the users
  • 36. Baylor College of Medicine - CHARGE A platform built by Baylor College of Medicine Human Genome Sequencing Center and DNANexus using the Mercury Pipeline for the Cohorts for Heart and Aging Research in Genomic Epidemiology (CHARGE) Consortium ! Supports 300+ researchers around the world ! Analyzed the genomes of over 14,000 individuals, encompassing 3,751 whole genomes and 10,940 whole exomes (~1PB of data) ! Used 3.3 million core hours over 4 weeks to complete the job 5.7x faster than what could have been accomplished on-premise ! The outcomes? 1. Easier collaboration 2. Faster time to science 3. Cost-effective: On-premise was prohibitively expensive 4. No longer constrained by on-premise capacity 5. Scientists focusing on Science as opposed to infrastructure ! https://aws.amazon.com/solutions/case-studies/baylor/ !
  • 38. Globally Distributed Compute for LHC on Amazon EC2 Spot http://www.hep.wisc.edu/~dan/talks/EC2SpotForCMS.pdf
  • 39. Globally Distributed Compute for LHC on Amazon EC2 Spot http://www.hep.wisc.edu/~dan/talks/EC2SpotForCMS.pdf
  • 41. Hybrid Computation Model <- Belle on AWS Price/Performance Benchmark • 170M events (3.6 TB) produced in 6 days • Amazon Spot Instances -> $0.20 / 10k events (May, 2010 pricing)
  • 42. High Throughput Computing at Cloud Scale
  • 44. The Square Kilometer Array SKA Mid and Survey Dishes SKA-low Dipoles
  • 45. The Square Kilometer Array SKA Mid and Survey Dishes SKA-low Dipoles
  • 46. SCALING MWA/ASKAP TO SKA1 MWA$ ASKAP$ SKA1)SURVEY$ SKA1)LOW$ SKA1)MID$ Digi$ser(Output( 0.12(Tbit/s( ( 1x( 69(Tbit/s( ( 575x( ( 184(Tbit/s( ( 1533x( 10(Tbit/s([3]( ( 83x( 17.1(Tbit/s([1]( ( 143x( ( Input(to( Science(Data( Processor( 3.2(Gbit/s( ( 1x( 20(Gbit/s( ( 6.3x( 37360(Gbit/s([2]( ( 11675x( 6736(Gbit/s([2]( ( 2105x( 26000(Gbit/s([2]( ( 8125x( ( Archived( Science(Data( Products( 3(PB/year( (25%(duty( cycle)([4]( 5(PB/year( ( Exabytes(per(year( ( Exabytes(per(year( ( Exabytes(per(year( ( [1] From SADT ConsorAum Technical Development Plan (SKA-­‐TEL.SADT-­‐PROP_TECH-­‐RED-­‐001) [2] From SKA1 System Baseline Design (SKA-­‐TEL-­‐SKO-­‐DD-­‐001) [3] From LFAA Technical DescripAon (AADC-­‐TEL.LFAA.SE.MGT-­‐AADC-­‐PL-­‐002) [4] MWA is archiving correlator output data, not science data processor output data
  • 47. What the SKA is now saying about AWS “No one’s ever built anything this big before, and we really don’t understand the ins and outs of operating it…Cloud systems — which provide on-demand, ‘elastic’ access to shared, remote computing resources — would provide an amount of flexibility for the project that buying dedicated hardware might not.” - SKA architect Tim Cornwell, Nature May 27, 2014 Professor Peter Quinn, Director of the International Center for Radio Astronomy Research
  • 50. SkyNet - ICRAR EPO System for the SKA ICRAR SkyNet Architecture ! http://aws.amazon.com/solutions/case-studies/icrar/
  • 51. National Database for Autism Research http://ndar.nih.gov/cloud_overview.html All autism research funded by NIMH must be publicly accessible ! NDAR provides a web interface to query the aggregate data set
  • 52. Download and Copy RDS images videos files binaries snapshots
  • 53. Download and Copy RDS images videos files binaries snapshots
  • 54. Download and Copy RDS images videos files binaries snapshots
  • 55. Download and Copy RDS images videos files binaries snapshots
  • 56. Access in the Cloud RDS images videos files binaries snapshots
  • 57. Access in the Cloud RDS images videos files binaries snapshots RDS
  • 58. Access in the Cloud RDS images videos files binaries snapshots RDS RDS
  • 59. Access in the Cloud RDS images videos files binaries snapshots RDS RDS RDS RDS RDS RDS RDS
  • 60. Computation in the Cloud RDS images videos files binaries snapshots
  • 61. Where is this Going? 1. Researcher conducts experiment
  • 62. Where is this Going? 1. Researcher conducts experiment 2. Experimental data and results uploaded to the cloud along with reproducible machine images images videos files binaries snapshots EC2 Amazon Elastic MapReduce EC2 Machine Amazon S3 Images
  • 63. Where is this Going? 1. Researcher conducts experiment 2. Experimental data and results uploaded to the cloud along with reproducible machine images 3. Reviewers leverage cloud resources to reproduce and validate results. images videos files binaries snapshots EC2 Amazon Elastic MapReduce EC2 Machine Amazon S3 Images aws.amazon.com/managementconsole Instance Type M3 Extra Large Number of Instances 1,000 Availability Zone US-West-2b Launch
  • 64. Where is this Going? 1. Researcher conducts experiment 2. Experimental data and results uploaded to the cloud along with reproducible machine images 3. Reviewers leverage cloud resources to reproduce and validate results. 4. Results published in a peer-reviewed journal, including references (e.g. DOIs) to cloud data and AMIs images videos files binaries snapshots EC2 Amazon Elastic MapReduce EC2 Machine Amazon S3 Images Scientific Journal
  • 65. Where is this Going? 1. Researcher conducts experiment 2. Experimental data and results uploaded to the cloud along with reproducible machine images 3. Reviewers leverage cloud resources to reproduce and validate results. 4. Results published in a peer-reviewed journal, including references (e.g. DOIs) to cloud data and AMIs 5. Other researchers use these resources as a jumping off point for further research, also publishing their results in the cloud.
  • 66. Where is this Going? 1. Researcher conducts experiment 2. Experimental data and results uploaded to the cloud along with reproducible machine images 3. Reviewers leverage cloud resources to reproduce and validate results. 4. Results published in a peer-reviewed journal, including references (e.g. DOIs) to cloud data and AMIs 5. Other researchers use these resources as a jumping off point for further research, also publishing their results in the cloud. 6. Automated workflows re-run the original researcher’s experiments on the new data using the original machine images. Interesting results trigger notifications and further review.
  • 67. AWS Research Grants AWS.amazon.com/grants
  • 68. AWS Public Data Sets AWS.amazon.com/datasets
  • 69. Multi-Spectrum Atlas of the Galactic Plane •Collaboration between AWS, Caltech/IPAC and USC/ISI •All images are publicly accessible via direct download and VAO APIs •16 wavelength infrared atlas spanning 1μm to 70μm •Datasets from GLIMPSE and MIPSGAL, 2MASS, MSX, WISE •Spatial sampling of 1 arcsec with ±180° longitude and ±20° latitude •Mosaics generated by Caltech’s Montage (http://montage.ipac.caltech.edu) •Compute resources coordinated by USC’s Pegasus (http://pegasus.isi.edu/)
  • 72. AWS and the NASA Earth eXchange (NEX) images videos files binaries snapshots ! •Data and Software now available to those without @nasa.gov email addresses ! •Enables crowd-sourced citizen science applications like those found on the Zooniverse EC2 Amazon Elastic MapReduce EC2 Machine Amazon S3 Images •National Climate Assesment datasets hosted on AWS ! •Machine images, tutorials and hosted workshops provided by NASA Climate Researchers
  • 76. EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 Ames On-demand access to effectively Specialized supercomputing limitless resources resources NASA Researcher Small-scale shared compute
  • 78. One server running for 1000 hours has the same cost as 1000 servers running for one hour.
  • 80. Cloud for Scalability ! Cloud for Global Collaboration ! Cloud for Big Data