SlideShare a Scribd company logo
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Darin Briskman
Data Scientist & Engineer
AWS Databases, Analytics, and Machine Learning
Building with Purpose-Built Databases
Managed services transform operations
Power, HVAC, net
Rack & stack
Server maintenance
OS patches
DB software patches
Database backups
High Availability
DB software installs
OS installation
Scaling
Operating
Databases
in AWS
App optimization
you
Power, HVAC, net
Rack & stack
Server maintenance
OS patches
DB software patches
Database backups
Scaling
High Availability
DB software installs
OS installation
you
App optimization
Operating
Databases
in the Old World
A one size fits all database doesn’t fit anyone
Modern Applications Need Purpose-Built Databases
Users: 1M+
Data volume: TB–PB–EB
Locality: Global
Performance: Milliseconds–microseconds
Request Rate: Millions
Access: Mobile, IoT, devices
Scale: Up-out-in
Economics: Pay as you go
Developer Access: Instant API access
Relational Key-value Document
In-memory Graph Search
AWS purpose-built strategy
The right tool for the right job
Relational
Non-Relational
Aurora RDS
ElastiCacheDynamoDB
Key-value Document
Neptune
Graph
Data models and common use cases
Relational Key-value Document In-memory Graph Search
Referential
integrity, ACID
transactions,
schema-on-write
Low-latency,
key look-ups with
high throughput
and fast ingestion
of data
Indexing and
storing
documents with
support
for query on
any attribute
Microseconds
latency,
key-based
queries, and
specialized
data structures
Creating and
navigating
relations
between data
easily
and quickly
Indexing and
searching
semistructured
logs and data
ERP, medical records,
CRM, finance
Real-time bidding,
shopping cart, IoT device
tracking
Content management,
personalization, mobile
Leaderboards, real-time
analytics, caching
Fraud detection, social
networking,
recommendation engine
Product catalog,
help/FAQs, full-text
Amazon Aurora
Amazon RDS
Amazon Redshift
Amazon Amazon Amazon
for Redis &
Memcached
Amazon Neptune Amazon
Elasticsearch
AWS databases and analytics
B r o a d a n d d e e p p o r t f o l i o , p u r p o s e - b u i l t f o r b u i l d e r s
Data Lake
S3/Glacier Glue
(ETL & Data Catalog)
Data Movement
Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams
Non-Relational Databases
DynamoDB
ElastiCache
(Redis, Memcached)
Neptune
(Graph)
Analytics
DW | Big Data Processing | Interactive
Redshift EMR Athena
Kinesis
Analytics
Elasticsearch
Service
Real-time
Relational Databases
RDS
Aurora
Business Intelligence & Machine Learning
QuickSight SageMaker Comprehend
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Old-guard commercial databases
Very
expensive
Proprietary Lock-in Punitive
licensing
You’ve
got mail
Moving to open source
database engines
+
Commercial-grade performance and reliability?
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scale compute
and storage with a few
clicks; minimal
downtime for your
application
Automatic Multi-AZ
data replication;
automated backup,
snapshots, and failover
Data encryption at rest
and in transit; industry
compliance and
assurance programs
Running many databases with Amazon RDS
Managed Relational Database Service with choice
Managed &Automated
Deploy and maintain
hardware, OS, and DB
software; built-in monitoring
Performant & scalable Available & durable Secure & compliant
Fully compatible with
PostgreSQL and MySQL,
with 3x – 5x the throughput
Storage volume striped across
hundreds of storage nodes
distributed over 3 different
availability zones
Six copies of data on SSD, two
copies in each availability zone, to
protect against AZ+1 failures
Continuous backup to Amazon
S3 (built for 99.999999999%
durability)
Master Replica Replica Replica
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
Large relational databases with Amazon Aurora
Scale-out, distributed, multi-tenant architecture
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Database Migration Service
Migrating
Databases
to AWS
90,000+
Databases migrated
Migrate between on-premises and AWS
Migrate between databases
Data replication for zero-downtime migration
Automated schema conversion
Amazon DynamoDB
Fully-managed nonrelational database for any scale
Secure
Encryption at rest and transit
Fine-grained access control
PCI, HIPAA, FIPS140-2 eligible
High performance
Fast, consistent performance
Virtually unlimited throughput
Virtually unlimited storage
Fully managed
Maintenance-free
Serverless
Auto scaling
Backup and restore
Global tables GlobalTables
High-performance, globally distributed
applications
Multi-region redundancy
and resiliency
Easy to set up and no application
rewrites required
Managed services for open source software
Redis, Memcached, Elasticsearch, Apache Hadoop, etc.
Fully managed
AWS manages all hardware
and software setup,
configuration, monitoring
Extreme performance
In-memory data store and cache
for sub-millisecond response times
Easily scalable
Non-disruptive scaling
up and down to
meet changing
demands
Amazon ElastiCache
Open and Secure
Direct access to open-source APIs
Secure access withVPC
Amazon Elasticsearch Service
Apache Hadoop Ecosystem
19 open-source frameworks
Low costs with S3 storage and Spot
Amazon EMR
Highly connected data best represented in a graph
Relational model
Foreign keys used to represent relationships
Queries can involve nesting & complex joins
Performance can degrade as datasets grow
Graph model
Relationships are first-order citizens
Write queries that navigate the graph
Results returned quickly, even on large datasets
Amazon Neptune
Fully managed graph database
Fast & Scalable ReliableFlexible
Store billions of relationships; query
with millisecond latency
Six replicas of your data
across three AZs with full
backup and restore
Build powerful queries
with
Gremlin and SPARQL
Supports Apache
TinkerPop & W3C RDF
graph models
Gremlin
SPARQL
Open Standards
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Analytics Services
Any analytic workload, any scale, at the lowest possible cost
Insights
Analytics
Data Lake
Data Movement
QuickSight SageMaker
Glue
(ETL & Data Catalog)
S3/Glacier
(Storage)
Redshift
+Spectrum
EMR Athena
Elasticsearch
service
Kinesis Data Analytics
Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams
Real-time
Comprehend
DW Big data processing Interactive
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Lakes on AWS
Most ways to bring data in
Unmatched durability and availability at EB scale
Best security, compliance, and audit capabilities
Run any analytics on same data without movement
Scale storage and compute independently
OLTP ERP CRM LOB
Data Warehouse
Business
Intelligence
Data Lake
10011000010010101110010
10101110010101000010111
11011010
0011110010110010110
0100011000010
Devices Web Sensors Social
Catalog
Machine
Learning
DW
Queries
Big data
processing
Interactive Real-time
Amazon Redshift Spectrum
Extend the data warehouse to exabytes of data in S3 data lake
• Exabyte Redshift SQL queries against Amazon
S3
• Join data across Redshift and S3
• Scale compute and storage separately
• Stable query performance and unlimited
concurrency
• CSV, ORC, Grok, Avro, & Parquet data formats
• Pay only for the amount of data scanned
S3 data lakeAmazon
Redshift data
Redshift Spectrum
query engine
Amazon Elasticsearch Service
Fully-managed.
Deploy production-ready
clusters in minutes
Open
Direct access to Amazon ES
open-source APIs; supports
Logstash and Kibana
Secure
Secure access with VPC to
keep all traffic within AWS
network
Available
Zone awareness replicates
data between two AZs;
automatically monitors &
replaces failed nodes
Managed service to deploy, secure, operate, and scale Amazon Elasticsearch Service
Customers use Amazon ES for log analytics, full-text search & application monitoring
Fully Managed
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon QuickSight
• Fast, easy to use, serverless analytics at 1/10th the cost of traditional BI
Empower
everyone
Seamless
connectivity
Fast analysis Serverless
400,000+ Customers using AWS DB & Analytics Services
M L F R A M E W O R K S
Put Machine Learning in the hands of
every developer
M L S E R V I C E S
A M A Z O N
S A G E M A K E R
A I S E R V I C E S
R E K O G N I T I O N R E K O G N I T I O N
V I D E O
P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E X
V isio n S p eec h L an g u ag e C h at b o t s &
C o n t ac t C en t er s
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you
Please rate my session.
https://amzn.to/ottawa-sessions
Track: Spotlight
Session: 10:00 AM - Building with Purpose-Built Databases
How did we do?
https://amzn.to/ottawa-summit

More Related Content

What's hot

Building a modern data platform in AWS
Building a modern data platform in AWSBuilding a modern data platform in AWS
Building a modern data platform in AWS
Amazon Web Services
 
Building-a-Modern-Data-Platform-in-the-Cloud.pdf
Building-a-Modern-Data-Platform-in-the-Cloud.pdfBuilding-a-Modern-Data-Platform-in-the-Cloud.pdf
Building-a-Modern-Data-Platform-in-the-Cloud.pdf
Amazon Web Services
 
Data Warehousing in the Cloud - AWS Summit Sydney
Data Warehousing in the Cloud - AWS Summit SydneyData Warehousing in the Cloud - AWS Summit Sydney
Data Warehousing in the Cloud - AWS Summit Sydney
Amazon Web Services
 
Using Data Lakes
Using Data LakesUsing Data Lakes
Using Data Lakes
Amazon Web Services
 
Building Data Lake on AWS | AWS Floor28
Building Data Lake on AWS | AWS Floor28Building Data Lake on AWS | AWS Floor28
Building Data Lake on AWS | AWS Floor28
Amazon Web Services
 
Best Practices for Building Your Data Lake on AWS
Best Practices for Building Your Data Lake on AWSBest Practices for Building Your Data Lake on AWS
Best Practices for Building Your Data Lake on AWS
Amazon Web Services
 
Visualize your data in Data Lake with AWS Athena and AWS Quicksight Hands-on ...
Visualize your data in Data Lake with AWS Athena and AWS Quicksight Hands-on ...Visualize your data in Data Lake with AWS Athena and AWS Quicksight Hands-on ...
Visualize your data in Data Lake with AWS Athena and AWS Quicksight Hands-on ...
Amazon Web Services
 
Big Data@Scale
 Big Data@Scale Big Data@Scale
Big Data@Scale
Amazon Web Services
 
Modern Cloud Data Warehousing ft. Intuit: Optimize Analytics Practices (ANT20...
Modern Cloud Data Warehousing ft. Intuit: Optimize Analytics Practices (ANT20...Modern Cloud Data Warehousing ft. Intuit: Optimize Analytics Practices (ANT20...
Modern Cloud Data Warehousing ft. Intuit: Optimize Analytics Practices (ANT20...
Amazon Web Services
 
What's New with Amazon Redshift ft. Dow Jones (ANT350-R) - AWS re:Invent 2018
What's New with Amazon Redshift ft. Dow Jones (ANT350-R) - AWS re:Invent 2018What's New with Amazon Redshift ft. Dow Jones (ANT350-R) - AWS re:Invent 2018
What's New with Amazon Redshift ft. Dow Jones (ANT350-R) - AWS re:Invent 2018
Amazon Web Services
 
Preparing Data for the Lake
Preparing Data for the LakePreparing Data for the Lake
Preparing Data for the Lake
Amazon Web Services
 
Build a High-Performance, Cloud-Native, Open-Source Platform on AWS & Save Mi...
Build a High-Performance, Cloud-Native, Open-Source Platform on AWS & Save Mi...Build a High-Performance, Cloud-Native, Open-Source Platform on AWS & Save Mi...
Build a High-Performance, Cloud-Native, Open-Source Platform on AWS & Save Mi...
Amazon Web Services
 
Implementazione di una soluzione Data Lake.pdf
Implementazione di una soluzione Data Lake.pdfImplementazione di una soluzione Data Lake.pdf
Implementazione di una soluzione Data Lake.pdfAmazon Web Services
 
The AWS Big Data Platform – Overview
The AWS Big Data Platform – OverviewThe AWS Big Data Platform – Overview
The AWS Big Data Platform – Overview
Amazon Web Services
 
ABD206-Building Visualizations and Dashboards with Amazon QuickSight
ABD206-Building Visualizations and Dashboards with Amazon QuickSightABD206-Building Visualizations and Dashboards with Amazon QuickSight
ABD206-Building Visualizations and Dashboards with Amazon QuickSight
Amazon Web Services
 
Building Your Data Lake on AWS - Level 200
Building Your Data Lake on AWS - Level 200Building Your Data Lake on AWS - Level 200
Building Your Data Lake on AWS - Level 200
Amazon Web Services
 
Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...
Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...
Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...
Amazon Web Services
 
Building Data Lakes with AWS
Building Data Lakes with AWSBuilding Data Lakes with AWS
Building Data Lakes with AWS
Amazon Web Services
 
Building-a-Data-Lake-on-AWS
Building-a-Data-Lake-on-AWSBuilding-a-Data-Lake-on-AWS
Building-a-Data-Lake-on-AWS
Amazon Web Services
 
Build Data Engineering Platforms with Amazon EMR (ANT204) - AWS re:Invent 2018
Build Data Engineering Platforms with Amazon EMR (ANT204) - AWS re:Invent 2018Build Data Engineering Platforms with Amazon EMR (ANT204) - AWS re:Invent 2018
Build Data Engineering Platforms with Amazon EMR (ANT204) - AWS re:Invent 2018
Amazon Web Services
 

What's hot (20)

Building a modern data platform in AWS
Building a modern data platform in AWSBuilding a modern data platform in AWS
Building a modern data platform in AWS
 
Building-a-Modern-Data-Platform-in-the-Cloud.pdf
Building-a-Modern-Data-Platform-in-the-Cloud.pdfBuilding-a-Modern-Data-Platform-in-the-Cloud.pdf
Building-a-Modern-Data-Platform-in-the-Cloud.pdf
 
Data Warehousing in the Cloud - AWS Summit Sydney
Data Warehousing in the Cloud - AWS Summit SydneyData Warehousing in the Cloud - AWS Summit Sydney
Data Warehousing in the Cloud - AWS Summit Sydney
 
Using Data Lakes
Using Data LakesUsing Data Lakes
Using Data Lakes
 
Building Data Lake on AWS | AWS Floor28
Building Data Lake on AWS | AWS Floor28Building Data Lake on AWS | AWS Floor28
Building Data Lake on AWS | AWS Floor28
 
Best Practices for Building Your Data Lake on AWS
Best Practices for Building Your Data Lake on AWSBest Practices for Building Your Data Lake on AWS
Best Practices for Building Your Data Lake on AWS
 
Visualize your data in Data Lake with AWS Athena and AWS Quicksight Hands-on ...
Visualize your data in Data Lake with AWS Athena and AWS Quicksight Hands-on ...Visualize your data in Data Lake with AWS Athena and AWS Quicksight Hands-on ...
Visualize your data in Data Lake with AWS Athena and AWS Quicksight Hands-on ...
 
Big Data@Scale
 Big Data@Scale Big Data@Scale
Big Data@Scale
 
Modern Cloud Data Warehousing ft. Intuit: Optimize Analytics Practices (ANT20...
Modern Cloud Data Warehousing ft. Intuit: Optimize Analytics Practices (ANT20...Modern Cloud Data Warehousing ft. Intuit: Optimize Analytics Practices (ANT20...
Modern Cloud Data Warehousing ft. Intuit: Optimize Analytics Practices (ANT20...
 
What's New with Amazon Redshift ft. Dow Jones (ANT350-R) - AWS re:Invent 2018
What's New with Amazon Redshift ft. Dow Jones (ANT350-R) - AWS re:Invent 2018What's New with Amazon Redshift ft. Dow Jones (ANT350-R) - AWS re:Invent 2018
What's New with Amazon Redshift ft. Dow Jones (ANT350-R) - AWS re:Invent 2018
 
Preparing Data for the Lake
Preparing Data for the LakePreparing Data for the Lake
Preparing Data for the Lake
 
Build a High-Performance, Cloud-Native, Open-Source Platform on AWS & Save Mi...
Build a High-Performance, Cloud-Native, Open-Source Platform on AWS & Save Mi...Build a High-Performance, Cloud-Native, Open-Source Platform on AWS & Save Mi...
Build a High-Performance, Cloud-Native, Open-Source Platform on AWS & Save Mi...
 
Implementazione di una soluzione Data Lake.pdf
Implementazione di una soluzione Data Lake.pdfImplementazione di una soluzione Data Lake.pdf
Implementazione di una soluzione Data Lake.pdf
 
The AWS Big Data Platform – Overview
The AWS Big Data Platform – OverviewThe AWS Big Data Platform – Overview
The AWS Big Data Platform – Overview
 
ABD206-Building Visualizations and Dashboards with Amazon QuickSight
ABD206-Building Visualizations and Dashboards with Amazon QuickSightABD206-Building Visualizations and Dashboards with Amazon QuickSight
ABD206-Building Visualizations and Dashboards with Amazon QuickSight
 
Building Your Data Lake on AWS - Level 200
Building Your Data Lake on AWS - Level 200Building Your Data Lake on AWS - Level 200
Building Your Data Lake on AWS - Level 200
 
Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...
Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...
Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...
 
Building Data Lakes with AWS
Building Data Lakes with AWSBuilding Data Lakes with AWS
Building Data Lakes with AWS
 
Building-a-Data-Lake-on-AWS
Building-a-Data-Lake-on-AWSBuilding-a-Data-Lake-on-AWS
Building-a-Data-Lake-on-AWS
 
Build Data Engineering Platforms with Amazon EMR (ANT204) - AWS re:Invent 2018
Build Data Engineering Platforms with Amazon EMR (ANT204) - AWS re:Invent 2018Build Data Engineering Platforms with Amazon EMR (ANT204) - AWS re:Invent 2018
Build Data Engineering Platforms with Amazon EMR (ANT204) - AWS re:Invent 2018
 

Similar to Building with Purpose - Built Databases: Match Your Workloads to the Right Database

AWS Database and Analytics State of the Union
AWS Database and Analytics State of the UnionAWS Database and Analytics State of the Union
AWS Database and Analytics State of the Union
Amazon Web Services
 
Using AWS Purpose-Built Databases to Modernize your Applications
Using AWS Purpose-Built Databases to Modernize your ApplicationsUsing AWS Purpose-Built Databases to Modernize your Applications
Using AWS Purpose-Built Databases to Modernize your Applications
Amazon Web Services
 
AWS Database and Analytics State of the Union
AWS Database and Analytics State of the UnionAWS Database and Analytics State of the Union
AWS Database and Analytics State of the Union
Amazon Web Services
 
Architecting a Serverless Data Lake on AWS
Architecting a Serverless Data Lake on AWSArchitecting a Serverless Data Lake on AWS
Architecting a Serverless Data Lake on AWS
Amazon Web Services
 
Technology Trends in Data Processing - DAT311 - re:Invent 2017
Technology Trends in Data Processing - DAT311 - re:Invent 2017Technology Trends in Data Processing - DAT311 - re:Invent 2017
Technology Trends in Data Processing - DAT311 - re:Invent 2017
Amazon Web Services
 
How to Choose The Right Database on AWS - Berlin Summit - 2019
How to Choose The Right Database on AWS - Berlin Summit - 2019How to Choose The Right Database on AWS - Berlin Summit - 2019
How to Choose The Right Database on AWS - Berlin Summit - 2019
Randall Hunt
 
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Amazon Web Services
 
AWS reInvent 2018 recap edition
AWS reInvent 2018 recap editionAWS reInvent 2018 recap edition
AWS reInvent 2018 recap edition
Amazon Web Services
 
Building with Purpose-Built Databases: Match Your workload to the Right Database
Building with Purpose-Built Databases: Match Your workload to the Right DatabaseBuilding with Purpose-Built Databases: Match Your workload to the Right Database
Building with Purpose-Built Databases: Match Your workload to the Right Database
AWS Summits
 
AWS Database Services @ Scale
AWS Database Services @ ScaleAWS Database Services @ Scale
AWS Database Services @ Scale
Amazon Web Services
 
Choosing the Right Database (Database Freedom)
Choosing the Right Database (Database Freedom)Choosing the Right Database (Database Freedom)
Choosing the Right Database (Database Freedom)Amazon Web Services
 
Building with Purpose-Built Databases: Match Your Workload to the Right Database
Building with Purpose-Built Databases: Match Your Workload to the Right DatabaseBuilding with Purpose-Built Databases: Match Your Workload to the Right Database
Building with Purpose-Built Databases: Match Your Workload to the Right Database
Amazon Web Services
 
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Amazon Web Services
 
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Amazon Web Services
 
Databases - EBC on the road Brazil Edition [Portuguese]
Databases - EBC on the road Brazil Edition [Portuguese]Databases - EBC on the road Brazil Edition [Portuguese]
Databases - EBC on the road Brazil Edition [Portuguese]
Amazon Web Services
 
Managed Relational Databases
Managed Relational DatabasesManaged Relational Databases
Managed Relational Databases
Amazon Web Services
 
Build Data Lakes and Analytics on AWS: Patterns & Best Practices
Build Data Lakes and Analytics on AWS: Patterns & Best PracticesBuild Data Lakes and Analytics on AWS: Patterns & Best Practices
Build Data Lakes and Analytics on AWS: Patterns & Best Practices
Amazon Web Services
 
Build Data Lakes & Analytics on AWS: Patterns & Best Practices
Build Data Lakes & Analytics on AWS: Patterns & Best PracticesBuild Data Lakes & Analytics on AWS: Patterns & Best Practices
Build Data Lakes & Analytics on AWS: Patterns & Best Practices
Amazon Web Services
 
Building low latency apps with a serverless architecture and in-memory data I...
Building low latency apps with a serverless architecture and in-memory data I...Building low latency apps with a serverless architecture and in-memory data I...
Building low latency apps with a serverless architecture and in-memory data I...
AWS Germany
 

Similar to Building with Purpose - Built Databases: Match Your Workloads to the Right Database (20)

AWS Database and Analytics State of the Union
AWS Database and Analytics State of the UnionAWS Database and Analytics State of the Union
AWS Database and Analytics State of the Union
 
Using AWS Purpose-Built Databases to Modernize your Applications
Using AWS Purpose-Built Databases to Modernize your ApplicationsUsing AWS Purpose-Built Databases to Modernize your Applications
Using AWS Purpose-Built Databases to Modernize your Applications
 
AWS Database and Analytics State of the Union
AWS Database and Analytics State of the UnionAWS Database and Analytics State of the Union
AWS Database and Analytics State of the Union
 
Architecting a Serverless Data Lake on AWS
Architecting a Serverless Data Lake on AWSArchitecting a Serverless Data Lake on AWS
Architecting a Serverless Data Lake on AWS
 
Technology Trends in Data Processing - DAT311 - re:Invent 2017
Technology Trends in Data Processing - DAT311 - re:Invent 2017Technology Trends in Data Processing - DAT311 - re:Invent 2017
Technology Trends in Data Processing - DAT311 - re:Invent 2017
 
How to Choose The Right Database on AWS - Berlin Summit - 2019
How to Choose The Right Database on AWS - Berlin Summit - 2019How to Choose The Right Database on AWS - Berlin Summit - 2019
How to Choose The Right Database on AWS - Berlin Summit - 2019
 
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
 
AWS reInvent 2018 recap edition
AWS reInvent 2018 recap editionAWS reInvent 2018 recap edition
AWS reInvent 2018 recap edition
 
Data_Analytics_and_AI_ML
Data_Analytics_and_AI_MLData_Analytics_and_AI_ML
Data_Analytics_and_AI_ML
 
Building with Purpose-Built Databases: Match Your workload to the Right Database
Building with Purpose-Built Databases: Match Your workload to the Right DatabaseBuilding with Purpose-Built Databases: Match Your workload to the Right Database
Building with Purpose-Built Databases: Match Your workload to the Right Database
 
AWS Database Services @ Scale
AWS Database Services @ ScaleAWS Database Services @ Scale
AWS Database Services @ Scale
 
Choosing the Right Database (Database Freedom)
Choosing the Right Database (Database Freedom)Choosing the Right Database (Database Freedom)
Choosing the Right Database (Database Freedom)
 
Building with Purpose-Built Databases: Match Your Workload to the Right Database
Building with Purpose-Built Databases: Match Your Workload to the Right DatabaseBuilding with Purpose-Built Databases: Match Your Workload to the Right Database
Building with Purpose-Built Databases: Match Your Workload to the Right Database
 
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
 
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
 
Databases - EBC on the road Brazil Edition [Portuguese]
Databases - EBC on the road Brazil Edition [Portuguese]Databases - EBC on the road Brazil Edition [Portuguese]
Databases - EBC on the road Brazil Edition [Portuguese]
 
Managed Relational Databases
Managed Relational DatabasesManaged Relational Databases
Managed Relational Databases
 
Build Data Lakes and Analytics on AWS: Patterns & Best Practices
Build Data Lakes and Analytics on AWS: Patterns & Best PracticesBuild Data Lakes and Analytics on AWS: Patterns & Best Practices
Build Data Lakes and Analytics on AWS: Patterns & Best Practices
 
Build Data Lakes & Analytics on AWS: Patterns & Best Practices
Build Data Lakes & Analytics on AWS: Patterns & Best PracticesBuild Data Lakes & Analytics on AWS: Patterns & Best Practices
Build Data Lakes & Analytics on AWS: Patterns & Best Practices
 
Building low latency apps with a serverless architecture and in-memory data I...
Building low latency apps with a serverless architecture and in-memory data I...Building low latency apps with a serverless architecture and in-memory data I...
Building low latency apps with a serverless architecture and in-memory data I...
 

More from Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Amazon Web Services
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Amazon Web Services
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
Amazon Web Services
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
Amazon Web Services
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
Amazon Web Services
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
Amazon Web Services
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Amazon Web Services
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
Amazon Web Services
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Amazon Web Services
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
Amazon Web Services
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
Amazon Web Services
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Amazon Web Services
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
Amazon Web Services
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Amazon Web Services
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWSAmazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckAmazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without serversAmazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
Amazon Web Services
 

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

Building with Purpose - Built Databases: Match Your Workloads to the Right Database

  • 1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Darin Briskman Data Scientist & Engineer AWS Databases, Analytics, and Machine Learning Building with Purpose-Built Databases
  • 2. Managed services transform operations Power, HVAC, net Rack & stack Server maintenance OS patches DB software patches Database backups High Availability DB software installs OS installation Scaling Operating Databases in AWS App optimization you Power, HVAC, net Rack & stack Server maintenance OS patches DB software patches Database backups Scaling High Availability DB software installs OS installation you App optimization Operating Databases in the Old World
  • 3. A one size fits all database doesn’t fit anyone Modern Applications Need Purpose-Built Databases Users: 1M+ Data volume: TB–PB–EB Locality: Global Performance: Milliseconds–microseconds Request Rate: Millions Access: Mobile, IoT, devices Scale: Up-out-in Economics: Pay as you go Developer Access: Instant API access Relational Key-value Document In-memory Graph Search
  • 4. AWS purpose-built strategy The right tool for the right job Relational Non-Relational Aurora RDS ElastiCacheDynamoDB Key-value Document Neptune Graph
  • 5. Data models and common use cases Relational Key-value Document In-memory Graph Search Referential integrity, ACID transactions, schema-on-write Low-latency, key look-ups with high throughput and fast ingestion of data Indexing and storing documents with support for query on any attribute Microseconds latency, key-based queries, and specialized data structures Creating and navigating relations between data easily and quickly Indexing and searching semistructured logs and data ERP, medical records, CRM, finance Real-time bidding, shopping cart, IoT device tracking Content management, personalization, mobile Leaderboards, real-time analytics, caching Fraud detection, social networking, recommendation engine Product catalog, help/FAQs, full-text Amazon Aurora Amazon RDS Amazon Redshift Amazon Amazon Amazon for Redis & Memcached Amazon Neptune Amazon Elasticsearch
  • 6. AWS databases and analytics B r o a d a n d d e e p p o r t f o l i o , p u r p o s e - b u i l t f o r b u i l d e r s Data Lake S3/Glacier Glue (ETL & Data Catalog) Data Movement Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams Non-Relational Databases DynamoDB ElastiCache (Redis, Memcached) Neptune (Graph) Analytics DW | Big Data Processing | Interactive Redshift EMR Athena Kinesis Analytics Elasticsearch Service Real-time Relational Databases RDS Aurora Business Intelligence & Machine Learning QuickSight SageMaker Comprehend
  • 7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Old-guard commercial databases Very expensive Proprietary Lock-in Punitive licensing You’ve got mail
  • 8. Moving to open source database engines + Commercial-grade performance and reliability?
  • 9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Scale compute and storage with a few clicks; minimal downtime for your application Automatic Multi-AZ data replication; automated backup, snapshots, and failover Data encryption at rest and in transit; industry compliance and assurance programs Running many databases with Amazon RDS Managed Relational Database Service with choice Managed &Automated Deploy and maintain hardware, OS, and DB software; built-in monitoring Performant & scalable Available & durable Secure & compliant
  • 10. Fully compatible with PostgreSQL and MySQL, with 3x – 5x the throughput Storage volume striped across hundreds of storage nodes distributed over 3 different availability zones Six copies of data on SSD, two copies in each availability zone, to protect against AZ+1 failures Continuous backup to Amazon S3 (built for 99.999999999% durability) Master Replica Replica Replica Availability Zone 1 Availability Zone 2 Availability Zone 3 Large relational databases with Amazon Aurora Scale-out, distributed, multi-tenant architecture
  • 11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Database Migration Service Migrating Databases to AWS 90,000+ Databases migrated Migrate between on-premises and AWS Migrate between databases Data replication for zero-downtime migration Automated schema conversion
  • 12. Amazon DynamoDB Fully-managed nonrelational database for any scale Secure Encryption at rest and transit Fine-grained access control PCI, HIPAA, FIPS140-2 eligible High performance Fast, consistent performance Virtually unlimited throughput Virtually unlimited storage Fully managed Maintenance-free Serverless Auto scaling Backup and restore Global tables GlobalTables High-performance, globally distributed applications Multi-region redundancy and resiliency Easy to set up and no application rewrites required
  • 13. Managed services for open source software Redis, Memcached, Elasticsearch, Apache Hadoop, etc. Fully managed AWS manages all hardware and software setup, configuration, monitoring Extreme performance In-memory data store and cache for sub-millisecond response times Easily scalable Non-disruptive scaling up and down to meet changing demands Amazon ElastiCache Open and Secure Direct access to open-source APIs Secure access withVPC Amazon Elasticsearch Service Apache Hadoop Ecosystem 19 open-source frameworks Low costs with S3 storage and Spot Amazon EMR
  • 14. Highly connected data best represented in a graph Relational model Foreign keys used to represent relationships Queries can involve nesting & complex joins Performance can degrade as datasets grow Graph model Relationships are first-order citizens Write queries that navigate the graph Results returned quickly, even on large datasets
  • 15. Amazon Neptune Fully managed graph database Fast & Scalable ReliableFlexible Store billions of relationships; query with millisecond latency Six replicas of your data across three AZs with full backup and restore Build powerful queries with Gremlin and SPARQL Supports Apache TinkerPop & W3C RDF graph models Gremlin SPARQL Open Standards
  • 16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Analytics Services Any analytic workload, any scale, at the lowest possible cost Insights Analytics Data Lake Data Movement QuickSight SageMaker Glue (ETL & Data Catalog) S3/Glacier (Storage) Redshift +Spectrum EMR Athena Elasticsearch service Kinesis Data Analytics Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams Real-time Comprehend DW Big data processing Interactive
  • 17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Data Lakes on AWS Most ways to bring data in Unmatched durability and availability at EB scale Best security, compliance, and audit capabilities Run any analytics on same data without movement Scale storage and compute independently OLTP ERP CRM LOB Data Warehouse Business Intelligence Data Lake 10011000010010101110010 10101110010101000010111 11011010 0011110010110010110 0100011000010 Devices Web Sensors Social Catalog Machine Learning DW Queries Big data processing Interactive Real-time
  • 18. Amazon Redshift Spectrum Extend the data warehouse to exabytes of data in S3 data lake • Exabyte Redshift SQL queries against Amazon S3 • Join data across Redshift and S3 • Scale compute and storage separately • Stable query performance and unlimited concurrency • CSV, ORC, Grok, Avro, & Parquet data formats • Pay only for the amount of data scanned S3 data lakeAmazon Redshift data Redshift Spectrum query engine
  • 19. Amazon Elasticsearch Service Fully-managed. Deploy production-ready clusters in minutes Open Direct access to Amazon ES open-source APIs; supports Logstash and Kibana Secure Secure access with VPC to keep all traffic within AWS network Available Zone awareness replicates data between two AZs; automatically monitors & replaces failed nodes Managed service to deploy, secure, operate, and scale Amazon Elasticsearch Service Customers use Amazon ES for log analytics, full-text search & application monitoring Fully Managed
  • 20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon QuickSight • Fast, easy to use, serverless analytics at 1/10th the cost of traditional BI Empower everyone Seamless connectivity Fast analysis Serverless
  • 21. 400,000+ Customers using AWS DB & Analytics Services
  • 22. M L F R A M E W O R K S Put Machine Learning in the hands of every developer M L S E R V I C E S A M A Z O N S A G E M A K E R A I S E R V I C E S R E K O G N I T I O N R E K O G N I T I O N V I D E O P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E X V isio n S p eec h L an g u ag e C h at b o t s & C o n t ac t C en t er s
  • 23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you Please rate my session. https://amzn.to/ottawa-sessions Track: Spotlight Session: 10:00 AM - Building with Purpose-Built Databases How did we do? https://amzn.to/ottawa-summit

Editor's Notes

  1. The Cloud is a fully managed environment Using managed services frees you to focus on your mission instead of minutia of operational details AWS customers use a broad and deep selection of fully managed services to support work at any scale
  2. Using a single database for every purpose doesn’t work in today’s world of large scale Developers choose relational databases, nonrelational databases, analytical databases, machine learning, visualization and other tools to do the job AWS customers need flexibility, scale, and performance Application requirements are changing, and a one-size-fits-all approach of using a relational database as the only data store for your application no longer works. An increasing number of developers now choose relational and nonrelational databases that are purpose-built to meet their application’s specific needs, like storing key-value pairs and documents. If you are building an online retail site, you might choose a relational database to help ensure financial transactions related to an order are 100% correct. If you want a shopping cart that can provide consistent single-digit-millisecond latency with virtually limitless scale to handle the likes of Amazon Prime Day, you can choose a key-value database. If you want to show more personalized recommendations like accessories that friends of your users purchased, you can choose a graph database. The characteristics of cloud applications is driving why different database services exist today. Developers are always looking for the right tool for the job and because they are so easy to gain access to, developers can enjoy a rich development flexibility without sacrificing scale and performance.
  3. No one truck is right for every job, which is why there are tractor-trailers, pickup trucks, earth movers, and delivery trucks No one data tool is right for every job, which is why there are AWS services for both relational and nonrelational data This is one way to think about the different database choices developers have, as they think about using the right tool for the job often considering speed, scale, & programmability. If I were standing here today, saying we have one database, that can literally do everything, it might be like me saying, you can use one vehicle that is a utility, earth mover, delivery truck and long-haul cargo mover that is equally efficient in all aspects of the job its being used for.
  4. Relational data is important for many customers A lot of new development uses nonrelational data The key is using the right tool for the job
  5. AWS offers services across the full range of data tools Business Intelligence and Machine Learning tools help make sense of data Database services are the right tools for relational, nonrelational, and analytic jobs The data lake combines data tools with scalable storage and data governance Data movement tools let you get data between different formats and places
  6. Many customers are still trapped using old-guard databases such as Oracle, Microsoft SQL Server, or IBM Db2 These databases are expensive, with proprietary lock-in and punitive licensing Old-guard vendors will conduct audits (“you’ve got mail – audit coming!”) whenever they want to force extra payments AWS helps customers escape for all of these limitations The relational database world has been an unpleasant place for most customers. These customers have had to deal with old-guard database providers that are expensive, proprietary, have high lock-in, and impose punitive licensing terms. And, you occasionally get an email that says you’re being audited!
  7. Open Source relational databases are widely-used and well supported AWS customers want the low cost and community support of Open Source and the high performance and reliability of commercial databases You can get fully managed Open Source with performance and reliability with Amazon RDS and Amazon Aurora However, getting the same performance on open source databases as you get on commercial-grade databases is difficult. We have done this at Amazon.com, but it has required a lot of tuning. Customers that are moving to open source databases have asked us for the performance of commercial-grade databases with the pricing, freedom, and flexibility of open source databases. That's why we spent a few years building Amazon Aurora.
  8. RDS is fully managed, automating patching, backup, high availability, encryption, and security. With up to 16TB per database instance, run hundreds or thousands of DB instances w/out large staff commitments. You can use both Open Source (MySQL, MariaDB, PostgreSQL) and Commercial (Oracle, Microsoft SQL Server) databases
  9. Aurora combines Open Source interfaces of MySQL and PostgreSQL with enterprise-class scalability, performance, and reliability All data is stored in six copies across three independent physical facilities (we call these Availability Zones) Aurora is high performance, with 3x – 5x the throughput of MySQL or PostgreSQL
  10. DMS enables customers to copy and move databases without downtime No lock-in at AWS: you can move data to the Cloud, off of the Cloud, and between Clouds AWS customers have used DMS to migrate over 90,000 databases In addition to offering a broad portfolio of purpose-built database services, AWS makes it easy for you to migrate your database to the cloud. The AWS Database Migration Service (DMS) helps customers securely migrate their databases to AWS with minimal or no downtime. The source database remains fully operational during the migration, causing no interruption to applications that rely on that database. DMS can migrate your data from most widely used commercial and open-source databases. DMS supports migrations such as Oracle to Oracle migrations, as well as migrations between different database platforms, such as Oracle to Amazon Aurora. DMS offers six months of free usage for migrations to Amazon Aurora, Amazon DynamoDB, and Amazon Redshift. For large databases, where terabytes of data need to be migrated, you can use AWS Snowball, a petabyte-scale data transport service that uses secure appliances to transfer data into and out of AWS.
  11. Amazon.com runs our own business largely with DynamoDB DynamoDB is fully managed, with consistent high performance at any scale, with some customers storing over 1 PB in a single DynamoDB table Global Tables enable true active-active databases across the world Amazon DynamoDB is a fully managed NoSQL database service running in the AWS Cloud. The complexity of running a massively scalable, distributed NoSQL database is managed by the service itself, allowing software developers to focus on building applications rather than managing infrastructure. NoSQL databases are designed for scale, but their architectures are sophisticated, and there can be significant operational overhead in running a large NoSQL cluster. Instead of having to become experts in advanced distributed computing concepts, the developer need only to learn DynamoDB’s straightforward API using the SDK for the programming language of choice. In addition to being easy to use, DynamoDB is also cost effective. With DynamoDB, you pay for the storage you are consuming and the IO throughput you have provisioned. It is designed to scale elastically while maintaining high performance. When the storage and throughput requirements of an application are low, only a small amount of capacity needs to be provisioned in the DynamoDB service. As the number of users of an application grows and the required IO  throughput increases, additional capacity can be provisioned on the fly. This enables an application to seamlessly grow to support millions of users making thousands of concurrent requests to the database every second. Finally, DynamoDB is secure with support for end to end encryption and fine grained access control.
  12. AWS also has fully managed solutions for other popular Open Source packages ElastiCache provides managed redis and memcached for sub-millisecond in-memory data Elasticsearch is a managed search engine, both for text search and log analytics Nineteen Apache Hadoop packages are managed with Amazon EMR, including Hbase, Spark, Presto, Hive, and others While millisecond latency works for many applications, microsecond latency is required by real-time, data-intensive applications. For example, gaming leaderboards capture the scores of millions of online players every time their scores change and re-rank the players in real-time. A common solution for this is an in-memory data store where millions of data records can be written and accessed in microseconds. In-memory data stores can also function as stand-alone databases for transient data such as website user authentication tokens that expire at the end of the user session. Redis and Memcached are two popular choices for in-memory data stores. Redis is an open-source, in-memory, key-value store that offers a variety of built-in data structures such as sorted sets, lists, and geospatial data, making it faster to develop applications. Memcached is an open-source in-memory caching system that is easy to use. However, Redis and Memcached lack enterprise features such as scalability and reliability, and that's why we built Amazon ElastiCache.   Amazon ElastiCache offers Redis and Memcached as fully managed services. It automates management tasks such as hardware provisioning, software patching, setup, configuration, monitoring, and backups, making it easy to run Redis and Memcached on AWS. ElastiCache can scale-out, scale-in, and scale-up to meet changing application demands. ElastiCache for Redis allows you add up to five read replicas across multiple availability zones, enabling you to easily scale read capacity. And, if the primary read/write node fails, ElastiCache for Redis automatically promotes one of the read replicas to be the primary node, making your application more reliable. For scaling write capacity, ElastiCache for Redis lets you partition your data across multiple primary nodes, and distributes write requests across these nodes. ElastiCache for Redis provides encryption-at-rest and encryption-in-transit, helping you secure your data. Key benefits of Amazon ElastiCache include: Redis and Memcached Compatible With Amazon ElastiCache, you get native access to Redis or Memcached in-memory environments. This enables compatibility with your existing tools and applications. Extreme Performance Amazon ElastiCache works as an in-memory data store and cache to support the most demanding applications requiring sub-millisecond response times. By utilizing an end-to-end optimized stack running on customer dedicated nodes, Amazon Elasticache provides you secure, blazing fast performance. Fully Managed You no longer need to perform management tasks such as hardware provisioning, software patching, setup, configuration, monitoring, failure recovery, and backups. ElastiCache continuously monitors your clusters to keep your workloads up and running so that you can focus on higher value application development. Easily Scalable Amazon ElastiCache can scale-out, scale-in, and scale-up to meet fluctuating application demands.  Write and memory scaling is supported with sharding. Replicas provide read scaling.
  13. Relational models, ironically, are not great for representing the relationships between data Graph models are great for highly connected data, such as recommendation engines and social networks The mathematics behind graph databases go back to the 1700’s, but actually implementing them has been difficult and expensive Now consider an app like for recommendations, where someone wants to recommend some kind of organization, entity or sites of a certain type, in a particular city, that for example some of their connections also liked. To do this, you need to put together a lot of connected datasets. To know the users, their connections & their likes. You also need to know the organizations, entities and their attributes, such as museums, or schools, or places to eat. In a rel. model, you end up with mult. tables, mult foreign keys, and soon, queries slow down & maint. is most difficult. Alternatively, you can use an open source graph database, which are hard to scale and lack ent capabilities such as HA Or commercial graph databases which are expensive, often proprietary, and you have to choose from graph models. What we want is a graph DB compat. w/ldg graph models, open APIs, & also fast, reliable, scalable, & cost effective.
  14. Amazon Neptune enable very large graph databases at low cost, high performance, and reliability Just like the other databases, Neptune is fully managed, with AWS providing patching, backup, high availability, and high performance Amazon Neptune is a fast, reliable, fully-man. graph DB. It makes it EZ to build & run apps that work w/highly conn. datasets. It has a purpose-built, high-perf graph DB engine optimized for storing B’s of relationships & querying the graph w/ms latency. Neptune supports the popular graph models, Property Graph and W3C's RDF And their respective query languages Apache TinkerPop Gremlin and SPARQL. Neptune is fully mang’d w/HA, read replicas, point-in-time recovery, continuous b/up to Amazon S3, & repl. across AZs. Neptune is secure, w/support for encrypt. @ rest & in transit.
  15. As data sizes grow, so does the need for analytics AWS provides services for the full range of analytics, from visualization to queries to storage to security Customers like NETFLIX, Zillow, NASDAQ, Yelp, iRobot, and FINRA trust AWS to run their analytics workloads. AWS Big Data and Analytic services enable customers to easily run any analytic workload (batch, ad-hoc, real-time, IoT and predictive analytics) at any scale (GB to TB to PB to EB), in a secure fashion, at the lowest possible cost. AWS provides a highly scalable, available, secure, and cost effective data store that lets you store data in its native format and easily extract value from your data. (what people call a Data Lake). This is particularly true now that many customers see much of their new data created directly in the cloud, with Amazon S3 being home to the vast majority of it. With much more operating experience and scale, and a broader set analytics services available than anywhere else, S3 and our portfolio of Big Data & Analytics services is the clear number one choice for you to build your data lake and analytics solutions with.
  16. Data Lakes extend analytics to any scale, from Gigabytes to Exabytes With a Data Lake, you can use any analytical approach, from dashboards to reporting to predictive analytics powered by machine learning These enable cust’s to build cloud data lakes to analyze all their data w/broadest set of analytical approaches including ML. As a result, there are more organizations running their data lakes and analytics on AWS than anywhere else.
  17. Redshift includes Spectrum, a feature that enables queries against data stored in files Spectrum allows queries of very large data sets (Petabytes and Exabytes) at a low cost Amazon Redshift Spectrum lets you to run Amazon Redshift SQL queries against exabytes of data in Amazon S3. Extending the analytic power of Amazon Redshift beyond data stored on local disks in your DW. You can query vast am’ts of unstruct data in your Amazon S3 “Data Lake” – w/out having to load or transform any data. It uses sophisticated query optimization across 000s of nodes so results are fast – even w/large data sets & complex queries. It dir. queries data in S3 using open data fmts like CSV, Grok, ORC, Parquet, RCFile, SequenceFile, TextFile, TSV, & others. It supports the SQL syntax of Amazon Redshift so you can run sophisticated queries using the same BI tools you use today. You can also run queries that span data stored locally in Amazon Redshift and your full data sets in Amazon S3. You only pay for queries you run, w/S3 rates for data storage and Amazon Redshift instance rates for the clusters used.
  18. Elasticsearch is an Open Source search engine that is popular, but hard to install and maintain Amazon Elasticsearch service is fully managed, making it easy to deploy, secure, operate and scale Elasticsearch Customers use Elasticsearch both for text search and for log analytics Amazon Elasticsearch Service makes it easy to deploy, secure, operate, and scale Elasticsearch This is for log analytics, full text search, application monitoring, and more. It is a fully managed service that delivers Elasticsearch’s easy-to-use APIs and real-time analytics capabilities But with the availability, scalability, and security that production workloads require. It has built-in integrations w/Kibana, Logstash, & AWS so you can go from raw data to actionable insights quickly & securely. These AWS integrations include Amazon Virtual Private Cloud (VPC), Amazon Kinesis Firehose, AWS Lambda, and Amazon CloudWatch  You get direct access to the Elasticsearch open-source API so existing Elasticsearch environments work seamlessly.
  19. Humans use data better with pictures. QuickSight makes it easy to make data understandable for everyone QuickSight can connect to data from almost any source, from AWS services to traditional BI services off-the-Cloud, to Excel spreadsheets QuickSight is low cost and serverless, so you only pay for what you use, as you use it Amazon QuickSight is a fast, cloud-powered business analytics service that makes it easy to build visualizations, perform ad-hoc analysis, and quickly get business insights from your data. Using our cloud-based service you can easily connect to your data, perform advanced analysis, and create stunning visualizations and rich dashboards that can be accessed from any browser or mobile device. Insights for everyone: QuickSight enables self-serve/decentralized analytics in your organization better than any other system out there.  As a Business Analyst can take an Analysis from concept to reality without depending on data engineers.  You can create and prepare datasets, build your analysis, and share/collaborate with little to no intervention from IT or data engineers. Seamless connectivity: Connecting to a data source (especially AWS data source) doesn’t involve back-end coding or complex setups.  You simply click on the data source and enter your credentials, and QuickSight will auto-discover tables that you can select from – taking out the guesswork from selecting the right table to create your datasets.  Then there is “Schedule Refresh”.  If you setup your datasets to Refresh every day, every week  - then you are assured of the latest data when you are looking at your analysis. Fast analysis: Fast interactions with your charts and graphs – The charts and graphs built on SPICE data set are highly responsive.  You can zoom-in/zoom-out, drill through, add filters on the fly with little to no time delay.  Serverless: With QuickSight it is completely serverless. Not only do you not anything installed or deployed for QuickSight, but in combination with S3, and Athena, you can have an end-to-end analytics solution without ever starting or managing servers.
  20. Over 400,000 customers use AWS database and analytics. While the database and analytics markets have been around for a while, with many mature offerings for customers to choose from. We continue to see customers move to the cloud for a number of reasons and our recent growth in the database market is evidence of how rapidly the landscape is changing. Customers move to the cloud to minimize time spent managing infrastructure Customers are choosing the cloud and migrating more and more of their workloads to it. In the next 10 to 15 years, the majority of computing is going to be done in the cloud. In the fullness of time, very few companies will want to own their own data centers, manage infrastructure whether it is compute, storage, databases or analytics. Customers move to the cloud for performance, scale, reliability and cost Increasingly, new applications need to be globally distributed, support millions of users and devices, work with petabytes of data, run 24/7 and be responsive. As customers move to the cloud and to micro-service architectures, developers are increasingly the ones making technology decisions As customers move from monolithic apps to micro-service architectures with loosely coupled components and DevOps cultures. The developers are increasingly making decisions as part of their application development lifecycle on what frameworks and components do they use.
  21. For Developers, AWS offers a number of AI services AI services don’t require any knowledge of AI or ML Now, at the next level up, we have a set of AI services. These are really designed for application developers who don't want to get into the weeds of how machine learning operates. They don't want to have to become deep learning experts. They don't want to have to go and label a whole bunch of data. They just want to get stuff done. And you can see right off the bat you get this broad set of capabilities available to you. For computer vision, using services such as Amazon Recognition, which provides image analysis and facial detection; Recognition Video, which provides video analysis, [pathing], face identification; Speech, using Polly, which is a text-to-speech service, the same service that we use to generate the voice of Alexa; or Transcribe, which does it the other way around. It takes speech and turns it into text.   Or in terms of languages, Translate, which takes text in one language and translates it into another; or Comprehend, which looks inside the document at all of the text, takes all of the context, and then allows you to derive better insights and understanding of what that natural language text looks like, for completely unstructured information. All the way through to Amazon Lex, which is a natural language understanding and speech recognition system that a lot of customers are using to build Chatbots or use in context centers as IVR systems.   And Lex is the exact same beating heart that we use to run Alexa for the Echo and other Alexa-enabled devices. So whilst all of these services are designed to work independently and have a broad set of capabilities themselves, the real magic comes as customers start to pull them together. And they can be joined together for some common use cases.