SlideShare a Scribd company logo
1 of 75
Download to read offline
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Leadership Session:
AWS Databases and Analytics
Raju Gulabani
Vice President
Databases, Analytics, Machine Learning, & Blockchain
AWS
D A T 2 0 6 - L
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What to expect
Hear
about the new
announcements,
with an emphasis on
news not covered in
Andy’s keynote
1
Understand
our database and
analytics strategy;
our portfolio of
various services &
how they work
together
2
Plan
how you would use
these services by
appreciating how
others use them
3
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Our strategy & our beliefs
1. There is going to be an explosion in data.
2. Cloud will enable a different architecture.
3. One size does not fit all—databases should be
purpose-built.
2010
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Analytics
Our portfolio
Broad and deep portfolio, purpose-built for builders
Redshift
Data warehousing
EMR
Hadoop + Spark
Athena
Interactive analytics
Kinesis Data Analytics
Real time
Elasticsearch Service
Operational Analytics
QuickSight SageMaker
S3/Glacier
Glue
ETL & Data Catalog
Lake Formation
Data Lakes
Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams
Data Movement
Business Intelligence & Machine Learning
Data Lake
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Analytics
Our portfolio
Broad and deep portfolio, purpose-built for builders
QuickSight SageMaker
S3/Glacier
Glue
ETL & Data Catalog
Lake Formation
Data Lakes
Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams
Data Movement
Business Intelligence & Machine Learning
Data Lake
Redshift
Data warehousing
EMR
Hadoop + Spark
Kinesis Data Analytics
Real time
Elasticsearch Service
Operational Analytics
Athena
Interactive analytics
RDS
MySQL, PostgreSQL, MariaDB,
Oracle, SQL Server
Aurora
MySQL, PostgreSQL
DynamoDB
Key value, Document
ElastiCache
Redis, Memcached
Neptune
Graph
Timestream
Time Series
QLDB
Ledger Database
RDS on VMware
Databases
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Our portfolio
Broad and deep portfolio, purpose-built for builders
Redshift
Data warehousing
EMR
Hadoop + Spark
Athena
Interactive analytics
Kinesis Data Analytics
Real time
Elasticsearch Service
Operational Analytics
RDS
MySQL, PostgreSQL, MariaDB,
Oracle, SQL Server
Aurora
MySQL, PostgreSQL
QuickSight SageMaker
DynamoDB
Key value, Document
ElastiCache
Redis, Memcached
Neptune
Graph
Timestream
Time Series
QLDB
Ledger Database
S3/Glacier
Glue
ETL & Data Catalog
Lake Formation
Data Lakes
Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams
Data Movement
Analytics Databases
Business Intelligence & Machine Learning
Data Lake
Managed
Blockchain
Blockchain
Templates
Blockchain
RDS on VMware
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Three type of projects
Quickly build new
apps in the cloud
Gain new
insights
“Lift and shift” existing
apps to the cloud
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Three type of projects
Quickly build new
apps in the cloud
Gain new
insights
“Lift and shift” existing
apps to the cloud
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Traditionally, analytics looked like this
Relational data
GBs-TBs scale [not designed for PB/EBs]
Expensive: Large initial capex + $10K-$50K/TB/year
90% of data was thrown away because of cost
OLTP ERP CRM LOB
Data Warehouse
Business Intelligence
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Our beliefs
1. All data has value. No data should be thrown away.
2. All employees should have access to all data (subject
to company access rules).
2010
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Snowball
Snowmobile Kinesis
Data Firehose
Kinesis
Data Streams
S3
Redshift
EMR
Athena Kinesis
Elasticsearch Service
Data lakes on AWS
Kinesis
Video Streams
AI Services
QuickSight
Exabyte scale
Store and analyze relational and non-relational data
Purpose-built analytics tools
Cost effective
• Store at 2.3 cents per GB-month in Amazon S3
• Query with Amazon Athena at ½ cent per GB scanned
• DW with Amazon Redshift for $1,000/TB/year
Give access to everyone
• Amazon QuickSight: $0.30 for 30 minutes of use
CHALLENGE
Need to create constant feedback
loop for designers.
Gain up-to-the-minute
understanding of gamer
satisfaction to guarantee gamers
are engaged, resulting in the most
popular game played in the world.
Fortnite | 125+ million players
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Epic Games uses data lakes and analytics
Entire analytics platform running on AWS
Amazon S3 leveraged as a data lake
All telemetry data is collected with Amazon Kinesis
Real-time analytics done through Spark on Amazon EMR,
DynamoDB to create scoreboards and real-time queries
Use Amazon EMR for large batch data processing
Game designers use data to inform their decisions
Game
clients
Game
servers
Launcher
Game
services
N E A R R E A L T I M E P I P E L I N E
N E A R R E A L T I M E P I P E L I N E
Grafana
Scoreboards API
Limited raw data
(real time ad-hoc SQL)
User ETL
(metric definition)
Spark on EMR DynamoDB
NEAR REAL-TIME PIPELINES
BATCH PIPELINES
ETL using
EMR
Tableau/BI
Ad-hoc SQLS3
(Data lake)
Kinesis
APIs
Databases
S3
Other
sources
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Lake Formation (sign up for the preview)
Build a secure data lake in days
Move, store, catalog, and
clean your data faster
Move, store, catalog,
and clean your data faster
with machine learning
Enforce security policies
across multiple services
Enforce security policies across
multiple services
Gain and manage new
insights
Empower analyst and data
scientist to gain and manage
new insights
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How it works
Data lakes and analytics on AWS
S3
IAM KMS
OLTP
ERP
CRM
LOB
Devices
Web
Sensors
Social Kinesis
Build data lakes quickly
• Identify, crawl, and catalog sources
• Ingest and clean data
• Transform into optimal formats
Simplify security management
• Enforce encryption
• Define access policies
• Implement audit login
Enable self-service and combined analytics
• Analysts discover all data available for analysis
from a single data catalog
• Use multiple analytics tools over the same data
Athena
Redshift
AI Services
EMR
QuickSight
Data
catalog
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift
Highly scalable cloud data warehouse at 10x the performance and 1/10th the cost
of traditional data warehouses
Virtually unlimited
concurrency
Extends your
data lake
Dynamically scales to
support virtually unlimited
number of concurrent users
and growing data volumes
Analyze exabytes of data in the
Amazon S3 data lake together
with petabytes of data loaded
into Amazon Redshift’s high
performance SSDs
10x performance 1/10th the cost
Get faster time-to-insight for all
types of analytics workloads;
powered by machine learning,
columnar storage and MPP
Start at $0.25 per hour,
scale out as low as $1,000
per terabyte per year
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Automatically spins up additional clusters on-demand
Handles virtually unlimited number of concurrent users
Accrued minutes make it free for most customers
Amazon Redshift Concurrency Scaling (Preview)
Consistently fast performance at virtually unlimited concurrency
Redshift Managed
S3
Cluster Leader
Node
Data Data
Caching Layer
Cluster Leader
Node
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift Spectrum
Extend the data warehouse to exabytes of data in S3 data lake
S3 data lakeRedshift data
Amazon Redshift Spectrum
query engine
Exabyte Redshift SQL queries against Amazon S3
Join data across Amazon Redshift and Amazon S3
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift Elastic Resize (GA)
Quickly scale up or down to increase performance on-demand
Cluster Leader
Node
Data Data Data Data
Redshift Managed
S3
Add/remove additional nodes to cluster in minutes
Available today
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift is >3x faster than 6 months ago
100%
181%
237%
284%
350%
Redshift 6 months ago Redshift July 2018 Redshift Aug 2018 Redshift Sep 2018 Redshift Oct 2018
Queriesperhour
asa%ofRedshift6monthsago
Normalized Queries Per Hour (QPH)
(assuming Redshift’s QPH 6 months ago = 100%; higher is better)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
“20 percent of our queries
now complete in less than one
second. Best of all, we didn’t
have to change anything to
get this speed-up with
Redshift, which supports our
mission-critical workloads.”
-Greg Rokita, Executive Director of
Technology, Edmunds
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fastest: up to 16x faster
100%
34%
6%
As a % of Amazon Redshift’s queries per hour
Based on the cloud DW benchmark derived from TPC-DS 3 TB dataset, 4-node cluster
Queries per hour
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Most cost effective: 1/10 the cost
$ per hour
(As a % of Amazon Redshift’s $ per hour with a 3yr RI)
Based on the cloud DW benchmark derived from TPC-DS 3 TB dataset, 4-node cluster
2683%
500%
320%
211%
100%
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CHALLENGE
Needed to analyze data to find
insights, identify opportunities, and
evaluate business performance.
The Oracle DW did not scale, was
difficult to maintain, and costly.
SOLUTION
Deployed a data lake with Amazon S3,
and run analytics with Amazon
Redshift, Amazon Redshift Spectrum,
and Amazon EMR.
Result: They doubled the data stored
(100PB), lowered costs, and was able
to gain insights faster.
50 PB of data
600,000 analytics jobs/day
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Equinox Fitness migrated from Teradata to Redshift
Maximilian
(ELT scripts)
Spark
on EMR
Redshift
S3
Clickstream
Cycling logs
Club
management
software
Applications
Social
Redshift
Spectrum
EMR
Athena
Equinox
apps
3rd party
apps
Migrated from Teradata data warehouse
Built a DW with Redshift and data lake with S3
Analytics on data lake with Amazon Athena,
Amazon Redshift Spectrum, and Amazon EMR
Increased user productivity to move faster
Amazon Redshift costs ~20% of its original
Teradata maintenance & support
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon QuickSight
First BI service with pay-per-session pricing for everyone in your organization
Serverless, cloud-powered BI service (no servers to manage)
Scale from 10s of users to 100s of thousands of users
Pay only for what you use
• Readers: $0.30/30 min session with a $5/user/month max
• Authors: $18/month/Author
Integrates with S3, Athena, Redshift, RDS, Aurora, & EMR
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Directory Service
Microsoft AD
Custom Date Format Dashboard Save As Aggregate Calculations Readers Groups
Private VPC
25 GB SPICE
tables
Spark and Presto Connector Scheduled refresh Just In Time Provisioning One-click upgrade
Search Totals Excel Custom Range
100+
new features released since
launch
Federated SSO
Athena connector Export to CSV S3 Analytics
Week Aggregation Aurora PostgreSQL Calculations in SPICE
Cross Account
S3 Access
Aggregate Filters Hourly refresh
Row level security Hourly refresh
10K Filter Values On-screen controls
Redshift Spectrum
Support
KPI Chart
Spark Connector
AWS Directory Service
AD Connector
Tabular Reports Data labels
URL Actions
Combo Charts
Audit logging
with CloudTrail
Geospatial maps Count Distinct Parameters Relative Date Filters Filter Groups
Table calculations Snowflake Connector SaaS Connectors Teradata Connector HIPAA PCI compliance
Amazon QuickSight has been innovating quickly
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon QuickSight—embedded dashboards
Supercharge your applications with embedded dashboards
Fully interactive with drill down, filtering, & external links
No servers to manage, no long-term commitments
Pay for usage with pay-per-session reader pricing
Easy embedding with JavaScript SDK
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Embedded NFL Next Gen Stats Dashboards
“With the Amazon QuickSight Readers and
pay-per-session pricing, we are able to
extend these secure, customized and easy
to use dashboards for each club without
having to provision servers or manage
infrastructure – all while only paying for
actual usage.”
Matt Swensson
Vice President, Emerging Products and Technology
Real-time stats for NFL games
Embedded in NFL Next Gen Stats Portal
Shared with 100s of users across NFL,
32 clubs and broadcast partners
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon QuickSight is used by customers at the largest scale
One of the world’s largest
metals and mining companies
deployed Amazon QuickSight
with its critical risk
management (CRM) solution
to ensure employee safety.
Thousands of employees
use its CRM globally.
Uses Amazon QuickSight
embedded in its Converge
Platform, a governance, risk,
and compliance healthcare
solution. Tens of thousands
of users across 900
healthcare organizations
use this platform.
Amazon.com is using
Amazon QuickSight
company-wide
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon QuickSight—ML Insights (Preview)
Automated business insights powered by ML and natural language
ML-powered anomaly detection
ML-powered forecasting
Auto-narratives
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Discover all the hidden trends and
anomalies on millions of metrics
Amazon QuickSight—ML Insights
Example: anomaly detection
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
“Sales for office supplies in APAC
was 15% above expected.”
Amazon QuickSight—ML Insights
Example: anomaly detection
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
“SMB Segment was the top
contributor.”
Amazon QuickSight—ML Insights
Example: anomaly detection
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
“It’s significant because SMB
typically only accounts for 30% of
sales.”
Amazon QuickSight—ML Insights
Example: anomaly detection
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
QuickSight ML-powered forecasting Traditional BI forecasting
Captures seasonality and upward trends
Automatically excludes bad data
High confidence band
Captures only seasonality
Missing upward trend
Confidence band influenced by bad data
QuickSight ML Insights vs. traditional BI forecasting
VS.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Insights in plain language narrative
Embedded within your dashboard
No more staring at dashboards for hours!
Fully customizable to meet every need
No coding needed. Easy-to-use UI templates.
Amazon QuickSight—ML Insights
Auto-narratives
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS analytics
Any analytic workload, any scale, at the lowest possible cost
Redshift
Data warehousing
EMR
Hadoop + Spark
Athena
Interactive analytics
Kinesis Data Analytics
Real time
Elasticsearch Service
Operational Analytics
Analytics
QuickSight
Business Intelligence
S3/Glacier Glue
ETL & Data Catalog
Lake Formation
Data Lakes
Data Lake
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
More places to learn about analytics services
Amazon Athena
Amazon
Redshift
Amazon
Elasticsearch
Service
AWS Lake
Formation
ANT401-R2: Deep Dive and Best Practices for
Amazon Redshift | Fri 11:30
ANT 401-R1: Deep Dive and Best Practices for
Amazon Redshift | Thu 4:00
ANT202-R1: Modern Cloud Data Warehousing ft.
Intuit | Thu 2:30
ANT350-R1: What's New with Amazon Redshift ft.
McDonald's | Thu 3:15
Sessions that already occured: ANT202-R,
ANT350-R
ANT323-R1: Build Your Own Log Analytics
Solutions on AWS | Thur 11:30
Sessions that already occured: ANT334-R,
ANT334-R1, ANT323-RANT203
Introduction to AWS Lake Formation - Build a
Secure Data Lake in Days | Wed 7:00pm | Venetian,
Level 4, Delfino 4005.
Sessions that already occurred: ANT205
ANT340-R1: A Deep Dive into What's New with
Amazon EMR | Fri 3:00
Sessions that already occurred: ANT204, ANT312,
ANT340-R
Amazon Kinesis
Amazon
QuickSight
AWS Glue
Introducing Amazon Kinesis Data Analytics for
Java applications | Thu 12:15pm | MGM, Level 1,
South Concourse 105
ANT322-R1: High Performance Data Streaming
with Amazon Kinesis: Best Practices | Thu 1:00
ANT 310: Architecting for Real-Time Insights
with Amazon Kinesis | Thu 3:15
Sessions that already occurred: ANT 208,
ANT322-R
Introducing ML-powered insights with Amazon
QuickSight | Wed 1:00pm | Aria East, Level 1,
Joshua 9
ANT311: NFL and Forwood Safety Deploy
Business Analytics at Scale with Amazon
QuickSight | Fri 11:30
Amazon EMR
Sessions that already occured: ANT309, ANT308Sessions from prior days: ANT324
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Customers tell us: they have three type of projects
Quickly build new
apps in the cloud
Gain new
insights
“Lift and shift” existing
apps to the cloud
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Characteristics of modern applications
Internet-scale and transactional
Users: 1M+
Data volume: TB–PB–EB
Locality: Global
Performance: Milliseconds–microseconds
Request Rate: Millions
Access: Mobile, IoT, devices
Scale: Up-out-in
Economics: Pay-as-you-go
Developer access: Instant API accessSocial mediaRide hailing Media streaming Dating
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS databases services
Purpose-built for all your app needs
DynamoDB NeptuneRDS
Aurora CommercialCommunity
Timestream QLDBElastiCache
Relational Key-value Document In-memory Graph Time series Ledger
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon DynamoDB
Fast and flexible key value database service for any scale
Comprehensive
security
Encrypts all data by
default and fully integrates
with AWS Identity and
Access Management for
robust security
Performance at scale
Consistent, single-digit
millisecond response times at
any scale; build applications with
virtually unlimited throughput
Global database for
global users and apps
Build global applications with
fast access to local data by easily
replicating tables across multiple
AWS Regions
Serverless
No hardware provisioning,
software patching, or
upgrades; scales up or down
automatically; continuously
backs up your data
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DynamoDB powers the world’s largest applications
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The need for transactions support
Scenario: Customer purchases an item
“PUT”:
“TableName”: “Orders”,
“OrderStatus”: “Sold”,
“Item”: “Bike”,
“Quantity”: “1”
“PUT”:
“TableName”: “Inventory”,
“Item”: “Bike”,
“Quantity”: “- 1”
Transact-write-items
{
“PUT”:
“TableName”: “Orders”,
“OrderStatus”: “Sold”,
“Item”: “Bike”,
“Quantity”: “1”
“PUT”:
“TableName”: “Inventory”,
“Item”: “Bike”,
“Quantity”: “- 1”
}
If code fails here, we inserted the
order but did not update inventory
If code fails here, we updated some
of our inventory, but not all
Developer needs to write code to
undo a partial operation
Transaction support: The system
ensures everything happens (or not)
VS.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon DynamoDB Transactions (GA)
Build internet-scale apps with ACID transactions
Simplify application
code with ACID
guarantees
Run transactions
for large scale
workloads
Accelerate legacy
migrations
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DynamoDB—read/write capacity on-demand (GA)
No more capacity planning—pay only for what you use
No capacity planning
No need to specify how much
read/write throughput you expect to
use
Ideal for unpredictable workloads
Ramp from zero to tens of thousands of requests
per second on demand
Pay only for what you use
Pay-per-request pricing
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Timestream (sign up for the preview)
Fast, scalable, fully managed time series database
1,000x faster and 1/10th the
cost of relational databases
Collect data at the rate of
millions of inserts per
second (10M/second)
Trillions of
daily events
Adaptive query processing
engine maintains steady,
predictable performance
Analytics optimized
for time series data
Built-in functions for
interpolation, smoothing,
and approximation
Serverless
Automated setup,
configuration, server
provisioning, software patching
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Quantum Ledger Database (QLDB) (Preview)
Fully managed ledger database
Track and verify history of all changes made to your application’s data
Immutable
Maintains a sequenced record of
all changes to your data, which
cannot be deleted or modified;
you have the ability to query and
analyze the full history
Cryptographically
verifiable
Uses cryptography to
generate a secure output
file of your data’s history
Easy to use
Easy to use, letting you
use familiar database
capabilities like SQL APIs
for querying the data
Highly scalable
Executes 2–3x as many
transactions than ledgers
in common blockchain
frameworks
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Managed Blockchain (preview)
Create and manage scalable blockchain networks
Choice of Hyperledger
Fabric or Ethereum
Hyperledger Fabric available
today; Ethereum coming soon
Fully managed
Create blockchain networks
with a few clicks; Manage
them with simple API calls
Easily analyze
blockchain activity
Easy to move data into
QLDB for further analysis
Scalable and
secure
Support thousands of client
applications running
millions of transaction;
integrates with AWS KMS
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon QLBD vs. Amazon Managed Blockchain
Amazon QLDB Amazon Managed Blockchain
Central trusted authority
Track and verify transactions with centralized
ownership
Ledger is immutable and cryptographically
verifiable
Ledger is owned by a trusted and centralized
authority
No central trusted authority
Execute transactions and contracts with
decentralized ownership
Ledger is immutable and cryptographically
verifiable
Each party maintains their own copy of the
ledger
Use Ethereum or Hyperledger Fabric to build
blockchain networks
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
More places to learn about purpose-built databases
Building modern applications using
DynamoDB Transactions | Wed 7:00pm |
MGM, Level 3, Premier Ballroom 310
Sessions that already occurred: DAT321,
DAT401, DAT201, DAT303, DAT314,
DAT325, DAT320, DAT332
Amazon
DynamoDB
Amazon
Neptune
Amazon
ElastiCache
Amazon
QLDB
Amazon
Managed
Blockchain
Sessions that already
occurred: DAT202, DAT319,
DAT302-R1, DAT302-R
Sessions that already
occurred: DAT403, DAT315,
DAT316
Use Cases for Amazon QLDB | Fri 9:15 |
Mirage, Montego D
Deep Dive on Amazon Blockchain |
Wed 3:15 | Venetian, Level 2,
Titian 2205 - T2
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Customers tell us: they have three type of projects
Quickly build new
apps in the cloud
Gain new
insights
“Lift and shift” existing
apps to the cloud
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Database Migration Service
M I G R A T I N G
D A T A B A S E S
T O A W S
Migrate between on-premises and AWS
Migrate between databases
Automated schema conversion
Data replication for zero
downtime migration
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Customers want to lift and shift to the cloud
Relational
databases
Non-relational
databases
Data
warehouses
Hadoop
and Spark
Redshift EMR
Operational
analytics
Elasticsearch
ServiceAurora DynamoDB
Business
Intelligence
QuickSightRDS
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
>100,000 databases migrated with DMS
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Customers are migrating their workloads to AWS
Verizon is migrating over 1,000 business-critical applications and database backend systems to AWS,
several of which also include the migration of production databases to Amazon Aurora.
Wappa migrated from their Oracle database to Amazon Aurora and improved their
reporting time per user by 75 percent.
Trimble migrated their Oracle databases to Amazon RDS and project they will pay about 1/4th
of what they paid when managing their private infrastructure.
Intuit migrated from Microsoft SQL Server to Amazon Redshift to reduce data-processing timelines
and get insights to decision makers faster and more frequently.
Equinox Fitness migrated its Teradata on-premises data warehouse to Amazon Redshift. They went
from static reports to a modern data lake that delivers dynamic reports.
Eventbrite moved from Cloudera to Amazon EMR and were able to cut costs dramatically, spinning
clusters up/down on-demand and using Spot (saving > 80%) and Reserved Instances.
By December 2018, Amazon.com will have migrated 88% of their Oracle DBs (and 97% of
critical system DBs) moved to Amazon Aurora and Amazon DynamoDB. They also migrated their
50 PB Oracle Data Warehouse to AWS (Amazon S3, Amazon Redshift, and Amazon EMR).
Samsung Electronics migrated their Cassandra clusters to Amazon DynamoDB for their Samsung
Cloud workload with 70% cost savings.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Aurora
MySQL and PostgreSQL compatible relational database built for the cloud
Performance and availability of commercial-grade databases at 1/10th the cost
Availability
and durability
Fault-tolerant, self-healing
storage; six copies of data
across three AZs; continuous
backup to S3
Fully managed
Managed by RDS:
no hardware provisioning,
software patching, setup,
configuration, or backups
Highly secure
Network isolation,
encryption at
rest/transit
Performance
and scalability
5x throughput of standard
MySQL and 3x of standard
PostgreSQL; scale-out up to
15 read replicas
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aurora Global Database (GA)
High-performance database for globally-distributed applications
Single Global Database with cross region replication
Replication typically completes in less than a second
No impact on database performance
Write master in one region and read replicas in other regions
Cross-region disaster recovery
Local read latency for applications with global users
Primary Region Secondary Region
Application
Storage Storage
Replication <1s
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon RDS
Managed relational database service with a choice of six popular database engines
Available & durable
Automatic Multi-AZ data
replication; automated backup,
snapshots, failover
Easy to administer
No need for infrastructure
provisioning, installing and
maintaining DB software
Highly scalable
Scale database compute
and storage with a few
clicks with no
application downtime
Fast & secure
SSD storage and guaranteed
provisioned I/O; data
encryption at rest and
in transit
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Databases in private data centers are still difficult
and expensive to set up and manage
Difficult to set up and
manage databases for
high availability across
multiple nodes
Personnel needed to create
the database image, install
operating system,
packages, and setup
Burdensome to support
multiple versions and
applying patching
?
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon RDS on VMware (Preview)
Managed service for on-premises databases
RDS deployed as a service in on-premises VMware private data centers (vSphere)
Automates management of on-premises databases and hybrid backup and scaling
Available and
durable
Enable hybrid features
and tap into AWS for
high availability, backup,
and restore
Secure and
compliant
Automate management of
databases for workloads that
must remain on-premises to
adhere to strict data policies
Fully managed
Easy to provision, monitor, and
operate relational databases in
your private data center
Scalability and
performance
Scale storage, compute, and
memory of on-premises
databases from a single,
simple interface
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How it works
Amazon RDS on VMware
RDS on VMware
Deploy RDS on VMware
in your private data
center and manage on-
premises databases using
a single RDS interface
RDS interface
Use the Amazon RDS
console, APIs, or CLI to
provision databases
Log in or create
AWS account
Find RDS on VMware in
the console and choose
AWS region
Download and install
the connector
Download & install the RDS
connector in your VMware
vSphere environment to
establish secure VPN
connection between AWS and
your private data center
Automate
database
management
RDS on VMware
automates database
management tasks
including provisioning,
patching, backups and
failover
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
More places to learn about Aurora, RDS, and DMS
Sessions that already occurred:
DAT204-R1, DAT313, DAT318,
DAT204-R, DAT304-R, DAT304-R1,
DAT305-R, DAT305-R1, DAT336
Amazon Aurora
DMS
Amazon RDS Deep Dive on Amazon RDS on VMware
| Thu 1:45 | Aria East, Plaza Level,
Orovada 2
Chalk Talk on Amazon RDS on
VMware | Fri 10:45 | Mirage,
Martinique A
DAT323: Best Practices for Running
SQL Server on Amazon RDS | Thu 1:00
DAT402: Using Performance Insights
to Optimize Database Performance |
Thu 12:15
Sessions that already occurred:
DAT203, DAT322, DAT324, DAT317
Sessions that already
occurred: DAT207
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
When to Use Which Services
Situation Solution
Existing application Use your existing engine on RDS
• MySQL Amazon Aurora, RDS for MySQL
• PostgreSQL Amazon Aurora, RDS for PostgreSQL
• MariaDB Amazon Aurora, RDS for MariaDB
• Oracle Use SCT to determine complexity Amazon Aurora, RDS for Oracle
• SQL Server Use SCT to determine complexity Amazon Aurora, RDS for SQL Server
New application • If you can avoid relational features DynamoDB
• If you need relational features Amazon Aurora
In-memory store/cache • Amazon ElastiCache
Time series data • Amazon Timestream
Track every application change, crypto verifiable.
Have a central trust authority
• Amazon Quantum Ledger Database (QLDB)
Don’t have a trusted central authority • Amazon Managed Blockchain
Data Warehouse & BI • Amazon Redshift, Amazon Redshift Spectrum, and Amazon QuickSight
Adhoc analysis of data in S3 • Amazon Athena and Amazon QuickSight
Apache Spark, Hadoop, HBase (needle in a
haystack type queries)
• Amazon EMR
Log analytics, operational monitoring, & search • Amazon Elasticsearch Service and Amazon Kinesis
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Architect services ground-up for the cloud
and for the explosion of data
Offer a portfolio of purpose-built
services, optimized for your workloads
Help you innovate faster through
managed services
Our approach
Provide services that help you migrate
existing apps and databases to the cloud
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EA is supporting their most
mission critical workloads
with Amazon Redshift.
They were able to increase
query throughput by 2x
over the last 12 months.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Kinesis—real time
Easily collect, process, and analyze video and data streams in real time
Capture, process,
and store video
streams for
analytics
Load data streams
into AWS
data stores
Analyze data
streams with SQL
Build custom
applications that
analyze data
streams
Kinesis Video
Streams
Kinesis
Data Streams
Kinesis Data
Firehose
Kinesis Data
Analytics
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Build real-time streaming analytics in your Java apps
Use Java Integrated Development Environment of choice
Libraries include pre-built stream processing operators
Real-time application can be built in hours
Kinesis Data Analytics—Java support (GA)
Devices
Sensors
IoT
Kinesis Data
Streams
Kinesis Data
Analytics
Java IDE
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tailored recommendations to increase performance & reduce cost
Redshift’s machine learning engine uncovers optimizations
Operations such as vacuum and analyze run behind the scenes
Available today
Redshift ML based auto-tuning (GA)
Clusters always optimized for best performance and lowest cost
AUTO
AUTO
AUTO
ADVISE
ADVISE
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Most enterprise database & analytics cloud customers
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Most startup database & analytics cloud customers

More Related Content

What's hot

Under the Hood: How Amazon Uses AWS Services for Analytics at a Massive Scale...
Under the Hood: How Amazon Uses AWS Services for Analytics at a Massive Scale...Under the Hood: How Amazon Uses AWS Services for Analytics at a Massive Scale...
Under the Hood: How Amazon Uses AWS Services for Analytics at a Massive Scale...Amazon Web Services
 
Build Your First Big Data Application on AWS (ANT213-R1) - AWS re:Invent 2018
Build Your First Big Data Application on AWS (ANT213-R1) - AWS re:Invent 2018Build Your First Big Data Application on AWS (ANT213-R1) - AWS re:Invent 2018
Build Your First Big Data Application on AWS (ANT213-R1) - AWS re:Invent 2018Amazon Web Services
 
Visualization with Amazon QuickSight
Visualization with Amazon QuickSightVisualization with Amazon QuickSight
Visualization with Amazon QuickSightAmazon Web Services
 
Building Data Lake on AWS | AWS Floor28
Building Data Lake on AWS | AWS Floor28Building Data Lake on AWS | AWS Floor28
Building Data Lake on AWS | AWS Floor28Amazon Web Services
 
Building a Modern Data Warehouse - Deep Dive on Amazon Redshift
Building a Modern Data Warehouse - Deep Dive on Amazon RedshiftBuilding a Modern Data Warehouse - Deep Dive on Amazon Redshift
Building a Modern Data Warehouse - Deep Dive on Amazon RedshiftAmazon Web Services
 
The Open Data Lake Platform Brief - Data Sheets | Whitepaper
The Open Data Lake Platform Brief - Data Sheets | WhitepaperThe Open Data Lake Platform Brief - Data Sheets | Whitepaper
The Open Data Lake Platform Brief - Data Sheets | WhitepaperVasu S
 
Aurora Serverless: Scalable, Cost-Effective Application Deployment (DAT336) -...
Aurora Serverless: Scalable, Cost-Effective Application Deployment (DAT336) -...Aurora Serverless: Scalable, Cost-Effective Application Deployment (DAT336) -...
Aurora Serverless: Scalable, Cost-Effective Application Deployment (DAT336) -...Amazon Web Services
 
Extending Analytics Beyond the Data Warehouse, ft. Warner Bros. Analytics (AN...
Extending Analytics Beyond the Data Warehouse, ft. Warner Bros. Analytics (AN...Extending Analytics Beyond the Data Warehouse, ft. Warner Bros. Analytics (AN...
Extending Analytics Beyond the Data Warehouse, ft. Warner Bros. Analytics (AN...Amazon Web Services
 
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...Amazon Web Services
 
Building Data Lakes That Cost Less and Deliver Results Faster - AWS Online Te...
Building Data Lakes That Cost Less and Deliver Results Faster - AWS Online Te...Building Data Lakes That Cost Less and Deliver Results Faster - AWS Online Te...
Building Data Lakes That Cost Less and Deliver Results Faster - AWS Online Te...Amazon Web Services
 
Build Data Engineering Platforms with Amazon EMR (ANT204) - AWS re:Invent 2018
Build Data Engineering Platforms with Amazon EMR (ANT204) - AWS re:Invent 2018Build Data Engineering Platforms with Amazon EMR (ANT204) - AWS re:Invent 2018
Build Data Engineering Platforms with Amazon EMR (ANT204) - AWS re:Invent 2018Amazon Web Services
 
Social Media Analytics with Amazon QuickSight (ANT370) - AWS re:Invent 2018
Social Media Analytics with Amazon QuickSight (ANT370) - AWS re:Invent 2018Social Media Analytics with Amazon QuickSight (ANT370) - AWS re:Invent 2018
Social Media Analytics with Amazon QuickSight (ANT370) - AWS re:Invent 2018Amazon Web Services
 
Data preparation and transformation - Spin your straw into gold - Tel Aviv Su...
Data preparation and transformation - Spin your straw into gold - Tel Aviv Su...Data preparation and transformation - Spin your straw into gold - Tel Aviv Su...
Data preparation and transformation - Spin your straw into gold - Tel Aviv Su...Amazon Web Services
 
Building Serverless Analytics Pipelines with AWS Glue (ANT308) - AWS re:Inven...
Building Serverless Analytics Pipelines with AWS Glue (ANT308) - AWS re:Inven...Building Serverless Analytics Pipelines with AWS Glue (ANT308) - AWS re:Inven...
Building Serverless Analytics Pipelines with AWS Glue (ANT308) - AWS re:Inven...Amazon Web Services
 
Modern Cloud Data Warehousing ft. Intuit: Optimize Analytics Practices (ANT20...
Modern Cloud Data Warehousing ft. Intuit: Optimize Analytics Practices (ANT20...Modern Cloud Data Warehousing ft. Intuit: Optimize Analytics Practices (ANT20...
Modern Cloud Data Warehousing ft. Intuit: Optimize Analytics Practices (ANT20...Amazon Web Services
 

What's hot (20)

Under the Hood: How Amazon Uses AWS Services for Analytics at a Massive Scale...
Under the Hood: How Amazon Uses AWS Services for Analytics at a Massive Scale...Under the Hood: How Amazon Uses AWS Services for Analytics at a Massive Scale...
Under the Hood: How Amazon Uses AWS Services for Analytics at a Massive Scale...
 
Build Your First Big Data Application on AWS (ANT213-R1) - AWS re:Invent 2018
Build Your First Big Data Application on AWS (ANT213-R1) - AWS re:Invent 2018Build Your First Big Data Application on AWS (ANT213-R1) - AWS re:Invent 2018
Build Your First Big Data Application on AWS (ANT213-R1) - AWS re:Invent 2018
 
Visualization with Amazon QuickSight
Visualization with Amazon QuickSightVisualization with Amazon QuickSight
Visualization with Amazon QuickSight
 
Building Data Lake on AWS | AWS Floor28
Building Data Lake on AWS | AWS Floor28Building Data Lake on AWS | AWS Floor28
Building Data Lake on AWS | AWS Floor28
 
Building a Modern Data Warehouse - Deep Dive on Amazon Redshift
Building a Modern Data Warehouse - Deep Dive on Amazon RedshiftBuilding a Modern Data Warehouse - Deep Dive on Amazon Redshift
Building a Modern Data Warehouse - Deep Dive on Amazon Redshift
 
Using Data Lakes
Using Data LakesUsing Data Lakes
Using Data Lakes
 
Data Warehouses and Data Lakes
Data Warehouses and Data LakesData Warehouses and Data Lakes
Data Warehouses and Data Lakes
 
The Open Data Lake Platform Brief - Data Sheets | Whitepaper
The Open Data Lake Platform Brief - Data Sheets | WhitepaperThe Open Data Lake Platform Brief - Data Sheets | Whitepaper
The Open Data Lake Platform Brief - Data Sheets | Whitepaper
 
Aurora Serverless: Scalable, Cost-Effective Application Deployment (DAT336) -...
Aurora Serverless: Scalable, Cost-Effective Application Deployment (DAT336) -...Aurora Serverless: Scalable, Cost-Effective Application Deployment (DAT336) -...
Aurora Serverless: Scalable, Cost-Effective Application Deployment (DAT336) -...
 
Extending Analytics Beyond the Data Warehouse, ft. Warner Bros. Analytics (AN...
Extending Analytics Beyond the Data Warehouse, ft. Warner Bros. Analytics (AN...Extending Analytics Beyond the Data Warehouse, ft. Warner Bros. Analytics (AN...
Extending Analytics Beyond the Data Warehouse, ft. Warner Bros. Analytics (AN...
 
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
 
Preparing Data for the Lake
Preparing Data for the LakePreparing Data for the Lake
Preparing Data for the Lake
 
Building Data Lakes That Cost Less and Deliver Results Faster - AWS Online Te...
Building Data Lakes That Cost Less and Deliver Results Faster - AWS Online Te...Building Data Lakes That Cost Less and Deliver Results Faster - AWS Online Te...
Building Data Lakes That Cost Less and Deliver Results Faster - AWS Online Te...
 
Build Data Engineering Platforms with Amazon EMR (ANT204) - AWS re:Invent 2018
Build Data Engineering Platforms with Amazon EMR (ANT204) - AWS re:Invent 2018Build Data Engineering Platforms with Amazon EMR (ANT204) - AWS re:Invent 2018
Build Data Engineering Platforms with Amazon EMR (ANT204) - AWS re:Invent 2018
 
Social Media Analytics with Amazon QuickSight (ANT370) - AWS re:Invent 2018
Social Media Analytics with Amazon QuickSight (ANT370) - AWS re:Invent 2018Social Media Analytics with Amazon QuickSight (ANT370) - AWS re:Invent 2018
Social Media Analytics with Amazon QuickSight (ANT370) - AWS re:Invent 2018
 
Data preparation and transformation - Spin your straw into gold - Tel Aviv Su...
Data preparation and transformation - Spin your straw into gold - Tel Aviv Su...Data preparation and transformation - Spin your straw into gold - Tel Aviv Su...
Data preparation and transformation - Spin your straw into gold - Tel Aviv Su...
 
Data Warehouses and Data Lakes
Data Warehouses and Data LakesData Warehouses and Data Lakes
Data Warehouses and Data Lakes
 
Building Serverless Analytics Pipelines with AWS Glue (ANT308) - AWS re:Inven...
Building Serverless Analytics Pipelines with AWS Glue (ANT308) - AWS re:Inven...Building Serverless Analytics Pipelines with AWS Glue (ANT308) - AWS re:Inven...
Building Serverless Analytics Pipelines with AWS Glue (ANT308) - AWS re:Inven...
 
How Amazon uses AWS Analytics
How Amazon uses AWS AnalyticsHow Amazon uses AWS Analytics
How Amazon uses AWS Analytics
 
Modern Cloud Data Warehousing ft. Intuit: Optimize Analytics Practices (ANT20...
Modern Cloud Data Warehousing ft. Intuit: Optimize Analytics Practices (ANT20...Modern Cloud Data Warehousing ft. Intuit: Optimize Analytics Practices (ANT20...
Modern Cloud Data Warehousing ft. Intuit: Optimize Analytics Practices (ANT20...
 

Similar to AWS Databases and Analytics Leadership Session

Big Data@Scale_AWSPSSummit_Singapore
Big Data@Scale_AWSPSSummit_SingaporeBig Data@Scale_AWSPSSummit_Singapore
Big Data@Scale_AWSPSSummit_SingaporeAmazon Web Services
 
Amazon Redshift Update and How Equinox Fitness Clubs Migrated to a Modern Dat...
Amazon Redshift Update and How Equinox Fitness Clubs Migrated to a Modern Dat...Amazon Redshift Update and How Equinox Fitness Clubs Migrated to a Modern Dat...
Amazon Redshift Update and How Equinox Fitness Clubs Migrated to a Modern Dat...Amazon Web Services
 
BDA306 Building a Modern Data Warehouse: Deep Dive on Amazon Redshift
BDA306 Building a Modern Data Warehouse: Deep Dive on Amazon RedshiftBDA306 Building a Modern Data Warehouse: Deep Dive on Amazon Redshift
BDA306 Building a Modern Data Warehouse: Deep Dive on Amazon RedshiftAmazon Web Services
 
Analyze your Data Lake, Fast @ Any Scale - AWS Online Tech Talks
Analyze your Data Lake, Fast @ Any Scale - AWS Online Tech TalksAnalyze your Data Lake, Fast @ Any Scale - AWS Online Tech Talks
Analyze your Data Lake, Fast @ Any Scale - AWS Online Tech TalksAmazon Web Services
 
Building a Modern Data Warehouse: Deep Dive on Amazon Redshift - SRV337 - Chi...
Building a Modern Data Warehouse: Deep Dive on Amazon Redshift - SRV337 - Chi...Building a Modern Data Warehouse: Deep Dive on Amazon Redshift - SRV337 - Chi...
Building a Modern Data Warehouse: Deep Dive on Amazon Redshift - SRV337 - Chi...Amazon Web Services
 
AWS Data Lake: data analysis @ scale
AWS Data Lake: data analysis @ scaleAWS Data Lake: data analysis @ scale
AWS Data Lake: data analysis @ scaleAmazon Web Services
 
Implementazione di una soluzione Data Lake.pdf
Implementazione di una soluzione Data Lake.pdfImplementazione di una soluzione Data Lake.pdf
Implementazione di una soluzione Data Lake.pdfAmazon Web Services
 
Value of Data Beyond Analytics by Darin Briskman
 Value of Data Beyond Analytics by Darin Briskman Value of Data Beyond Analytics by Darin Briskman
Value of Data Beyond Analytics by Darin BriskmanSameer Kenkare
 
SRV309 AWS Purpose-Built Database Strategy: The Right Tool for the Right Job
 SRV309 AWS Purpose-Built Database Strategy: The Right Tool for the Right Job SRV309 AWS Purpose-Built Database Strategy: The Right Tool for the Right Job
SRV309 AWS Purpose-Built Database Strategy: The Right Tool for the Right JobAmazon Web Services
 
Using data lakes to quench your analytics fire - AWS Summit Cape Town 2018
Using data lakes to quench your analytics fire - AWS Summit Cape Town 2018Using data lakes to quench your analytics fire - AWS Summit Cape Town 2018
Using data lakes to quench your analytics fire - AWS Summit Cape Town 2018Amazon Web Services
 
How TrueCar Gains Actionable Insights with Splunk Cloud PPT
How TrueCar Gains Actionable Insights with Splunk Cloud PPTHow TrueCar Gains Actionable Insights with Splunk Cloud PPT
How TrueCar Gains Actionable Insights with Splunk Cloud PPTAmazon Web Services
 
AWS Floor 28 - Building Data lake on AWS
AWS Floor 28 - Building Data lake on AWSAWS Floor 28 - Building Data lake on AWS
AWS Floor 28 - Building Data lake on AWSAdir Sharabi
 
Using AWS Purpose-Built Databases to Modernize your Applications
Using AWS Purpose-Built Databases to Modernize your ApplicationsUsing AWS Purpose-Built Databases to Modernize your Applications
Using AWS Purpose-Built Databases to Modernize your ApplicationsAmazon Web Services
 
Database Freedom. Database migration approaches to get to the Cloud - Marcus ...
Database Freedom. Database migration approaches to get to the Cloud - Marcus ...Database Freedom. Database migration approaches to get to the Cloud - Marcus ...
Database Freedom. Database migration approaches to get to the Cloud - Marcus ...Amazon Web Services
 
BDA308 Deep Dive: Log Analytics with Amazon Elasticsearch Service
BDA308 Deep Dive: Log Analytics with Amazon Elasticsearch ServiceBDA308 Deep Dive: Log Analytics with Amazon Elasticsearch Service
BDA308 Deep Dive: Log Analytics with Amazon Elasticsearch ServiceAmazon Web Services
 
Connecting the dots - How Amazon Neptune and Graph Databases can transform yo...
Connecting the dots - How Amazon Neptune and Graph Databases can transform yo...Connecting the dots - How Amazon Neptune and Graph Databases can transform yo...
Connecting the dots - How Amazon Neptune and Graph Databases can transform yo...Amazon Web Services
 
Building a Data Lake in Amazon S3 & Amazon Glacier (STG401-R1) - AWS re:Inven...
Building a Data Lake in Amazon S3 & Amazon Glacier (STG401-R1) - AWS re:Inven...Building a Data Lake in Amazon S3 & Amazon Glacier (STG401-R1) - AWS re:Inven...
Building a Data Lake in Amazon S3 & Amazon Glacier (STG401-R1) - AWS re:Inven...Amazon Web Services
 
Immersion Day - Como simplificar o acesso ao seu ambiente analítico
Immersion Day - Como simplificar o acesso ao seu ambiente analíticoImmersion Day - Como simplificar o acesso ao seu ambiente analítico
Immersion Day - Como simplificar o acesso ao seu ambiente analíticoAmazon Web Services LATAM
 
What's New with Amazon Redshift ft. McDonald's (ANT350-R1) - AWS re:Invent 2018
What's New with Amazon Redshift ft. McDonald's (ANT350-R1) - AWS re:Invent 2018What's New with Amazon Redshift ft. McDonald's (ANT350-R1) - AWS re:Invent 2018
What's New with Amazon Redshift ft. McDonald's (ANT350-R1) - AWS re:Invent 2018Amazon Web Services
 

Similar to AWS Databases and Analytics Leadership Session (20)

Big Data@Scale_AWSPSSummit_Singapore
Big Data@Scale_AWSPSSummit_SingaporeBig Data@Scale_AWSPSSummit_Singapore
Big Data@Scale_AWSPSSummit_Singapore
 
Amazon Redshift Update and How Equinox Fitness Clubs Migrated to a Modern Dat...
Amazon Redshift Update and How Equinox Fitness Clubs Migrated to a Modern Dat...Amazon Redshift Update and How Equinox Fitness Clubs Migrated to a Modern Dat...
Amazon Redshift Update and How Equinox Fitness Clubs Migrated to a Modern Dat...
 
BDA306 Building a Modern Data Warehouse: Deep Dive on Amazon Redshift
BDA306 Building a Modern Data Warehouse: Deep Dive on Amazon RedshiftBDA306 Building a Modern Data Warehouse: Deep Dive on Amazon Redshift
BDA306 Building a Modern Data Warehouse: Deep Dive on Amazon Redshift
 
Analyze your Data Lake, Fast @ Any Scale - AWS Online Tech Talks
Analyze your Data Lake, Fast @ Any Scale - AWS Online Tech TalksAnalyze your Data Lake, Fast @ Any Scale - AWS Online Tech Talks
Analyze your Data Lake, Fast @ Any Scale - AWS Online Tech Talks
 
Building a Modern Data Warehouse: Deep Dive on Amazon Redshift - SRV337 - Chi...
Building a Modern Data Warehouse: Deep Dive on Amazon Redshift - SRV337 - Chi...Building a Modern Data Warehouse: Deep Dive on Amazon Redshift - SRV337 - Chi...
Building a Modern Data Warehouse: Deep Dive on Amazon Redshift - SRV337 - Chi...
 
AWS Data Lake: data analysis @ scale
AWS Data Lake: data analysis @ scaleAWS Data Lake: data analysis @ scale
AWS Data Lake: data analysis @ scale
 
Implementazione di una soluzione Data Lake.pdf
Implementazione di una soluzione Data Lake.pdfImplementazione di una soluzione Data Lake.pdf
Implementazione di una soluzione Data Lake.pdf
 
Value of Data Beyond Analytics by Darin Briskman
 Value of Data Beyond Analytics by Darin Briskman Value of Data Beyond Analytics by Darin Briskman
Value of Data Beyond Analytics by Darin Briskman
 
SRV309 AWS Purpose-Built Database Strategy: The Right Tool for the Right Job
 SRV309 AWS Purpose-Built Database Strategy: The Right Tool for the Right Job SRV309 AWS Purpose-Built Database Strategy: The Right Tool for the Right Job
SRV309 AWS Purpose-Built Database Strategy: The Right Tool for the Right Job
 
Using data lakes to quench your analytics fire - AWS Summit Cape Town 2018
Using data lakes to quench your analytics fire - AWS Summit Cape Town 2018Using data lakes to quench your analytics fire - AWS Summit Cape Town 2018
Using data lakes to quench your analytics fire - AWS Summit Cape Town 2018
 
Big Data@Scale
 Big Data@Scale Big Data@Scale
Big Data@Scale
 
How TrueCar Gains Actionable Insights with Splunk Cloud PPT
How TrueCar Gains Actionable Insights with Splunk Cloud PPTHow TrueCar Gains Actionable Insights with Splunk Cloud PPT
How TrueCar Gains Actionable Insights with Splunk Cloud PPT
 
AWS Floor 28 - Building Data lake on AWS
AWS Floor 28 - Building Data lake on AWSAWS Floor 28 - Building Data lake on AWS
AWS Floor 28 - Building Data lake on AWS
 
Using AWS Purpose-Built Databases to Modernize your Applications
Using AWS Purpose-Built Databases to Modernize your ApplicationsUsing AWS Purpose-Built Databases to Modernize your Applications
Using AWS Purpose-Built Databases to Modernize your Applications
 
Database Freedom. Database migration approaches to get to the Cloud - Marcus ...
Database Freedom. Database migration approaches to get to the Cloud - Marcus ...Database Freedom. Database migration approaches to get to the Cloud - Marcus ...
Database Freedom. Database migration approaches to get to the Cloud - Marcus ...
 
BDA308 Deep Dive: Log Analytics with Amazon Elasticsearch Service
BDA308 Deep Dive: Log Analytics with Amazon Elasticsearch ServiceBDA308 Deep Dive: Log Analytics with Amazon Elasticsearch Service
BDA308 Deep Dive: Log Analytics with Amazon Elasticsearch Service
 
Connecting the dots - How Amazon Neptune and Graph Databases can transform yo...
Connecting the dots - How Amazon Neptune and Graph Databases can transform yo...Connecting the dots - How Amazon Neptune and Graph Databases can transform yo...
Connecting the dots - How Amazon Neptune and Graph Databases can transform yo...
 
Building a Data Lake in Amazon S3 & Amazon Glacier (STG401-R1) - AWS re:Inven...
Building a Data Lake in Amazon S3 & Amazon Glacier (STG401-R1) - AWS re:Inven...Building a Data Lake in Amazon S3 & Amazon Glacier (STG401-R1) - AWS re:Inven...
Building a Data Lake in Amazon S3 & Amazon Glacier (STG401-R1) - AWS re:Inven...
 
Immersion Day - Como simplificar o acesso ao seu ambiente analítico
Immersion Day - Como simplificar o acesso ao seu ambiente analíticoImmersion Day - Como simplificar o acesso ao seu ambiente analítico
Immersion Day - Como simplificar o acesso ao seu ambiente analítico
 
What's New with Amazon Redshift ft. McDonald's (ANT350-R1) - AWS re:Invent 2018
What's New with Amazon Redshift ft. McDonald's (ANT350-R1) - AWS re:Invent 2018What's New with Amazon Redshift ft. McDonald's (ANT350-R1) - AWS re:Invent 2018
What's New with Amazon Redshift ft. McDonald's (ANT350-R1) - AWS re:Invent 2018
 

More from Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateAmazon Web Services
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSAmazon Web Services
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareAmazon Web Services
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAmazon Web Services
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWSAmazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckAmazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without serversAmazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceAmazon Web Services
 

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

AWS Databases and Analytics Leadership Session

  • 1.
  • 2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Leadership Session: AWS Databases and Analytics Raju Gulabani Vice President Databases, Analytics, Machine Learning, & Blockchain AWS D A T 2 0 6 - L
  • 3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. What to expect Hear about the new announcements, with an emphasis on news not covered in Andy’s keynote 1 Understand our database and analytics strategy; our portfolio of various services & how they work together 2 Plan how you would use these services by appreciating how others use them 3
  • 4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Our strategy & our beliefs 1. There is going to be an explosion in data. 2. Cloud will enable a different architecture. 3. One size does not fit all—databases should be purpose-built. 2010
  • 5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Analytics Our portfolio Broad and deep portfolio, purpose-built for builders Redshift Data warehousing EMR Hadoop + Spark Athena Interactive analytics Kinesis Data Analytics Real time Elasticsearch Service Operational Analytics QuickSight SageMaker S3/Glacier Glue ETL & Data Catalog Lake Formation Data Lakes Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams Data Movement Business Intelligence & Machine Learning Data Lake
  • 6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Analytics Our portfolio Broad and deep portfolio, purpose-built for builders QuickSight SageMaker S3/Glacier Glue ETL & Data Catalog Lake Formation Data Lakes Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams Data Movement Business Intelligence & Machine Learning Data Lake Redshift Data warehousing EMR Hadoop + Spark Kinesis Data Analytics Real time Elasticsearch Service Operational Analytics Athena Interactive analytics RDS MySQL, PostgreSQL, MariaDB, Oracle, SQL Server Aurora MySQL, PostgreSQL DynamoDB Key value, Document ElastiCache Redis, Memcached Neptune Graph Timestream Time Series QLDB Ledger Database RDS on VMware Databases
  • 7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Our portfolio Broad and deep portfolio, purpose-built for builders Redshift Data warehousing EMR Hadoop + Spark Athena Interactive analytics Kinesis Data Analytics Real time Elasticsearch Service Operational Analytics RDS MySQL, PostgreSQL, MariaDB, Oracle, SQL Server Aurora MySQL, PostgreSQL QuickSight SageMaker DynamoDB Key value, Document ElastiCache Redis, Memcached Neptune Graph Timestream Time Series QLDB Ledger Database S3/Glacier Glue ETL & Data Catalog Lake Formation Data Lakes Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams Data Movement Analytics Databases Business Intelligence & Machine Learning Data Lake Managed Blockchain Blockchain Templates Blockchain RDS on VMware
  • 8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Three type of projects Quickly build new apps in the cloud Gain new insights “Lift and shift” existing apps to the cloud
  • 9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Three type of projects Quickly build new apps in the cloud Gain new insights “Lift and shift” existing apps to the cloud
  • 10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Traditionally, analytics looked like this Relational data GBs-TBs scale [not designed for PB/EBs] Expensive: Large initial capex + $10K-$50K/TB/year 90% of data was thrown away because of cost OLTP ERP CRM LOB Data Warehouse Business Intelligence
  • 11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Our beliefs 1. All data has value. No data should be thrown away. 2. All employees should have access to all data (subject to company access rules). 2010
  • 12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Snowball Snowmobile Kinesis Data Firehose Kinesis Data Streams S3 Redshift EMR Athena Kinesis Elasticsearch Service Data lakes on AWS Kinesis Video Streams AI Services QuickSight Exabyte scale Store and analyze relational and non-relational data Purpose-built analytics tools Cost effective • Store at 2.3 cents per GB-month in Amazon S3 • Query with Amazon Athena at ½ cent per GB scanned • DW with Amazon Redshift for $1,000/TB/year Give access to everyone • Amazon QuickSight: $0.30 for 30 minutes of use
  • 13. CHALLENGE Need to create constant feedback loop for designers. Gain up-to-the-minute understanding of gamer satisfaction to guarantee gamers are engaged, resulting in the most popular game played in the world. Fortnite | 125+ million players
  • 14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Epic Games uses data lakes and analytics Entire analytics platform running on AWS Amazon S3 leveraged as a data lake All telemetry data is collected with Amazon Kinesis Real-time analytics done through Spark on Amazon EMR, DynamoDB to create scoreboards and real-time queries Use Amazon EMR for large batch data processing Game designers use data to inform their decisions Game clients Game servers Launcher Game services N E A R R E A L T I M E P I P E L I N E N E A R R E A L T I M E P I P E L I N E Grafana Scoreboards API Limited raw data (real time ad-hoc SQL) User ETL (metric definition) Spark on EMR DynamoDB NEAR REAL-TIME PIPELINES BATCH PIPELINES ETL using EMR Tableau/BI Ad-hoc SQLS3 (Data lake) Kinesis APIs Databases S3 Other sources
  • 15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Lake Formation (sign up for the preview) Build a secure data lake in days Move, store, catalog, and clean your data faster Move, store, catalog, and clean your data faster with machine learning Enforce security policies across multiple services Enforce security policies across multiple services Gain and manage new insights Empower analyst and data scientist to gain and manage new insights
  • 16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. How it works Data lakes and analytics on AWS S3 IAM KMS OLTP ERP CRM LOB Devices Web Sensors Social Kinesis Build data lakes quickly • Identify, crawl, and catalog sources • Ingest and clean data • Transform into optimal formats Simplify security management • Enforce encryption • Define access policies • Implement audit login Enable self-service and combined analytics • Analysts discover all data available for analysis from a single data catalog • Use multiple analytics tools over the same data Athena Redshift AI Services EMR QuickSight Data catalog
  • 17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Redshift Highly scalable cloud data warehouse at 10x the performance and 1/10th the cost of traditional data warehouses Virtually unlimited concurrency Extends your data lake Dynamically scales to support virtually unlimited number of concurrent users and growing data volumes Analyze exabytes of data in the Amazon S3 data lake together with petabytes of data loaded into Amazon Redshift’s high performance SSDs 10x performance 1/10th the cost Get faster time-to-insight for all types of analytics workloads; powered by machine learning, columnar storage and MPP Start at $0.25 per hour, scale out as low as $1,000 per terabyte per year
  • 18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Automatically spins up additional clusters on-demand Handles virtually unlimited number of concurrent users Accrued minutes make it free for most customers Amazon Redshift Concurrency Scaling (Preview) Consistently fast performance at virtually unlimited concurrency Redshift Managed S3 Cluster Leader Node Data Data Caching Layer Cluster Leader Node
  • 19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Redshift Spectrum Extend the data warehouse to exabytes of data in S3 data lake S3 data lakeRedshift data Amazon Redshift Spectrum query engine Exabyte Redshift SQL queries against Amazon S3 Join data across Amazon Redshift and Amazon S3
  • 20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Redshift Elastic Resize (GA) Quickly scale up or down to increase performance on-demand Cluster Leader Node Data Data Data Data Redshift Managed S3 Add/remove additional nodes to cluster in minutes Available today
  • 21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Redshift is >3x faster than 6 months ago 100% 181% 237% 284% 350% Redshift 6 months ago Redshift July 2018 Redshift Aug 2018 Redshift Sep 2018 Redshift Oct 2018 Queriesperhour asa%ofRedshift6monthsago Normalized Queries Per Hour (QPH) (assuming Redshift’s QPH 6 months ago = 100%; higher is better)
  • 22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. “20 percent of our queries now complete in less than one second. Best of all, we didn’t have to change anything to get this speed-up with Redshift, which supports our mission-critical workloads.” -Greg Rokita, Executive Director of Technology, Edmunds
  • 23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fastest: up to 16x faster 100% 34% 6% As a % of Amazon Redshift’s queries per hour Based on the cloud DW benchmark derived from TPC-DS 3 TB dataset, 4-node cluster Queries per hour
  • 24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Most cost effective: 1/10 the cost $ per hour (As a % of Amazon Redshift’s $ per hour with a 3yr RI) Based on the cloud DW benchmark derived from TPC-DS 3 TB dataset, 4-node cluster 2683% 500% 320% 211% 100%
  • 25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. CHALLENGE Needed to analyze data to find insights, identify opportunities, and evaluate business performance. The Oracle DW did not scale, was difficult to maintain, and costly. SOLUTION Deployed a data lake with Amazon S3, and run analytics with Amazon Redshift, Amazon Redshift Spectrum, and Amazon EMR. Result: They doubled the data stored (100PB), lowered costs, and was able to gain insights faster. 50 PB of data 600,000 analytics jobs/day
  • 26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Equinox Fitness migrated from Teradata to Redshift Maximilian (ELT scripts) Spark on EMR Redshift S3 Clickstream Cycling logs Club management software Applications Social Redshift Spectrum EMR Athena Equinox apps 3rd party apps Migrated from Teradata data warehouse Built a DW with Redshift and data lake with S3 Analytics on data lake with Amazon Athena, Amazon Redshift Spectrum, and Amazon EMR Increased user productivity to move faster Amazon Redshift costs ~20% of its original Teradata maintenance & support
  • 27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon QuickSight First BI service with pay-per-session pricing for everyone in your organization Serverless, cloud-powered BI service (no servers to manage) Scale from 10s of users to 100s of thousands of users Pay only for what you use • Readers: $0.30/30 min session with a $5/user/month max • Authors: $18/month/Author Integrates with S3, Athena, Redshift, RDS, Aurora, & EMR
  • 28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Directory Service Microsoft AD Custom Date Format Dashboard Save As Aggregate Calculations Readers Groups Private VPC 25 GB SPICE tables Spark and Presto Connector Scheduled refresh Just In Time Provisioning One-click upgrade Search Totals Excel Custom Range 100+ new features released since launch Federated SSO Athena connector Export to CSV S3 Analytics Week Aggregation Aurora PostgreSQL Calculations in SPICE Cross Account S3 Access Aggregate Filters Hourly refresh Row level security Hourly refresh 10K Filter Values On-screen controls Redshift Spectrum Support KPI Chart Spark Connector AWS Directory Service AD Connector Tabular Reports Data labels URL Actions Combo Charts Audit logging with CloudTrail Geospatial maps Count Distinct Parameters Relative Date Filters Filter Groups Table calculations Snowflake Connector SaaS Connectors Teradata Connector HIPAA PCI compliance Amazon QuickSight has been innovating quickly
  • 29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon QuickSight—embedded dashboards Supercharge your applications with embedded dashboards Fully interactive with drill down, filtering, & external links No servers to manage, no long-term commitments Pay for usage with pay-per-session reader pricing Easy embedding with JavaScript SDK
  • 30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Embedded NFL Next Gen Stats Dashboards “With the Amazon QuickSight Readers and pay-per-session pricing, we are able to extend these secure, customized and easy to use dashboards for each club without having to provision servers or manage infrastructure – all while only paying for actual usage.” Matt Swensson Vice President, Emerging Products and Technology Real-time stats for NFL games Embedded in NFL Next Gen Stats Portal Shared with 100s of users across NFL, 32 clubs and broadcast partners
  • 31.
  • 32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon QuickSight is used by customers at the largest scale One of the world’s largest metals and mining companies deployed Amazon QuickSight with its critical risk management (CRM) solution to ensure employee safety. Thousands of employees use its CRM globally. Uses Amazon QuickSight embedded in its Converge Platform, a governance, risk, and compliance healthcare solution. Tens of thousands of users across 900 healthcare organizations use this platform. Amazon.com is using Amazon QuickSight company-wide
  • 33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon QuickSight—ML Insights (Preview) Automated business insights powered by ML and natural language ML-powered anomaly detection ML-powered forecasting Auto-narratives
  • 34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Discover all the hidden trends and anomalies on millions of metrics Amazon QuickSight—ML Insights Example: anomaly detection
  • 35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. “Sales for office supplies in APAC was 15% above expected.” Amazon QuickSight—ML Insights Example: anomaly detection
  • 36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. “SMB Segment was the top contributor.” Amazon QuickSight—ML Insights Example: anomaly detection
  • 37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. “It’s significant because SMB typically only accounts for 30% of sales.” Amazon QuickSight—ML Insights Example: anomaly detection
  • 38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. QuickSight ML-powered forecasting Traditional BI forecasting Captures seasonality and upward trends Automatically excludes bad data High confidence band Captures only seasonality Missing upward trend Confidence band influenced by bad data QuickSight ML Insights vs. traditional BI forecasting VS.
  • 39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Insights in plain language narrative Embedded within your dashboard No more staring at dashboards for hours! Fully customizable to meet every need No coding needed. Easy-to-use UI templates. Amazon QuickSight—ML Insights Auto-narratives
  • 40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS analytics Any analytic workload, any scale, at the lowest possible cost Redshift Data warehousing EMR Hadoop + Spark Athena Interactive analytics Kinesis Data Analytics Real time Elasticsearch Service Operational Analytics Analytics QuickSight Business Intelligence S3/Glacier Glue ETL & Data Catalog Lake Formation Data Lakes Data Lake
  • 41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. More places to learn about analytics services Amazon Athena Amazon Redshift Amazon Elasticsearch Service AWS Lake Formation ANT401-R2: Deep Dive and Best Practices for Amazon Redshift | Fri 11:30 ANT 401-R1: Deep Dive and Best Practices for Amazon Redshift | Thu 4:00 ANT202-R1: Modern Cloud Data Warehousing ft. Intuit | Thu 2:30 ANT350-R1: What's New with Amazon Redshift ft. McDonald's | Thu 3:15 Sessions that already occured: ANT202-R, ANT350-R ANT323-R1: Build Your Own Log Analytics Solutions on AWS | Thur 11:30 Sessions that already occured: ANT334-R, ANT334-R1, ANT323-RANT203 Introduction to AWS Lake Formation - Build a Secure Data Lake in Days | Wed 7:00pm | Venetian, Level 4, Delfino 4005. Sessions that already occurred: ANT205 ANT340-R1: A Deep Dive into What's New with Amazon EMR | Fri 3:00 Sessions that already occurred: ANT204, ANT312, ANT340-R Amazon Kinesis Amazon QuickSight AWS Glue Introducing Amazon Kinesis Data Analytics for Java applications | Thu 12:15pm | MGM, Level 1, South Concourse 105 ANT322-R1: High Performance Data Streaming with Amazon Kinesis: Best Practices | Thu 1:00 ANT 310: Architecting for Real-Time Insights with Amazon Kinesis | Thu 3:15 Sessions that already occurred: ANT 208, ANT322-R Introducing ML-powered insights with Amazon QuickSight | Wed 1:00pm | Aria East, Level 1, Joshua 9 ANT311: NFL and Forwood Safety Deploy Business Analytics at Scale with Amazon QuickSight | Fri 11:30 Amazon EMR Sessions that already occured: ANT309, ANT308Sessions from prior days: ANT324
  • 42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Customers tell us: they have three type of projects Quickly build new apps in the cloud Gain new insights “Lift and shift” existing apps to the cloud
  • 43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Characteristics of modern applications Internet-scale and transactional Users: 1M+ Data volume: TB–PB–EB Locality: Global Performance: Milliseconds–microseconds Request Rate: Millions Access: Mobile, IoT, devices Scale: Up-out-in Economics: Pay-as-you-go Developer access: Instant API accessSocial mediaRide hailing Media streaming Dating
  • 44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS databases services Purpose-built for all your app needs DynamoDB NeptuneRDS Aurora CommercialCommunity Timestream QLDBElastiCache Relational Key-value Document In-memory Graph Time series Ledger
  • 45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon DynamoDB Fast and flexible key value database service for any scale Comprehensive security Encrypts all data by default and fully integrates with AWS Identity and Access Management for robust security Performance at scale Consistent, single-digit millisecond response times at any scale; build applications with virtually unlimited throughput Global database for global users and apps Build global applications with fast access to local data by easily replicating tables across multiple AWS Regions Serverless No hardware provisioning, software patching, or upgrades; scales up or down automatically; continuously backs up your data
  • 46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. DynamoDB powers the world’s largest applications
  • 47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. The need for transactions support Scenario: Customer purchases an item “PUT”: “TableName”: “Orders”, “OrderStatus”: “Sold”, “Item”: “Bike”, “Quantity”: “1” “PUT”: “TableName”: “Inventory”, “Item”: “Bike”, “Quantity”: “- 1” Transact-write-items { “PUT”: “TableName”: “Orders”, “OrderStatus”: “Sold”, “Item”: “Bike”, “Quantity”: “1” “PUT”: “TableName”: “Inventory”, “Item”: “Bike”, “Quantity”: “- 1” } If code fails here, we inserted the order but did not update inventory If code fails here, we updated some of our inventory, but not all Developer needs to write code to undo a partial operation Transaction support: The system ensures everything happens (or not) VS.
  • 48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon DynamoDB Transactions (GA) Build internet-scale apps with ACID transactions Simplify application code with ACID guarantees Run transactions for large scale workloads Accelerate legacy migrations
  • 49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. DynamoDB—read/write capacity on-demand (GA) No more capacity planning—pay only for what you use No capacity planning No need to specify how much read/write throughput you expect to use Ideal for unpredictable workloads Ramp from zero to tens of thousands of requests per second on demand Pay only for what you use Pay-per-request pricing
  • 50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Timestream (sign up for the preview) Fast, scalable, fully managed time series database 1,000x faster and 1/10th the cost of relational databases Collect data at the rate of millions of inserts per second (10M/second) Trillions of daily events Adaptive query processing engine maintains steady, predictable performance Analytics optimized for time series data Built-in functions for interpolation, smoothing, and approximation Serverless Automated setup, configuration, server provisioning, software patching
  • 51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Quantum Ledger Database (QLDB) (Preview) Fully managed ledger database Track and verify history of all changes made to your application’s data Immutable Maintains a sequenced record of all changes to your data, which cannot be deleted or modified; you have the ability to query and analyze the full history Cryptographically verifiable Uses cryptography to generate a secure output file of your data’s history Easy to use Easy to use, letting you use familiar database capabilities like SQL APIs for querying the data Highly scalable Executes 2–3x as many transactions than ledgers in common blockchain frameworks
  • 52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Managed Blockchain (preview) Create and manage scalable blockchain networks Choice of Hyperledger Fabric or Ethereum Hyperledger Fabric available today; Ethereum coming soon Fully managed Create blockchain networks with a few clicks; Manage them with simple API calls Easily analyze blockchain activity Easy to move data into QLDB for further analysis Scalable and secure Support thousands of client applications running millions of transaction; integrates with AWS KMS
  • 53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon QLBD vs. Amazon Managed Blockchain Amazon QLDB Amazon Managed Blockchain Central trusted authority Track and verify transactions with centralized ownership Ledger is immutable and cryptographically verifiable Ledger is owned by a trusted and centralized authority No central trusted authority Execute transactions and contracts with decentralized ownership Ledger is immutable and cryptographically verifiable Each party maintains their own copy of the ledger Use Ethereum or Hyperledger Fabric to build blockchain networks
  • 54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. More places to learn about purpose-built databases Building modern applications using DynamoDB Transactions | Wed 7:00pm | MGM, Level 3, Premier Ballroom 310 Sessions that already occurred: DAT321, DAT401, DAT201, DAT303, DAT314, DAT325, DAT320, DAT332 Amazon DynamoDB Amazon Neptune Amazon ElastiCache Amazon QLDB Amazon Managed Blockchain Sessions that already occurred: DAT202, DAT319, DAT302-R1, DAT302-R Sessions that already occurred: DAT403, DAT315, DAT316 Use Cases for Amazon QLDB | Fri 9:15 | Mirage, Montego D Deep Dive on Amazon Blockchain | Wed 3:15 | Venetian, Level 2, Titian 2205 - T2
  • 55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Customers tell us: they have three type of projects Quickly build new apps in the cloud Gain new insights “Lift and shift” existing apps to the cloud
  • 56. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Database Migration Service M I G R A T I N G D A T A B A S E S T O A W S Migrate between on-premises and AWS Migrate between databases Automated schema conversion Data replication for zero downtime migration
  • 57. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Customers want to lift and shift to the cloud Relational databases Non-relational databases Data warehouses Hadoop and Spark Redshift EMR Operational analytics Elasticsearch ServiceAurora DynamoDB Business Intelligence QuickSightRDS
  • 58. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. >100,000 databases migrated with DMS
  • 59. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Customers are migrating their workloads to AWS Verizon is migrating over 1,000 business-critical applications and database backend systems to AWS, several of which also include the migration of production databases to Amazon Aurora. Wappa migrated from their Oracle database to Amazon Aurora and improved their reporting time per user by 75 percent. Trimble migrated their Oracle databases to Amazon RDS and project they will pay about 1/4th of what they paid when managing their private infrastructure. Intuit migrated from Microsoft SQL Server to Amazon Redshift to reduce data-processing timelines and get insights to decision makers faster and more frequently. Equinox Fitness migrated its Teradata on-premises data warehouse to Amazon Redshift. They went from static reports to a modern data lake that delivers dynamic reports. Eventbrite moved from Cloudera to Amazon EMR and were able to cut costs dramatically, spinning clusters up/down on-demand and using Spot (saving > 80%) and Reserved Instances. By December 2018, Amazon.com will have migrated 88% of their Oracle DBs (and 97% of critical system DBs) moved to Amazon Aurora and Amazon DynamoDB. They also migrated their 50 PB Oracle Data Warehouse to AWS (Amazon S3, Amazon Redshift, and Amazon EMR). Samsung Electronics migrated their Cassandra clusters to Amazon DynamoDB for their Samsung Cloud workload with 70% cost savings.
  • 60. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Aurora MySQL and PostgreSQL compatible relational database built for the cloud Performance and availability of commercial-grade databases at 1/10th the cost Availability and durability Fault-tolerant, self-healing storage; six copies of data across three AZs; continuous backup to S3 Fully managed Managed by RDS: no hardware provisioning, software patching, setup, configuration, or backups Highly secure Network isolation, encryption at rest/transit Performance and scalability 5x throughput of standard MySQL and 3x of standard PostgreSQL; scale-out up to 15 read replicas
  • 61. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Aurora Global Database (GA) High-performance database for globally-distributed applications Single Global Database with cross region replication Replication typically completes in less than a second No impact on database performance Write master in one region and read replicas in other regions Cross-region disaster recovery Local read latency for applications with global users Primary Region Secondary Region Application Storage Storage Replication <1s
  • 62. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon RDS Managed relational database service with a choice of six popular database engines Available & durable Automatic Multi-AZ data replication; automated backup, snapshots, failover Easy to administer No need for infrastructure provisioning, installing and maintaining DB software Highly scalable Scale database compute and storage with a few clicks with no application downtime Fast & secure SSD storage and guaranteed provisioned I/O; data encryption at rest and in transit
  • 63. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Databases in private data centers are still difficult and expensive to set up and manage Difficult to set up and manage databases for high availability across multiple nodes Personnel needed to create the database image, install operating system, packages, and setup Burdensome to support multiple versions and applying patching ?
  • 64. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon RDS on VMware (Preview) Managed service for on-premises databases RDS deployed as a service in on-premises VMware private data centers (vSphere) Automates management of on-premises databases and hybrid backup and scaling Available and durable Enable hybrid features and tap into AWS for high availability, backup, and restore Secure and compliant Automate management of databases for workloads that must remain on-premises to adhere to strict data policies Fully managed Easy to provision, monitor, and operate relational databases in your private data center Scalability and performance Scale storage, compute, and memory of on-premises databases from a single, simple interface
  • 65. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. How it works Amazon RDS on VMware RDS on VMware Deploy RDS on VMware in your private data center and manage on- premises databases using a single RDS interface RDS interface Use the Amazon RDS console, APIs, or CLI to provision databases Log in or create AWS account Find RDS on VMware in the console and choose AWS region Download and install the connector Download & install the RDS connector in your VMware vSphere environment to establish secure VPN connection between AWS and your private data center Automate database management RDS on VMware automates database management tasks including provisioning, patching, backups and failover
  • 66. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. More places to learn about Aurora, RDS, and DMS Sessions that already occurred: DAT204-R1, DAT313, DAT318, DAT204-R, DAT304-R, DAT304-R1, DAT305-R, DAT305-R1, DAT336 Amazon Aurora DMS Amazon RDS Deep Dive on Amazon RDS on VMware | Thu 1:45 | Aria East, Plaza Level, Orovada 2 Chalk Talk on Amazon RDS on VMware | Fri 10:45 | Mirage, Martinique A DAT323: Best Practices for Running SQL Server on Amazon RDS | Thu 1:00 DAT402: Using Performance Insights to Optimize Database Performance | Thu 12:15 Sessions that already occurred: DAT203, DAT322, DAT324, DAT317 Sessions that already occurred: DAT207
  • 67. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. When to Use Which Services Situation Solution Existing application Use your existing engine on RDS • MySQL Amazon Aurora, RDS for MySQL • PostgreSQL Amazon Aurora, RDS for PostgreSQL • MariaDB Amazon Aurora, RDS for MariaDB • Oracle Use SCT to determine complexity Amazon Aurora, RDS for Oracle • SQL Server Use SCT to determine complexity Amazon Aurora, RDS for SQL Server New application • If you can avoid relational features DynamoDB • If you need relational features Amazon Aurora In-memory store/cache • Amazon ElastiCache Time series data • Amazon Timestream Track every application change, crypto verifiable. Have a central trust authority • Amazon Quantum Ledger Database (QLDB) Don’t have a trusted central authority • Amazon Managed Blockchain Data Warehouse & BI • Amazon Redshift, Amazon Redshift Spectrum, and Amazon QuickSight Adhoc analysis of data in S3 • Amazon Athena and Amazon QuickSight Apache Spark, Hadoop, HBase (needle in a haystack type queries) • Amazon EMR Log analytics, operational monitoring, & search • Amazon Elasticsearch Service and Amazon Kinesis
  • 68. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  • 69. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Architect services ground-up for the cloud and for the explosion of data Offer a portfolio of purpose-built services, optimized for your workloads Help you innovate faster through managed services Our approach Provide services that help you migrate existing apps and databases to the cloud
  • 70. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. EA is supporting their most mission critical workloads with Amazon Redshift. They were able to increase query throughput by 2x over the last 12 months.
  • 71. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Kinesis—real time Easily collect, process, and analyze video and data streams in real time Capture, process, and store video streams for analytics Load data streams into AWS data stores Analyze data streams with SQL Build custom applications that analyze data streams Kinesis Video Streams Kinesis Data Streams Kinesis Data Firehose Kinesis Data Analytics
  • 72. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Build real-time streaming analytics in your Java apps Use Java Integrated Development Environment of choice Libraries include pre-built stream processing operators Real-time application can be built in hours Kinesis Data Analytics—Java support (GA) Devices Sensors IoT Kinesis Data Streams Kinesis Data Analytics Java IDE
  • 73. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Tailored recommendations to increase performance & reduce cost Redshift’s machine learning engine uncovers optimizations Operations such as vacuum and analyze run behind the scenes Available today Redshift ML based auto-tuning (GA) Clusters always optimized for best performance and lowest cost AUTO AUTO AUTO ADVISE ADVISE
  • 74. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Most enterprise database & analytics cloud customers
  • 75. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Most startup database & analytics cloud customers