Come along and learn about the enhancements that we have made to Amazon Redshift, including features around performance, scalability and cluster management. We will also explore why a cloud native Data Warehouse solution allows AWS to innovate faster and deliver customer outcomes that are not possible in a more traditional on premises solution.

1. S U M M I T
SYDNEY

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Data warehousing in the cloud
Greg Khairallah
Head of Business Development, DB & Analytics
Amazon Web Services

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Agenda
• Quick Recap of Amazon Redshift
• Overview of updates in the past 6-12 months
• Updates in speed
• Updates in scalability
• Updates in simplicity
• Updates in security

4. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Redshift
Fastest
Get faster time-to-insight
for all types of analytics
workloads; powered by
machine learning, columnar
storage and MPP
Unlimited scale Extends your
data lake
1/10th
the cost
Dynamically scale up to
guarantee performance
even with unpredictable
analytical demands and
data volumes
Analyse data in the Amazon
S3 data lake in-place and in
open formats, together with
data loaded into Amazon
Redshift’s high performance
SSDs
Start at $0.25 per hour,
save costs with automated
administration tasks and
eliminate business impact
due to downtime; as low as
$1,000 per terabyte per year
Fast, simple, cost-effective DW
that extends queries across your data lake
Analyse data in open formats
such as Parquet, ORC, and JSON, using SQL tools

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Redshift
Architecture
Load
Unload
Backup
Restore
SQL clients/BI tools
128GB RAM
16TB disk
16 cores
JDBC/ODBC
128gb ram
16TB disk
16 coresCompute
node
128gb ram
16TB disk
16 coresCompute
node
128gb ram
16TB disk
16 coresCompute
node
Leader
node
Amazon S3

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Redshift
Architecture
Load
Unload
Backup
Restore
SQL clients/BI tools
128GB RAM
16TB disk
16 cores
JDBC/ODBC
128gb ram
16TB disk
16 coresCompute
node
128gb ram
16TB disk
16 coresCompute
node
128gb ram
16TB disk
16 coresCompute
node
Leader
node
Amazon S3
...
1 2 3 4 N
Amazon
Redshift
Spectrum
Load
Query
AWS Lake Formation
Catalogue

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Query All Your Data
Amazon Redshift’s Spectrum Feature Extends Amazon Redshift Queries to an Amazon S3 Data Lake
Amazon
Redshift Spectrum
Redshift Query
Engine
Redshift Data
Data Lake
Amazon S3
• Directly query exabytes in S3,
no loading required
• Query across Amazon Redshift and Amazon
S3
• Scale compute and storage separately
• High concurrency
• Support for Parquet, ORC, Avro, CSV,
JSON, Grok, and other open file formats
• Pay only for the amount of data scanned

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS
Snowball
AWS
Snowmobile
Amazon
Kinesis
Data Firehose
Amazon Kinesis
Data Streams
S3
Amazon
Redshift
Amazon
EMR
Amazon
Athena
Amazon
Kinesis
Amazon Elasticsearch
Service
Amazon
Kinesis
Video
Streams
AI Services
Amazon
QuickSight
Exabyte scale
Store and analyse relational and non-relational data
Purpose-built analytics tools
Cost effective
• Store at 2.3 cents per GB-month in Amazon S3
• Query with Amazon Athena at ½ cent per GB scanned
• DW with Amazon Redshift for $1,000/TB/year
Give access to everyone
• Amazon QuickSight: $0.30 for 30 minutes of use
Data Warehouse and Data Lake working together

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Use the ETL, SQL and BI tools you love
Data integration Business intelligence Systems integrators

11. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Redshift: what’s new and what’s coming
We’re innovating across the 4 things that matter most to customers
Speed SecuritySimplicity
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scale

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Unload
to Parquet
Amazon Redshift
New features
Speed
Scale
WLM
Concurrency
Setting
Simplicity
Amazon Lake
Formation
integration
Security
Auto-
Vacuum &
Auto-
Analyse
Auto Data
Distribution
Deferred
Maintenance
Snapshot
Scheduler
Spectrum
Request
Accelerator
10x average
performance
improvement
Elastic
resize
Concurrency
Scaling
Improving
short query
acceleration
Support for
stored
procedures

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
*Since re:Invent 2017
Compiled code cache
Support for lateral
column alias reference
Resource management for
memory-intensive queries
Late materialisation
Result caching
Joins involving large numbers of
NULL values in a join key column
Queries with intermediate subquery
results that can be distributed
Cluster
resize operations
Queries that refer to stable
functions with constant expressions
Short query
acceleration
Queries operating over CHAR
and VARCHAR columns
Single-row inserts
Improvements to speed
Expressions on the partition
columns of external tablesFaster string manipulation
Complex EXCEPT
subqueries
Commit processing
enhancements
DC2 nodes
2x the number of tables
in a cluster
Hash join memory utilisation
optimisations and cache line
prefetching
COPY operation when
ingesting data from Parquet
and ORC formats
Performance improvement for
queries that refer to stable functions
over constant expressions
Improvements for the COPY
operation when ingesting data
from Parquet and ORC formats
Query processing
improvements
Query rewrites that pushdown selective joins
into a subquery
Query planning
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
https://www.amazon.com
10x faster
than it was two years ago
Amazon Redshift is now
from over 200 features and enhancements released due to lessons learnt from
more than 10,000 customer deployments processing over 2 exabytes of data every day

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
1 2 4
Daily cluster
queue time
per day
Remaining 13% have
bursts of activity averaging
10 minutes at a time
of Amazon
Redshift customers don’t have
significant wait times

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Caching Layer
Concurrency Scaling
Automatically
creates more
clusters on-
demand
Consistently
fast
performance
even with
thousands of
concurrent queries
No advance
hydration
required
Quickly scale
to serve changing
query workload
New!
Backup
Redshift Managed S3
1
2 3

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
For every 24 hours your main
cluster is in use, we’ll provide a
one-hour credit for concurrent
cluster usage
97% of users will never see a
charge for auto-scale resources
0
2000
4000
6000
8000
10000
12000
1 2 3 4 5 6
QueriesperHour(QpH)
Number of concurrently active users
Throughput scales linearly
Amazon Redshift’s throughput scales with concurrent
users

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Concurrency Scaling is a powerful
new feature of Amazon Redshift.
We were really impressed by the
performance of the feature,
especially with its ability to
instantly add transient capacity
with nothing to manage on our
end. As Redshift administrators at
Yelp, we think that Concurrency
Scaling will keep our many users
happy, even under peak load. We’re
excited that Concurrency Scaling
provides the flexibility to handle
significant variance in our
workloads over the course of a day.
-Shahid Chohan,
Software Engineer, Yelp

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Performance
improvements
in query speed
- Minero Aoki
Senior Data Engineer, Cookpad Inc.
Redshift’s query performance
and scalability has been
increasing, even though our data
has grown. In the last 10
months, we have seen commit
performance increase by 500%
without any increase in cost.

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
*Since re:Invent 2017
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Improvements to scale
Integrate seamlessly with your data lake
DATE data type
Retrieving metadata for late-binding
viewsSupport for Enhanced VPC Routing
IN-list predicate processing
in Spectrum scans
Query external tables
during a resize operation
Specify the root of an
S3 bucket as the source
for an existing table
Spectrum queries with
aggregations on partition columns
Renaming external
table columns
Table property to specify the file
compression type for external tables
Push the LENGTH()
string function to
Spectrum
ALTER TABLE ADD/DROP
COLUMN for external tables is now
supported via standard JDBC calls
Map datatypes in
Spectrum to contain
arrays
Support for Parquet, ORC, Avro,
CSV, and other open file formats
New Spectrum
regions
Spectrum support for
JSON and ION
Spectrum support
for nested data
Arrays of arrays and
arrays of maps

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Redshift Spectrum
Amazon Redshift Spectrum
query engine
Query across
Amazon Redshift
and Amazon S3
Amazon
Redshift data
Amazon S3
data lake
Extend the data warehouse to exabytes of data in S3 Data Lake
No loading required
Scale compute and storage separately
Directly query data stored in S3
Parquet, ORC, Avro, Grok, and CSV data formats
 Unload to Parquet
Spectrum Request Accelerator
Coming
Soon!

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Redshift Elastic Resize (GA)
Adds
additional
nodes
to Redshift cluster
Run queries
faster
in busy periods
Minimal
transition time
Scale compute
and storage on-
demand
Scale up and down in minutes
Amazon Redshift
Cluster
Amazon Redshift Managed S3
JDBC/ODBC
Leader Node
CN2CN1 CN3 CN4
Backup

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Improvements to
simplicity
CloudWatch metrics for
Workload Execution
Breakdown
Current and trailing tracks
for release updates
Lateral column
alias reference
CloudWatch metrics
for Query Duration
by WLM Queues
Cluster resize operations
CloudWatch
Query Runtime Breakdown metric
Stream real-time data in
Parquet or ORC formats
using Kinesis Data Firehose
DISTSTYLE AUTO
distribution style
Free upgrade from for DC1
RIs to DC2
Query Monitoring Rules (QMR)
now support 3x more rules
Short query
acceleration is
self-optimising
Redshift Advisor for best practice
recommendationsCloudWatch metrics
for Query Throughput
by WLM Queues
Cluster resize Query Editor
Enhancements to
VACUUM DELETE
Manage components
of a multi-part query
in the AWS console
Automatic vacuum delete
Efficiency of backup performance
CloudWatch metrics for Query
Throughput, Query Duration

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Redshift Query Editor
Query data
directly from
the AWS console
Results are instantly
visible within the console
No need to install
and setup an external
JDBC/ODBC client
Launched in October!

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
>96% of
clusters
have tailored
feedback
Provides
automated
recommendations
to help optimise database
performance and
decrease operating costs
Actionable
WLM
COPY, storage,
and system
maintenance advice
for tuning based
on continuous
workload analysis
Intelligent
recommendations
Amazon Redshift Advisor

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Redshift performs maintenance automatically
VacuumAnalyse WLM
Concurrency
Setting
AutoAuto Auto
Maintenance processes like vacuum
and analyse will automatically run in
the background.
Amazon Redshift automatically adjusts
the WLM concurrency setting to deliver
optimal throughput.
Near zero-
maintenance.

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Redshift performs administration automatically
Automates data
distribution in tables for
improved performance
and disk space utilisation.
Provides intelligent
recommendations for tuning
based on continuous
workload analysis.
ALL
keyA keyB keyC keyD
Node 1
Slice 1 Slice 2
Node 2
Slice 3 Slice 4
EVEN
Node 1
Slice 1 Slice 2
Node 2
Slice 3 Slice 4
KEY
Node 1
Slice 1 Slice 2
Node 2
Slice 3 Slice 4
Auto
distribution key
No more messing
with distkeys!

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
*Since re:Invent 2015
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Improvements
to security
Encrypt your previously unencrypted
cluster with 1-click
Enhanced
VPC Routing
Utilisation
alerts for RIs
SAS integration
enhancements
Superusers to grant users
access to all rows in
selected system tablesEncrypt unloaded data using S3
server-side encryption with AWS
KMS keys
IAM roles with COPY
and UNLOAD
commands
Tag-based
permissions
Default access
privileges
Federated
authentication with
single sign-on
Cross-region backups for
KMS-encrypted clusters

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Security is built-in
Select compliance certifications*
10 GigE (HPC)
Customer
VPC
Internal
VPC
JDBC/ODBC
Compute
Nodes
Leader
Node
Network Isolation
End-to-end encryption
Integration with AWS Key
Management Service
Amazon S3

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Integration with Amazon Lake Formation Coming Soon!
KinesisSocial Web
Sensors Devices
LOBCRM
ERPOLTP
IAM KMS
Data
Catalog
Amazon Athena
Amazon EMR
Amazon Elasticsearch
AI Services
Amazon QuickSight
Amazon
Redshift

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Migrating from on-premises DWs is easy

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Migrate with minimal risk
AWS provides the broadest range of tools for easy, fast,
and secure data movement to and from the AWS cloud
AWS Direct
Connect
AWS
Snowball
Database
Migration
Service
AWS Storage
Gateway
Amazon S3
Transfer
Acceleration
Amazon Kinesis
Firehose
ISV
Connectors

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Self-service migration to Amazon Redshift
AWS Database Migration Service
DMS usage is free for 6 months
For more information: https://aws.amazon.com/dms/free-dms/
“AWS Database Migration Service is
the most impressive migration
service we’ve seen.”
AWS DMS Amazon
Redshift
2. Relational Databases
1. Non-Relational Databases
3. Other sources
Azure SQL DB

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Run stored procedures in
Amazon Redshift
Bring your existing Stored
Procedure and run in Redshift.
Amazon Redshift will support Stored
Procedure in PL/pgSQL format,
enabling you to bring your existing
Stored Procedure to Amazon Redshift.
Migrating to Amazon
Redshift is even
easier!
Coming Soon!
where the data
is to efficiently run
ETL, data validation,
and custom
business logic.

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Migrate with Systems and Data Integration partners
AWS partners bring invaluable technical expertise and
resources to help with complex migration projects

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
for their data
warehouse
workloads than
anyone else
More than 15K
customers use
Amazon

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
More places to learn about Amazon Redshift
Modern Data Warehousing on AWS ebook
Sign up for Concurrency Scaling
Amazon Redshift and the art of performance optimization in the cloud by Werner Vogels
Performance matters: Amazon Redshift is now up to 3.5x faster for real-world workloads
Amazon Redshift customer use cases
Building a Proof of Concept for Amazon Redshift
Try it out for yourself: https://aws.amazon.com/redshift/

39. Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Greg Khairallah