Learn about the latest and hottest features of Amazon Redshift. We’ll deep dive into the architecture and inner workings of Amazon Redshift and discuss how the recent availability, performance, and manageability improvements we’ve made can significantly enhance your user experience. We’ll also share glimpse of what we are working on and our plans for the future. McDonald's will join us to share how they leverage a data lake powered by Redshift, Redshift spectrum and Athena to get quick insights. 

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s New with Amazon Redshift
ft. McDonald’s
A N T 3 5 0 - R 1
Abhi Bhatt
Director, Global Data & Analytics
McDonalds
Vidhya Srinivasan
General Manager, Amazon Redshift
AWS

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
million orders
in 100+ countries
every day
We feed
of the total
global population
every day
different menu items
and infinite variations

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Change drivers
Customers
are going
digital
Experience
is as important
as food
McDelivery
/UberEats
Global
menu

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data challenges
Limited data
availability
Siloed
Data
Limited scale for quick
data demands and
high fixed cost
Limited
Scale
Mostly descriptive
analytics focusing on
what happened
Limited
Analytics
IT collects and
maintains data
Lack of
Self-service

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cloud model to solve challenges
Removes
data silos and eliminates data movement
Scale
from terabytes to exabytes
Can use
a variety of analytical engines to gain insight
Unified access
and governance
Self-service model
Analytical engines
of choice

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
we measured
across considered solutions
Elasticity Cost Maturity Time to
implementation
Flexibility Self-service A solution that would allow for an
iterative approach over the next
few years

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Migration to AWS
Our goal is to build a
with a focus
on enabling four key
capabilities.
Well-Architected Platform
Self-Service Enablement
Governance & Data Quality
Data Catalog & Search

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Global Data & Analytics Platform
Data Lake Operational/Known Workloads
Data Science/Analytics Workloads
RedshiftEMR
Self-Service
Workloads
Athena
EMR SageMaker
v
Data
Catalog
AWS Glue
Amazon
Kineses Data
Firehose
Operational
Reporting and
Dashboarding
Users
Ad-hoc/Self-
service Users
Data Science,
ML/AI UsersAmazon
EC2

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Global data lake organization
Batch and
real-time raw data
from source systems
No logic or business
rules applied
Folders by
subject areas
Business rules applied
to data
Metadata for data files
available to enable
self-service
Folders by
subject areas
Outbound data feeds
in the platform
Folders for
third party

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Global data lake benefits
Data Lake
on AWS
Redshift EMR Athena
AI
Services
Collection and storage
of all data at scale and low cost
De-couple
storage and compute
Flexibility
in using data engines by use cases/workloads

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift data engine
Amazon
Redshift
Data Lake
AWS Glue Data Catalog
SensorsWebDevicesLOBCRMERPOLTP
One of the many AWS data engines
used to enable McDonald’s workloads
Amazon Redshift runs
global known workloads for operational reporting
and dashboards
McDonald’s contributes
to Redshift’s product roadmap
Unknown/unplanned workloads
de-coupled from Amazon Redshift using the data
lake, AWS Glue Data Catalog and Amazon Athena
Redshift
Queries
Athena

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Integrated and Trusted Data Platform
Descriptive, predictive and prescriptive analytics
Self-service delivery model
takes hours/days, not weeks/months
Data enables faster business insights and growth
On-demand scale with cloud, usage-based cost
Results

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s next?
Optimize
cloud work loads to
enable transient and
serverless processing,
improving performance
and self-service
Automation
Deploy
data product
across the globe
with single-click
deployment
Implement
chargeback model

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Warehouse
Business
Intelligence
Predictive
Data Catalog
DW Queries
Big data
processing
Interactive Real-time
OLTP ERP CRM LOB Devices Web Sensors Social
Self-service delivery model
AWS Glue is a great tool for
cataloging and data transformation
Amazon Redshift for running
global workloads for fast,
operational reporting
Use the best analytics tool for the job
Thank you.
Data Lake

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
2012
Analytics
Portfolio
Analytics
Storage
Data
movement
S3
REDSHIFT EMR
DATA PIPELINE
Data warehousing Big data processing

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS databases and analytics
Broad and deep portfolio, built for builders
AWS Marketplace
Redshift
Data warehousing
EMR
Hadoop + Spark
Athena
Interactive analytics
Kinesis Analytics
Real-time
Elasticsearch service
Operational Analytics
RDS
MySQL, PostgreSQL, MariaDB,
Oracle, SQL Server
Aurora
MySQL, PostgreSQL
QuickSight SageMaker
DynamoDB
Key value, Document
ElastiCache
Redis, Memcached
Neptune
Graph
Timestream
Time Series
QLDB
Ledger Database
S3/Glacier
Glue
ETL & Data Catalog
Lake Formation
Data Lakes
Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams | Data Pipeline | Direct Connect
Data Movement
AnalyticsDatabases
Business Intelligence & Machine Learning
Data Lake
Managed
Blockchain
Blockchain
Templates
Blockchain
Comprehend Rekognition Lex Transcribe DeepLens 250+ solutions
730+ Database
solutions
600+ Analytics
solutions
25+ Blockchain
solutions
20+ Data lake
solutions
30+ solutions
RDS on VMWare

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data
every 5 years
There is more data
than people think
15
years
live for
Data platforms need to
1,000x
scale
>10x
grows

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hadoop Elasticsearch
There are more
ways to analyze data
than ever before
Years ago
11 8 5 4
Presto Spark
Didn’t exist

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
There are more
people working
with data than
ever before
How do I provide democratized
access to data to enable
informed decisions while at the
same time enforce data
governance and prevent
mismanagement of the data?
Democratization
of data
Governance
& Control

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Managing the evolving data landscape
Flexible
Open APIs
and open
data formats
Choice
Use the best
analytic tool for
the job, without
data movement
Scale
Platforms
that scale up
to 1,000x
Secure
Full auditability,
access controls,
and data
governance
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift
The 4 things that matter most
Speed Scale SecuritySimplicity
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
for their data
warehouse
workloads than
anyone else
Amazon
Redshift

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Let’s dig into what we’ve done
in the past several months
and what’s coming …

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
features and
enhancements
released*
Amazon Redshift is growing fast and innovating faster
Automatically enabled
short query acceleration
Support for lateral column
alias reference
New Quick Starts
New CloudWatch metrics
Customized
Recommendations
with Advisor
Current and trailing tracks
for release update
Federated authentication
with single sign-on
Improved performance
for commits
COPY from Parquet and
ORC file formats
Additional Spectrum regions
Support for Scalar JSON
and Ion data types
Late materialization for
faster query processing
Support for DATE data
type with Spectrum
Short Query
Acceleration
Utilization reports
Machine learning integration
to accelerate dashboards
and interactive analysis
Improved resource
management for
memory-intensive queries
Faster string manipulation
Support for Parquet and
ORC in Kinesis Data Firehose
Improved workload
management console
experience
Query Editor
Support for late-binding views
SQL Scalar user-defined
functions
Integration with AWS Glue
Support for Nested
Data with Spectrum
Spectrum support
for DATE data type
Improved performance
for UNION ALL queries
Free upgrade from
DC1 to DC2 RIs
Query monitoring rules (QMR)
Support for Zstandard high
compression encoding
Query processing
improvements
Support for Python
UDF logging module
Enhanced VPC routing
Automatically hopping
queries without restarts
Support for uppercase
column names
Result Caching for
Repeat Queries
Support for LISTAGG DISTINCT
Support for ORC and
Grok file formats
Integration with QuickSight
DMS support with Redshift
3.5x Improved
Throughput
Improved performance
for repeat queries
Since we last spoke…
*since re:Invent 2017

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Improvements
in availability
since we
last talked
NOV DEC JAN FEB MAR APR JUN JUL AUG SEP OCT NOV
20182017
Weekly Database Restarts
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
*Since re:Invent 2017
Compiled code cache
Support for lateral
column alias reference
Resource management for
memory-intensive queries
Late materialization
Result caching
Joins involving large numbers of
NULL values in a join key column
Queries with intermediate subquery
results that can be distributed
Cluster
resize operations
Queries that refer to stable
functions with constant expressions
Short query
acceleration
Queries operating over CHAR
and VARCHAR columns
Single-row inserts
Improvements to speed
Expressions on the partition
columns of external tablesFaster string manipulation
Complex EXCEPT
subqueries
Commit processing
enhancements
DC2 nodes
2x the number of tables
in a cluster
Hash join memory utilization
optimizations and cache line
prefetching
COPY operation when
ingesting data from Parquet
and ORC formats
Performance improvement for
queries that refer to stable functions
over constant expressions
Improvements for the COPY
operation when ingesting data
from Parquet and ORC formats
Query processing
improvements
Query rewrites that pushdown selective joins
into a subquery
Query planning
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Increases in
performance in
real-world workloads
How do we
improve real-world
performance?
for repetitive
queries
for bulk-deletes for single-row
inserts
for commits
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How we leverage fleet telemetry

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Performance
improvements
in query speed
- Minero Aoki
Senior Data Engineer, Cookpad Inc.
Redshift query performance and
scalability has been increasing,
even though our data has
grown. In the last 10 months, we
have seen commit performance
increase by 500% without any
increase in cost.

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift is
now over 3x faster
on standard
benchmarks than
6 months ago
Normalized Queries Per Hour (QPH)
Assuming Redshift’s QPH 6 months ago=100%
Queriesperhour
Asa%ofredshift6monthsago
JUL 2018 AUG 2018 SEP 2018 OCT 2018MAY 2018
100%
181%
237%
284%
350%
Higher is better
115%
JUN 2018
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift fasterup to
Higher is betterHigher is better
Based on the cloud DW benchmark derived
from TPC-DS 3 TB dataset, 4-node cluster
82%
6%
34%
REDSHIFT VENDOR 1 VENDOR 2 VENDOR 3
TPC-DS 3TB queries per hour
QueriesPerHour
(Asa%ofAmazonRedshift’sQueriesperhour)
61%
113%
40%
REDSHIFT VENDOR 1 VENDOR 2 VENDOR 3
TPC-H 3TB queries per hour
QueriesPerHour
(Asa%ofAmazonRedshift’sQueriesperhour)
Based on the cloud DW benchmark derived
from TPC-H 3 TB dataset, 4-node cluster

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
$/YrforRedshiftisbasedonthe1
yearReservedInstance(RI)price
Amazon Redshift is the most cost-effective cloud data warehouse
The best price-to-performance
The only data warehouse with
reserved instances saving
up to 75%$560,640
$264,902
$944,941
REDSHIFT VENDOR 1 VENDOR 2 VENDOR 3
Price per year
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lower is better

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fleet telemetry on query wait times
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
<1 minute 15 minutes >20 minutes
Daily cluster
queue time
per day
Remaining 13% have
bursts of activity averaging
10 minutes at a time
of Amazon
Redshift customers don’t have
significant wait times

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Caching Layer
Concurrency Scaling for
bursts of user activity (Preview)
Automatically
creates more
clusters on-
demand
Consistently
fast
performance
even with
thousands of
concurrent queries
No advance
hydration
required
Quickly scale
to serve changing
query workload
New!
Backup
Redshift Managed S3

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Results with Concurrency Scaling
For every 24 hours your
main cluster is in use,
we’ll provide a one-hour
credit for concurrent
cluster usage.
Concurrency Scaling is
free for more than 97%
of Redshift customers.
Auto-scaling resources for bursts of user activity
Redshift Redshift with auto-scaling
Higher is better
Queriesperhour
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
*Since re:Invent 2017
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Improvements
to simplicity
CloudWatch metrics for
Workload Execution
Breakdown
Current and trailing tracks
for release updates
Lateral column
alias reference
CloudWatch metrics
for Query Duration
by WLM Queues
Cluster resize operations
CloudWatch
Query Runtime Breakdown metric
Stream real-time data in
Parquet or ORC formats
using Kinesis Data Firehose
DISTSTYLE AUTO
distribution style
Free upgrade from for DC1
RIs to DC2
Query Monitoring Rules (QMR)
now support 3x more rules
Short query
acceleration is
self-optimizing
Redshift Advisor for best
practice recommendationsCloudWatch metrics
for Query Throughput
by WLM Queues
Cluster resize Query Editor
Enhancements to
VACUUM DELETE
Manage components
of a multi-part query
in the AWS console
Automatic vacuum delete
Efficiency of backup performance
CloudWatch metrics for Query
Throughput, Query Duration

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift Elastic Resize (GA)
Adds
additional
nodes
to Redshift cluster
Distributes
data
across new
configuration
in minutes
Minimal
transition time
Scale compute
and storage on-
demand
Scale up and down in minutes
New!
Redshift
Cluster
Redshift Managed S3
JDBC/ODBC
Leader Node
CN2CN1 CN3 CN4
Backup

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Redshift Query Editor
Query data
directly from
the AWS console
Results are instantly
visible within the console
No need to install
and setup an external
JDBC/ODBC client
Launched in October!

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift Advisor
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
>96% of
clusters
have tailored
feedback
Provides
automated
recommendations
to help optimize database
performance and
decrease operating costs
Actionable
WLM
COPY, storage,
and system
maintenance advice
for tuning based
on continuous
workload analysis
Intelligent
recommendations
Launched in July!

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift intelligent
administration
Automates data
distribution in tables for
improved performance
and disk space
utilization.
Provides intelligent
recommendations for tuning
based on continuous
workload analysis.
ALL
keyA keyB keyC keyD
Node 1
Slice 1 Slice 2
Node 2
Slice 3 Slice 4
EVEN
Node 1
Slice 1 Slice 2
Node 2
Slice 3 Slice 4
KEY
Node 1
Slice 1 Slice 2
Node 2
Slice 3 Slice 4
recommended
distribution key
No more messing
with distkeys!
Coming Soon!
Advise

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift intelligent maintenance
VacuumAnalyze WLM
Concurrency
Setting
AutoAuto Auto
Maintenance processes like
vacuum and analyze will
automatically run in the
background.
Amazon Redshift will automatically
adjust the WLM concurrency setting to
deliver optimal throughput.
Moving towards
zero-maintenance.
Coming Soon!

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Run stored procedures in
Amazon Redshift
Bring your existing Stored
Procedure and run in
Redshift.
Amazon Redshift will support Stored
Procedure in PL/pgSQL format,
enabling you to bring your existing
Stored Procedure to Amazon Redshift.
Migrating to Redshift
is even easier!
Coming Soon!
where the data is to
efficiently run ETL,
data validation, and
custom business
logic.

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
*Since re:Invent 2017
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Improvements to scale
Integrate seamlessly with your data lake
DATE data type
Retrieving metadata for late-binding
viewsSupport for Enhanced VPC Routing
IN-list predicate processing
in Spectrum scans
Query external tables
during a resize operation
Specify the root of an
S3 bucket as the source
for an existing table
Spectrum queries with
aggregations on partition columns
Renaming external
table columns
Table property to specify the file
compression type for external tables
Push the LENGTH()
string function to
Spectrum
ALTER TABLE ADD/DROP
COLUMN for external tables is now
supported via standard JDBC calls
Map datatypes in
Spectrum to contain
arrays
Support for Parquet, ORC, Avro,
CSV, and other open file formats
New Spectrum
regions
Spectrum support
for JSON and ION
Spectrum support
for nested data
Arrays of arrays and
arrays of maps

47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift Spectrum
Redshift Spectrum
query engine
Query across
Amazon Redshift
and Amazon S3
Redshift
data
S3
data lake
Extend the data warehouse to exabytes of data in S3 Data Lake
No loading required
Scale compute and storage separately
Directly query data stored in Amazon S3
Parquet, ORC, Avro, Grok, and CSV data formats
 Unload to Parquet
Spectrum Request Accelerator
Coming
Soon!

48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
*Since re:Invent 2015
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Improvements
to security
Encrypt your previously
unencrypted cluster with 1-click
Enhanced
VPC Routing
SAS integration
enhancements
Superusers to grant users
access to all rows in
selected system tables
Encrypt unloaded data using S3
server-side encryption with AWS
KMS keys
IAM roles with COPY
and UNLOAD
commands
Tag-based
permissionsDefault access
privileges
Federated
authentication with
single sign-on
Cross-region backups for
KMS-encrypted clusters

49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Security is built-in
Select compliance certifications*
10 GigE (HPC)
Customer
VPC
Internal
VPC
JDBC/ODBC
Compute
Nodes
Leader
Node
Network Isolation
End-to-end encryption
Integration with AWS Key
Management Service
Amazon S3

50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Integration with AWS Lake Formation Coming Soon!
KinesisSocial Web
Sensors Devices
LOBCRM
ERPOLTP
IAM KMS
Data
Catalog
Athena
EMR
Elasticsearch
AI Services
QuickSight
Redshift

51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Unload
to Parquet
Amazon Redshift
New features
Speed
Scale
WLM
Concurrency
Setting
Simplicity
AWS Lake
Formation
integration
Security
Auto-
Vacuum &
Auto-
Analyze
Auto Data
Distribution
Deferred
Maintenance
Snapshot
Scheduler
Spectrum
Request
Accelerator
Auto data
distribution
Elastic
resize
Concurrency
Scaling
Improving
short query
acceleration
Support for
stored
procedures

52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Try it out for yourself:
Modern Data
Warehousing
on AWS ebook
Blog on
performance
matters:
Amazon Redshift is
now up to 3.5x
faster for real-
world workloads
Sign up for
Concurrency
Scaling
Amazon Redshift
and the art of
performance
optimization
in the cloud
by Werner Vogels
Amazon Redshift
customer
use cases
Building
a Proof of
Concept
for Amazon
Redshift
More places
to learn about
Amazon Redshift

53. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Vidhya Srinivasan
Vid@amazon.com

54. Please complete the session
survey in the mobile app.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.