In the third webinar in our Amazon Redshift Spectrum Series, learn more about how you can accessing your data lake assets from Amazon Redshift Spectrum. Spectrum allows you to use the analytic capabilities of Amazon Redshift beyond the data which is in your data warehouse to query large amounts of semistructured and structured data in your “data lake,” without having to load or transform it into Amazon Redshift. In this webinar learn: - How to launch Amazon Redshift On-Demand - About Amazon Redshift Spectrum from Greg Khairallah (AWS) - More about Matillion's Spectrum features - Through demonstration, how to accessing your data lake assets from Amazon Redshift Spectrum Watch the full webinar: https://youtu.be/mOKWovh5l4g

1. 1
www.matillion.com
© 2017 Matillion. All rights reserved.
Presented by:
Copyright © 2017. All rights reserved. Matillion, trademarks, registered trademarks
or service marks are property of their respective owners. 10/27/2017
Spectrum Webinar Series
Part 3: Accessing your Data Lake assets from Amazon Redshift Spectrum
James Johnson, Ian Funnell and Greg Khairallah

2. 2
www.matillion.com
© 2017 Matillion. All rights reserved.
• Launch Redshift on demand
• Presentation from Greg Khairallah of AWS
• Recap on Amazon Redshift Spectrum
• Recap of Matillion's Spectrum features
• Demonstration, including Redshift on demand
Introduction

3. 3
www.matillion.com
© 2017 Matillion. All rights reserved.
Launch On Demand Redshift

4. 4
www.matillion.com
© 2017 Matillion. All rights reserved.
State of Data Warehousing
Data Warehousing Challenges Today
Exponential Data Growth Varying Data Types Need Data Analyzed Faster

5. 5
www.matillion.com
© 2017 Matillion. All rights reserved.
Amazon Redshift is used for mission-critical workloads
Financial and
management reporting
Payments to suppliers
and billing workflows
Web/Mobile clickstream
and event analysis
Recommendation and
predictive analytics

6. 6
www.matillion.com
© 2017 Matillion. All rights reserved.
The Forrester Wave™: Big Data Warehouse, Q2 2017
The Forrester Wave™ is copyrighted
by Forrester Research, Inc. Forrester
and Forrester Wave™ are trademarks
of Forrester Research, Inc. The
Forrester Wave™ is a graphical
representation of Forrester's call on a
market and is plotted using a detailed
spreadsheet with exposed scores,
weightings, and comments. Forrester
does not endorse any vendor, product,
or service depicted in the Forrester
Wave. Information is based on best
available resources. Opinions reflect
judgment at the time and are subject to
change.

7. 7
www.matillion.com
© 2017 Matillion. All rights reserved.
Benefits of Using Amazon Redshift
Amazon Redshift is a fast, fully managed, petabyte-scale data warehouse that makes it
simple and cost-effective to analyze all your data using your existing business intelligence
tools. Amazon Redshift also includes Redshift Spectrum, allowing you to directly query
exabytes of unstructured data in Amazon S3.
Amazon Redshift is:
Fast Simple Elastic Secure Compatible Low Cost

8. 8
www.matillion.com
© 2017 Matillion. All rights reserved.
Amazon Redshift
• Fully managed, petabyte-scale data warehousing
service
• High performance SQL, Massively Parallel Processing
• Fully managed with SSD and HDD platforms. Built-in
end to end security, including customer-managed keys
• Data automatically backed up to Amazon S3 with
cross region backup capability for global disaster
recovery
• $1,000/TB/Year; start at $0.25/hour
Query tool
Amazon
Redshift

9. 9
www.matillion.com
© 2017 Matillion. All rights reserved.
Paradigm Shift Enabled by Redshift Spectrum

10. 10
www.matillion.com
© 2017 Matillion. All rights reserved.
Paradigm Shift Enabled by Redshift Spectrum
• Extends Redshift to query Exabyte-scale
data lakes in S3
• Intelligent query optimizer
• Elastic scale-out fleet to process S3 data
• Directly query data in open formats
• Multiple Redshift clusters against same data
• Pay for just the queries you run ($5/TB scanned)
Query tool
Amazon
Redshift
...
1 2 3 4
Spectrum
n
S3

11. 11
www.matillion.com
© 2017 Matillion. All rights reserved.
Let's build an analytic query - #1
• An author is releasing the 8th book in her
popular series. How many should we order
for Seattle? What were prior first few day
sales?
• Let’s get the prior books she’s written
• 1 Table
• 2 Filters
• SELECT
• P.ASIN,
• P.TITLE
• FROM
• products P
• WHERE
• P.TITLE LIKE ‘%POTTER%’ AND
• P.AUTHOR = ‘J. K. Rowling’

12. 12
www.matillion.com
© 2017 Matillion. All rights reserved.
Let's build an analytic query - #2
• An author is releasing the 8th book in her
popular series. How many should we order
for Seattle? What were prior first few day
sales?
• Let's compute the sales of the prior books
she’s written in this series and return the top
20 values
• SELECT
• P.ASIN,
• P.TITLE,
• SUM(D.QUANTITY *
D.OUR_PRICE) AS SALES_sum
• FROM
• s3.d_customer_order_item_det
ails D,
• products P
• WHERE
• D.ASIN = P.ASIN AND
• P.TITLE LIKE '%Potter%' AND
• P.AUTHOR = 'J. K. Rowling'
AND
• GROUP BY P.ASIN, P.TITLE
• ORDER BY SALES_sum DESC
• LIMIT 20;
• 2 Tables (1 S3, 1 local)
• 2 Filters
• 1 Join
• 2 Group By columns
• 1 Order By
• 1 Limit
• 1 Aggregation

13. 13
www.matillion.com
© 2017 Matillion. All rights reserved.
Let's build an analytic query - #3
• An author is releasing the 8th book in her
popular series. How many should we order
for Seattle? What were prior first few day
sales?
• Let's compute the sales of the prior books
she’s written in this series and return the top
20 values, just for the first three days of sales
of first editions
• SELECT
• P.ASIN,
• P.TITLE,
• P.RELEASE_DATE,
• SUM(D.QUANTITY * D.OUR_PRICE) AS
SALES_sum
• FROM
• s3.d_customer_order_item_details D,
• asin_attributes A,
• products P
• WHERE
• D.ASIN = P.ASIN AND
• P.ASIN = A.ASIN AND
• A.EDITION LIKE '%FIRST%' AND
• P.TITLE LIKE '%Potter%' AND
• P.AUTHOR = 'J. K. Rowling' AND
• D.ORDER_DAY :: DATE >=
P.RELEASE_DATE AND
• D.ORDER_DAY :: DATE < dateadd(day,
3, P.RELEASE_DATE)
• GROUP BY P.ASIN, P.TITLE, P.RELEASE_DATE
• ORDER BY SALES_sum DESC
• LIMIT 20;
• 3 Tables (1 S3, 2 local)
• 5 Filters
• 2 Joins
• 3 Group By columns
• 1 Order By
• 1 Limit
• 1 Aggregation
• 1 Function
• 2 Casts

14. 14
www.matillion.com
© 2017 Matillion. All rights reserved.
Let's build an analytic query - #4
• An author is releasing the 8th book in her
popular series. How many should we
order for Seattle? What were prior first
few day sales?
• Let's compute the sales of the prior
books she’s written in this series and
return the top 20 values, just for the first
three days of sales of first editions in the
city of Seattle, WA, USA
• 4 Tables (1 S3, 3 local)
• 8 Filters
• 3 Joins
• 4 Group By columns
• 1 Order By
• 1 Limit
• 1 Aggregation
• 1 Function
• 2 Casts
• SELECT
• P.ASIN,
• P.TITLE,
• R.POSTAL_CODE,
• P.RELEASE_DATE,
• SUM(D.QUANTITY * D.OUR_PRICE) AS SALES_sum
• FROM
• s3.d_customer_order_item_details D,
• asin_attributes A,
• products P,
• regions R
• WHERE
• D.ASIN = P.ASIN AND
• P.ASIN = A.ASIN AND
• D.REGION_ID = R.REGION_ID AND
• A.EDITION LIKE '%FIRST%' AND
• P.TITLE LIKE '%Potter%' AND
• P.AUTHOR = 'J. K. Rowling' AND
• R.COUNTRY_CODE = ‘US’ AND
• R.CITY = ‘Seattle’ AND
• R.STATE = ‘WA’ AND
• D.ORDER_DAY :: DATE >= P.RELEASE_DATE AND
• D.ORDER_DAY :: DATE < dateadd(day, 3,
P.RELEASE_DATE)
• GROUP BY P.ASIN, P.TITLE, R.POSTAL_CODE, P.RELEASE_DATE
• ORDER BY SALES_sum DESC
• LIMIT 20;

15. 15
www.matillion.com
© 2017 Matillion. All rights reserved.
Now let’s run that query over an exabyte of data in S3
Roughly 140 TB of customer item order detail records for
each day over past 20 years.
190 million files across 15,000 partitions in S3. One partition
per day for USA and rest of world.
Need a billion-fold reduction in data processed.
Running this query using a 1000 node Hive cluster would
take over 5 years.*
• Compression ……………..….……..5X
• Columnar file format……….......…10X
• Scanning with 2500 nodes…....2500X
• Static partition elimination…............2X
• Dynamic partition elimination..….350X
• Redshift’s query optimizer……......40X
---------------------------------------------------
Total reduction……….…………3.5B X
* Estimated using 20 node Hive cluster & 1.4TB, assume linear
* Query used a 20 node DC1.8XLarge Amazon Redshift cluster
* Not actual sales data - generated for this demo based on data format
used by Amazon Retail.

16. 16
www.matillion.com
© 2017 Matillion. All rights reserved.
Accelerate Migrations from Legacy Systems
“AWS Database Migration Service is the most
impressive migration service we’ve seen.” – Gartner
Amazon Redshift
Migrate
Over 1,000 unique
migrations to Amazon
Redshift using DMS

17. 17
www.matillion.com
© 2017 Matillion. All rights reserved.
Recap on Amazon Redshift Spectrum
Catalog
JDBC/ODBC Client Application
Leader Node
Node 1 Node nNode 1
Spectrum Spectrum Spectrum Spectrum
S3
Independent Scaling
Redshift Scaling

18. 18
www.matillion.com
© 2017 Matillion. All rights reserved.
• No “load” into Redshift
• Independently-managed infrastructure
• Many formats
• Ordinary Redshift SQL
• Predicates and Aggregates are pushed down into Spectrum (but not Joins)
• You can visualize external data using e.g. Tableau
• Partitions are S3 prefixes
Recap on Amazon Redshift Spectrum

19. 19
www.matillion.com
© 2017 Matillion. All rights reserved.
• Other tools  S3 + Athena  Spectrum/Redshift
• Redshift on demand
• Athena
• Predicates, Aggregates, Joins
• Unload
Demonstration, including Redshift on demand

20. 20
www.matillion.com
© 2017 Matillion. All rights reserved.
Missed parts 1 and 2 of our Spectrum Series?
• September 20, 11AM EST: Getting started with Amazon Redshift Spectrum
 View an on-demand recording now (link in resource list)
• October 4, 11AM EST: Using Amazon Redshift Spectrum from Matillion ETL
 View an on-demand recording now (link in resource list)
 EST: Using Amazon Redshift Spectrum from Matillion ETL
• October 18, 11AM EST: Accessing your Data Lake assets from Amazon Redshift Spectrum

21. 21
www.matillion.com
© 2017 Matillion. All rights reserved.

22. 22
www.matillion.com
© 2017 Matillion. All rights reserved.
Presented by:
Copyright © 2017. All rights reserved. Matillion, trademarks, registered trademarks
or service marks are property of their respective owners. 10/27/2017
Thank You
James Johnson, Ian Funnell and Greg Khairallah