In the first webinar in our Amazon Redshift Spectrum Series, learn more about getting started with Spectrum. Spectrum allows you to use the analytic capabilities of Amazon Redshift beyond the data which is in your data warehouse to query large amounts of semistructured and structured data in your “data lake,” without having to load or transform it into Amazon Redshift. In this webinar learn: - What is Amazon Redshift Spectrum? - How to set up AWS Security for Amazon Redshift Spectrum - What are external schemas and external tables - How to query S3 data with Redshift SQL via Amazon Redshift Spectrum using Matillion ETL

1. 1
www.matillion.com
© 2017 Matillion. All rights reserved.
Presented by:
Copyright © 2017. All rights reserved. Matillion, trademarks, registered trademarks
or service marks are property of their respective owners. 9/21/2017
September 20th, 2017
Part 1: Getting Started with Amazon Redshift Spectrum
Spectrum Webinar Series
David Langton and James Johnson

2. 2
www.matillion.com
© 2017 Matillion. All rights reserved.
• What is Amazon Redshift Spectrum?
• Query Architecture
• External Schemas
• External Tables
• Queries and Plans
Introduction

3. 3
www.matillion.com
© 2017 Matillion. All rights reserved.
• Run Redshift SQL directly against exabytes of S3 data
• Store table data in Redshift or S3. Query from Redshift.
• Scales independently from Redshift
• Priced per-query
What is Amazon Redshift Spectrum?

4. 4
www.matillion.com
© 2017 Matillion. All rights reserved.
• On Premise
• Amazon Redshift on AWS
• Amazon Redshift on AWS with Spectrum
Data Warehouse Scalability

5. 5
www.matillion.com
© 2017 Matillion. All rights reserved.
Architecture of Amazon Redshift Spectrum
Catalog
JDBC/ODBC Client Application
Leader Node
Node 1 Node nNode 1
Spectrum Spectrum Spectrum Spectrum
S3
Independent Scaling
Redshift Scaling

6. 6
www.matillion.com
© 2017 Matillion. All rights reserved.
• Provide access to S3 data seamlessly from Redshift
• Queries against external schema still understand ordinary Redshift SQL
• External Schema is a Pointer to a Data Catalog
- Athena (legacy)
- AWS Glue (new)
• Requires an IAM Role
- Spectrum is an external service
- It does not run in your VPC.
- The role must be attached to your Cluster
External Schemas
Redshift Schema Data Catalog

7. 7
www.matillion.com
© 2017 Matillion. All rights reserved.
Demo 1 - Setup external schema

8. 8
www.matillion.com
© 2017 Matillion. All rights reserved.
• Data Formats
- Avro, CSV, Grok, ORC, Parquet, RCFile,
RegexSerDe, SequenceFile, TextFile, and TSV
• Table Definition Syntax
- STRUCT, ARRAY, and MAP types are
unsupported.
- May be partitioned on one or more keys
• Sort / Distribution Options
- Spectrum does not use the same concepts -
you cannot set sort and/or distribution options
for Spectrum
External Tables
CREATE EXTERNAL TABLE
"webinar"."webinar_airports"
(
"iata" varchar(255),
"airport" varchar(255),
"city" varchar(255),
"state" varchar(255),
"country" varchar(255),
"lat" decimal(12, 8),
"long" decimal(12, 8)
)
STORED AS TEXTFILE
location
's3://<bucket>/<path>';

9. 9
www.matillion.com
© 2017 Matillion. All rights reserved.
Demo 2 - Setup external tables

10. 10
www.matillion.com
© 2017 Matillion. All rights reserved.
• The query plan shows when Redshift pushes down all or part of a query to
Spectrum
- S3 Seq Scan, S3 HashAggregate, S3 Query Scan, Seq Scan PartitionInfo, Partition Loop
• This plan is for a query on a single Redshift tables:
XN Seq Scan on webinar_airports_redshift (cost=0.00..42.20 rows=2 width=78)
Filter: ((city)::text = 'New York'::text)
• This plan is when that table is instead in Spectrum:
XN S3 Query Scan webinar_airports (cost=0.00..225000000.00 rows=10000000000 width=2028)
-> S3 Seq Scan video.webinar_airports location:"s3://mtln-spectrum-data/webinar/demo1"
format:TEXT (cost=0.00..125000000.00 rows=10000000000 width=2028)
Filter: ((city)::text = 'New York'::text)
Querying Redshift Tables and Spectrum Tables

11. 11
www.matillion.com
© 2017 Matillion. All rights reserved.
Summary

12. 12
www.matillion.com
© 2017 Matillion. All rights reserved.
Coming soon…the next 2 webinars in our series
• October 4, 11AM EST: Using Amazon Redshift Spectrum from Matillion ETL
• October 18, 11AM EST: Accessing your Data Lake assets from Amazon Redshift
Spectrum
• October 4, 11AM EST: Using Amazon Redshift Spectrum from Matillion ETL
• October 18, 11AM EST: Accessing your Data Lake assets from Amazon Redshift Spectrum

13. 13
www.matillion.com
© 2017 Matillion. All rights reserved.

14. 14
www.matillion.com
© 2017 Matillion. All rights reserved.
Presented by:
Copyright © 2017. All rights reserved. Matillion, trademarks, registered trademarks
or service marks are property of their respective owners. 9/21/2017
Thank You
David Langton and James Johnson