Redshift is a petabyte-scale data warehouse that is a lot faster, a lot less expensive and a whole lot simpler to use. How can you get your data into Amazon Redshift? In this webinar, hear from representatives of Attunity (Amazon Redshift Partner), and AWS as they present many of the options available for data integration. Whether your data is in an on premise platform or a cloud based database like DynamoDB, we will show you how you can easily load your data in to Re dshift. Reasons to attend: - Learn about best practices to efficiently integrate data into Redshift. - Attend Q&A session with Redshift experts

1. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Data Integration into Amazon Redshift
Brad Helicher - Director of Cloud Business, Attunity
Reza Khan - Director of Global Support Services, Attunity
John Loughlin - Business Development Manager, Amazon Web Services

2. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Redshift Webinars
Various topics
• Overview: Introducing Redshift
• Best Practices 1: Data Loading and Key Choices
• Best Practices 2: Workload Migration and Space
Management
http://aws.amazon.com/resources/databaseservices/webin
ars

3. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Agenda
Data Integration in Redshift
• Integration with Amazon S3
• Integration with DynamoDB
• Partner Talk: Attunity Overview and Demo
• Wrap up
• Questions and Answers

4. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Getting data to the Amazon Cloud
Multi-part Upload
VPN
Direct Connect
Import Export

5. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Amazon Redshift Loading Data Overview
AWS CloudCorporate Data center
DynamoDB
Amazon S3
Data Volume
Amazon Elastic
MapReduce
Amazon RDS
Amazon
Redshift
Amazon
Glacier
logs / files
Source DBs
VPN
Connection
AWS Direct
Connect
S3 Multipart
Upload
AWS Import/
Export

6. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Native Integration
Load Data from DynamoDB
Load from Amazon S3
Data Pipeline

7. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Amazon Redshift Loading Data Overview
AWS CloudCorporate Data center
DynamoDB
Amazon S3
Data Volume
Amazon Elastic
MapReduce
Amazon RDS
Amazon
Redshift
Amazon
Glacier
logs / files
Source DBs
VPN
Connection
AWS Direct
Connect
S3 Multipart
Upload
AWS Import/
Export
Loading Data
from DynamoDB

8. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Loading data from a DynamoDB table
DynamoDB
Table
Amazon Redshift
COPY command
Amazon
Redshift
Copy orders from ‘dynamodb://orders’
Credentials ‘aws_access_key_id=<your-access-key>;
aws_secret_access_key=<your_secret_key>’
Readratio 50;

9. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
AWS CloudSocial Data
Redshift Data
Warehouse
Query & Report
DynamoDB
Online Registration
Web Apps
Reporting
and BI
DynamoDB Integration with Redshift

10. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Amazon Redshift Loading Data Overview
AWS CloudCorporate Data center
DynamoDB
Amazon S3
Data Volume
Amazon Elastic
MapReduce
Amazon RDS
Amazon
Redshift
Amazon
Glacier
logs / files
Source DBs
VPN
Connection
AWS Direct
Connect
S3 Multipart
Upload
AWS Import/
Export
Loading Data from S3

11. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Uploading Files to Amazon S3
Amazon
Redshiftmydata
Client.txt
Corporate Data center
Region
Ensure that your
data resides in the
same Region as
your Redshift
clusters
Split the data into
multiple files to
facilitate parallel
processing
Client.txt.
1
Client.txt.
2
Client.txt.
3
Client.txt.
4

12. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Unstructured Data and Redshift
transform
and enrich
S3 S3
EMR
Redshift
logs / files
Data
Pipeline
Reporting and
BI
exploratory
analytics

13. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Introduce Attunity

14. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Questions

15. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Amazon Redshift Partners
Data
Integration
Systems
Integrators
Business
Intelligence

16. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
References
Webinars on Best Practices, Redshift Overview and a
variety of database topics:
http://aws.amazon.com/resources/databaseservices/web
inars
Redshift partners:
http://aws.amazon.com/redshift/partners/

17. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
WEBINAR
Data Integration
into Amazon Redshift
www.attunitycloudbeam.co
m

18. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
STRUCTURED
SEMI-STRUCTURED
UNSTRUCTURED
Any Data
Any Time
Any Where
High Performance
Lower Total Cost
Quick Time to Value
Attunity
Moving the Data that Moves Your Business
WHERE DATA RESIDES
18
C-Level /
Management
Line of
Business
Analyst
Data
Warehouse
BI/Analytics
Server
Hadoop /
HDFS
Cloud
WHERE DATA NEEDS TO BE
ANALYTICS VALUEBIG DATA
CRM ERP
Content
Management
Web Logs
HR
Systems
Example Sources
APPLICATIONS
Sensors
OTHER
AND MORE…
www.attunitycloudbeam.co

19. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Moving Data into the DW is a Common Issue
Only 17%
of organizations are very satisfied
with the performance of their
data warehouse loading process.
– IDC Survey
19
“
”www.attunitycloudbeam.co

20. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Pains in Data Acquisition for the Cloud
1. Complexity
2. Takes too long
3. Costs too much
4. Not real-time
5. Lack of Developer Resources
20
www.attunitycloudbeam.co

21. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
The Easy Way
To Get Data Into Amazon Redshift
Data Value
Click-2-Load. Optimized. Affordable.
More Data
Less Time
Less Cost
• Easy, no coding, no complexity
• Fully automated, end to end
• Fast, high performance integration
• Incremental and/or Real-time
Loading 21
www.attunitycloudbeam.co

22. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Attunity CloudBeam for Amazon Redshift
Optimized, end-to-end solution for
accelerating data loading into Redshift
Automated solution, easy to set-up and
manage
Supports many on-premises source
DB’s:
22
Source
Database
Amazon
Redshift
(on-prem)
Data Source Full Load CDC
Oracle + +
SQL Server + +
DB2 LUW + +
DB2 for iSeries + +
DB2 for z/OS + +
Sybase + +*
mySQL +
Salesforce +
ODBC +
www.attunitycloudbeam.co

23. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
23
Web-based Designer and
Management Console
Target
Database
Replication Server
In Memory Processing
Transform
Filter
Persistent Store
Source
Database
Transaction
Log
Bulk Reader
CDC
Bulk Loader
Stream Loader
Data / Metadata
Data / Metadata
Attunity CloudBeam for Amazon
Redshift
Attunity Replicate – on premises
www.attunitycloudbeam.co

24. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Replication Server
Attunity CloudBeam for Amazon Redshift –
Full Load
24
Source Database
3a Execute ‘copy’ command
to load data tables from S3
1 Generate table files
3b ‘Copy’ data from S3
Table Files
(folder per table)
S3
Table Files
in customer’s S3 account
Amazon
Redshift
AWS Region
2a
Beam
files to S3
2b
Validate file
content upon
arrival
3b Receive Acknowledgment on
successful ‘copy’ and apply
www.attunitycloudbeam.co

25. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Replication Server
Attunity CloudBeam for Amazon Redshift –
Incremental Load (CDC)
25
Source Database
1 Generate change files
Change Files (CDC)
Net Changes file
S3
Change Files
in customer’s S3 account
3b ‘Copy’ data to CDC table
4
Execute SQL commands
‘merge’ change into data tables
Amazon
Redshift AWS Region
Data Tables
CDC Table
2a
Beam
files to S3
2b
Validate file
content upon
arrival
3a Execute ‘copy’ command
to load data tables from S3
3b Receive Acknowledgment on
successful ‘copy’ and apply
www.attunitycloudbeam.co

26. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Attunity CloudBeam – Replicate for Redshift
Performance Optimizations
Optimized transfer protocol
Data transfer technologies:
Leverages Amazon multi-part transfers
Concurrent Sessions / Transfers
Compression
Recoverability, Guaranteed Delivery
SSL Encryption
Performance Gains:
10-12x over Standard Copy
Common Variables:
Bandwidth
Hardware
Data set
26
www.attunitycloudbeam.co

27. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
DEMO
Loading Oracle Data On-Prem to Amazon Redshift
High-Performance Information Availability Solutions. Made 27

28. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
The Easy Way
To Get Data Into Amazon Redshift
Data Value
Click-2-Load. Optimized. Affordable.
More Data
Less Time
Less Cost
• Easy, no coding, no complexity
• Fully automated, end to end
• Fast, high performance integration
• Incremental and/or Real-time
Loading
28
www.attunitycloudbeam.co

29. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Start Today. Let us Help.
Sign up and check it out:
www.attunitycloudbeam.com
* on-demand subscription starts as low as $350/month
Contact us for more information:
Brad Helicher
954-946-2274, ext. 1105
Brad.helicher@attunity.com 29

30. WEBINAR
Data Integration
into Amazon Redshift
www.attunitycloudbeam.com

31. STRUCTURED
SEMI-STRUCTURED
UNSTRUCTURED
Any Data
Any Time
Any Where
High Performance
Lower Total Cost
Quick Time to Value
Attunity
Moving the Data that Moves Your Business
WHERE DATA RESIDES
31
C-Level /
Management
Line of
Business
Analyst
Data
Warehouse
BI/Analytics
Server
Hadoop /
HDFS
Cloud
WHERE DATA NEEDS TO BE
ANALYTICS VALUEBIG DATA
CRM ERP
Content
Management
Web Logs
HR
Systems
Example Sources
APPLICATIONS
Sensors
OTHER
AND MORE…
www.attunitycloudbeam.com

32. Moving Data into the DW is a Common Issue
Only 17%
of organizations are very satisfied
with the performance of their
data warehouse loading process.
– IDC Survey
32
“
”www.attunitycloudbeam.com

33. Pains in Data Acquisition for the Cloud
1. Complexity
2. Takes too long
3. Costs too much
4. Not real-time
5. Lack of Developer Resources
33
www.attunitycloudbeam.com

34. The Easy Way
To Get Data Into Amazon Redshift
Data Value
Click-2-Load. Optimized. Affordable.
More Data
Less Time
Less Cost
• Easy, no coding, no complexity
• Fully automated, end to end
• Fast, high performance integration
• Incremental and/or Real-time Loading
• Significantly lower cost 34
www.attunitycloudbeam.com

35. Attunity CloudBeam for Amazon Redshift
» Optimized, end-to-end solution for
accelerating data loading into Redshift
» Automated solution, easy to set-up and
manage
» Supports many on-premises source DB’s:
35
Source
Database
Amazon
Redshift
(on-prem)
Data Source Full Load CDC
Oracle + +
SQL Server + +
DB2 LUW + +
DB2 for iSeries + +
DB2 for z/OS + +
Sybase + +*
mySQL +
Salesforce +
ODBC +
www.attunitycloudbeam.com

36. 36
Web-based Designer and
Management Console
Target
Database
Replication Server
In Memory Processing
Transform
Filter
Persistent Store
Source
Database
Transaction
Log
Bulk Reader
CDC
Bulk Loader
Stream Loader
Data / Metadata
Data / Metadata
Attunity CloudBeam for Amazon Redshift
Attunity Replicate – on premises
www.attunitycloudbeam.com

37. Replication Server
Attunity CloudBeam for Amazon Redshift –
Full Load
37
Source Database
3a Execute ‘copy’ command
to load data tables from S3
1 Generate table files
3b ‘Copy’ data from S3
Table Files
(folder per table)
S3
Table Files
in customer’s S3 account
Amazon
Redshift
AWS Region
2a Beam
files to S3
2b Validate file
content upon
arrival
3b Receive Acknowledgment on
successful ‘copy’ and apply
www.attunitycloudbeam.com

38. Replication Server
Attunity CloudBeam for Amazon Redshift –
Incremental Load (CDC)
38
Source Database
1 Generate change files
Change Files (CDC)
Net Changes file
S3
Change Files
in customer’s S3 account
3b ‘Copy’ data to CDC table
4
Execute SQL commands
‘merge’ change into data tables
Amazon
Redshift AWS Region
Data Tables
CDC Table
2a Beam
files to S3
2b Validate file
content upon
arrival
3a Execute ‘copy’ command
to load data tables from S3
3b Receive Acknowledgment on
successful ‘copy’ and apply
www.attunitycloudbeam.com

39. Attunity CloudBeam – Replicate for Redshift
Performance Optimizations
» Optimized transfer protocol
» Data transfer technologies:
» Leverages Amazon multi-part transfers
» Concurrent Sessions / Transfers
» Compression
» Recoverability, Guaranteed Delivery
» SSL Encryption
» Performance Gains:
» 10-12x over Standard Copy
» Common Variables:
» Bandwidth
» Hardware
» Data set
39
www.attunitycloudbeam.com

40. DEMO
Loading Oracle Data On-Prem to Amazon Redshift
High-Performance Information Availability Solutions. Made Radically Simple.40

41. The Easy Way
To Get Data Into Amazon Redshift
Data Value
Click-2-Load. Optimized. Affordable.
More Data
Less Time
Less Cost
• Easy, no coding, no complexity
• Fully automated, end to end
• Fast, high performance integration
• Incremental and/or Real-time Loading
• Significantly lower cost
41
www.attunitycloudbeam.com

42. Start Today. Let us Help.
» Sign up and check it out:
www.attunitycloudbeam.com
* on-demand subscription starts as low as $350/month
» Contact us for more information:
Brad Helicher
954-946-2274, ext. 1105
Brad.helicher@attunity.com 42