© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
WEBINAR
Data Integration
into Amazon Redshift
www.attunitycloudbeam.com
STRUCTURED
SEMI-STRUCTURED
UNSTRUCTURED
Any Data
Any Time
Any Where
High Performance
Lower Total Cost
Quick Time to Value
...
Moving Data into the DW is a Common Issue
Only 17%
of organizations are very satisfied
with the performance of their
data ...
Pains in Data Acquisition for the Cloud
1. Complexity
2. Takes too long
3. Costs too much
4. Not real-time
5. Lack of Deve...
The Easy Way
To Get Data Into Amazon Redshift
Data Value
Click-2-Load. Optimized. Affordable.
More Data
Less Time
Less Cos...
Attunity CloudBeam for Amazon Redshift
» Optimized, end-to-end solution for
accelerating data loading into Redshift
» Auto...
36
Web-based Designer and
Management Console
Target
Database
Replication Server
In Memory Processing
Transform
Filter
Pers...
Replication Server
Attunity CloudBeam for Amazon Redshift –
Full Load
37
Source Database
3a Execute ‘copy’ command
to load...
Replication Server
Attunity CloudBeam for Amazon Redshift –
Incremental Load (CDC)
38
Source Database
1 Generate change fi...
Attunity CloudBeam – Replicate for Redshift
Performance Optimizations
» Optimized transfer protocol
» Data transfer techno...
DEMO
Loading Oracle Data On-Prem to Amazon Redshift
High-Performance Information Availability Solutions. Made Radically Si...
The Easy Way
To Get Data Into Amazon Redshift
Data Value
Click-2-Load. Optimized. Affordable.
More Data
Less Time
Less Cos...
Start Today. Let us Help.
» Sign up and check it out:
www.attunitycloudbeam.com
* on-demand subscription starts as low as ...
Upcoming SlideShare
Loading in...5
×

AWS Webcast - Data Integration into Amazon Redshift

2,800

Published on

Redshift is a petabyte-scale data warehouse that is a lot faster, a lot less expensive and a whole lot simpler to use. How can you get your data into Amazon Redshift? In this webinar, hear from representatives of Attunity (Amazon Redshift Partner), and AWS as they present many of the options available for data integration. Whether your data is in an on premise platform or a cloud based database like DynamoDB, we will show you how you can easily load your data in to Re
dshift.

Reasons to attend: - Learn about best practices to efficiently integrate data into Redshift. - Attend Q&A session with Redshift experts

Published in: Business, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,800
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
92
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

AWS Webcast - Data Integration into Amazon Redshift

  1. 1. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Data Integration into Amazon Redshift Brad Helicher - Director of Cloud Business, Attunity Reza Khan - Director of Global Support Services, Attunity John Loughlin - Business Development Manager, Amazon Web Services
  2. 2. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Redshift Webinars Various topics • Overview: Introducing Redshift • Best Practices 1: Data Loading and Key Choices • Best Practices 2: Workload Migration and Space Management http://aws.amazon.com/resources/databaseservices/webin ars
  3. 3. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Agenda Data Integration in Redshift • Integration with Amazon S3 • Integration with DynamoDB • Partner Talk: Attunity Overview and Demo • Wrap up • Questions and Answers
  4. 4. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Getting data to the Amazon Cloud Multi-part Upload VPN Direct Connect Import Export
  5. 5. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Amazon Redshift Loading Data Overview AWS CloudCorporate Data center DynamoDB Amazon S3 Data Volume Amazon Elastic MapReduce Amazon RDS Amazon Redshift Amazon Glacier logs / files Source DBs VPN Connection AWS Direct Connect S3 Multipart Upload AWS Import/ Export
  6. 6. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Native Integration Load Data from DynamoDB Load from Amazon S3 Data Pipeline
  7. 7. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Amazon Redshift Loading Data Overview AWS CloudCorporate Data center DynamoDB Amazon S3 Data Volume Amazon Elastic MapReduce Amazon RDS Amazon Redshift Amazon Glacier logs / files Source DBs VPN Connection AWS Direct Connect S3 Multipart Upload AWS Import/ Export Loading Data from DynamoDB
  8. 8. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Loading data from a DynamoDB table DynamoDB Table Amazon Redshift COPY command Amazon Redshift Copy orders from ‘dynamodb://orders’ Credentials ‘aws_access_key_id=<your-access-key>; aws_secret_access_key=<your_secret_key>’ Readratio 50;
  9. 9. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. AWS CloudSocial Data Redshift Data Warehouse Query & Report DynamoDB Online Registration Web Apps Reporting and BI DynamoDB Integration with Redshift
  10. 10. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Amazon Redshift Loading Data Overview AWS CloudCorporate Data center DynamoDB Amazon S3 Data Volume Amazon Elastic MapReduce Amazon RDS Amazon Redshift Amazon Glacier logs / files Source DBs VPN Connection AWS Direct Connect S3 Multipart Upload AWS Import/ Export Loading Data from S3
  11. 11. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Uploading Files to Amazon S3 Amazon Redshiftmydata Client.txt Corporate Data center Region Ensure that your data resides in the same Region as your Redshift clusters Split the data into multiple files to facilitate parallel processing Client.txt. 1 Client.txt. 2 Client.txt. 3 Client.txt. 4
  12. 12. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Unstructured Data and Redshift transform and enrich S3 S3 EMR Redshift logs / files Data Pipeline Reporting and BI exploratory analytics
  13. 13. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Introduce Attunity
  14. 14. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Questions
  15. 15. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Amazon Redshift Partners Data Integration Systems Integrators Business Intelligence
  16. 16. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. References Webinars on Best Practices, Redshift Overview and a variety of database topics: http://aws.amazon.com/resources/databaseservices/web inars Redshift partners: http://aws.amazon.com/redshift/partners/
  17. 17. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. WEBINAR Data Integration into Amazon Redshift www.attunitycloudbeam.co m
  18. 18. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. STRUCTURED SEMI-STRUCTURED UNSTRUCTURED Any Data Any Time Any Where High Performance Lower Total Cost Quick Time to Value Attunity Moving the Data that Moves Your Business WHERE DATA RESIDES 18 C-Level / Management Line of Business Analyst Data Warehouse BI/Analytics Server Hadoop / HDFS Cloud WHERE DATA NEEDS TO BE ANALYTICS VALUEBIG DATA CRM ERP Content Management Web Logs HR Systems Example Sources APPLICATIONS Sensors OTHER AND MORE… www.attunitycloudbeam.co
  19. 19. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Moving Data into the DW is a Common Issue Only 17% of organizations are very satisfied with the performance of their data warehouse loading process. – IDC Survey 19 “ ”www.attunitycloudbeam.co
  20. 20. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Pains in Data Acquisition for the Cloud 1. Complexity 2. Takes too long 3. Costs too much 4. Not real-time 5. Lack of Developer Resources 20 www.attunitycloudbeam.co
  21. 21. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. The Easy Way To Get Data Into Amazon Redshift Data Value Click-2-Load. Optimized. Affordable. More Data Less Time Less Cost • Easy, no coding, no complexity • Fully automated, end to end • Fast, high performance integration • Incremental and/or Real-time Loading 21 www.attunitycloudbeam.co
  22. 22. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Attunity CloudBeam for Amazon Redshift Optimized, end-to-end solution for accelerating data loading into Redshift Automated solution, easy to set-up and manage Supports many on-premises source DB’s: 22 Source Database Amazon Redshift (on-prem) Data Source Full Load CDC Oracle + + SQL Server + + DB2 LUW + + DB2 for iSeries + + DB2 for z/OS + + Sybase + +* mySQL + Salesforce + ODBC + www.attunitycloudbeam.co
  23. 23. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 23 Web-based Designer and Management Console Target Database Replication Server In Memory Processing Transform Filter Persistent Store Source Database Transaction Log Bulk Reader CDC Bulk Loader Stream Loader Data / Metadata Data / Metadata Attunity CloudBeam for Amazon Redshift Attunity Replicate – on premises www.attunitycloudbeam.co
  24. 24. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Replication Server Attunity CloudBeam for Amazon Redshift – Full Load 24 Source Database 3a Execute ‘copy’ command to load data tables from S3 1 Generate table files 3b ‘Copy’ data from S3 Table Files (folder per table) S3 Table Files in customer’s S3 account Amazon Redshift AWS Region 2a Beam files to S3 2b Validate file content upon arrival 3b Receive Acknowledgment on successful ‘copy’ and apply www.attunitycloudbeam.co
  25. 25. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Replication Server Attunity CloudBeam for Amazon Redshift – Incremental Load (CDC) 25 Source Database 1 Generate change files Change Files (CDC) Net Changes file S3 Change Files in customer’s S3 account 3b ‘Copy’ data to CDC table 4 Execute SQL commands ‘merge’ change into data tables Amazon Redshift AWS Region Data Tables CDC Table 2a Beam files to S3 2b Validate file content upon arrival 3a Execute ‘copy’ command to load data tables from S3 3b Receive Acknowledgment on successful ‘copy’ and apply www.attunitycloudbeam.co
  26. 26. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Attunity CloudBeam – Replicate for Redshift Performance Optimizations Optimized transfer protocol Data transfer technologies: Leverages Amazon multi-part transfers Concurrent Sessions / Transfers Compression Recoverability, Guaranteed Delivery SSL Encryption Performance Gains: 10-12x over Standard Copy Common Variables: Bandwidth Hardware Data set 26 www.attunitycloudbeam.co
  27. 27. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. DEMO Loading Oracle Data On-Prem to Amazon Redshift High-Performance Information Availability Solutions. Made 27
  28. 28. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. The Easy Way To Get Data Into Amazon Redshift Data Value Click-2-Load. Optimized. Affordable. More Data Less Time Less Cost • Easy, no coding, no complexity • Fully automated, end to end • Fast, high performance integration • Incremental and/or Real-time Loading 28 www.attunitycloudbeam.co
  29. 29. © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Start Today. Let us Help. Sign up and check it out: www.attunitycloudbeam.com * on-demand subscription starts as low as $350/month Contact us for more information: Brad Helicher 954-946-2274, ext. 1105 Brad.helicher@attunity.com 29
  30. 30. WEBINAR Data Integration into Amazon Redshift www.attunitycloudbeam.com
  31. 31. STRUCTURED SEMI-STRUCTURED UNSTRUCTURED Any Data Any Time Any Where High Performance Lower Total Cost Quick Time to Value Attunity Moving the Data that Moves Your Business WHERE DATA RESIDES 31 C-Level / Management Line of Business Analyst Data Warehouse BI/Analytics Server Hadoop / HDFS Cloud WHERE DATA NEEDS TO BE ANALYTICS VALUEBIG DATA CRM ERP Content Management Web Logs HR Systems Example Sources APPLICATIONS Sensors OTHER AND MORE… www.attunitycloudbeam.com
  32. 32. Moving Data into the DW is a Common Issue Only 17% of organizations are very satisfied with the performance of their data warehouse loading process. – IDC Survey 32 “ ”www.attunitycloudbeam.com
  33. 33. Pains in Data Acquisition for the Cloud 1. Complexity 2. Takes too long 3. Costs too much 4. Not real-time 5. Lack of Developer Resources 33 www.attunitycloudbeam.com
  34. 34. The Easy Way To Get Data Into Amazon Redshift Data Value Click-2-Load. Optimized. Affordable. More Data Less Time Less Cost • Easy, no coding, no complexity • Fully automated, end to end • Fast, high performance integration • Incremental and/or Real-time Loading • Significantly lower cost 34 www.attunitycloudbeam.com
  35. 35. Attunity CloudBeam for Amazon Redshift » Optimized, end-to-end solution for accelerating data loading into Redshift » Automated solution, easy to set-up and manage » Supports many on-premises source DB’s: 35 Source Database Amazon Redshift (on-prem) Data Source Full Load CDC Oracle + + SQL Server + + DB2 LUW + + DB2 for iSeries + + DB2 for z/OS + + Sybase + +* mySQL + Salesforce + ODBC + www.attunitycloudbeam.com
  36. 36. 36 Web-based Designer and Management Console Target Database Replication Server In Memory Processing Transform Filter Persistent Store Source Database Transaction Log Bulk Reader CDC Bulk Loader Stream Loader Data / Metadata Data / Metadata Attunity CloudBeam for Amazon Redshift Attunity Replicate – on premises www.attunitycloudbeam.com
  37. 37. Replication Server Attunity CloudBeam for Amazon Redshift – Full Load 37 Source Database 3a Execute ‘copy’ command to load data tables from S3 1 Generate table files 3b ‘Copy’ data from S3 Table Files (folder per table) S3 Table Files in customer’s S3 account Amazon Redshift AWS Region 2a Beam files to S3 2b Validate file content upon arrival 3b Receive Acknowledgment on successful ‘copy’ and apply www.attunitycloudbeam.com
  38. 38. Replication Server Attunity CloudBeam for Amazon Redshift – Incremental Load (CDC) 38 Source Database 1 Generate change files Change Files (CDC) Net Changes file S3 Change Files in customer’s S3 account 3b ‘Copy’ data to CDC table 4 Execute SQL commands ‘merge’ change into data tables Amazon Redshift AWS Region Data Tables CDC Table 2a Beam files to S3 2b Validate file content upon arrival 3a Execute ‘copy’ command to load data tables from S3 3b Receive Acknowledgment on successful ‘copy’ and apply www.attunitycloudbeam.com
  39. 39. Attunity CloudBeam – Replicate for Redshift Performance Optimizations » Optimized transfer protocol » Data transfer technologies: » Leverages Amazon multi-part transfers » Concurrent Sessions / Transfers » Compression » Recoverability, Guaranteed Delivery » SSL Encryption » Performance Gains: » 10-12x over Standard Copy » Common Variables: » Bandwidth » Hardware » Data set 39 www.attunitycloudbeam.com
  40. 40. DEMO Loading Oracle Data On-Prem to Amazon Redshift High-Performance Information Availability Solutions. Made Radically Simple.40
  41. 41. The Easy Way To Get Data Into Amazon Redshift Data Value Click-2-Load. Optimized. Affordable. More Data Less Time Less Cost • Easy, no coding, no complexity • Fully automated, end to end • Fast, high performance integration • Incremental and/or Real-time Loading • Significantly lower cost 41 www.attunitycloudbeam.com
  42. 42. Start Today. Let us Help. » Sign up and check it out: www.attunitycloudbeam.com * on-demand subscription starts as low as $350/month » Contact us for more information: Brad Helicher 954-946-2274, ext. 1105 Brad.helicher@attunity.com 42
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×