• Like
  • Save
AWS Webcast - Data Integration into Amazon Redshift
Upcoming SlideShare
Loading in...5
×
 

AWS Webcast - Data Integration into Amazon Redshift

on

  • 2,392 views

Redshift is a petabyte-scale data warehouse that is a lot faster, a lot less expensive and a whole lot simpler to use. How can you get your data into Amazon Redshift? In this webinar, hear from ...

Redshift is a petabyte-scale data warehouse that is a lot faster, a lot less expensive and a whole lot simpler to use. How can you get your data into Amazon Redshift? In this webinar, hear from representatives of Attunity (Amazon Redshift Partner), and AWS as they present many of the options available for data integration. Whether your data is in an on premise platform or a cloud based database like DynamoDB, we will show you how you can easily load your data in to Re
dshift.

Reasons to attend: - Learn about best practices to efficiently integrate data into Redshift. - Attend Q&A session with Redshift experts

Statistics

Views

Total Views
2,392
Views on SlideShare
2,392
Embed Views
0

Actions

Likes
1
Downloads
58
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    AWS Webcast - Data Integration into Amazon Redshift AWS Webcast - Data Integration into Amazon Redshift Presentation Transcript

    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Data Integration into Amazon Redshift Brad Helicher - Director of Cloud Business, Attunity Reza Khan - Director of Global Support Services, Attunity John Loughlin - Business Development Manager, Amazon Web Services
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Redshift Webinars Various topics • Overview: Introducing Redshift • Best Practices 1: Data Loading and Key Choices • Best Practices 2: Workload Migration and Space Management http://aws.amazon.com/resources/databaseservices/webin ars
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Agenda Data Integration in Redshift • Integration with Amazon S3 • Integration with DynamoDB • Partner Talk: Attunity Overview and Demo • Wrap up • Questions and Answers
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Getting data to the Amazon Cloud Multi-part Upload VPN Direct Connect Import Export
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Amazon Redshift Loading Data Overview AWS CloudCorporate Data center DynamoDB Amazon S3 Data Volume Amazon Elastic MapReduce Amazon RDS Amazon Redshift Amazon Glacier logs / files Source DBs VPN Connection AWS Direct Connect S3 Multipart Upload AWS Import/ Export
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Native Integration Load Data from DynamoDB Load from Amazon S3 Data Pipeline
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Amazon Redshift Loading Data Overview AWS CloudCorporate Data center DynamoDB Amazon S3 Data Volume Amazon Elastic MapReduce Amazon RDS Amazon Redshift Amazon Glacier logs / files Source DBs VPN Connection AWS Direct Connect S3 Multipart Upload AWS Import/ Export Loading Data from DynamoDB
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Loading data from a DynamoDB table DynamoDB Table Amazon Redshift COPY command Amazon Redshift Copy orders from ‘dynamodb://orders’ Credentials ‘aws_access_key_id=<your-access-key>; aws_secret_access_key=<your_secret_key>’ Readratio 50;
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. AWS CloudSocial Data Redshift Data Warehouse Query & Report DynamoDB Online Registration Web Apps Reporting and BI DynamoDB Integration with Redshift
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Amazon Redshift Loading Data Overview AWS CloudCorporate Data center DynamoDB Amazon S3 Data Volume Amazon Elastic MapReduce Amazon RDS Amazon Redshift Amazon Glacier logs / files Source DBs VPN Connection AWS Direct Connect S3 Multipart Upload AWS Import/ Export Loading Data from S3
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Uploading Files to Amazon S3 Amazon Redshiftmydata Client.txt Corporate Data center Region Ensure that your data resides in the same Region as your Redshift clusters Split the data into multiple files to facilitate parallel processing Client.txt. 1 Client.txt. 2 Client.txt. 3 Client.txt. 4
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Unstructured Data and Redshift transform and enrich S3 S3 EMR Redshift logs / files Data Pipeline Reporting and BI exploratory analytics
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Introduce Attunity
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Questions
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Amazon Redshift Partners Data Integration Systems Integrators Business Intelligence
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. References Webinars on Best Practices, Redshift Overview and a variety of database topics: http://aws.amazon.com/resources/databaseservices/web inars Redshift partners: http://aws.amazon.com/redshift/partners/
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. WEBINAR Data Integration into Amazon Redshift www.attunitycloudbeam.co m
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. STRUCTURED SEMI-STRUCTURED UNSTRUCTURED Any Data Any Time Any Where High Performance Lower Total Cost Quick Time to Value Attunity Moving the Data that Moves Your Business WHERE DATA RESIDES 18 C-Level / Management Line of Business Analyst Data Warehouse BI/Analytics Server Hadoop / HDFS Cloud WHERE DATA NEEDS TO BE ANALYTICS VALUEBIG DATA CRM ERP Content Management Web Logs HR Systems Example Sources APPLICATIONS Sensors OTHER AND MORE… www.attunitycloudbeam.co
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Moving Data into the DW is a Common Issue Only 17% of organizations are very satisfied with the performance of their data warehouse loading process. – IDC Survey 19 “ ”www.attunitycloudbeam.co
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Pains in Data Acquisition for the Cloud 1. Complexity 2. Takes too long 3. Costs too much 4. Not real-time 5. Lack of Developer Resources 20 www.attunitycloudbeam.co
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. The Easy Way To Get Data Into Amazon Redshift Data Value Click-2-Load. Optimized. Affordable. More Data Less Time Less Cost • Easy, no coding, no complexity • Fully automated, end to end • Fast, high performance integration • Incremental and/or Real-time Loading 21 www.attunitycloudbeam.co
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Attunity CloudBeam for Amazon Redshift Optimized, end-to-end solution for accelerating data loading into Redshift Automated solution, easy to set-up and manage Supports many on-premises source DB’s: 22 Source Database Amazon Redshift (on-prem) Data Source Full Load CDC Oracle + + SQL Server + + DB2 LUW + + DB2 for iSeries + + DB2 for z/OS + + Sybase + +* mySQL + Salesforce + ODBC + www.attunitycloudbeam.co
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 23 Web-based Designer and Management Console Target Database Replication Server In Memory Processing Transform Filter Persistent Store Source Database Transaction Log Bulk Reader CDC Bulk Loader Stream Loader Data / Metadata Data / Metadata Attunity CloudBeam for Amazon Redshift Attunity Replicate – on premises www.attunitycloudbeam.co
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Replication Server Attunity CloudBeam for Amazon Redshift – Full Load 24 Source Database 3a Execute ‘copy’ command to load data tables from S3 1 Generate table files 3b ‘Copy’ data from S3 Table Files (folder per table) S3 Table Files in customer’s S3 account Amazon Redshift AWS Region 2a Beam files to S3 2b Validate file content upon arrival 3b Receive Acknowledgment on successful ‘copy’ and apply www.attunitycloudbeam.co
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Replication Server Attunity CloudBeam for Amazon Redshift – Incremental Load (CDC) 25 Source Database 1 Generate change files Change Files (CDC) Net Changes file S3 Change Files in customer’s S3 account 3b ‘Copy’ data to CDC table 4 Execute SQL commands ‘merge’ change into data tables Amazon Redshift AWS Region Data Tables CDC Table 2a Beam files to S3 2b Validate file content upon arrival 3a Execute ‘copy’ command to load data tables from S3 3b Receive Acknowledgment on successful ‘copy’ and apply www.attunitycloudbeam.co
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Attunity CloudBeam – Replicate for Redshift Performance Optimizations Optimized transfer protocol Data transfer technologies: Leverages Amazon multi-part transfers Concurrent Sessions / Transfers Compression Recoverability, Guaranteed Delivery SSL Encryption Performance Gains: 10-12x over Standard Copy Common Variables: Bandwidth Hardware Data set 26 www.attunitycloudbeam.co
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. DEMO Loading Oracle Data On-Prem to Amazon Redshift High-Performance Information Availability Solutions. Made 27
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. The Easy Way To Get Data Into Amazon Redshift Data Value Click-2-Load. Optimized. Affordable. More Data Less Time Less Cost • Easy, no coding, no complexity • Fully automated, end to end • Fast, high performance integration • Incremental and/or Real-time Loading 28 www.attunitycloudbeam.co
    • © 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Start Today. Let us Help. Sign up and check it out: www.attunitycloudbeam.com * on-demand subscription starts as low as $350/month Contact us for more information: Brad Helicher 954-946-2274, ext. 1105 Brad.helicher@attunity.com 29
    • WEBINAR Data Integration into Amazon Redshift www.attunitycloudbeam.com
    • STRUCTURED SEMI-STRUCTURED UNSTRUCTURED Any Data Any Time Any Where High Performance Lower Total Cost Quick Time to Value Attunity Moving the Data that Moves Your Business WHERE DATA RESIDES 31 C-Level / Management Line of Business Analyst Data Warehouse BI/Analytics Server Hadoop / HDFS Cloud WHERE DATA NEEDS TO BE ANALYTICS VALUEBIG DATA CRM ERP Content Management Web Logs HR Systems Example Sources APPLICATIONS Sensors OTHER AND MORE… www.attunitycloudbeam.com
    • Moving Data into the DW is a Common Issue Only 17% of organizations are very satisfied with the performance of their data warehouse loading process. – IDC Survey 32 “ ”www.attunitycloudbeam.com
    • Pains in Data Acquisition for the Cloud 1. Complexity 2. Takes too long 3. Costs too much 4. Not real-time 5. Lack of Developer Resources 33 www.attunitycloudbeam.com
    • The Easy Way To Get Data Into Amazon Redshift Data Value Click-2-Load. Optimized. Affordable. More Data Less Time Less Cost • Easy, no coding, no complexity • Fully automated, end to end • Fast, high performance integration • Incremental and/or Real-time Loading • Significantly lower cost 34 www.attunitycloudbeam.com
    • Attunity CloudBeam for Amazon Redshift » Optimized, end-to-end solution for accelerating data loading into Redshift » Automated solution, easy to set-up and manage » Supports many on-premises source DB’s: 35 Source Database Amazon Redshift (on-prem) Data Source Full Load CDC Oracle + + SQL Server + + DB2 LUW + + DB2 for iSeries + + DB2 for z/OS + + Sybase + +* mySQL + Salesforce + ODBC + www.attunitycloudbeam.com
    • 36 Web-based Designer and Management Console Target Database Replication Server In Memory Processing Transform Filter Persistent Store Source Database Transaction Log Bulk Reader CDC Bulk Loader Stream Loader Data / Metadata Data / Metadata Attunity CloudBeam for Amazon Redshift Attunity Replicate – on premises www.attunitycloudbeam.com
    • Replication Server Attunity CloudBeam for Amazon Redshift – Full Load 37 Source Database 3a Execute ‘copy’ command to load data tables from S3 1 Generate table files 3b ‘Copy’ data from S3 Table Files (folder per table) S3 Table Files in customer’s S3 account Amazon Redshift AWS Region 2a Beam files to S3 2b Validate file content upon arrival 3b Receive Acknowledgment on successful ‘copy’ and apply www.attunitycloudbeam.com
    • Replication Server Attunity CloudBeam for Amazon Redshift – Incremental Load (CDC) 38 Source Database 1 Generate change files Change Files (CDC) Net Changes file S3 Change Files in customer’s S3 account 3b ‘Copy’ data to CDC table 4 Execute SQL commands ‘merge’ change into data tables Amazon Redshift AWS Region Data Tables CDC Table 2a Beam files to S3 2b Validate file content upon arrival 3a Execute ‘copy’ command to load data tables from S3 3b Receive Acknowledgment on successful ‘copy’ and apply www.attunitycloudbeam.com
    • Attunity CloudBeam – Replicate for Redshift Performance Optimizations » Optimized transfer protocol » Data transfer technologies: » Leverages Amazon multi-part transfers » Concurrent Sessions / Transfers » Compression » Recoverability, Guaranteed Delivery » SSL Encryption » Performance Gains: » 10-12x over Standard Copy » Common Variables: » Bandwidth » Hardware » Data set 39 www.attunitycloudbeam.com
    • DEMO Loading Oracle Data On-Prem to Amazon Redshift High-Performance Information Availability Solutions. Made Radically Simple.40
    • The Easy Way To Get Data Into Amazon Redshift Data Value Click-2-Load. Optimized. Affordable. More Data Less Time Less Cost • Easy, no coding, no complexity • Fully automated, end to end • Fast, high performance integration • Incremental and/or Real-time Loading • Significantly lower cost 41 www.attunitycloudbeam.com
    • Start Today. Let us Help. » Sign up and check it out: www.attunitycloudbeam.com * on-demand subscription starts as low as $350/month » Contact us for more information: Brad Helicher 954-946-2274, ext. 1105 Brad.helicher@attunity.com 42