High Performance Big Data Loading
for AWS: Deep Dive and Best
Practices from Informatica
Ajay Gandhi, VP Cloud Product Mar...
Why Are Customers Adopting Cloud and AWS?
1.
Cost savings
through economics
of scale
Don’t have to
guess on capacity
3.
Ag...
3
So, How Do You Try Redshift – Quickly & Easily?
Amazon Redshift
4
Amazon Redshift
ERP, CRM Apps
Files
Legacy, RDBMS
Firewall
Logs, JSONs, Social
SaaS Apps
Use New Cloud & Traditional Dat...
5
How To Manage Integration In This New World?
Amazon Redshift
ERP, CRM Apps
Files
Legacy, RDBMS
Firewall
Experiment.
Prot...
AWS RDS Staging, Redshift DW, Infa Cloud
ERP, CRM Apps
Files
Legacy, RDBMS
Amazon
RDS
Logs, JSONs, Social
SaaS Apps
Experi...
Map Once. Deploy Anywhere.
ON PREMISE HADOOP 3rd PARTY
APPLICATIONS
CLOUD
AWS EMR (Hadoop) and DynamoDB (NoSQL)
ERP, CRM Apps
Files
Legacy, RDBMS
Amazon
RDS
Amazon
Redshift
Amazon
EMR
Logs, JSONs,...
Growth Path to Hybrid Data Warehouse
ERP, CRM Apps
Files
Legacy, RDBMS
Amazon
RDS
Amazon
Redshift
Amazon
EMR
Logs, JSONs, ...
Informatica Cloud - Get it right. Go live. Grow flexibly.
Cloud
Data Integration
Cloud
Real-time
Integration
Cloud Test
Da...
Hundreds of Connectors
JDBC
Technical Innovations for AWS Data Loading
• Out-of-the-box integration for S3, DynamoDB, Kinesis, Redshift and
RDS availa...
Loading data into
REDSHIFT,
DYNAMODB and RDS
2
Informatica Cloud Architecture Overview- Redshift
4Secure
Agent
Your Company or VPC
Amazon
Redshift
1
Amazon
RDSAmazon S...
Informatica Cloud Amazon Redshift Architecture
Firewall
Informatica Cloud
Secure Agent
Metadata Mappings
Build mapping and...
REDSHIFT and RDS
DEMO!
REDSHIFT and
DYNAMODB DEMO!
Loading data into
KINESIS
1 0 1010
1 0 1010
1 0 1010
1 0 1010
1 0 1010
1 0 1010
KINESIS
IoT: Operational Intelligence
Documents and
files
pdf DOC XLS EDI
Documents and
files
pdf DOC XLS EDI
Machine device,
cloud
Machine device,
cloud
Social...
Documents and
files
pdf DOC XLS EDI
Documents and
files
pdf DOC XLS EDI
Machine device,
cloud
Machine device,
cloud
Social...
Streaming Collection: Vibe Data Stream
VD
S
VD
S
VD
S
• Central Monitoring Console for
Deployment
• Fault Tolerant
• High ...
KINESIS DEMO!
Try it today:
community.informatica.com/solutions/
vibe_data_stream_for_kinesis
Next Steps
• Visit us at Booth# 107 to
see more demos
• Try our 60-Day free trial
for Redshift
• www.informaticacloud.com
...
Q & A
InformaticaCloud.com
High Performance Big Data Loading for AWS: Deep Dive and Best Practices from Informatica
Upcoming SlideShare
Loading in...5
×

High Performance Big Data Loading for AWS: Deep Dive and Best Practices from Informatica

1,028
-1

Published on

Companies are increasingly dealing with large data sets and looking for ways to increase the scale and lower the cost of Big Data analysis with AWS. In this interactive session, you’ll learn how to:
* Integrate massive data volumes, from any on-premises or cloud data sources into AWS with Informatica’s high performance cloud integration connectors and Vibe Secure Agent technology.
* Transform and load data into RDS, Redshift, and S3 without the need for coding.
* Automate streaming data collection into Kinesis with built-in high availability and failover features.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,028
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
57
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

High Performance Big Data Loading for AWS: Deep Dive and Best Practices from Informatica

  1. 1. High Performance Big Data Loading for AWS: Deep Dive and Best Practices from Informatica Ajay Gandhi, VP Cloud Product Marketing Nicolas Brisoux, Sr. Cloud Platform Specialist Roderick Clemente, Product Specialist July 10th, 2014
  2. 2. Why Are Customers Adopting Cloud and AWS? 1. Cost savings through economics of scale Don’t have to guess on capacity 3. Agility, Speed to market & Flexibility 4. Global in minutes 5. 2. Trade capital expense for variable expense Security and Compliance 6.
  3. 3. 3 So, How Do You Try Redshift – Quickly & Easily? Amazon Redshift
  4. 4. 4 Amazon Redshift ERP, CRM Apps Files Legacy, RDBMS Firewall Logs, JSONs, Social SaaS Apps Use New Cloud & Traditional Data Sources
  5. 5. 5 How To Manage Integration In This New World? Amazon Redshift ERP, CRM Apps Files Legacy, RDBMS Firewall Experiment. Prototype. Repeat. Logs, JSONs, Social SaaS Apps
  6. 6. AWS RDS Staging, Redshift DW, Infa Cloud ERP, CRM Apps Files Legacy, RDBMS Amazon RDS Logs, JSONs, Social SaaS Apps Experiment. Prototype. Repeat. Amazon Redshift
  7. 7. Map Once. Deploy Anywhere. ON PREMISE HADOOP 3rd PARTY APPLICATIONS CLOUD
  8. 8. AWS EMR (Hadoop) and DynamoDB (NoSQL) ERP, CRM Apps Files Legacy, RDBMS Amazon RDS Amazon Redshift Amazon EMR Logs, JSONs, Social SaaS Apps Dynamo DB
  9. 9. Growth Path to Hybrid Data Warehouse ERP, CRM Apps Files Legacy, RDBMS Amazon RDS Amazon Redshift Amazon EMR Logs, JSONs, Social SaaS Apps Dynamo DB Traditional Staging DB Traditional Data Warehouse
  10. 10. Informatica Cloud - Get it right. Go live. Grow flexibly. Cloud Data Integration Cloud Real-time Integration Cloud Test Data Management Cloud Data Quality Cloud Master Data Management Secure Development Data Leverage Existing Bulk Data Cleanse and De-dupe Data Consolidate and Visualize Data Instant Access to Actionable Data “The Informatica Cloud Platform is the only complete solution for cloud integration and data management that allows SaaS application administrators, architects, and developers to easily power optimal processes connected with enterprise-ready data across cloud, on-premises, big data, social, and mobile environments.”
  11. 11. Hundreds of Connectors JDBC
  12. 12. Technical Innovations for AWS Data Loading • Out-of-the-box integration for S3, DynamoDB, Kinesis, Redshift and RDS available NOW! • Agile data loading for cloud data warehousing with Redshift • Create target using cloud designer and multiple source objects • High performance parallel data loading architecture • E.g. load data in parallel across all 32 nodes in a Redshift cluster • Push down optimization for increased throughput • Push data transformations down to optimal source/target database engine ©2013 Informatica. Proprietary and Confidential 12
  13. 13. Loading data into REDSHIFT, DYNAMODB and RDS
  14. 14. 2 Informatica Cloud Architecture Overview- Redshift 4Secure Agent Your Company or VPC Amazon Redshift 1 Amazon RDSAmazon S3 Amazon DynamoDB 3
  15. 15. Informatica Cloud Amazon Redshift Architecture Firewall Informatica Cloud Secure Agent Metadata Mappings Build mapping and execute job 1 1 Retrieve Account Data 2 2 3 Put Account Data into Flat File 4 Transfer compressed Flat File to S3 5 Initiate copy from S3 6 Load data into Amazon Redshift 6 3 54 Amazon S3 Amazon Redshift
  16. 16. REDSHIFT and RDS DEMO!
  17. 17. REDSHIFT and DYNAMODB DEMO!
  18. 18. Loading data into KINESIS
  19. 19. 1 0 1010 1 0 1010 1 0 1010 1 0 1010 1 0 1010 1 0 1010 KINESIS IoT: Operational Intelligence
  20. 20. Documents and files pdf DOC XLS EDI Documents and files pdf DOC XLS EDI Machine device, cloud Machine device, cloud Social media, web Logs Social media, web Logs Machine device, cloud Social media, web Logs Documents and files pdf DOC XLS EDI Documents and files pdf DOC XLS EDI Machine device, cloud Social media, web Logs Documents and files pdf DOC XLS EDI Machine device, cloud Social media, web Logs aws amazon kinesis
  21. 21. Documents and files pdf DOC XLS EDI Documents and files pdf DOC XLS EDI Machine device, cloud Machine device, cloud Social media, web Logs Social media, web Logs Machine device, cloud Social media, web Logs Documents and files pdf DOC XLS EDI Documents and files pdf DOC XLS EDI Machine device, cloud Social media, web Logs Documents and files pdf DOC XLS EDI Machine device, cloud Social media, web Logs aws amazon kinesis
  22. 22. Streaming Collection: Vibe Data Stream VD S VD S VD S • Central Monitoring Console for Deployment • Fault Tolerant • High Availability • Vertical & Horizontal Scaling • Ease of Configuration Industrial Systems IoT devices Social media, web Logs aws amazon kinesis HVAC
  23. 23. KINESIS DEMO!
  24. 24. Try it today: community.informatica.com/solutions/ vibe_data_stream_for_kinesis
  25. 25. Next Steps • Visit us at Booth# 107 to see more demos • Try our 60-Day free trial for Redshift • www.informaticacloud.com /cloud-trial-for-redshift 26
  26. 26. Q & A InformaticaCloud.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×