• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
BDT201 AWS Data Pipeline - AWS re: Invent 2012
 

BDT201 AWS Data Pipeline - AWS re: Invent 2012

on

  • 4,554 views

In this session, we'll review the features and architecture of the new AWS Data Pipeline service and explain how you can use it to better manage your data-driven workloads. We'll then go over a few ...

In this session, we'll review the features and architecture of the new AWS Data Pipeline service and explain how you can use it to better manage your data-driven workloads. We'll then go over a few examples of setting up and provisioning a pipeline in the system.

Statistics

Views

Total Views
4,554
Views on SlideShare
4,489
Embed Views
65

Actions

Likes
10
Downloads
22
Comments
1

9 Embeds 65

https://twitter.com 36
http://ams.activemailservice.com 9
http://ology4.com 6
http://ursw.ie 4
http://www.thinkmarket.fr 3
https://twimg0-a.akamaihd.net 3
https://si0.twimg.com 2
http://www.onlydoo.com 1
http://www.ursw.ie 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Pagumetropics does inquire every scientist to explain that why do the tropics rotates faster than the temperates?
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    BDT201 AWS Data Pipeline - AWS re: Invent 2012 BDT201 AWS Data Pipeline - AWS re: Invent 2012 Presentation Transcript

    • Amazon S3 Amazon RDS Amazon AmazonDynamoDB Redshift HDFS On (Amazon EMR) Premise
    • Amazon DynamoDB Amazon S3
    • Amazon S3 Amazon RDS Amazon AmazonDynamoDB Redshift HDFS On (Amazon EMR) Premise
    • Amazon S3 Amazon RDS Amazon AmazonDynamoDB Redshift HDFS On (Amazon EMR) Premise
    • Amazon S3 Amazon RDS Amazon AmazonDynamoDB Redshift HDFS On (Amazon EMR) Premise
    • Amazon S3 Amazon RDS Amazon AmazonDynamoDB Redshift HDFS On (Amazon EMR) Premise
    • Amazon S3 Amazon RDS Amazon AmazonDynamoDB Redshift HDFS On (Amazon EMR) Premise
    • Input DatanodeActivity[Output Datanode]
    • Input Datanode with precondition checkActivity with failure & delay notificationsOuput Datanode
    • Data DataData Stores Data Stores Compute Resources
    • Start Interval[End]
    • Noon Today 1 hour
    • 12-1pm X1-2pm2-3pm …..
    • 12-1pm X1-2pm2-3pm X 1 day …..
    • Monthly DailyHourly Quarterly Yearly Weekly
    • S3 logs (hourly) Geolocation data Per-geographyusage computation (hourly) Redshift results
    • S3 logs (hourly) Geolocation dataPrecondition: files exist Precondition: ./geo_available Per-geography usage computation (hourly) Redshift results
    • Dynamo RDSevent data demographics Hive-basedanalysis (hourly) Redshift results
    • Hourly click updates Hourly event analysis Daily reporting SQL
    • Custom Amazon RDS Amazon S3 Amazon demographics logs Precondition DynamoDB event data Hive scriptEMR usage-by-geo job Amazon Redshift DW table Amazon Redshift Amazon EC2 DW table report generation
    • Custom Amazon RDS Amazon S3 Amazon demographics logs Precondition DynamoDB event data Hive scriptEMR usage-by-geo job Amazon Redshift DW table Amazon Redshift Amazon EC2 DW table report generation
    • We Manage You Manage EMR Clusters EC2 EC2 Instances InstancesEMR Clusters On Premise Resources
    • { "objects" : [ { "name" : “My Copy”, "type" : “Copy Action”, “input”: {“ref” : “My RDS Data”}, “output”: {“ref” : “My S3 Data”}, ”runsOn” : {“ref”: “My Instance”}, "schedule" : { "ref" : “My Schedule" } }, { "name" : ”My Instance”, "type" : ”EC2Instance”, "instanceType" : "m1.small”, "schedule" : { "ref” : “My Schedule" } },…..}
    • On AWS On PremiseHigh $1/month $2.50/monthFrequencyLow Frequency $.60/month $1.50/month
    • We are sincerely eager to hear your feedback on thispresentation and on re:Invent. Please fill out an evaluation form when you have a chance.