SlideShare a Scribd company logo
1 of 24
Download to read offline
KINESIS

REDSHIFT
DYNAMO DB

EMR

2.5 quintillion bytes of
Data is generated
Everyday!

Big Data
Building Blocks
With AWS
How Do You Tackle These
Big Data Challenges?
Agenda

1

5

Big Data is getting
Bigger and Bigger !

3

2

Figuring Out the
Big Data Life Cycle

4

How AWS Building Blocks
can Help Tame Big Data!

Why is Cloud Big Data’s
Best Friend ?

Cloud IT Better

How Cloudlytics Uses
AWS Cloud for its Big Data

2
So What is Big Data ?

Simply put, Big Data is
data which cannot be
processed by the current
tools or technologies. Big
Data is too Big, too Fast
and too Varied.

Cloud IT Better

3

High Resolution images
from NASA, our place in
the cosmos!
The 3 V’s that make Big Data difficult to Tame!

Volume
Conventional
Databases allow
processing of
data in batches,
it could take
days weeks to
process one
batch of Big
Data.

Cloud IT Better

Variety
Data from social networks, sensors installed at
store entrances, traffic lights, in airplanes, Car
GPS and countless other sources !!

2.5 quintillion
bytes of Data is
generated
everyday!

4

Velocity
Twitter Generates 5 Giga Bytes of data/min
Facebook generates 7 Giga Bytes of data/min.
Big Data is Getting Bigger and BIGGER!
“ It is estimated that Walmart collects
more than 2.5 petabytes of data EVERY
HOUR from its customer transactions ”

“ More data
crosses the
internet EVERY
SECOND than
were stored in
the entire
internet just
20 years ago? “

“ Zuckerberg noted that 1
billion pieces of content
are shared via Facebook’s
Open Graph DAILY ! “
Cloud IT Better

5
Why is Cloud Big Data’s Best Friend ?
With Big Data, we Know
we want to Generate,
Store, Analyze & Share.

But How does Cloud
come in to Picture?
Cloud IT Better

6
Our IT Resources are Limited & Precious!

And, Cloud has
The Solution for this !!
Cloud IT Better

7
Cloud Has Many Advantages
Elasticity
Fast Time to
Market

On Demand

Flexible

Cost Effective

Pay
Per Use

Secure

Resilient

Cloud IT Better

No CapEx

Remote Access
8

Scalable
Pooled Resources
Cloud Optimizes Your IT Resources

Cloud Makes Sure that Your
Precious IT Resources are

OPTIMIZED
Cloud IT Better

9
Cloud makes it Easy!
Cloud Makes Big Data

Easier
To Handle
Image Courtesy: http://www.slideshare.net/AmazonWebServicesLATAM/big-data-on-aws?

Cloud IT Better

10
Let us Figure out the Big Data Life Cycle
Generation
In order to make the entire process of
Big Data more tangible, it is divided
into 4 stages:

Data
Collaboration
& Sharing

Collection
& Store

Analyze &
Computation

Cloud IT Better

11
Generating the Data
Financial
analysis

Scientific
simulations

Structured Data –
Employee Records
Semi Structured Data –
End User Logs

Bioinformatics
research

Data
warehousing

Generation

Data
Collaboration
& Sharing

Web based APIs can be used
to access this data and Store it.
12

Web indexing

Log file analysis

Data Mining

Unstructured Data –
Social User Profile images

Cloud IT Better

Machine learning

Collection
& Store

Analyze &
Computation
Transferring Your Data to AWS Cloud
To transfer your Data Sets on to the Cloud You can Use:

AWS Import/Export

AWS Storage Gateway

Move large amounts of data into and out of AWS
using portable storage devices for transport

Secure Integration between an On-premises
IT & AWS’s storage infrastructure

AWS Direct Connect
Establish a dedicated network connection
from your premises to AWS

Cloud IT Better

13
Collecting & Storing Data on AWS Cloud

AWS Relational
Database Service (RDS)
Simple Storage Service (S3)
Write, read, and delete objects
containing from 1 byte to 5
terabytes of data each.

A full featured relational databases giving you
access to capabilities of a MySQL, Oracle, SQL
Server, or PostgreSQL databases engines

AWS DynamoDB
A fast, fully managed NoSQL database service
making it simple & cost-effective to store & retrieve
any amount of data, and serve any level of request traffic.

Cloud IT Better

14
Data Analysis on AWS Cloud
Once You’ve
stored your
Content On
Cloud, It is
Time to
Analyze It !!
http://dorkutopia.com/wp-content/uploads/2013/06

Cloud IT Better

So if you’re Thinking
implementing a
Hadoop
Infrastructure ……

/
15
Data Analysis on AWS Cloud
Setting Up a
Hadoop
Infrastructure
is not that Easy,
But AWS Has the
Answer !
Image courtesy: http://globalgeeknews.com/wp-content/uploads/

Cloud IT Better

16
Data Analysis on AWS Cloud
Amazon Elastic Map Reduce (EMR)
• A managed Hadoop distribution by Amazon Web Services using customized
Apache Hadoop framework
• Using MapReduce, in which a data processing tasks are mapped to set of servers
in a cluster for processing.
• EMR integrates with AWS S3 (an alternative Storage
to HDFS) & EC2(Compute Instances).
• EMR allows you to tune the default Hadoop Job Flows to your custom needs.
• The various How To’s of Hadoop Architecture such as adding,
removing & configuring nodes is taken care of by EMR.

Cloud IT Better

17
AWS Redshift for Retrieval & Collaboration
Amazon Redshift is a fast, fully managed, petabyte-scale data warehouse service
making it simple & cost-effective to efficiently analyze all your data using your
existing business intelligence tools.

• Amazon Redshift has a massively parallel processing (MPP)
architecture, parallelizing and
distributing SQL operations.
• You can use AWS Redshift to Store and retrieve processed
data quickly, to generate custom based Reports.

AWS Redshift
Cloud IT Better

18
AWS Data Pipelines for Automation
AWS Data pipeline allows users to define a dependent chain
of data sources and destinations with an option to create data
processing activities called pipeline.
Input
Node

Activity

•
•
•
•
•

Can be implemented across all stages of Big Data Life Cycle.
Tasks Scheduled to perform Data movement and processing Activities.
Failure & Retry options in Data pipeline workflows also Available.
Input & Output Data nodes support S3 Bucket, DynamoDB, MySQL DB & SQL Data Source.
Activities currently supported are Copy, EMR, Hive & Shell Activity.

Output
Node

Cloud IT Better

19
AWS Kinesis (NEW)
Amazon Kinesis is a fully managed service for real-time processing of streaming data at
massive scale. Amazon Kinesis can collect and process hundreds of TBs of data/hr from hundreds of
thousands of sources.
• Real Time Processing allowing you to answer questions
about the current state of your data.
• Amazon Kinesis automatically provisions &
manages the storage required to reliably &
durably collect your data stream.
• You can add as many as kinesis Streams as desired based on
the volume & variety of Data.
• Your Kinesis Streams are connected to your Kinesis App
from which you can use DynamoDB or Redshift
to process complex queries at real Time.
Image courtesy: https://static.gosquared.com/images/liquidicity/kinesis/

Cloud IT Better

20
The Big Data Life cycle - Compiled
Generation
AWS S3
AWS RDS
AWS DynamoDB
AWS Redshift
AWS Data Pipeline

Data
Collaboration
& Sharing

Collection
& Store

Analyze &
Computation

AWS EMR
AWS Data Pipeline

Cloud IT Better

21

AWS S3
Component Description
AWS RDS
………………………
AWS DynamoDB ………………
AWS Data Pipeline
........
…………….
…….
Use Case - Cloudlytics
Cloudlytics is a Pay-as-you-Go, SaaS based Log Analytics Tool powered by AWS. It
Takes the Big Data Approach using AWS Components such as EMR & Redshift.

Processed
Data

Processing

Customer Log Files
Stored in S3
Customer
Reports

Cloud IT Better

22
Check out our Past Webinars

Cloud IT Better

23
Thank you
Follow Us On :
Our Blog :
Contact us :

http://blog.blazeclan.com/
info@blazeclan.com

www.blazeclan.com

Cloud IT Better

24

More Related Content

What's hot

Webinar | From Zero to 1 Million with Google Cloud Platform and DataStax
Webinar | From Zero to 1 Million with Google Cloud Platform and DataStaxWebinar | From Zero to 1 Million with Google Cloud Platform and DataStax
Webinar | From Zero to 1 Million with Google Cloud Platform and DataStaxDataStax
 
How Big Data and Hadoop Integrated into BMC ControlM at CARFAX
How Big Data and Hadoop Integrated into BMC ControlM at CARFAXHow Big Data and Hadoop Integrated into BMC ControlM at CARFAX
How Big Data and Hadoop Integrated into BMC ControlM at CARFAXBMC Software
 
IlOUG Tech Days 2016 - Big Data for Oracle Developers - Towards Spark, Real-T...
IlOUG Tech Days 2016 - Big Data for Oracle Developers - Towards Spark, Real-T...IlOUG Tech Days 2016 - Big Data for Oracle Developers - Towards Spark, Real-T...
IlOUG Tech Days 2016 - Big Data for Oracle Developers - Towards Spark, Real-T...Mark Rittman
 
Cost of Ownership for Hadoop Implementation
Cost of Ownership for Hadoop ImplementationCost of Ownership for Hadoop Implementation
Cost of Ownership for Hadoop ImplementationDataWorks Summit
 
Data & Analytics - Session 1 - Big Data Analytics
Data & Analytics - Session 1 -  Big Data AnalyticsData & Analytics - Session 1 -  Big Data Analytics
Data & Analytics - Session 1 - Big Data AnalyticsAmazon Web Services
 
Empowering you with Democratized Data Access, Data Science and Machine Learning
Empowering you with Democratized Data Access, Data Science and Machine LearningEmpowering you with Democratized Data Access, Data Science and Machine Learning
Empowering you with Democratized Data Access, Data Science and Machine LearningDataWorks Summit
 
Building the Enterprise Data Lake - Important Considerations Before You Jump In
Building the Enterprise Data Lake - Important Considerations Before You Jump InBuilding the Enterprise Data Lake - Important Considerations Before You Jump In
Building the Enterprise Data Lake - Important Considerations Before You Jump InSnapLogic
 
How to Operationalise Real-Time Hadoop in the Cloud
How to Operationalise Real-Time Hadoop in the CloudHow to Operationalise Real-Time Hadoop in the Cloud
How to Operationalise Real-Time Hadoop in the CloudAttunity
 
Architecting an Open Data Lake for the Enterprise
Architecting an Open Data Lake for the EnterpriseArchitecting an Open Data Lake for the Enterprise
Architecting an Open Data Lake for the EnterpriseAmazon Web Services
 
Democratizing Data Science on Kubernetes
Democratizing Data Science on Kubernetes Democratizing Data Science on Kubernetes
Democratizing Data Science on Kubernetes John Archer
 
Use .NET Core to create IoT Solutions
Use .NET Core to create IoT SolutionsUse .NET Core to create IoT Solutions
Use .NET Core to create IoT SolutionsJohn Chang
 
Hortonworks roadshow
Hortonworks roadshowHortonworks roadshow
Hortonworks roadshowAccenture
 
Microsoft and Hortonworks Delivers the Modern Data Architecture for Big Data
Microsoft and Hortonworks Delivers the Modern Data Architecture for Big DataMicrosoft and Hortonworks Delivers the Modern Data Architecture for Big Data
Microsoft and Hortonworks Delivers the Modern Data Architecture for Big DataHortonworks
 
C* Summit EU 2013: Leveraging the Power of Cassandra: Operational Reporting a...
C* Summit EU 2013: Leveraging the Power of Cassandra: Operational Reporting a...C* Summit EU 2013: Leveraging the Power of Cassandra: Operational Reporting a...
C* Summit EU 2013: Leveraging the Power of Cassandra: Operational Reporting a...DataStax Academy
 
Trivadis TechEvent 2016 DWH Modernization – in the Age of Big Data by Gregor ...
Trivadis TechEvent 2016 DWH Modernization – in the Age of Big Data by Gregor ...Trivadis TechEvent 2016 DWH Modernization – in the Age of Big Data by Gregor ...
Trivadis TechEvent 2016 DWH Modernization – in the Age of Big Data by Gregor ...Trivadis
 
Building a modern data warehouse
Building a modern data warehouseBuilding a modern data warehouse
Building a modern data warehouseJames Serra
 
A beginners guide to Cloudera Hadoop
A beginners guide to Cloudera HadoopA beginners guide to Cloudera Hadoop
A beginners guide to Cloudera HadoopDavid Yahalom
 

What's hot (19)

Webinar | From Zero to 1 Million with Google Cloud Platform and DataStax
Webinar | From Zero to 1 Million with Google Cloud Platform and DataStaxWebinar | From Zero to 1 Million with Google Cloud Platform and DataStax
Webinar | From Zero to 1 Million with Google Cloud Platform and DataStax
 
How Big Data and Hadoop Integrated into BMC ControlM at CARFAX
How Big Data and Hadoop Integrated into BMC ControlM at CARFAXHow Big Data and Hadoop Integrated into BMC ControlM at CARFAX
How Big Data and Hadoop Integrated into BMC ControlM at CARFAX
 
IlOUG Tech Days 2016 - Big Data for Oracle Developers - Towards Spark, Real-T...
IlOUG Tech Days 2016 - Big Data for Oracle Developers - Towards Spark, Real-T...IlOUG Tech Days 2016 - Big Data for Oracle Developers - Towards Spark, Real-T...
IlOUG Tech Days 2016 - Big Data for Oracle Developers - Towards Spark, Real-T...
 
Cost of Ownership for Hadoop Implementation
Cost of Ownership for Hadoop ImplementationCost of Ownership for Hadoop Implementation
Cost of Ownership for Hadoop Implementation
 
Data & Analytics - Session 1 - Big Data Analytics
Data & Analytics - Session 1 -  Big Data AnalyticsData & Analytics - Session 1 -  Big Data Analytics
Data & Analytics - Session 1 - Big Data Analytics
 
Empowering you with Democratized Data Access, Data Science and Machine Learning
Empowering you with Democratized Data Access, Data Science and Machine LearningEmpowering you with Democratized Data Access, Data Science and Machine Learning
Empowering you with Democratized Data Access, Data Science and Machine Learning
 
Building the Enterprise Data Lake - Important Considerations Before You Jump In
Building the Enterprise Data Lake - Important Considerations Before You Jump InBuilding the Enterprise Data Lake - Important Considerations Before You Jump In
Building the Enterprise Data Lake - Important Considerations Before You Jump In
 
How to Operationalise Real-Time Hadoop in the Cloud
How to Operationalise Real-Time Hadoop in the CloudHow to Operationalise Real-Time Hadoop in the Cloud
How to Operationalise Real-Time Hadoop in the Cloud
 
Architecting an Open Data Lake for the Enterprise
Architecting an Open Data Lake for the EnterpriseArchitecting an Open Data Lake for the Enterprise
Architecting an Open Data Lake for the Enterprise
 
Democratizing Data Science on Kubernetes
Democratizing Data Science on Kubernetes Democratizing Data Science on Kubernetes
Democratizing Data Science on Kubernetes
 
Use .NET Core to create IoT Solutions
Use .NET Core to create IoT SolutionsUse .NET Core to create IoT Solutions
Use .NET Core to create IoT Solutions
 
Hortonworks roadshow
Hortonworks roadshowHortonworks roadshow
Hortonworks roadshow
 
Microsoft and Hortonworks Delivers the Modern Data Architecture for Big Data
Microsoft and Hortonworks Delivers the Modern Data Architecture for Big DataMicrosoft and Hortonworks Delivers the Modern Data Architecture for Big Data
Microsoft and Hortonworks Delivers the Modern Data Architecture for Big Data
 
C* Summit EU 2013: Leveraging the Power of Cassandra: Operational Reporting a...
C* Summit EU 2013: Leveraging the Power of Cassandra: Operational Reporting a...C* Summit EU 2013: Leveraging the Power of Cassandra: Operational Reporting a...
C* Summit EU 2013: Leveraging the Power of Cassandra: Operational Reporting a...
 
Why Hadoop as a Service?
Why Hadoop as a Service?Why Hadoop as a Service?
Why Hadoop as a Service?
 
Trivadis TechEvent 2016 DWH Modernization – in the Age of Big Data by Gregor ...
Trivadis TechEvent 2016 DWH Modernization – in the Age of Big Data by Gregor ...Trivadis TechEvent 2016 DWH Modernization – in the Age of Big Data by Gregor ...
Trivadis TechEvent 2016 DWH Modernization – in the Age of Big Data by Gregor ...
 
Building a modern data warehouse
Building a modern data warehouseBuilding a modern data warehouse
Building a modern data warehouse
 
Big Data: Myths and Realities
Big Data: Myths and RealitiesBig Data: Myths and Realities
Big Data: Myths and Realities
 
A beginners guide to Cloudera Hadoop
A beginners guide to Cloudera HadoopA beginners guide to Cloudera Hadoop
A beginners guide to Cloudera Hadoop
 

Viewers also liked

[TechTalks] Learning Configuration Management with SaltStack (Advanced Concepts)
[TechTalks] Learning Configuration Management with SaltStack (Advanced Concepts)[TechTalks] Learning Configuration Management with SaltStack (Advanced Concepts)
[TechTalks] Learning Configuration Management with SaltStack (Advanced Concepts)Blazeclan Technologies Private Limited
 
Cloudlytics Reporting: Analyze Amazon CloudFront, S3 & ELB Logs - Part 2
Cloudlytics Reporting: Analyze Amazon CloudFront, S3 & ELB Logs - Part 2Cloudlytics Reporting: Analyze Amazon CloudFront, S3 & ELB Logs - Part 2
Cloudlytics Reporting: Analyze Amazon CloudFront, S3 & ELB Logs - Part 2Blazeclan Technologies Private Limited
 
[TechTalks] Effects of UI/ UX Designs on Customer Satisfaction & Loyalty
[TechTalks] Effects of UI/ UX Designs on Customer Satisfaction & Loyalty[TechTalks] Effects of UI/ UX Designs on Customer Satisfaction & Loyalty
[TechTalks] Effects of UI/ UX Designs on Customer Satisfaction & LoyaltyBlazeclan Technologies Private Limited
 

Viewers also liked (18)

How to Design for High Availability & Scale with AWS
How to Design for High Availability & Scale with AWSHow to Design for High Availability & Scale with AWS
How to Design for High Availability & Scale with AWS
 
Analyze Amazon CloudFront, S3 & ELB Logs with Cloudlytics - Part 1
Analyze Amazon CloudFront, S3 & ELB Logs with Cloudlytics - Part 1Analyze Amazon CloudFront, S3 & ELB Logs with Cloudlytics - Part 1
Analyze Amazon CloudFront, S3 & ELB Logs with Cloudlytics - Part 1
 
Life of data from generation to visualization using big data
Life of data from generation to visualization using big dataLife of data from generation to visualization using big data
Life of data from generation to visualization using big data
 
[TechTalks] Learning Configuration Management with SaltStack (Advanced Concepts)
[TechTalks] Learning Configuration Management with SaltStack (Advanced Concepts)[TechTalks] Learning Configuration Management with SaltStack (Advanced Concepts)
[TechTalks] Learning Configuration Management with SaltStack (Advanced Concepts)
 
Overview of AWS Services for Media Content
Overview of AWS Services for Media ContentOverview of AWS Services for Media Content
Overview of AWS Services for Media Content
 
Micro services on AWS
Micro services on AWSMicro services on AWS
Micro services on AWS
 
Architecting for Big Data with AWS
Architecting for Big Data with AWSArchitecting for Big Data with AWS
Architecting for Big Data with AWS
 
AWS RDS Migration Tool
AWS RDS Migration Tool AWS RDS Migration Tool
AWS RDS Migration Tool
 
Cloud stream webinar
Cloud stream webinarCloud stream webinar
Cloud stream webinar
 
Solving Big Data problems on AWS by Rajnish Malik
Solving Big Data problems on AWS by Rajnish MalikSolving Big Data problems on AWS by Rajnish Malik
Solving Big Data problems on AWS by Rajnish Malik
 
Productive Expansion on Amazon Web Services with BlazeClan
 Productive Expansion on Amazon Web Services with BlazeClan Productive Expansion on Amazon Web Services with BlazeClan
Productive Expansion on Amazon Web Services with BlazeClan
 
Cloudlytics Reporting: Analyze Amazon CloudFront, S3 & ELB Logs - Part 2
Cloudlytics Reporting: Analyze Amazon CloudFront, S3 & ELB Logs - Part 2Cloudlytics Reporting: Analyze Amazon CloudFront, S3 & ELB Logs - Part 2
Cloudlytics Reporting: Analyze Amazon CloudFront, S3 & ELB Logs - Part 2
 
Enterprise Cloud for your Business Applications
Enterprise Cloud for your Business ApplicationsEnterprise Cloud for your Business Applications
Enterprise Cloud for your Business Applications
 
Hurix case study
Hurix case study Hurix case study
Hurix case study
 
Overview of AWS Services for your Enterprise
Overview of AWS Services for your Enterprise Overview of AWS Services for your Enterprise
Overview of AWS Services for your Enterprise
 
[TechTalks] Effects of UI/ UX Designs on Customer Satisfaction & Loyalty
[TechTalks] Effects of UI/ UX Designs on Customer Satisfaction & Loyalty[TechTalks] Effects of UI/ UX Designs on Customer Satisfaction & Loyalty
[TechTalks] Effects of UI/ UX Designs on Customer Satisfaction & Loyalty
 
Solving Big Data Industry Use Cases with AWS Cloud Computing
Solving Big Data Industry Use Cases with AWS Cloud ComputingSolving Big Data Industry Use Cases with AWS Cloud Computing
Solving Big Data Industry Use Cases with AWS Cloud Computing
 
Amazon CloudFront Complete with Blazeclan's Media Solution Stack
Amazon CloudFront Complete with Blazeclan's Media Solution StackAmazon CloudFront Complete with Blazeclan's Media Solution Stack
Amazon CloudFront Complete with Blazeclan's Media Solution Stack
 

Similar to Big Data Building Blocks with AWS Cloud

Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...
Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...
Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...Precisely
 
Building your First Big Data Application on AWS
Building your First Big Data Application on AWSBuilding your First Big Data Application on AWS
Building your First Big Data Application on AWSAmazon Web Services
 
Architecting Data Lakes on AWS
Architecting Data Lakes on AWSArchitecting Data Lakes on AWS
Architecting Data Lakes on AWSSajith Appukuttan
 
Building Your Data Lake on AWS - Level 200
Building Your Data Lake on AWS - Level 200Building Your Data Lake on AWS - Level 200
Building Your Data Lake on AWS - Level 200Amazon Web Services
 
Immersion Day - Como simplificar o acesso ao seu ambiente analítico
Immersion Day - Como simplificar o acesso ao seu ambiente analíticoImmersion Day - Como simplificar o acesso ao seu ambiente analítico
Immersion Day - Como simplificar o acesso ao seu ambiente analíticoAmazon Web Services LATAM
 
Slides: Proven Strategies for Hybrid Cloud Computing with Mainframes — From A...
Slides: Proven Strategies for Hybrid Cloud Computing with Mainframes — From A...Slides: Proven Strategies for Hybrid Cloud Computing with Mainframes — From A...
Slides: Proven Strategies for Hybrid Cloud Computing with Mainframes — From A...DATAVERSITY
 
AWS Summit Singapore - Architecting a Serverless Data Lake on AWS
AWS Summit Singapore - Architecting a Serverless Data Lake on AWSAWS Summit Singapore - Architecting a Serverless Data Lake on AWS
AWS Summit Singapore - Architecting a Serverless Data Lake on AWSAmazon Web Services
 
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...Amazon Web Services
 
Finding Meaning in the Noise: Understanding Big Data with AWS Analytics
Finding Meaning in the Noise: Understanding Big Data with AWS AnalyticsFinding Meaning in the Noise: Understanding Big Data with AWS Analytics
Finding Meaning in the Noise: Understanding Big Data with AWS AnalyticsAmazon Web Services
 
AWS tutorial-Part59:AWS Cloud Database Products-2nd Intro Session
AWS tutorial-Part59:AWS Cloud Database Products-2nd Intro SessionAWS tutorial-Part59:AWS Cloud Database Products-2nd Intro Session
AWS tutorial-Part59:AWS Cloud Database Products-2nd Intro SessionSaM theCloudGuy
 
AWS Webcast - Attunity Couchsurfing
AWS Webcast - Attunity CouchsurfingAWS Webcast - Attunity Couchsurfing
AWS Webcast - Attunity CouchsurfingAmazon Web Services
 
Database Freedom - ADB304 - Santa Clara AWS Summit
Database Freedom - ADB304 - Santa Clara AWS SummitDatabase Freedom - ADB304 - Santa Clara AWS Summit
Database Freedom - ADB304 - Santa Clara AWS SummitAmazon Web Services
 
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...Amazon Web Services
 
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...Amazon Web Services
 
(BDT201) Big Data and HPC State of the Union | AWS re:Invent 2014
(BDT201) Big Data and HPC State of the Union | AWS re:Invent 2014(BDT201) Big Data and HPC State of the Union | AWS re:Invent 2014
(BDT201) Big Data and HPC State of the Union | AWS re:Invent 2014Amazon Web Services
 
5 Reasons to Move Your BI to the Cloud
5 Reasons to Move Your BI to the Cloud5 Reasons to Move Your BI to the Cloud
5 Reasons to Move Your BI to the CloudTableau Software
 
Fast Track to Your Data Lake on AWS
Fast Track to Your Data Lake on AWSFast Track to Your Data Lake on AWS
Fast Track to Your Data Lake on AWSAmazon Web Services
 
Why Scale Matters and How the Cloud is Really Different (at scale)
Why Scale Matters and How the Cloud is Really Different (at scale)Why Scale Matters and How the Cloud is Really Different (at scale)
Why Scale Matters and How the Cloud is Really Different (at scale)Amazon Web Services
 
From raw data to business insights. A modern data lake
From raw data to business insights. A modern data lakeFrom raw data to business insights. A modern data lake
From raw data to business insights. A modern data lakejavier ramirez
 
Analisi dei dati con AWS: una panoramica degli strumenti disponibili
Analisi dei dati con AWS: una panoramica degli strumenti disponibiliAnalisi dei dati con AWS: una panoramica degli strumenti disponibili
Analisi dei dati con AWS: una panoramica degli strumenti disponibiliAmazon Web Services
 

Similar to Big Data Building Blocks with AWS Cloud (20)

Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...
Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...
Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...
 
Building your First Big Data Application on AWS
Building your First Big Data Application on AWSBuilding your First Big Data Application on AWS
Building your First Big Data Application on AWS
 
Architecting Data Lakes on AWS
Architecting Data Lakes on AWSArchitecting Data Lakes on AWS
Architecting Data Lakes on AWS
 
Building Your Data Lake on AWS - Level 200
Building Your Data Lake on AWS - Level 200Building Your Data Lake on AWS - Level 200
Building Your Data Lake on AWS - Level 200
 
Immersion Day - Como simplificar o acesso ao seu ambiente analítico
Immersion Day - Como simplificar o acesso ao seu ambiente analíticoImmersion Day - Como simplificar o acesso ao seu ambiente analítico
Immersion Day - Como simplificar o acesso ao seu ambiente analítico
 
Slides: Proven Strategies for Hybrid Cloud Computing with Mainframes — From A...
Slides: Proven Strategies for Hybrid Cloud Computing with Mainframes — From A...Slides: Proven Strategies for Hybrid Cloud Computing with Mainframes — From A...
Slides: Proven Strategies for Hybrid Cloud Computing with Mainframes — From A...
 
AWS Summit Singapore - Architecting a Serverless Data Lake on AWS
AWS Summit Singapore - Architecting a Serverless Data Lake on AWSAWS Summit Singapore - Architecting a Serverless Data Lake on AWS
AWS Summit Singapore - Architecting a Serverless Data Lake on AWS
 
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
 
Finding Meaning in the Noise: Understanding Big Data with AWS Analytics
Finding Meaning in the Noise: Understanding Big Data with AWS AnalyticsFinding Meaning in the Noise: Understanding Big Data with AWS Analytics
Finding Meaning in the Noise: Understanding Big Data with AWS Analytics
 
AWS tutorial-Part59:AWS Cloud Database Products-2nd Intro Session
AWS tutorial-Part59:AWS Cloud Database Products-2nd Intro SessionAWS tutorial-Part59:AWS Cloud Database Products-2nd Intro Session
AWS tutorial-Part59:AWS Cloud Database Products-2nd Intro Session
 
AWS Webcast - Attunity Couchsurfing
AWS Webcast - Attunity CouchsurfingAWS Webcast - Attunity Couchsurfing
AWS Webcast - Attunity Couchsurfing
 
Database Freedom - ADB304 - Santa Clara AWS Summit
Database Freedom - ADB304 - Santa Clara AWS SummitDatabase Freedom - ADB304 - Santa Clara AWS Summit
Database Freedom - ADB304 - Santa Clara AWS Summit
 
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
 
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...
 
(BDT201) Big Data and HPC State of the Union | AWS re:Invent 2014
(BDT201) Big Data and HPC State of the Union | AWS re:Invent 2014(BDT201) Big Data and HPC State of the Union | AWS re:Invent 2014
(BDT201) Big Data and HPC State of the Union | AWS re:Invent 2014
 
5 Reasons to Move Your BI to the Cloud
5 Reasons to Move Your BI to the Cloud5 Reasons to Move Your BI to the Cloud
5 Reasons to Move Your BI to the Cloud
 
Fast Track to Your Data Lake on AWS
Fast Track to Your Data Lake on AWSFast Track to Your Data Lake on AWS
Fast Track to Your Data Lake on AWS
 
Why Scale Matters and How the Cloud is Really Different (at scale)
Why Scale Matters and How the Cloud is Really Different (at scale)Why Scale Matters and How the Cloud is Really Different (at scale)
Why Scale Matters and How the Cloud is Really Different (at scale)
 
From raw data to business insights. A modern data lake
From raw data to business insights. A modern data lakeFrom raw data to business insights. A modern data lake
From raw data to business insights. A modern data lake
 
Analisi dei dati con AWS: una panoramica degli strumenti disponibili
Analisi dei dati con AWS: una panoramica degli strumenti disponibiliAnalisi dei dati con AWS: una panoramica degli strumenti disponibili
Analisi dei dati con AWS: una panoramica degli strumenti disponibili
 

More from Blazeclan Technologies Private Limited

Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring ReportsCloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring ReportsBlazeclan Technologies Private Limited
 

More from Blazeclan Technologies Private Limited (12)

2020 Recap | Clan's Transformational Journey In The New Normal
2020 Recap | Clan's Transformational Journey In The New Normal2020 Recap | Clan's Transformational Journey In The New Normal
2020 Recap | Clan's Transformational Journey In The New Normal
 
Reminiscing 2019 And Heading Toward A Brighter Future!
Reminiscing 2019 And Heading Toward A Brighter Future!Reminiscing 2019 And Heading Toward A Brighter Future!
Reminiscing 2019 And Heading Toward A Brighter Future!
 
AWS Managed Services - BlazeClan Technologies
AWS Managed Services - BlazeClan TechnologiesAWS Managed Services - BlazeClan Technologies
AWS Managed Services - BlazeClan Technologies
 
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring ReportsCloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
 
Amazon Reshift as your Data Warehouse Solution
Amazon Reshift as your Data Warehouse SolutionAmazon Reshift as your Data Warehouse Solution
Amazon Reshift as your Data Warehouse Solution
 
Testing Framework on AWS Cloud - Solution Set
Testing Framework on AWS Cloud - Solution SetTesting Framework on AWS Cloud - Solution Set
Testing Framework on AWS Cloud - Solution Set
 
Cloud for Media - A Complete Solution Stack for Faster Cloud Adoption
Cloud for Media - A Complete Solution Stack for Faster Cloud AdoptionCloud for Media - A Complete Solution Stack for Faster Cloud Adoption
Cloud for Media - A Complete Solution Stack for Faster Cloud Adoption
 
5 Points to Consider - Enterprise Road Map to AWS Cloud
5 Points to Consider  - Enterprise Road Map to AWS Cloud5 Points to Consider  - Enterprise Road Map to AWS Cloud
5 Points to Consider - Enterprise Road Map to AWS Cloud
 
How cloud is fueling growth for online gaming
How cloud is fueling growth for online gamingHow cloud is fueling growth for online gaming
How cloud is fueling growth for online gaming
 
A guide on Aws Security Token Service
A guide on Aws Security Token ServiceA guide on Aws Security Token Service
A guide on Aws Security Token Service
 
Working and Features of HTML5 and PhoneGap - An Overview
Working and Features of HTML5 and PhoneGap - An OverviewWorking and Features of HTML5 and PhoneGap - An Overview
Working and Features of HTML5 and PhoneGap - An Overview
 
Cloud Migration Strategy - IT Transformation with Cloud
Cloud Migration Strategy - IT Transformation with CloudCloud Migration Strategy - IT Transformation with Cloud
Cloud Migration Strategy - IT Transformation with Cloud
 

Recently uploaded

20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdfJamie (Taka) Wang
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.francesco barbera
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfAnna Loughnan Colquhoun
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServicePicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServiceRenan Moreira de Oliveira
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 

Recently uploaded (20)

20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdf
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServicePicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 

Big Data Building Blocks with AWS Cloud

  • 1. KINESIS REDSHIFT DYNAMO DB EMR 2.5 quintillion bytes of Data is generated Everyday! Big Data Building Blocks With AWS How Do You Tackle These Big Data Challenges?
  • 2. Agenda 1 5 Big Data is getting Bigger and Bigger ! 3 2 Figuring Out the Big Data Life Cycle 4 How AWS Building Blocks can Help Tame Big Data! Why is Cloud Big Data’s Best Friend ? Cloud IT Better How Cloudlytics Uses AWS Cloud for its Big Data 2
  • 3. So What is Big Data ? Simply put, Big Data is data which cannot be processed by the current tools or technologies. Big Data is too Big, too Fast and too Varied. Cloud IT Better 3 High Resolution images from NASA, our place in the cosmos!
  • 4. The 3 V’s that make Big Data difficult to Tame! Volume Conventional Databases allow processing of data in batches, it could take days weeks to process one batch of Big Data. Cloud IT Better Variety Data from social networks, sensors installed at store entrances, traffic lights, in airplanes, Car GPS and countless other sources !! 2.5 quintillion bytes of Data is generated everyday! 4 Velocity Twitter Generates 5 Giga Bytes of data/min Facebook generates 7 Giga Bytes of data/min.
  • 5. Big Data is Getting Bigger and BIGGER! “ It is estimated that Walmart collects more than 2.5 petabytes of data EVERY HOUR from its customer transactions ” “ More data crosses the internet EVERY SECOND than were stored in the entire internet just 20 years ago? “ “ Zuckerberg noted that 1 billion pieces of content are shared via Facebook’s Open Graph DAILY ! “ Cloud IT Better 5
  • 6. Why is Cloud Big Data’s Best Friend ? With Big Data, we Know we want to Generate, Store, Analyze & Share. But How does Cloud come in to Picture? Cloud IT Better 6
  • 7. Our IT Resources are Limited & Precious! And, Cloud has The Solution for this !! Cloud IT Better 7
  • 8. Cloud Has Many Advantages Elasticity Fast Time to Market On Demand Flexible Cost Effective Pay Per Use Secure Resilient Cloud IT Better No CapEx Remote Access 8 Scalable Pooled Resources
  • 9. Cloud Optimizes Your IT Resources Cloud Makes Sure that Your Precious IT Resources are OPTIMIZED Cloud IT Better 9
  • 10. Cloud makes it Easy! Cloud Makes Big Data Easier To Handle Image Courtesy: http://www.slideshare.net/AmazonWebServicesLATAM/big-data-on-aws? Cloud IT Better 10
  • 11. Let us Figure out the Big Data Life Cycle Generation In order to make the entire process of Big Data more tangible, it is divided into 4 stages: Data Collaboration & Sharing Collection & Store Analyze & Computation Cloud IT Better 11
  • 12. Generating the Data Financial analysis Scientific simulations Structured Data – Employee Records Semi Structured Data – End User Logs Bioinformatics research Data warehousing Generation Data Collaboration & Sharing Web based APIs can be used to access this data and Store it. 12 Web indexing Log file analysis Data Mining Unstructured Data – Social User Profile images Cloud IT Better Machine learning Collection & Store Analyze & Computation
  • 13. Transferring Your Data to AWS Cloud To transfer your Data Sets on to the Cloud You can Use: AWS Import/Export AWS Storage Gateway Move large amounts of data into and out of AWS using portable storage devices for transport Secure Integration between an On-premises IT & AWS’s storage infrastructure AWS Direct Connect Establish a dedicated network connection from your premises to AWS Cloud IT Better 13
  • 14. Collecting & Storing Data on AWS Cloud AWS Relational Database Service (RDS) Simple Storage Service (S3) Write, read, and delete objects containing from 1 byte to 5 terabytes of data each. A full featured relational databases giving you access to capabilities of a MySQL, Oracle, SQL Server, or PostgreSQL databases engines AWS DynamoDB A fast, fully managed NoSQL database service making it simple & cost-effective to store & retrieve any amount of data, and serve any level of request traffic. Cloud IT Better 14
  • 15. Data Analysis on AWS Cloud Once You’ve stored your Content On Cloud, It is Time to Analyze It !! http://dorkutopia.com/wp-content/uploads/2013/06 Cloud IT Better So if you’re Thinking implementing a Hadoop Infrastructure …… / 15
  • 16. Data Analysis on AWS Cloud Setting Up a Hadoop Infrastructure is not that Easy, But AWS Has the Answer ! Image courtesy: http://globalgeeknews.com/wp-content/uploads/ Cloud IT Better 16
  • 17. Data Analysis on AWS Cloud Amazon Elastic Map Reduce (EMR) • A managed Hadoop distribution by Amazon Web Services using customized Apache Hadoop framework • Using MapReduce, in which a data processing tasks are mapped to set of servers in a cluster for processing. • EMR integrates with AWS S3 (an alternative Storage to HDFS) & EC2(Compute Instances). • EMR allows you to tune the default Hadoop Job Flows to your custom needs. • The various How To’s of Hadoop Architecture such as adding, removing & configuring nodes is taken care of by EMR. Cloud IT Better 17
  • 18. AWS Redshift for Retrieval & Collaboration Amazon Redshift is a fast, fully managed, petabyte-scale data warehouse service making it simple & cost-effective to efficiently analyze all your data using your existing business intelligence tools. • Amazon Redshift has a massively parallel processing (MPP) architecture, parallelizing and distributing SQL operations. • You can use AWS Redshift to Store and retrieve processed data quickly, to generate custom based Reports. AWS Redshift Cloud IT Better 18
  • 19. AWS Data Pipelines for Automation AWS Data pipeline allows users to define a dependent chain of data sources and destinations with an option to create data processing activities called pipeline. Input Node Activity • • • • • Can be implemented across all stages of Big Data Life Cycle. Tasks Scheduled to perform Data movement and processing Activities. Failure & Retry options in Data pipeline workflows also Available. Input & Output Data nodes support S3 Bucket, DynamoDB, MySQL DB & SQL Data Source. Activities currently supported are Copy, EMR, Hive & Shell Activity. Output Node Cloud IT Better 19
  • 20. AWS Kinesis (NEW) Amazon Kinesis is a fully managed service for real-time processing of streaming data at massive scale. Amazon Kinesis can collect and process hundreds of TBs of data/hr from hundreds of thousands of sources. • Real Time Processing allowing you to answer questions about the current state of your data. • Amazon Kinesis automatically provisions & manages the storage required to reliably & durably collect your data stream. • You can add as many as kinesis Streams as desired based on the volume & variety of Data. • Your Kinesis Streams are connected to your Kinesis App from which you can use DynamoDB or Redshift to process complex queries at real Time. Image courtesy: https://static.gosquared.com/images/liquidicity/kinesis/ Cloud IT Better 20
  • 21. The Big Data Life cycle - Compiled Generation AWS S3 AWS RDS AWS DynamoDB AWS Redshift AWS Data Pipeline Data Collaboration & Sharing Collection & Store Analyze & Computation AWS EMR AWS Data Pipeline Cloud IT Better 21 AWS S3 Component Description AWS RDS ……………………… AWS DynamoDB ……………… AWS Data Pipeline ........ ……………. …….
  • 22. Use Case - Cloudlytics Cloudlytics is a Pay-as-you-Go, SaaS based Log Analytics Tool powered by AWS. It Takes the Big Data Approach using AWS Components such as EMR & Redshift. Processed Data Processing Customer Log Files Stored in S3 Customer Reports Cloud IT Better 22
  • 23. Check out our Past Webinars Cloud IT Better 23
  • 24. Thank you Follow Us On : Our Blog : Contact us : http://blog.blazeclan.com/ info@blazeclan.com www.blazeclan.com Cloud IT Better 24