SlideShare a Scribd company logo
Big Data
Prepared By
Md Salman Ahmed
SalmanAhmedims@gmail.com
Content
 Introduction
 What is Big Data
 Storing, selecting and processing of Big Data
 Why Big Data
 How it is Different
 Big Data sources
 Tools used in Big Data
 Application of Big Data
 Risks of Big Data
 Benefits of Big Data
 How Big Data Impact on IT
 Future of Big Data
Introduction
• Everyone seems to be talking about it, but what is big
data really? How is it changing the way researchers at
companies, non-profits, governments, institutions, and
other organizations are learning about the world around
them? Where is this data coming from, how is it being
processed, and how are the results being used? And why
is open source so important to answering these
questions?
What is BIG DATA?
• In order to understand 'Big Data', we first need to know what 'data'
is.
Oxford dictionary defines 'data' as –
So, 'Big Data' is also a data but with a huge size. 'Big Data' is a term
used to describe collection of data that is huge in size and yet
growing exponentially with time.In short, such a data is so large and
complex that none of the traditional data management tools are
able to store it or process it efficiently.
What is BIG DATA?
• Walmart handles more than 1 million customer transactions every
hour.
• Facebook handles 40 billion photos from its user base.
• Decoding the human genome originally took 10years to process;
now it can be achieved in one week.
Characteristic of Big Data
Characteristic of Big Data
 Big Data Volume
• A typical PC might have had 10 gigabytes of storage in 2000.
• Today, Facebook ingests 500 terabytes of new data every day. Boeing
737 will generate 240 terabytes of flight data during a single flight
across the US.
• The smart phones, the data they create and consume; sensors
embedded into everyday objects will soon result in billions of new,
constantly-updated data feeds containing environmental, location,
and other information, including video.
Characteristic of Big Data
 Big Data Velocity
• Clickstreams and ad impressions capture user behavior at millions of
events per second
• high-frequency stock trading algorithms reflect market changes
within microseconds
• machine to machine processes exchange data between billions of
devices
• infrastructure and sensors generate massive log data in realtime
• on-line gaming systems support millions of concurrent users, each
producing multiple inputs per second.
Characteristic of Big Data
 Big Data Variety
• Big Data isn't just numbers, dates, and strings. Big Data is also
geospatial data, 3D data, audio and video, and unstructured text,
including log files and social media.
• Traditional database systems were designed to address smaller
volumes of structured data, fewer updates or a predictable,
consistent data structure.
• Big Data analysis includes different types of data
Storing, selecting and processing of Big Data
Analyzing your data characteristics
• Selecting data sources for analysis
• Eliminating redundant data
• Establishing the role of NoSQL
Storing, selecting and processing of Big Data
Overview of Big Data stores
• Data models: key value, graph, document, column-family •
Hadoop Distributed File System • HBase • Hive
Storing, selecting and processing of Big Data
Selecting Big Data stores
• Choosing the correct data stores based on your data
characteristics Moving code to data
• Implementing polyglot data store solutions
• Aligning business goals to the appropriate data store
Storing, selecting and processing of Big Data
Integrating disparate data stores
• Mapping data to the programming framework • Connecting and
extracting data from storage
• Transforming data for processing • Subdividing data in preparation
for Hadoop MapReduce
Storing, selecting and processing of Big Data
Employing Hadoop MapReduce
• Creating the components of Hadoop MapReduce jobs
• Distributing data processing across server farms
• Executing Hadoop MapReduce jobs • Monitoring the progress of
job flows
Why Big Data?
• FB generates 10TB daily
• Twitter generates 7TB of data Daily
• IBM claims 90% of today’s stored data was generated in just the
last two years.
How Is Big Data Different?
Automatically generated by a machine (e.g. Sensor embedded in
an engine)
Typically an entirely new source of data (e.g. Use of the internet)
Not designed to be friendly (e.g. Text streams) 4) May not have
much values
Big Data Analytics
• Examining large amount of data
• Appropriate information
• Identification of hidden patterns, unknown correlations
• Competitive advantage
• Better business decisions: strategic and operational
• Effective marketing, customer satisfaction, increased revenue
Types of tools used in Big-Data
Where processing is hosted?
– Distributed Servers / Cloud (e.g. Amazon EC2)
 Where data is stored?
– Distributed Storage (e.g. Amazon S3)
 What is the programming model?
– Distributed Processing (e.g. MapReduce)
 How data is stored & indexed?
– High-performance schema-free databases (e.g. MongoDB)
 What operations are performed on data?
– Analytic / Semantic Processing
Application Of Big Data analytics
Smarter Healthcare
Homeland Security
Traffic Control
Manufacturing
Multi-channel sales
Telecom
Trading Analytics
Search Quality
Risks of Big Data
Will be so overwhelmed
 Need the right people and solve the right problems
Costs escalate too fast
 Isn’t necessary to capture 100%
Many sources of big data is privacy
 self-regulation
 Legal regulation 22
Leading Technology Vendors
 Example Vendors
• IBM – Netezza
• EMC – Greenplum
• Oracle – Exadata
 Commonality
• MPP architectures
• Commodity Hardware
• RDBMS based
• Full SQL compliance
How Big data impacts on IT ?
Big data is a troublesome force presenting opportunities with
challenges to IT organizations.
By 2015 4.4 million IT jobs in Big Data ; 1.9 million is in US itself
India will require a minimum of 1 lakh data scientists in the next
couple of years in addition to data analysts and data managers to
support the Big Data space.
Benefits of Big Data
Real-time big data isn’t just a process for storing petabytes or exabytes
of data in a data warehouse, It’s about the ability to make better
decisions and take meaningful actions at the right time.
Fast forward to the present and technologies like Hadoop give you the
scale and flexibility to store data before you know how you are going to
process it.
Technologies such as MapReduce,Hive and Impala enable you to run
queries without changing the data structures underneath.
Benefits of Big Data
Our newest research finds that organizations are using big data to target
customer-centric outcomes, tap into internal data and build a better
information ecosystem.
Big Data is already an important part of the $64 billion database and data
analytics market
It offers commercial opportunities of a comparable scale to enterprise software
in the late 1980s
And the Internet boom of the 1990s, and the social media explosion of today.
Future of Big Data
 $15 billion on software firms only specializing in data management and
analytics.
This industry on its own is worth more than $100 billion and growing at almost
10% a year which is roughly twice as fast as the software business as a whole.
In February 2012, the open source analyst firm Wikibon released the first
market forecast for Big Data , listing $5.1B revenue in 2012 with growth to
$53.4B in 2017
The McKinsey Global Institute estimates that data volume is growing 40% per
year, and will grow 44x between 2009 and 2020.
References
www.google.com
www.Slideshare.com
www.wikipedia.com
www.computereducation.org
Presentation on Big Data

More Related Content

What's hot

Big Data & Analytics (Conceptual and Practical Introduction)
Big Data & Analytics (Conceptual and Practical Introduction)Big Data & Analytics (Conceptual and Practical Introduction)
Big Data & Analytics (Conceptual and Practical Introduction)
Yaman Hajja, Ph.D.
 
Big Data Analytics
Big Data AnalyticsBig Data Analytics
Big Data Analytics
Ghulam Imaduddin
 
Big data
Big dataBig data
Big data
Nimish Kochhar
 
Presentation on Big Data
Presentation on Big DataPresentation on Big Data
Presentation on Big Data
Maruf Abdullah (Rion)
 
Presentation About Big Data (DBMS)
Presentation About Big Data (DBMS)Presentation About Big Data (DBMS)
Presentation About Big Data (DBMS)
SiamAhmed16
 
Big data
Big dataBig data
Big Data PPT by Rohit Dubey
Big Data PPT by Rohit DubeyBig Data PPT by Rohit Dubey
Big Data PPT by Rohit Dubey
Rohit Dubey
 
Big data
Big dataBig data
Big Data ppt
Big Data pptBig Data ppt
Big Data ppt
Vivek Gautam
 
Ppt for Application of big data
Ppt for Application of big dataPpt for Application of big data
Ppt for Application of big data
Prashant Sharma
 
Big Data
Big DataBig Data
Big Data
Rohit Jain
 
Big data
Big dataBig data
Big data
Pooja Shah
 
Big data in telecom
Big data in telecomBig data in telecom
Big data in telecom
Shubham Bathe
 
Big Data - Applications and Technologies Overview
Big Data - Applications and Technologies OverviewBig Data - Applications and Technologies Overview
Big Data - Applications and Technologies Overview
Sivashankar Ganapathy
 
Big Data & Hadoop Introduction
Big Data & Hadoop IntroductionBig Data & Hadoop Introduction
Big Data & Hadoop Introduction
Jayant Mukherjee
 
Big Data
Big DataBig Data
Big Data
Seminar Links
 
Big Data
Big DataBig Data
Presentation on Big Data Analytics
Presentation on Big Data AnalyticsPresentation on Big Data Analytics
Presentation on Big Data Analytics
S P Sajjan
 
Big data Presentation
Big data PresentationBig data Presentation
Big data Presentation
Aswadmehar
 
Big Data
Big DataBig Data
Big Data
Vinayak Kamath
 

What's hot (20)

Big Data & Analytics (Conceptual and Practical Introduction)
Big Data & Analytics (Conceptual and Practical Introduction)Big Data & Analytics (Conceptual and Practical Introduction)
Big Data & Analytics (Conceptual and Practical Introduction)
 
Big Data Analytics
Big Data AnalyticsBig Data Analytics
Big Data Analytics
 
Big data
Big dataBig data
Big data
 
Presentation on Big Data
Presentation on Big DataPresentation on Big Data
Presentation on Big Data
 
Presentation About Big Data (DBMS)
Presentation About Big Data (DBMS)Presentation About Big Data (DBMS)
Presentation About Big Data (DBMS)
 
Big data
Big dataBig data
Big data
 
Big Data PPT by Rohit Dubey
Big Data PPT by Rohit DubeyBig Data PPT by Rohit Dubey
Big Data PPT by Rohit Dubey
 
Big data
Big dataBig data
Big data
 
Big Data ppt
Big Data pptBig Data ppt
Big Data ppt
 
Ppt for Application of big data
Ppt for Application of big dataPpt for Application of big data
Ppt for Application of big data
 
Big Data
Big DataBig Data
Big Data
 
Big data
Big dataBig data
Big data
 
Big data in telecom
Big data in telecomBig data in telecom
Big data in telecom
 
Big Data - Applications and Technologies Overview
Big Data - Applications and Technologies OverviewBig Data - Applications and Technologies Overview
Big Data - Applications and Technologies Overview
 
Big Data & Hadoop Introduction
Big Data & Hadoop IntroductionBig Data & Hadoop Introduction
Big Data & Hadoop Introduction
 
Big Data
Big DataBig Data
Big Data
 
Big Data
Big DataBig Data
Big Data
 
Presentation on Big Data Analytics
Presentation on Big Data AnalyticsPresentation on Big Data Analytics
Presentation on Big Data Analytics
 
Big data Presentation
Big data PresentationBig data Presentation
Big data Presentation
 
Big Data
Big DataBig Data
Big Data
 

Similar to Presentation on Big Data

Content1. Introduction2. What is Big Data3. Characte.docx
Content1. Introduction2. What is Big Data3. Characte.docxContent1. Introduction2. What is Big Data3. Characte.docx
Content1. Introduction2. What is Big Data3. Characte.docx
dickonsondorris
 
Special issues on big data
Special issues on big dataSpecial issues on big data
Special issues on big data
Vedanand Singh
 
Big data ppt
Big  data pptBig  data ppt
Big data ppt
Nasrin Hussain
 
Big data
Big dataBig data
Big data
Mahmudul Alam
 
Bigdatappt 140225061440-phpapp01
Bigdatappt 140225061440-phpapp01Bigdatappt 140225061440-phpapp01
Bigdatappt 140225061440-phpapp01
nayanbhatia2
 
Big_Data_ppt[1] (1).pptx
Big_Data_ppt[1] (1).pptxBig_Data_ppt[1] (1).pptx
Big_Data_ppt[1] (1).pptx
TanguturiAvinash
 
Big data Analytics
Big data Analytics Big data Analytics
Big data Analytics
Guduru Lakshmi Kiranmai
 
Big data ppt
Big data pptBig data ppt
ppt final.pptx
ppt final.pptxppt final.pptx
ppt final.pptx
kalai75
 
Kartikey tripathi
Kartikey tripathiKartikey tripathi
Kartikey tripathi
KARTIKEY TRIPATHI
 
big-data-8722-m8RQ3h1.pptx
big-data-8722-m8RQ3h1.pptxbig-data-8722-m8RQ3h1.pptx
big-data-8722-m8RQ3h1.pptx
VaishnavGhadge1
 
Big data and analytics
Big data and analyticsBig data and analytics
Big data and analytics
Bohitesh Misra, PMP
 
Big data
Big dataBig data
Big data
SaraRao3
 
Bigdata " new level"
Bigdata " new level"Bigdata " new level"
Bigdata " new level"
Vamshikrishna Goud
 
Big data and data mining
Big data and data miningBig data and data mining
Big data and data mining
Emran Hossain
 
BigDataFinal.pptx
BigDataFinal.pptxBigDataFinal.pptx
BigDataFinal.pptx
PentaTech
 
Big data seminor
Big data seminorBig data seminor
Big data seminor
berasrujana
 
BIGDATAPrepared ByMuhammad Abrar UddinIntrodu.docx
BIGDATAPrepared ByMuhammad Abrar UddinIntrodu.docxBIGDATAPrepared ByMuhammad Abrar UddinIntrodu.docx
BIGDATAPrepared ByMuhammad Abrar UddinIntrodu.docx
tangyechloe
 
Bigdata
BigdataBigdata
Bigdata
sayan sarker
 
bigdata.pptx
bigdata.pptxbigdata.pptx
bigdata.pptx
KammetaJoshna
 

Similar to Presentation on Big Data (20)

Content1. Introduction2. What is Big Data3. Characte.docx
Content1. Introduction2. What is Big Data3. Characte.docxContent1. Introduction2. What is Big Data3. Characte.docx
Content1. Introduction2. What is Big Data3. Characte.docx
 
Special issues on big data
Special issues on big dataSpecial issues on big data
Special issues on big data
 
Big data ppt
Big  data pptBig  data ppt
Big data ppt
 
Big data
Big dataBig data
Big data
 
Bigdatappt 140225061440-phpapp01
Bigdatappt 140225061440-phpapp01Bigdatappt 140225061440-phpapp01
Bigdatappt 140225061440-phpapp01
 
Big_Data_ppt[1] (1).pptx
Big_Data_ppt[1] (1).pptxBig_Data_ppt[1] (1).pptx
Big_Data_ppt[1] (1).pptx
 
Big data Analytics
Big data Analytics Big data Analytics
Big data Analytics
 
Big data ppt
Big data pptBig data ppt
Big data ppt
 
ppt final.pptx
ppt final.pptxppt final.pptx
ppt final.pptx
 
Kartikey tripathi
Kartikey tripathiKartikey tripathi
Kartikey tripathi
 
big-data-8722-m8RQ3h1.pptx
big-data-8722-m8RQ3h1.pptxbig-data-8722-m8RQ3h1.pptx
big-data-8722-m8RQ3h1.pptx
 
Big data and analytics
Big data and analyticsBig data and analytics
Big data and analytics
 
Big data
Big dataBig data
Big data
 
Bigdata " new level"
Bigdata " new level"Bigdata " new level"
Bigdata " new level"
 
Big data and data mining
Big data and data miningBig data and data mining
Big data and data mining
 
BigDataFinal.pptx
BigDataFinal.pptxBigDataFinal.pptx
BigDataFinal.pptx
 
Big data seminor
Big data seminorBig data seminor
Big data seminor
 
BIGDATAPrepared ByMuhammad Abrar UddinIntrodu.docx
BIGDATAPrepared ByMuhammad Abrar UddinIntrodu.docxBIGDATAPrepared ByMuhammad Abrar UddinIntrodu.docx
BIGDATAPrepared ByMuhammad Abrar UddinIntrodu.docx
 
Bigdata
BigdataBigdata
Bigdata
 
bigdata.pptx
bigdata.pptxbigdata.pptx
bigdata.pptx
 

Recently uploaded

Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Neo4j
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Antonios Katsarakis
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
Enterprise Knowledge
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
Fwdays
 
ScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking ReplicationScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking Replication
ScyllaDB
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
DanBrown980551
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
Safe Software
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Pablo Gómez Abajo
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving
 
What is an RPA CoE? Session 2 – CoE Roles
What is an RPA CoE?  Session 2 – CoE RolesWhat is an RPA CoE?  Session 2 – CoE Roles
What is an RPA CoE? Session 2 – CoE Roles
DianaGray10
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
Fwdays
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
LizaNolte
 

Recently uploaded (20)

Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
 
ScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking ReplicationScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking Replication
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
 
What is an RPA CoE? Session 2 – CoE Roles
What is an RPA CoE?  Session 2 – CoE RolesWhat is an RPA CoE?  Session 2 – CoE Roles
What is an RPA CoE? Session 2 – CoE Roles
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
 

Presentation on Big Data

  • 1. Big Data Prepared By Md Salman Ahmed SalmanAhmedims@gmail.com
  • 2. Content  Introduction  What is Big Data  Storing, selecting and processing of Big Data  Why Big Data  How it is Different  Big Data sources  Tools used in Big Data  Application of Big Data  Risks of Big Data  Benefits of Big Data  How Big Data Impact on IT  Future of Big Data
  • 3. Introduction • Everyone seems to be talking about it, but what is big data really? How is it changing the way researchers at companies, non-profits, governments, institutions, and other organizations are learning about the world around them? Where is this data coming from, how is it being processed, and how are the results being used? And why is open source so important to answering these questions?
  • 4. What is BIG DATA? • In order to understand 'Big Data', we first need to know what 'data' is. Oxford dictionary defines 'data' as – So, 'Big Data' is also a data but with a huge size. 'Big Data' is a term used to describe collection of data that is huge in size and yet growing exponentially with time.In short, such a data is so large and complex that none of the traditional data management tools are able to store it or process it efficiently.
  • 5. What is BIG DATA? • Walmart handles more than 1 million customer transactions every hour. • Facebook handles 40 billion photos from its user base. • Decoding the human genome originally took 10years to process; now it can be achieved in one week.
  • 7. Characteristic of Big Data  Big Data Volume • A typical PC might have had 10 gigabytes of storage in 2000. • Today, Facebook ingests 500 terabytes of new data every day. Boeing 737 will generate 240 terabytes of flight data during a single flight across the US. • The smart phones, the data they create and consume; sensors embedded into everyday objects will soon result in billions of new, constantly-updated data feeds containing environmental, location, and other information, including video.
  • 8. Characteristic of Big Data  Big Data Velocity • Clickstreams and ad impressions capture user behavior at millions of events per second • high-frequency stock trading algorithms reflect market changes within microseconds • machine to machine processes exchange data between billions of devices • infrastructure and sensors generate massive log data in realtime • on-line gaming systems support millions of concurrent users, each producing multiple inputs per second.
  • 9. Characteristic of Big Data  Big Data Variety • Big Data isn't just numbers, dates, and strings. Big Data is also geospatial data, 3D data, audio and video, and unstructured text, including log files and social media. • Traditional database systems were designed to address smaller volumes of structured data, fewer updates or a predictable, consistent data structure. • Big Data analysis includes different types of data
  • 10. Storing, selecting and processing of Big Data Analyzing your data characteristics • Selecting data sources for analysis • Eliminating redundant data • Establishing the role of NoSQL
  • 11. Storing, selecting and processing of Big Data Overview of Big Data stores • Data models: key value, graph, document, column-family • Hadoop Distributed File System • HBase • Hive
  • 12. Storing, selecting and processing of Big Data Selecting Big Data stores • Choosing the correct data stores based on your data characteristics Moving code to data • Implementing polyglot data store solutions • Aligning business goals to the appropriate data store
  • 13. Storing, selecting and processing of Big Data Integrating disparate data stores • Mapping data to the programming framework • Connecting and extracting data from storage • Transforming data for processing • Subdividing data in preparation for Hadoop MapReduce
  • 14. Storing, selecting and processing of Big Data Employing Hadoop MapReduce • Creating the components of Hadoop MapReduce jobs • Distributing data processing across server farms • Executing Hadoop MapReduce jobs • Monitoring the progress of job flows
  • 15. Why Big Data? • FB generates 10TB daily • Twitter generates 7TB of data Daily • IBM claims 90% of today’s stored data was generated in just the last two years.
  • 16. How Is Big Data Different? Automatically generated by a machine (e.g. Sensor embedded in an engine) Typically an entirely new source of data (e.g. Use of the internet) Not designed to be friendly (e.g. Text streams) 4) May not have much values
  • 17. Big Data Analytics • Examining large amount of data • Appropriate information • Identification of hidden patterns, unknown correlations • Competitive advantage • Better business decisions: strategic and operational • Effective marketing, customer satisfaction, increased revenue
  • 18. Types of tools used in Big-Data Where processing is hosted? – Distributed Servers / Cloud (e.g. Amazon EC2)  Where data is stored? – Distributed Storage (e.g. Amazon S3)  What is the programming model? – Distributed Processing (e.g. MapReduce)  How data is stored & indexed? – High-performance schema-free databases (e.g. MongoDB)  What operations are performed on data? – Analytic / Semantic Processing
  • 19. Application Of Big Data analytics Smarter Healthcare Homeland Security Traffic Control Manufacturing Multi-channel sales Telecom Trading Analytics Search Quality
  • 20. Risks of Big Data Will be so overwhelmed  Need the right people and solve the right problems Costs escalate too fast  Isn’t necessary to capture 100% Many sources of big data is privacy  self-regulation  Legal regulation 22
  • 21. Leading Technology Vendors  Example Vendors • IBM – Netezza • EMC – Greenplum • Oracle – Exadata  Commonality • MPP architectures • Commodity Hardware • RDBMS based • Full SQL compliance
  • 22. How Big data impacts on IT ? Big data is a troublesome force presenting opportunities with challenges to IT organizations. By 2015 4.4 million IT jobs in Big Data ; 1.9 million is in US itself India will require a minimum of 1 lakh data scientists in the next couple of years in addition to data analysts and data managers to support the Big Data space.
  • 23. Benefits of Big Data Real-time big data isn’t just a process for storing petabytes or exabytes of data in a data warehouse, It’s about the ability to make better decisions and take meaningful actions at the right time. Fast forward to the present and technologies like Hadoop give you the scale and flexibility to store data before you know how you are going to process it. Technologies such as MapReduce,Hive and Impala enable you to run queries without changing the data structures underneath.
  • 24. Benefits of Big Data Our newest research finds that organizations are using big data to target customer-centric outcomes, tap into internal data and build a better information ecosystem. Big Data is already an important part of the $64 billion database and data analytics market It offers commercial opportunities of a comparable scale to enterprise software in the late 1980s And the Internet boom of the 1990s, and the social media explosion of today.
  • 25. Future of Big Data  $15 billion on software firms only specializing in data management and analytics. This industry on its own is worth more than $100 billion and growing at almost 10% a year which is roughly twice as fast as the software business as a whole. In February 2012, the open source analyst firm Wikibon released the first market forecast for Big Data , listing $5.1B revenue in 2012 with growth to $53.4B in 2017 The McKinsey Global Institute estimates that data volume is growing 40% per year, and will grow 44x between 2009 and 2020.