SlideShare a Scribd company logo
Paper Presentation
on
Challenges of Big Data to Big Data Mining
with their Processing Framework
Kamlesh Kumar Pandey
Dept. of Computer Science & Applications
Dr. Hari Singh Gour Vishwavidyalaya, Sagar, M.P
E-mail: kamleshamk@gmail.com
International Conference on Communication Systems and Network Technologies 2018
Content
• Big Data
• Big Data Mining
• Data challenges
• Process challenges
• Management Challenges
• Big Data Mining Processing Framework
BIG DATA
• Diebold et Al. (2000) is a first writer who discussed the word Big Data
in his research paper. All of these authors define Big Data there means
if the data set is large then gigabyte then these type of data set is
known as Big Data.
• Doug Laney et al (2001) was the first person who gave a proper
definition for Big Data. He gave three characteristics Volume, Variety,
and Velocity of Big Data and these characteristics known as 3 V’s of
Big Data Management. Basically, these 3 V’s describe the framework
of Big Data.
• Gartner (2012), “Big data is high-volume, high-velocity and high-
variety information assets that demand cost-effective, innovative
forms of information processing for enhanced insight and decision
making”
BIG DATA V’s
• In present time seven V’s used for Big Data where the first three V’s Volume,
Variety, and Velocity are the main characteristics of big data. In addition to
Variability, Value, Veracity, and Visualization are depending on the
organization.
BIG DATA MINING
• Big Data Mining fetching on the requested information, uncovering
hidden relationship or patterns or extracting for the needed
information or knowledge from a dataset these datasets have to meet
three V’s of Big Data with higher complexity.
CHALLENGES OF BIG DATA MINING
• Data challenges,
• Process challenges
• Management challenges
• Data challenges are based on the basic characteristics such as volume,
variety, velocity, veracity etc. of the Big Data. These type of challenges differ
from traditional data characteristics.
• Process challenges are based on the technique for data mining, data
processing or analysis in which algorithms are used to mining or analysis,
integration, transform, preprocessing on data etc.
• Management challenges are cover to data management related challenges like
privacy, security, governance, and other aspects.
DATA CHALLENGES
• Roberto V. Zicari et al. (2014) and Uthayasankar Sivarajah et al.
(2017) are categorizing data challenges in seven categories.
• Volume
• Variety
• Velocity
• Variability
• Value
• Veracity
• Visualization
PROCESS CHALLENGES
• Kaisler et al. (2013) and Uthayasankar Sivarajah et al. (2017) identify
data processing related challenges that can be classify into five steps for
data mining.
• Data acquisition and warehousing
• Data cleaning
• Data analysis and Mining
• Data integration and aggregation
• Data querying and indexing
MANAGEMENT CHALLENGES
• Uthayasankar Sivarajah et al (2017) has discussed various
Management challenges which are ensuring data are used correctly,
data access where used by only authorized person, without any
permission data are not accessible, which maintains privacy, given
higher security from external and internal attack, the proper way of
transformed and derived data etc.
• Privacy
• Security
• Data and information sharing
• Cost/operational expenditures
• Data ownership
BIG DATA MINING PROCESSING FRAMEWORK
• Wu Xindong et al. (2014) presents a HACE theorem and big data
processing model for big data mining process and challenges
perspective. This big data mining processing model cover to data and
management driven challenges.
References
• Fan Wei and Bifet Albert (2012): “Mining Big Data: Current Status, and Forecast to the Future”, ACM SIGKDD Explorations Newsletter, V-14, I-2, pp1-5.
• K.U. Jaseena and David M. (2014): “Issue Challenges and Solution: Big Data Mining”, Published in the Proc. Of SMTP-2014, Published By AIRCC Publishing Corporation, held in Chennai, India on 27-28
Dec 2014, pp 131-140.
• Landset Sara, Khoshgoftaar Taghi M, Richter Aaron N. and Hasanin Tawfiq(2015): “A survey of open source tools for machine learning with big data in the Hadoop ecosystem”, Journal of Big Data, 2:
24K. Elissa, “Title of paper if known,” unpublished.
• Sivarajah Uthayasankar and Mustafa Kamal Muhammad (2017): “Critical analysis of Big Data challenges and analytical methods”, Journal of Business Research (Elsevier), V-70, PP 263-286.
• Najafabadi Maryam M, Villanustre Flavio, Khoshgoftaar Taghi M, Seliya Naeem, Wald Randall and Muharemagic Edin (2015): “Deep learning applications and challenges in big data analytics”, Journal
of Big Data, 2:1.
• Bifet Albert, (2013), “Mining Big data in Real-time”, Informatica, V-37, I-1, PP 15-20.
• Che Dunren, Safran Mejdl and Peng Zhiyong (2013): “From Big Data to Big Data Mining: Challenges, Issues, and Opportunities”, Published in the Proc. Of International Conference on Database Systems
for Advanced Applications Organized & Published by Springer held in Suzhou, China in March 2017, PP 1 to 15.
• Gandomi Amir and Haider Murtaza(2015): “Beyond the hype: Big data concepts, methods, and analytics”, International Journal of Information Management, Published By Springer, V-35, PP 137 to
144.
• Pandey Kamlesh (2018),: “Mining on Relationship in Big Data era Using Apriori Algorithm”, Published in the Proc. Of National Conference on Data Analytics, Machine Learning and Security to be held on
15-16 February 2018 by Department of CSIT, GGV, Bilaspur, C.G, India, ISBN: 978-93-5291-457-9.
• Fayyad Usama and Piatetsky-Shapiro Gregory (1996): “From Data Mining to Knowledge Discovery in Databases” Artificial Intelligence Magazine, V-17, I-3, PP-37-54.
• Pandey Kamlesh(2014): “An Analytical and Comparative Study of Various Data Preprocessing Method in Data Mining” International Journal of Emerging Technology and Advanced Engineering (ISSN
2250-2459), V-4, I-10, PP 174 to 180.
• Zicari, R. (2014): “Big Data: Challenges and Opportunities”, Chapman and Hall/CRC, pp. 103–128.
• Kaisler Stephen, Armour Frank and Espinosa J. Alberto (2013), “Big Data: Issues and Challenges Moving Forward”, Published in the Proc. Of 46th Hawaii International Conference on System Sciences
Published by IEEE held in Wailea, Maui, HI, the USA at 7-10 Jan. 2013.
• Chen Min, Mao Shiwen, Zhang Yin, and Leung Victor C.M. (2014): “Big Data Related Technologies, Challenges, and Future Prospects”, Springer Briefs in Computer Science, ISSN 2191-5776 (electronic).
• Wu Xindong, Zhu Xingquan, Wu Gong-Qing, and Ding Wei (2014), “Data Mining with Big Data”, IEEE Transactions on Knowledge & Data Engineering, VOL. 26, NO. 1, pp 97-107.
• R. Kamala, MaryGladence L. (2015), “An optimal approach for social data analysis in Big Data”, Published in the Proc. of ICCPEIC Published by IEEE held in 22-23 April 2015 at Chennai, India, pp 205-
208.
challenges of big data to big data mining with their processing framework

More Related Content

What's hot

Challenges of Big Data Research
Challenges of Big Data ResearchChallenges of Big Data Research
Challenges of Big Data Research
Regional Science Academy
 
Data Mining and Big Data Challenges and Research Opportunities
Data Mining and Big Data Challenges and Research OpportunitiesData Mining and Big Data Challenges and Research Opportunities
Data Mining and Big Data Challenges and Research Opportunities
Kathirvel Ayyaswamy
 
Motivation for big data
Motivation for big dataMotivation for big data
Motivation for big data
Arockiaraj Durairaj
 
An Comprehensive Study of Big Data Environment and its Challenges.
An Comprehensive Study of Big Data Environment and its Challenges.An Comprehensive Study of Big Data Environment and its Challenges.
An Comprehensive Study of Big Data Environment and its Challenges.
ijceronline
 
Elementary Concepts of data minig
Elementary Concepts of data minigElementary Concepts of data minig
Elementary Concepts of data minig
Dr Anjan Krishnamurthy
 
elgendy2014.pdf
elgendy2014.pdfelgendy2014.pdf
elgendy2014.pdf
Akuhuruf
 
Data minig with Big data analysis
Data minig with Big data analysisData minig with Big data analysis
Data minig with Big data analysis
Poonam Kshirsagar
 
Data mining & big data presentation 01
Data mining & big data presentation 01Data mining & big data presentation 01
Data mining & big data presentation 01
Aseem Chakrabarthy
 
Data quality - The True Big Data Challenge
Data quality - The True Big Data ChallengeData quality - The True Big Data Challenge
Data quality - The True Big Data Challenge
Stefan Kühn
 
Big data analysis
Big data analysisBig data analysis
Big data analysis
SAishwaryaDinesh
 
Big Data Analytics
Big Data AnalyticsBig Data Analytics
Big Data Analytics
Ghulam Imaduddin
 
Issues, challenges, and solutions
Issues, challenges, and solutionsIssues, challenges, and solutions
Issues, challenges, and solutions
csandit
 
Sameer Kumar Das International Conference Paper 53
Sameer Kumar Das International Conference Paper 53Sameer Kumar Das International Conference Paper 53
Sameer Kumar Das International Conference Paper 53
Mr.Sameer Kumar Das
 
000 introduction to big data analytics 2021
000   introduction to big data analytics  2021000   introduction to big data analytics  2021
000 introduction to big data analytics 2021
Dendej Sawarnkatat
 
Big data Analytics in Information Technology
Big data Analytics in Information TechnologyBig data Analytics in Information Technology
Big data Analytics in Information Technology
technakama
 
Introduction to big data
Introduction to big dataIntroduction to big data
Introduction to big data
Richard Vidgen
 
¿En qué se parece el Gobierno del Dato a un parque de atracciones?
¿En qué se parece el Gobierno del Dato a un parque de atracciones?¿En qué se parece el Gobierno del Dato a un parque de atracciones?
¿En qué se parece el Gobierno del Dato a un parque de atracciones?
Denodo
 
M.Florence Dayana
M.Florence DayanaM.Florence Dayana
M.Florence Dayana
Dr.Florence Dayana
 
Data mining on big data
Data mining on big dataData mining on big data
Data mining on big data
Swapnil Chaudhari
 
A Model Design of Big Data Processing using HACE Theorem
A Model Design of Big Data Processing using HACE TheoremA Model Design of Big Data Processing using HACE Theorem
A Model Design of Big Data Processing using HACE Theorem
AnthonyOtuonye
 

What's hot (20)

Challenges of Big Data Research
Challenges of Big Data ResearchChallenges of Big Data Research
Challenges of Big Data Research
 
Data Mining and Big Data Challenges and Research Opportunities
Data Mining and Big Data Challenges and Research OpportunitiesData Mining and Big Data Challenges and Research Opportunities
Data Mining and Big Data Challenges and Research Opportunities
 
Motivation for big data
Motivation for big dataMotivation for big data
Motivation for big data
 
An Comprehensive Study of Big Data Environment and its Challenges.
An Comprehensive Study of Big Data Environment and its Challenges.An Comprehensive Study of Big Data Environment and its Challenges.
An Comprehensive Study of Big Data Environment and its Challenges.
 
Elementary Concepts of data minig
Elementary Concepts of data minigElementary Concepts of data minig
Elementary Concepts of data minig
 
elgendy2014.pdf
elgendy2014.pdfelgendy2014.pdf
elgendy2014.pdf
 
Data minig with Big data analysis
Data minig with Big data analysisData minig with Big data analysis
Data minig with Big data analysis
 
Data mining & big data presentation 01
Data mining & big data presentation 01Data mining & big data presentation 01
Data mining & big data presentation 01
 
Data quality - The True Big Data Challenge
Data quality - The True Big Data ChallengeData quality - The True Big Data Challenge
Data quality - The True Big Data Challenge
 
Big data analysis
Big data analysisBig data analysis
Big data analysis
 
Big Data Analytics
Big Data AnalyticsBig Data Analytics
Big Data Analytics
 
Issues, challenges, and solutions
Issues, challenges, and solutionsIssues, challenges, and solutions
Issues, challenges, and solutions
 
Sameer Kumar Das International Conference Paper 53
Sameer Kumar Das International Conference Paper 53Sameer Kumar Das International Conference Paper 53
Sameer Kumar Das International Conference Paper 53
 
000 introduction to big data analytics 2021
000   introduction to big data analytics  2021000   introduction to big data analytics  2021
000 introduction to big data analytics 2021
 
Big data Analytics in Information Technology
Big data Analytics in Information TechnologyBig data Analytics in Information Technology
Big data Analytics in Information Technology
 
Introduction to big data
Introduction to big dataIntroduction to big data
Introduction to big data
 
¿En qué se parece el Gobierno del Dato a un parque de atracciones?
¿En qué se parece el Gobierno del Dato a un parque de atracciones?¿En qué se parece el Gobierno del Dato a un parque de atracciones?
¿En qué se parece el Gobierno del Dato a un parque de atracciones?
 
M.Florence Dayana
M.Florence DayanaM.Florence Dayana
M.Florence Dayana
 
Data mining on big data
Data mining on big dataData mining on big data
Data mining on big data
 
A Model Design of Big Data Processing using HACE Theorem
A Model Design of Big Data Processing using HACE TheoremA Model Design of Big Data Processing using HACE Theorem
A Model Design of Big Data Processing using HACE Theorem
 

Similar to challenges of big data to big data mining with their processing framework

data mining
data miningdata mining
data mining
mahsa rezaei
 
NCME Big Data in Education
NCME Big Data  in EducationNCME Big Data  in Education
NCME Big Data in Education
Philip Piety
 
Big data trends in 2020
Big data trends in 2020Big data trends in 2020
Big data trends in 2020
AIRCC Publishing Corporation
 
Big data characteristics, value chain and challenges
Big data characteristics, value chain and challengesBig data characteristics, value chain and challenges
Big data characteristics, value chain and challenges
Musfiqur Rahman
 
BIG DATA ANALYTICS.pptx
BIG DATA ANALYTICS.pptxBIG DATA ANALYTICS.pptx
BIG DATA ANALYTICS.pptx
Vikas Bhowate
 
Big Data and Data Mining - Lecture 3 in Introduction to Computational Social ...
Big Data and Data Mining - Lecture 3 in Introduction to Computational Social ...Big Data and Data Mining - Lecture 3 in Introduction to Computational Social ...
Big Data and Data Mining - Lecture 3 in Introduction to Computational Social ...
Lauri Eloranta
 
Big data
Big dataBig data
Data Warehouse
Data WarehouseData Warehouse
Data Warehouse
Sana Alvi
 
01-introduction.ppt the paper that you can unless you want to join me because...
01-introduction.ppt the paper that you can unless you want to join me because...01-introduction.ppt the paper that you can unless you want to join me because...
01-introduction.ppt the paper that you can unless you want to join me because...
teodroscampaus
 
Data Mining : Concepts and Techniques
Data Mining : Concepts and TechniquesData Mining : Concepts and Techniques
Data Mining : Concepts and Techniques
DeepaR42
 
00-01 DSnDA.pdf
00-01 DSnDA.pdf00-01 DSnDA.pdf
00-01 DSnDA.pdf
SugumarSarDurai
 
Mining on Relationships in Big Data era using Improve Apriori Algorithm with ...
Mining on Relationships in Big Data era using Improve Apriori Algorithm with ...Mining on Relationships in Big Data era using Improve Apriori Algorithm with ...
Mining on Relationships in Big Data era using Improve Apriori Algorithm with ...
KamleshKumar394
 
DataScience.pptx
DataScience.pptxDataScience.pptx
DataScience.pptx
M Vishnuvardhan Reddy
 
A Review Of Data Mining Literature
A Review Of Data Mining LiteratureA Review Of Data Mining Literature
A Review Of Data Mining Literature
Addison Coleman
 
New research articles 2020 august issue- international journal of computer ...
New research articles   2020 august issue- international journal of computer ...New research articles   2020 august issue- international journal of computer ...
New research articles 2020 august issue- international journal of computer ...
AIRCC Publishing Corporation
 
Automating Data Science over a Human Genomics Knowledge Base
Automating Data Science over a Human Genomics Knowledge BaseAutomating Data Science over a Human Genomics Knowledge Base
Automating Data Science over a Human Genomics Knowledge Base
Vaticle
 
A New Paradigm on Analytic-Driven Information and Automation V2.pdf
A New Paradigm on Analytic-Driven Information and Automation V2.pdfA New Paradigm on Analytic-Driven Information and Automation V2.pdf
A New Paradigm on Analytic-Driven Information and Automation V2.pdf
ArmyTrilidiaDevegaSK
 
Data mining Introduction
Data mining IntroductionData mining Introduction
Data mining Introduction
VijayasankariS
 
Crowdsourcing Approaches to Big Data Curation - Rio Big Data Meetup
Crowdsourcing Approaches to Big Data Curation - Rio Big Data MeetupCrowdsourcing Approaches to Big Data Curation - Rio Big Data Meetup
Crowdsourcing Approaches to Big Data Curation - Rio Big Data Meetup
Edward Curry
 
Big data
Big dataBig data
Big data
nikki135
 

Similar to challenges of big data to big data mining with their processing framework (20)

data mining
data miningdata mining
data mining
 
NCME Big Data in Education
NCME Big Data  in EducationNCME Big Data  in Education
NCME Big Data in Education
 
Big data trends in 2020
Big data trends in 2020Big data trends in 2020
Big data trends in 2020
 
Big data characteristics, value chain and challenges
Big data characteristics, value chain and challengesBig data characteristics, value chain and challenges
Big data characteristics, value chain and challenges
 
BIG DATA ANALYTICS.pptx
BIG DATA ANALYTICS.pptxBIG DATA ANALYTICS.pptx
BIG DATA ANALYTICS.pptx
 
Big Data and Data Mining - Lecture 3 in Introduction to Computational Social ...
Big Data and Data Mining - Lecture 3 in Introduction to Computational Social ...Big Data and Data Mining - Lecture 3 in Introduction to Computational Social ...
Big Data and Data Mining - Lecture 3 in Introduction to Computational Social ...
 
Big data
Big dataBig data
Big data
 
Data Warehouse
Data WarehouseData Warehouse
Data Warehouse
 
01-introduction.ppt the paper that you can unless you want to join me because...
01-introduction.ppt the paper that you can unless you want to join me because...01-introduction.ppt the paper that you can unless you want to join me because...
01-introduction.ppt the paper that you can unless you want to join me because...
 
Data Mining : Concepts and Techniques
Data Mining : Concepts and TechniquesData Mining : Concepts and Techniques
Data Mining : Concepts and Techniques
 
00-01 DSnDA.pdf
00-01 DSnDA.pdf00-01 DSnDA.pdf
00-01 DSnDA.pdf
 
Mining on Relationships in Big Data era using Improve Apriori Algorithm with ...
Mining on Relationships in Big Data era using Improve Apriori Algorithm with ...Mining on Relationships in Big Data era using Improve Apriori Algorithm with ...
Mining on Relationships in Big Data era using Improve Apriori Algorithm with ...
 
DataScience.pptx
DataScience.pptxDataScience.pptx
DataScience.pptx
 
A Review Of Data Mining Literature
A Review Of Data Mining LiteratureA Review Of Data Mining Literature
A Review Of Data Mining Literature
 
New research articles 2020 august issue- international journal of computer ...
New research articles   2020 august issue- international journal of computer ...New research articles   2020 august issue- international journal of computer ...
New research articles 2020 august issue- international journal of computer ...
 
Automating Data Science over a Human Genomics Knowledge Base
Automating Data Science over a Human Genomics Knowledge BaseAutomating Data Science over a Human Genomics Knowledge Base
Automating Data Science over a Human Genomics Knowledge Base
 
A New Paradigm on Analytic-Driven Information and Automation V2.pdf
A New Paradigm on Analytic-Driven Information and Automation V2.pdfA New Paradigm on Analytic-Driven Information and Automation V2.pdf
A New Paradigm on Analytic-Driven Information and Automation V2.pdf
 
Data mining Introduction
Data mining IntroductionData mining Introduction
Data mining Introduction
 
Crowdsourcing Approaches to Big Data Curation - Rio Big Data Meetup
Crowdsourcing Approaches to Big Data Curation - Rio Big Data MeetupCrowdsourcing Approaches to Big Data Curation - Rio Big Data Meetup
Crowdsourcing Approaches to Big Data Curation - Rio Big Data Meetup
 
Big data
Big dataBig data
Big data
 

Recently uploaded

一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
g4dpvqap0
 
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
nyfuhyz
 
Everything you wanted to know about LIHTC
Everything you wanted to know about LIHTCEverything you wanted to know about LIHTC
Everything you wanted to know about LIHTC
Roger Valdez
 
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
nuttdpt
 
一比一原版(UCSF文凭证书)旧金山分校毕业证如何办理
一比一原版(UCSF文凭证书)旧金山分校毕业证如何办理一比一原版(UCSF文凭证书)旧金山分校毕业证如何办理
一比一原版(UCSF文凭证书)旧金山分校毕业证如何办理
nuttdpt
 
Experts live - Improving user adoption with AI
Experts live - Improving user adoption with AIExperts live - Improving user adoption with AI
Experts live - Improving user adoption with AI
jitskeb
 
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
Social Samosa
 
State of Artificial intelligence Report 2023
State of Artificial intelligence Report 2023State of Artificial intelligence Report 2023
State of Artificial intelligence Report 2023
kuntobimo2016
 
A presentation that explain the Power BI Licensing
A presentation that explain the Power BI LicensingA presentation that explain the Power BI Licensing
A presentation that explain the Power BI Licensing
AlessioFois2
 
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
Aggregage
 
一比一原版(Harvard毕业证书)哈佛大学毕业证如何办理
一比一原版(Harvard毕业证书)哈佛大学毕业证如何办理一比一原版(Harvard毕业证书)哈佛大学毕业证如何办理
一比一原版(Harvard毕业证书)哈佛大学毕业证如何办理
zsjl4mimo
 
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
apvysm8
 
Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......
Sachin Paul
 
一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理
aqzctr7x
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
vikram sood
 
The Building Blocks of QuestDB, a Time Series Database
The Building Blocks of QuestDB, a Time Series DatabaseThe Building Blocks of QuestDB, a Time Series Database
The Building Blocks of QuestDB, a Time Series Database
javier ramirez
 
My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.
rwarrenll
 
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataPredictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Kiwi Creative
 
Intelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicineIntelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicine
AndrzejJarynowski
 
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
Timothy Spann
 

Recently uploaded (20)

一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
 
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
 
Everything you wanted to know about LIHTC
Everything you wanted to know about LIHTCEverything you wanted to know about LIHTC
Everything you wanted to know about LIHTC
 
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
 
一比一原版(UCSF文凭证书)旧金山分校毕业证如何办理
一比一原版(UCSF文凭证书)旧金山分校毕业证如何办理一比一原版(UCSF文凭证书)旧金山分校毕业证如何办理
一比一原版(UCSF文凭证书)旧金山分校毕业证如何办理
 
Experts live - Improving user adoption with AI
Experts live - Improving user adoption with AIExperts live - Improving user adoption with AI
Experts live - Improving user adoption with AI
 
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
 
State of Artificial intelligence Report 2023
State of Artificial intelligence Report 2023State of Artificial intelligence Report 2023
State of Artificial intelligence Report 2023
 
A presentation that explain the Power BI Licensing
A presentation that explain the Power BI LicensingA presentation that explain the Power BI Licensing
A presentation that explain the Power BI Licensing
 
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
 
一比一原版(Harvard毕业证书)哈佛大学毕业证如何办理
一比一原版(Harvard毕业证书)哈佛大学毕业证如何办理一比一原版(Harvard毕业证书)哈佛大学毕业证如何办理
一比一原版(Harvard毕业证书)哈佛大学毕业证如何办理
 
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
 
Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......
 
一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
 
The Building Blocks of QuestDB, a Time Series Database
The Building Blocks of QuestDB, a Time Series DatabaseThe Building Blocks of QuestDB, a Time Series Database
The Building Blocks of QuestDB, a Time Series Database
 
My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.
 
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataPredictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
 
Intelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicineIntelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicine
 
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
 

challenges of big data to big data mining with their processing framework

  • 1. Paper Presentation on Challenges of Big Data to Big Data Mining with their Processing Framework Kamlesh Kumar Pandey Dept. of Computer Science & Applications Dr. Hari Singh Gour Vishwavidyalaya, Sagar, M.P E-mail: kamleshamk@gmail.com International Conference on Communication Systems and Network Technologies 2018
  • 2. Content • Big Data • Big Data Mining • Data challenges • Process challenges • Management Challenges • Big Data Mining Processing Framework
  • 3. BIG DATA • Diebold et Al. (2000) is a first writer who discussed the word Big Data in his research paper. All of these authors define Big Data there means if the data set is large then gigabyte then these type of data set is known as Big Data. • Doug Laney et al (2001) was the first person who gave a proper definition for Big Data. He gave three characteristics Volume, Variety, and Velocity of Big Data and these characteristics known as 3 V’s of Big Data Management. Basically, these 3 V’s describe the framework of Big Data. • Gartner (2012), “Big data is high-volume, high-velocity and high- variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making”
  • 4. BIG DATA V’s • In present time seven V’s used for Big Data where the first three V’s Volume, Variety, and Velocity are the main characteristics of big data. In addition to Variability, Value, Veracity, and Visualization are depending on the organization.
  • 5. BIG DATA MINING • Big Data Mining fetching on the requested information, uncovering hidden relationship or patterns or extracting for the needed information or knowledge from a dataset these datasets have to meet three V’s of Big Data with higher complexity.
  • 6. CHALLENGES OF BIG DATA MINING • Data challenges, • Process challenges • Management challenges • Data challenges are based on the basic characteristics such as volume, variety, velocity, veracity etc. of the Big Data. These type of challenges differ from traditional data characteristics. • Process challenges are based on the technique for data mining, data processing or analysis in which algorithms are used to mining or analysis, integration, transform, preprocessing on data etc. • Management challenges are cover to data management related challenges like privacy, security, governance, and other aspects.
  • 7. DATA CHALLENGES • Roberto V. Zicari et al. (2014) and Uthayasankar Sivarajah et al. (2017) are categorizing data challenges in seven categories. • Volume • Variety • Velocity • Variability • Value • Veracity • Visualization
  • 8. PROCESS CHALLENGES • Kaisler et al. (2013) and Uthayasankar Sivarajah et al. (2017) identify data processing related challenges that can be classify into five steps for data mining. • Data acquisition and warehousing • Data cleaning • Data analysis and Mining • Data integration and aggregation • Data querying and indexing
  • 9. MANAGEMENT CHALLENGES • Uthayasankar Sivarajah et al (2017) has discussed various Management challenges which are ensuring data are used correctly, data access where used by only authorized person, without any permission data are not accessible, which maintains privacy, given higher security from external and internal attack, the proper way of transformed and derived data etc. • Privacy • Security • Data and information sharing • Cost/operational expenditures • Data ownership
  • 10. BIG DATA MINING PROCESSING FRAMEWORK • Wu Xindong et al. (2014) presents a HACE theorem and big data processing model for big data mining process and challenges perspective. This big data mining processing model cover to data and management driven challenges.
  • 11. References • Fan Wei and Bifet Albert (2012): “Mining Big Data: Current Status, and Forecast to the Future”, ACM SIGKDD Explorations Newsletter, V-14, I-2, pp1-5. • K.U. Jaseena and David M. (2014): “Issue Challenges and Solution: Big Data Mining”, Published in the Proc. Of SMTP-2014, Published By AIRCC Publishing Corporation, held in Chennai, India on 27-28 Dec 2014, pp 131-140. • Landset Sara, Khoshgoftaar Taghi M, Richter Aaron N. and Hasanin Tawfiq(2015): “A survey of open source tools for machine learning with big data in the Hadoop ecosystem”, Journal of Big Data, 2: 24K. Elissa, “Title of paper if known,” unpublished. • Sivarajah Uthayasankar and Mustafa Kamal Muhammad (2017): “Critical analysis of Big Data challenges and analytical methods”, Journal of Business Research (Elsevier), V-70, PP 263-286. • Najafabadi Maryam M, Villanustre Flavio, Khoshgoftaar Taghi M, Seliya Naeem, Wald Randall and Muharemagic Edin (2015): “Deep learning applications and challenges in big data analytics”, Journal of Big Data, 2:1. • Bifet Albert, (2013), “Mining Big data in Real-time”, Informatica, V-37, I-1, PP 15-20. • Che Dunren, Safran Mejdl and Peng Zhiyong (2013): “From Big Data to Big Data Mining: Challenges, Issues, and Opportunities”, Published in the Proc. Of International Conference on Database Systems for Advanced Applications Organized & Published by Springer held in Suzhou, China in March 2017, PP 1 to 15. • Gandomi Amir and Haider Murtaza(2015): “Beyond the hype: Big data concepts, methods, and analytics”, International Journal of Information Management, Published By Springer, V-35, PP 137 to 144. • Pandey Kamlesh (2018),: “Mining on Relationship in Big Data era Using Apriori Algorithm”, Published in the Proc. Of National Conference on Data Analytics, Machine Learning and Security to be held on 15-16 February 2018 by Department of CSIT, GGV, Bilaspur, C.G, India, ISBN: 978-93-5291-457-9. • Fayyad Usama and Piatetsky-Shapiro Gregory (1996): “From Data Mining to Knowledge Discovery in Databases” Artificial Intelligence Magazine, V-17, I-3, PP-37-54. • Pandey Kamlesh(2014): “An Analytical and Comparative Study of Various Data Preprocessing Method in Data Mining” International Journal of Emerging Technology and Advanced Engineering (ISSN 2250-2459), V-4, I-10, PP 174 to 180. • Zicari, R. (2014): “Big Data: Challenges and Opportunities”, Chapman and Hall/CRC, pp. 103–128. • Kaisler Stephen, Armour Frank and Espinosa J. Alberto (2013), “Big Data: Issues and Challenges Moving Forward”, Published in the Proc. Of 46th Hawaii International Conference on System Sciences Published by IEEE held in Wailea, Maui, HI, the USA at 7-10 Jan. 2013. • Chen Min, Mao Shiwen, Zhang Yin, and Leung Victor C.M. (2014): “Big Data Related Technologies, Challenges, and Future Prospects”, Springer Briefs in Computer Science, ISSN 2191-5776 (electronic). • Wu Xindong, Zhu Xingquan, Wu Gong-Qing, and Ding Wei (2014), “Data Mining with Big Data”, IEEE Transactions on Knowledge & Data Engineering, VOL. 26, NO. 1, pp 97-107. • R. Kamala, MaryGladence L. (2015), “An optimal approach for social data analysis in Big Data”, Published in the Proc. of ICCPEIC Published by IEEE held in 22-23 April 2015 at Chennai, India, pp 205- 208.