Submit Search
Upload
Big Data Processing with Hadoop : A Review
•
0 likes
•
30 views
IRJET Journal
Follow
https://irjet.net/archives/V4/i2/IRJET-V4I285.pdf
Read less
Read more
Engineering
Report
Share
Report
Share
1 of 4
Download now
Download to read offline
Recommended
E018142329
E018142329
IOSR Journals
U0 vqmtq3m tc=
U0 vqmtq3m tc=
International Journal of Science and Research (IJSR)
Big Data
Big Data
Kirubaburi R
IJARCCE_49
IJARCCE_49
Mr.Sameer Kumar Das
Data Ware House System in Cloud Environment
Data Ware House System in Cloud Environment
IJERA Editor
IRJET- A Comparative Study on Big Data Analytics Approaches and Tools
IRJET- A Comparative Study on Big Data Analytics Approaches and Tools
IRJET Journal
Big data Presentation
Big data Presentation
himanshu arora
IRJET - Survey Paper on Map Reduce Processing using HADOOP
IRJET - Survey Paper on Map Reduce Processing using HADOOP
IRJET Journal
Recommended
E018142329
E018142329
IOSR Journals
U0 vqmtq3m tc=
U0 vqmtq3m tc=
International Journal of Science and Research (IJSR)
Big Data
Big Data
Kirubaburi R
IJARCCE_49
IJARCCE_49
Mr.Sameer Kumar Das
Data Ware House System in Cloud Environment
Data Ware House System in Cloud Environment
IJERA Editor
IRJET- A Comparative Study on Big Data Analytics Approaches and Tools
IRJET- A Comparative Study on Big Data Analytics Approaches and Tools
IRJET Journal
Big data Presentation
Big data Presentation
himanshu arora
IRJET - Survey Paper on Map Reduce Processing using HADOOP
IRJET - Survey Paper on Map Reduce Processing using HADOOP
IRJET Journal
A Comprehensive Study on Big Data Applications and Challenges
A Comprehensive Study on Big Data Applications and Challenges
ijcisjournal
IRJET- Systematic Review: Progression Study on BIG DATA articles
IRJET- Systematic Review: Progression Study on BIG DATA articles
IRJET Journal
Seminar Report Vaibhav
Seminar Report Vaibhav
Vaibhav Dhattarwal
A Review Paper on Big Data and Hadoop for Data Science
A Review Paper on Big Data and Hadoop for Data Science
ijtsrd
Big data analytics
Big data analytics
Dr.Bhuvaneswari Velumani
hadoop seminar training report
hadoop seminar training report
Sarvesh Meena
Infrastructure Considerations for Analytical Workloads
Infrastructure Considerations for Analytical Workloads
Cognizant
The Big Data Importance – Tools and their Usage
The Big Data Importance – Tools and their Usage
IRJET Journal
IRJET- Survey of Big Data with Hadoop
IRJET- Survey of Big Data with Hadoop
IRJET Journal
Big data analysis concepts and references by Cloud Security Alliance
Big data analysis concepts and references by Cloud Security Alliance
Information Security Awareness Group
IRJET-Unraveling the Data Structures of Big data, the HDFS Architecture and I...
IRJET-Unraveling the Data Structures of Big data, the HDFS Architecture and I...
IRJET Journal
Hadoop
Hadoop
Veera Sundari
big data
big data
Jisha Aravind
Bigdata
Bigdata
NithiDazz
DOCUMENT SELECTION USING MAPREDUCE
DOCUMENT SELECTION USING MAPREDUCE
ijsptm
Big data analysis concepts and references
Big data analysis concepts and references
Information Security Awareness Group
Big Data Mining, Techniques, Handling Technologies and Some Related Issues: A...
Big Data Mining, Techniques, Handling Technologies and Some Related Issues: A...
IJSRD
A STUDY ON GRAPH STORAGE DATABASE OF NOSQL
A STUDY ON GRAPH STORAGE DATABASE OF NOSQL
ijscai
Big Data Testing Using Hadoop Platform
Big Data Testing Using Hadoop Platform
IRJET Journal
Big Data Summarization : Framework, Challenges and Possible Solutions
Big Data Summarization : Framework, Challenges and Possible Solutions
aciijournal
Big Data Summarization : Framework, Challenges and Possible Solutions
Big Data Summarization : Framework, Challenges and Possible Solutions
aciijournal
BIG DATA SUMMARIZATION: FRAMEWORK, CHALLENGES AND POSSIBLE SOLUTIONS
BIG DATA SUMMARIZATION: FRAMEWORK, CHALLENGES AND POSSIBLE SOLUTIONS
aciijournal
More Related Content
What's hot
A Comprehensive Study on Big Data Applications and Challenges
A Comprehensive Study on Big Data Applications and Challenges
ijcisjournal
IRJET- Systematic Review: Progression Study on BIG DATA articles
IRJET- Systematic Review: Progression Study on BIG DATA articles
IRJET Journal
Seminar Report Vaibhav
Seminar Report Vaibhav
Vaibhav Dhattarwal
A Review Paper on Big Data and Hadoop for Data Science
A Review Paper on Big Data and Hadoop for Data Science
ijtsrd
Big data analytics
Big data analytics
Dr.Bhuvaneswari Velumani
hadoop seminar training report
hadoop seminar training report
Sarvesh Meena
Infrastructure Considerations for Analytical Workloads
Infrastructure Considerations for Analytical Workloads
Cognizant
The Big Data Importance – Tools and their Usage
The Big Data Importance – Tools and their Usage
IRJET Journal
IRJET- Survey of Big Data with Hadoop
IRJET- Survey of Big Data with Hadoop
IRJET Journal
Big data analysis concepts and references by Cloud Security Alliance
Big data analysis concepts and references by Cloud Security Alliance
Information Security Awareness Group
IRJET-Unraveling the Data Structures of Big data, the HDFS Architecture and I...
IRJET-Unraveling the Data Structures of Big data, the HDFS Architecture and I...
IRJET Journal
Hadoop
Hadoop
Veera Sundari
big data
big data
Jisha Aravind
Bigdata
Bigdata
NithiDazz
DOCUMENT SELECTION USING MAPREDUCE
DOCUMENT SELECTION USING MAPREDUCE
ijsptm
Big data analysis concepts and references
Big data analysis concepts and references
Information Security Awareness Group
Big Data Mining, Techniques, Handling Technologies and Some Related Issues: A...
Big Data Mining, Techniques, Handling Technologies and Some Related Issues: A...
IJSRD
A STUDY ON GRAPH STORAGE DATABASE OF NOSQL
A STUDY ON GRAPH STORAGE DATABASE OF NOSQL
ijscai
What's hot
(18)
A Comprehensive Study on Big Data Applications and Challenges
A Comprehensive Study on Big Data Applications and Challenges
IRJET- Systematic Review: Progression Study on BIG DATA articles
IRJET- Systematic Review: Progression Study on BIG DATA articles
Seminar Report Vaibhav
Seminar Report Vaibhav
A Review Paper on Big Data and Hadoop for Data Science
A Review Paper on Big Data and Hadoop for Data Science
Big data analytics
Big data analytics
hadoop seminar training report
hadoop seminar training report
Infrastructure Considerations for Analytical Workloads
Infrastructure Considerations for Analytical Workloads
The Big Data Importance – Tools and their Usage
The Big Data Importance – Tools and their Usage
IRJET- Survey of Big Data with Hadoop
IRJET- Survey of Big Data with Hadoop
Big data analysis concepts and references by Cloud Security Alliance
Big data analysis concepts and references by Cloud Security Alliance
IRJET-Unraveling the Data Structures of Big data, the HDFS Architecture and I...
IRJET-Unraveling the Data Structures of Big data, the HDFS Architecture and I...
Hadoop
Hadoop
big data
big data
Bigdata
Bigdata
DOCUMENT SELECTION USING MAPREDUCE
DOCUMENT SELECTION USING MAPREDUCE
Big data analysis concepts and references
Big data analysis concepts and references
Big Data Mining, Techniques, Handling Technologies and Some Related Issues: A...
Big Data Mining, Techniques, Handling Technologies and Some Related Issues: A...
A STUDY ON GRAPH STORAGE DATABASE OF NOSQL
A STUDY ON GRAPH STORAGE DATABASE OF NOSQL
Similar to Big Data Processing with Hadoop : A Review
Big Data Testing Using Hadoop Platform
Big Data Testing Using Hadoop Platform
IRJET Journal
Big Data Summarization : Framework, Challenges and Possible Solutions
Big Data Summarization : Framework, Challenges and Possible Solutions
aciijournal
Big Data Summarization : Framework, Challenges and Possible Solutions
Big Data Summarization : Framework, Challenges and Possible Solutions
aciijournal
BIG DATA SUMMARIZATION: FRAMEWORK, CHALLENGES AND POSSIBLE SOLUTIONS
BIG DATA SUMMARIZATION: FRAMEWORK, CHALLENGES AND POSSIBLE SOLUTIONS
aciijournal
Big Data Summarization : Framework, Challenges and Possible Solutions
Big Data Summarization : Framework, Challenges and Possible Solutions
aciijournal
Big Data with Hadoop – For Data Management, Processing and Storing
Big Data with Hadoop – For Data Management, Processing and Storing
IRJET Journal
IRJET- Big Data-A Review Study with Comparitive Analysis of Hadoop
IRJET- Big Data-A Review Study with Comparitive Analysis of Hadoop
IRJET Journal
Big data - what, why, where, when and how
Big data - what, why, where, when and how
bobosenthil
Social Media Market Trender with Dache Manager Using Hadoop and Visualization...
Social Media Market Trender with Dache Manager Using Hadoop and Visualization...
IRJET Journal
Survey Paper on Big Data and Hadoop
Survey Paper on Big Data and Hadoop
IRJET Journal
Unstructured Datasets Analysis: Thesaurus Model
Unstructured Datasets Analysis: Thesaurus Model
Editor IJCATR
TSE_Pres12.pptx
TSE_Pres12.pptx
ssuseracaaae2
IRJET- Secured Hadoop Environment
IRJET- Secured Hadoop Environment
IRJET Journal
Big Data A Review
Big Data A Review
ijtsrd
SURVEY ON BIG DATA PROCESSING USING HADOOP, MAP REDUCE
SURVEY ON BIG DATA PROCESSING USING HADOOP, MAP REDUCE
AM Publications,India
Unit-1 -2-3- BDA PIET 6 AIDS.pptx
Unit-1 -2-3- BDA PIET 6 AIDS.pptx
YashiBatra1
[IJCT-V3I2P32] Authors: Amarbir Singh, Palwinder Singh
[IJCT-V3I2P32] Authors: Amarbir Singh, Palwinder Singh
IJET - International Journal of Engineering and Techniques
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
IRJET Journal
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
IRJET Journal
U - 2 Emerging.pptx
U - 2 Emerging.pptx
MulukenTamrat2
Similar to Big Data Processing with Hadoop : A Review
(20)
Big Data Testing Using Hadoop Platform
Big Data Testing Using Hadoop Platform
Big Data Summarization : Framework, Challenges and Possible Solutions
Big Data Summarization : Framework, Challenges and Possible Solutions
Big Data Summarization : Framework, Challenges and Possible Solutions
Big Data Summarization : Framework, Challenges and Possible Solutions
BIG DATA SUMMARIZATION: FRAMEWORK, CHALLENGES AND POSSIBLE SOLUTIONS
BIG DATA SUMMARIZATION: FRAMEWORK, CHALLENGES AND POSSIBLE SOLUTIONS
Big Data Summarization : Framework, Challenges and Possible Solutions
Big Data Summarization : Framework, Challenges and Possible Solutions
Big Data with Hadoop – For Data Management, Processing and Storing
Big Data with Hadoop – For Data Management, Processing and Storing
IRJET- Big Data-A Review Study with Comparitive Analysis of Hadoop
IRJET- Big Data-A Review Study with Comparitive Analysis of Hadoop
Big data - what, why, where, when and how
Big data - what, why, where, when and how
Social Media Market Trender with Dache Manager Using Hadoop and Visualization...
Social Media Market Trender with Dache Manager Using Hadoop and Visualization...
Survey Paper on Big Data and Hadoop
Survey Paper on Big Data and Hadoop
Unstructured Datasets Analysis: Thesaurus Model
Unstructured Datasets Analysis: Thesaurus Model
TSE_Pres12.pptx
TSE_Pres12.pptx
IRJET- Secured Hadoop Environment
IRJET- Secured Hadoop Environment
Big Data A Review
Big Data A Review
SURVEY ON BIG DATA PROCESSING USING HADOOP, MAP REDUCE
SURVEY ON BIG DATA PROCESSING USING HADOOP, MAP REDUCE
Unit-1 -2-3- BDA PIET 6 AIDS.pptx
Unit-1 -2-3- BDA PIET 6 AIDS.pptx
[IJCT-V3I2P32] Authors: Amarbir Singh, Palwinder Singh
[IJCT-V3I2P32] Authors: Amarbir Singh, Palwinder Singh
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
U - 2 Emerging.pptx
U - 2 Emerging.pptx
More from IRJET Journal
TUNNELING IN HIMALAYAS WITH NATM METHOD: A SPECIAL REFERENCES TO SUNGAL TUNNE...
TUNNELING IN HIMALAYAS WITH NATM METHOD: A SPECIAL REFERENCES TO SUNGAL TUNNE...
IRJET Journal
STUDY THE EFFECT OF RESPONSE REDUCTION FACTOR ON RC FRAMED STRUCTURE
STUDY THE EFFECT OF RESPONSE REDUCTION FACTOR ON RC FRAMED STRUCTURE
IRJET Journal
A COMPARATIVE ANALYSIS OF RCC ELEMENT OF SLAB WITH STARK STEEL (HYSD STEEL) A...
A COMPARATIVE ANALYSIS OF RCC ELEMENT OF SLAB WITH STARK STEEL (HYSD STEEL) A...
IRJET Journal
Effect of Camber and Angles of Attack on Airfoil Characteristics
Effect of Camber and Angles of Attack on Airfoil Characteristics
IRJET Journal
A Review on the Progress and Challenges of Aluminum-Based Metal Matrix Compos...
A Review on the Progress and Challenges of Aluminum-Based Metal Matrix Compos...
IRJET Journal
Dynamic Urban Transit Optimization: A Graph Neural Network Approach for Real-...
Dynamic Urban Transit Optimization: A Graph Neural Network Approach for Real-...
IRJET Journal
Structural Analysis and Design of Multi-Storey Symmetric and Asymmetric Shape...
Structural Analysis and Design of Multi-Storey Symmetric and Asymmetric Shape...
IRJET Journal
A Review of “Seismic Response of RC Structures Having Plan and Vertical Irreg...
A Review of “Seismic Response of RC Structures Having Plan and Vertical Irreg...
IRJET Journal
A REVIEW ON MACHINE LEARNING IN ADAS
A REVIEW ON MACHINE LEARNING IN ADAS
IRJET Journal
Long Term Trend Analysis of Precipitation and Temperature for Asosa district,...
Long Term Trend Analysis of Precipitation and Temperature for Asosa district,...
IRJET Journal
P.E.B. Framed Structure Design and Analysis Using STAAD Pro
P.E.B. Framed Structure Design and Analysis Using STAAD Pro
IRJET Journal
A Review on Innovative Fiber Integration for Enhanced Reinforcement of Concre...
A Review on Innovative Fiber Integration for Enhanced Reinforcement of Concre...
IRJET Journal
Survey Paper on Cloud-Based Secured Healthcare System
Survey Paper on Cloud-Based Secured Healthcare System
IRJET Journal
Review on studies and research on widening of existing concrete bridges
Review on studies and research on widening of existing concrete bridges
IRJET Journal
React based fullstack edtech web application
React based fullstack edtech web application
IRJET Journal
A Comprehensive Review of Integrating IoT and Blockchain Technologies in the ...
A Comprehensive Review of Integrating IoT and Blockchain Technologies in the ...
IRJET Journal
A REVIEW ON THE PERFORMANCE OF COCONUT FIBRE REINFORCED CONCRETE.
A REVIEW ON THE PERFORMANCE OF COCONUT FIBRE REINFORCED CONCRETE.
IRJET Journal
Optimizing Business Management Process Workflows: The Dynamic Influence of Mi...
Optimizing Business Management Process Workflows: The Dynamic Influence of Mi...
IRJET Journal
Multistoried and Multi Bay Steel Building Frame by using Seismic Design
Multistoried and Multi Bay Steel Building Frame by using Seismic Design
IRJET Journal
Cost Optimization of Construction Using Plastic Waste as a Sustainable Constr...
Cost Optimization of Construction Using Plastic Waste as a Sustainable Constr...
IRJET Journal
More from IRJET Journal
(20)
TUNNELING IN HIMALAYAS WITH NATM METHOD: A SPECIAL REFERENCES TO SUNGAL TUNNE...
TUNNELING IN HIMALAYAS WITH NATM METHOD: A SPECIAL REFERENCES TO SUNGAL TUNNE...
STUDY THE EFFECT OF RESPONSE REDUCTION FACTOR ON RC FRAMED STRUCTURE
STUDY THE EFFECT OF RESPONSE REDUCTION FACTOR ON RC FRAMED STRUCTURE
A COMPARATIVE ANALYSIS OF RCC ELEMENT OF SLAB WITH STARK STEEL (HYSD STEEL) A...
A COMPARATIVE ANALYSIS OF RCC ELEMENT OF SLAB WITH STARK STEEL (HYSD STEEL) A...
Effect of Camber and Angles of Attack on Airfoil Characteristics
Effect of Camber and Angles of Attack on Airfoil Characteristics
A Review on the Progress and Challenges of Aluminum-Based Metal Matrix Compos...
A Review on the Progress and Challenges of Aluminum-Based Metal Matrix Compos...
Dynamic Urban Transit Optimization: A Graph Neural Network Approach for Real-...
Dynamic Urban Transit Optimization: A Graph Neural Network Approach for Real-...
Structural Analysis and Design of Multi-Storey Symmetric and Asymmetric Shape...
Structural Analysis and Design of Multi-Storey Symmetric and Asymmetric Shape...
A Review of “Seismic Response of RC Structures Having Plan and Vertical Irreg...
A Review of “Seismic Response of RC Structures Having Plan and Vertical Irreg...
A REVIEW ON MACHINE LEARNING IN ADAS
A REVIEW ON MACHINE LEARNING IN ADAS
Long Term Trend Analysis of Precipitation and Temperature for Asosa district,...
Long Term Trend Analysis of Precipitation and Temperature for Asosa district,...
P.E.B. Framed Structure Design and Analysis Using STAAD Pro
P.E.B. Framed Structure Design and Analysis Using STAAD Pro
A Review on Innovative Fiber Integration for Enhanced Reinforcement of Concre...
A Review on Innovative Fiber Integration for Enhanced Reinforcement of Concre...
Survey Paper on Cloud-Based Secured Healthcare System
Survey Paper on Cloud-Based Secured Healthcare System
Review on studies and research on widening of existing concrete bridges
Review on studies and research on widening of existing concrete bridges
React based fullstack edtech web application
React based fullstack edtech web application
A Comprehensive Review of Integrating IoT and Blockchain Technologies in the ...
A Comprehensive Review of Integrating IoT and Blockchain Technologies in the ...
A REVIEW ON THE PERFORMANCE OF COCONUT FIBRE REINFORCED CONCRETE.
A REVIEW ON THE PERFORMANCE OF COCONUT FIBRE REINFORCED CONCRETE.
Optimizing Business Management Process Workflows: The Dynamic Influence of Mi...
Optimizing Business Management Process Workflows: The Dynamic Influence of Mi...
Multistoried and Multi Bay Steel Building Frame by using Seismic Design
Multistoried and Multi Bay Steel Building Frame by using Seismic Design
Cost Optimization of Construction Using Plastic Waste as a Sustainable Constr...
Cost Optimization of Construction Using Plastic Waste as a Sustainable Constr...
Recently uploaded
Oxy acetylene welding presentation note.
Oxy acetylene welding presentation note.
eptoze12
Heart Disease Prediction using machine learning.pptx
Heart Disease Prediction using machine learning.pptx
PoojaBan
Churning of Butter, Factors affecting .
Churning of Butter, Factors affecting .
Satyam Kumar
What are the advantages and disadvantages of membrane structures.pptx
What are the advantages and disadvantages of membrane structures.pptx
wendy cai
APPLICATIONS-AC/DC DRIVES-OPERATING CHARACTERISTICS
APPLICATIONS-AC/DC DRIVES-OPERATING CHARACTERISTICS
KurinjimalarL3
Call Us -/9953056974- Call Girls In Vikaspuri-/- Delhi NCR
Call Us -/9953056974- Call Girls In Vikaspuri-/- Delhi NCR
9953056974 Low Rate Call Girls In Saket, Delhi NCR
Call Girls Narol 7397865700 Independent Call Girls
Call Girls Narol 7397865700 Independent Call Girls
ssuser7cb4ff
Exploring_Network_Security_with_JA3_by_Rakesh Seal.pptx
Exploring_Network_Security_with_JA3_by_Rakesh Seal.pptx
null - The Open Security Community
Architect Hassan Khalil Portfolio for 2024
Architect Hassan Khalil Portfolio for 2024
hassan khalil
Current Transformer Drawing and GTP for MSETCL
Current Transformer Drawing and GTP for MSETCL
DeelipZope
Introduction-To-Agricultural-Surveillance-Rover.pptx
Introduction-To-Agricultural-Surveillance-Rover.pptx
k795866
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
Asst.prof M.Gokilavani
Artificial-Intelligence-in-Electronics (K).pptx
Artificial-Intelligence-in-Electronics (K).pptx
britheesh05
power system scada applications and uses
power system scada applications and uses
DevarapalliHaritha
INFLUENCE OF NANOSILICA ON THE PROPERTIES OF CONCRETE
INFLUENCE OF NANOSILICA ON THE PROPERTIES OF CONCRETE
roselinkalist12
complete construction, environmental and economics information of biomass com...
complete construction, environmental and economics information of biomass com...
asadnawaz62
young call girls in Green Park🔝 9953056974 🔝 escort Service
young call girls in Green Park🔝 9953056974 🔝 escort Service
9953056974 Low Rate Call Girls In Saket, Delhi NCR
Sachpazis Costas: Geotechnical Engineering: A student's Perspective Introduction
Sachpazis Costas: Geotechnical Engineering: A student's Perspective Introduction
Dr.Costas Sachpazis
Decoding Kotlin - Your guide to solving the mysterious in Kotlin.pptx
Decoding Kotlin - Your guide to solving the mysterious in Kotlin.pptx
João Esperancinha
main PPT.pptx of girls hostel security using rfid
main PPT.pptx of girls hostel security using rfid
NikhilNagaraju
Recently uploaded
(20)
Oxy acetylene welding presentation note.
Oxy acetylene welding presentation note.
Heart Disease Prediction using machine learning.pptx
Heart Disease Prediction using machine learning.pptx
Churning of Butter, Factors affecting .
Churning of Butter, Factors affecting .
What are the advantages and disadvantages of membrane structures.pptx
What are the advantages and disadvantages of membrane structures.pptx
APPLICATIONS-AC/DC DRIVES-OPERATING CHARACTERISTICS
APPLICATIONS-AC/DC DRIVES-OPERATING CHARACTERISTICS
Call Us -/9953056974- Call Girls In Vikaspuri-/- Delhi NCR
Call Us -/9953056974- Call Girls In Vikaspuri-/- Delhi NCR
Call Girls Narol 7397865700 Independent Call Girls
Call Girls Narol 7397865700 Independent Call Girls
Exploring_Network_Security_with_JA3_by_Rakesh Seal.pptx
Exploring_Network_Security_with_JA3_by_Rakesh Seal.pptx
Architect Hassan Khalil Portfolio for 2024
Architect Hassan Khalil Portfolio for 2024
Current Transformer Drawing and GTP for MSETCL
Current Transformer Drawing and GTP for MSETCL
Introduction-To-Agricultural-Surveillance-Rover.pptx
Introduction-To-Agricultural-Surveillance-Rover.pptx
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
Artificial-Intelligence-in-Electronics (K).pptx
Artificial-Intelligence-in-Electronics (K).pptx
power system scada applications and uses
power system scada applications and uses
INFLUENCE OF NANOSILICA ON THE PROPERTIES OF CONCRETE
INFLUENCE OF NANOSILICA ON THE PROPERTIES OF CONCRETE
complete construction, environmental and economics information of biomass com...
complete construction, environmental and economics information of biomass com...
young call girls in Green Park🔝 9953056974 🔝 escort Service
young call girls in Green Park🔝 9953056974 🔝 escort Service
Sachpazis Costas: Geotechnical Engineering: A student's Perspective Introduction
Sachpazis Costas: Geotechnical Engineering: A student's Perspective Introduction
Decoding Kotlin - Your guide to solving the mysterious in Kotlin.pptx
Decoding Kotlin - Your guide to solving the mysterious in Kotlin.pptx
main PPT.pptx of girls hostel security using rfid
main PPT.pptx of girls hostel security using rfid
Big Data Processing with Hadoop : A Review
1.
International Research Journal
of Engineering and Technology (IRJET) e-ISSN: 2395 -0056 Volume: 04 Issue: 02 | Feb -2017 www.irjet.net p-ISSN: 2395-0072 © 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 448 Big Data Processing with Hadoop : A Review Gayathri Ravichandran Student, Department of Computer Science , M.S Ramaiah Institute of Technology, Bangalore, India ---------------------------------------------------------------------***--------------------------------------------------------------------- Abstract – We live in an era where data is being generated by everything around us. The rate of data generation is so alarming, that it has engendered apressing needtoimplement easy and cost-effectivedatastorageandretrievalmechanisms. Furthermore, big data needs to be analyzed for insights and attribute relationships, which can lead to better decision- making and efficient business strategies. In this paper, we will describe a formal definition of Big Data and look into its industrial applications. Further, we will understand how traditional mechanisms proveinadequatefor dataprocessing due to the sheer volume, velocity and variety of big data. We will then look into the Hadoop Architecture and its underlying functionalities. This will include delineations on the HDFSand MapReduce Framework. We will then review the Hadoop Ecosystem, and explain each component in detail. Key Words: Big Data, Hadoop, MapReduce, Hadoop Components, HDFS 1. INTRODUCTION 1.1 Big Data: Definition Big data is a collection of large datasets- structured, unstructured or semi-structured that is being generated from multiple sources at an alarming rate. Key enablers for the growth of big data are – increasing storage capacities, increasing processing power and availability of data. It is thus important to develop mechanisms for easy storage and retrieval. Some of the fields that come under the umbrella of big data are - stock exchange data ( includes buying and selling decisions), social media data (Facebook andTwitter), power grid data ( contains information about the power consumed by each node in a power station) and search engine data ( Google). Structured data may include relational databases like MySQL. Unstructured data may include text files in .doc, .pdf formats as well as media files. 1.2 Benefits of Big Data Analysis of big data helps in improving business trends, finding innovative solutions, customer profiling and in sentimental analysis. It also helps in identifying the root causes for failures and re-evaluating risk portfolios. In addition, it also personalizes customer and interaction. 1. Valuable Insights Valuable insights can be derived from big datasets by employing proper tools and methodologies. This data includes those stored in the company database, or those obtained from social media and other third party sources. When data is processed and analyzed,onecandrawvaluable relationships between various attributes that can improve the quality of decision making. Statistics and industrial knowledge can be combined to obtain useful insights 2. New Products and Services Analyzing big data helps theorganizationtounderstandhow customers perceive their products and services. This aids in developing new products that are concurrentwithcustomer needs and demands. In addition, it also facilitates re- developing of currently existing products to suit customer requirements. 3. Smart cities Population increase begets demand. To help cities deal with the consequences of rapid expansion, big data is being used for the benefit of the citizens and the environment. For example, the city of Portland, Oregon adopted a mechanism for optimizing traffic signals in response to high congestion. This not only reduced traffic jams in the city, but was also significant in eliminating 157,000 metric tons of carbon dioxide emissions. 4. Risk Analysis Risk is defined as the probability of injury or loss. Risk management is a very crucial process which is often over- looked. Frequent analysis of the data will help mitigate potential risks. Predictive analysis aids the organization to keep up to date with recent technologies, services and products. It also identifies the risks involved, and how they can be mitigated. 5. Miscellaneous Big data also aids Media, Government, Technology,Scientific Research and Healthcare in making crucial decisions and predictions. For example, Google Flu Trends (GFT)provided estimates of influenza activity for more than 25 countries. It made accurate predictions about flu activity. 1.3 Challenges of Big Data 1. Volume Data is being generated at an alarming rate. The sheer volume of data being generated makes the issue of data
2.
International Research Journal
of Engineering and Technology (IRJET) e-ISSN: 2395 -0056 Volume: 04 Issue: 02 | Feb -2017 www.irjet.net p-ISSN: 2395-0072 © 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 449 processing a complicated task. Organizations collect and generate data from a variety of sources and with the help of technologies such as Hadoop, storage and retrieval of data has become easier. 2. Velocity Velocity refers to the rate at which data is being processed. Sometimes, data may arrive at unprecedented speeds, and thus it must be dealt with in a timely manner. It should be processed in such a speed that is compatible for real time applications. 3. Variety Data is being generated from various sources , including social media data, stock exchange data and black box data. Furthermore, the data canassumevariousforms – numerals, text, media files, etc. Thus, big data processing mechanisms must know how to deal with eclectic data. 4. Variability Data flow can be inconsistent which can be challenging to manage. 5. Complexity The relationships between various attributes in a dataset, hierarchies and data linkages add to the complexity of data 2. LIMITATIONS OF TRADITIONAL APPROACH The traditional approach consists of a computertostoreand process big data. Data is stored in a Relational Database like MySQL and Oracle. This approach works well when the volume of data is less. However, when dealing with larger volumes of data, it becomes tedious to process it through a database server. Hence, this calls for a more sophisticated approach. We will now look into Hadoop – its modules, framework and ecosystem. 3. HADOOP Apache Hadoop is an open source software framework for storing and processing large clusters of data. It has extensive processing power and it consists of large networks of computer clusters. Hadoop makes it possible to handle thousands of terabytes of data. Hardware failures are automatically handled by the framework. Apache Hadoop consists of 4 modules: a. Hadoop Distributed File System(HDFS) b. Hadoop MapReduce c. Hadoop YARN d. Hadoop Common This paper will primarily concentrate on the former two modules. 3.1 Hadoop Distributed File System (HDFS) Apache Hadoop uses the HadoopDistributedFileSystem.Itis highly fault tolerant and uses minimal cost hardware. It consists of a cluster of machines, and files are stored across them. It also provides file permissions and authentication, and streaming access to system data. The following figure depictsthegeneralarchitectureofHDFS Figure 1: HDFS Architecture HDFS follows the Master- Slave Architecture. It has the following components. 1. Name node The HDFS consists of a single name node, which acts as the master node. It controls and manages the file system namespace. A file system namespace consists of a hierarchy of files and directories, where users can create, remove or move files based on their privilege. A file is split into one or more blocks and each block is stored in a Data node. HDFS consists of more than one Data Nodes. The roles of the name node are as follows: a. Mapping blocks to their data nodes. b. Managing of file system namespace c. Executing file system operations- opening, closing and renaming of files. 2. Data node The HDFS consists of more than one data node. The data nodes store the file blocks that are mapped onto it by the Name node. The data nodes are responsible for performing read and write operations from file systems as per client
3.
International Research Journal
of Engineering and Technology (IRJET) e-ISSN: 2395 -0056 Volume: 04 Issue: 02 | Feb -2017 www.irjet.net p-ISSN: 2395-0072 © 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 450 request. They also perform block creation and replication. The minimum amount of data that the system can write or read is called a block. This value however is not fixed, and it can be increased. 3.2 Hadoop MapReduce Framework Hadoop uses the MapReduce framework for distributed computing applications to process large amounts of data.Itisadistributedprogrammingmodel based on the Java Programming language. The data processing frameworks are called mappers and reducers. The MapReduce framework is attractive due to its scalability. It consists of two important tasks : Map and Reduce Figure 2: MapReduce Framework 1. Map stage The map function takes in a set of data as the input, and returns a key-value pair as the output. The input may be in the form of a file or directory. The output of the map stage serves as input to the reduce stage. 2. Reduce stage The reduce function will combine the data tuples into a smaller set. The map task always precedes the reduce task. The output of reduce stage is stored in the HDFS. 3.3 Hadoop Ecosystem Figure 3: Hadoop Ecosystem 1. HBase HBASE or Hadoop Database is a NoSQL Database, i.e., it is non-relational. It is built on top of the HDFS System written in Java. It is the underlying technology of social media websites like Facebook. 2. Hive Hive is a structured Query Language. It uses the Hive Query Language (HQL), and it deals with structured data. It runs MapReduce Algorithm as its backend, and it is a data warehousing framework. 3. Pig Pig also deals with structured data, and it uses the Pig Latin Language. It consists of a series of operations applied to input data, and it uses MapReduce in the back-end. It adds a level of abstraction to data processing. 4. Mahout It is an open source Apache MachineLearninglibraryinJava. It has modules for clustering, categorization, collective filtering and mining of frequent patterns.
4.
International Research Journal
of Engineering and Technology (IRJET) e-ISSN: 2395 -0056 Volume: 04 Issue: 02 | Feb -2017 www.irjet.net p-ISSN: 2395-0072 © 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 451 4. CONCLUSION This paper starts off by giving a formaldefinitiontoBig Data. Then, the challenges of handling big data are examined, followed by the limitations of using the traditional big data processing approach. We then delve into the details of Hadoop and its components, and its MapReduce framework. REFERENCES [1] Dean, J. and Ghemawat, S., “MapReduce: a flexible data processing tool” Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989. [2] Varsha B.Bobade, “Survey Paper on Big Data and Hadoop”, IRJET, Volume 3, Issue 1, January 2016 [3] Bijesh Dhyani,Anurag Barthwal, “Big Data Analytics using Hadoop”, International Journal of Computing Applications, Volume 108, No.12, December 2014 [4] Ms. Gurpreet Kaur,Ms.ManpreetKaur, “ReviewPaperon Big Data using Hadoop”, International Journal of Computing Engineering and Technology, Volume 6, Issue 12, Dec 2015, pp. 65-71 [5] Harshwardhan S. Bhosale et al, “Review paper on Big Data using Hadoop”, International Journal of Scientific and Research Publications, Volume 4, Issue 10, October 2014 [6] Poonam S. Patil et al. “Survey Paper on Big Data Processing and Hadoop Components”, International Journal of Science and Research, Volume 3, Issue 10, October 2014. [7] Apache HBase. Available at http://hbase.apache.org [8] Apache Hive. Available at http://hive.apache.org [9] Abhishek S, “Big Data and Hadoop”, White Paper [10] Konstantin Shvachko et.al, “The HadoopDistributedFile System”
Download now