Submit Search
Upload
An Efficient Approach to Manage Small Files in Distributed File Systems
•
0 likes
•
42 views
IRJET Journal
Follow
https://www.irjet.net/archives/V4/i3/IRJET-V4I3601.pdf
Read less
Read more
Engineering
Report
Share
Report
Share
1 of 5
Download now
Download to read offline
Recommended
BFC: High-Performance Distributed Big-File Cloud Storage Based On Key-Value S...
BFC: High-Performance Distributed Big-File Cloud Storage Based On Key-Value S...
dbpublications
Dynamic Metadata Management in Semantic File Systems
Dynamic Metadata Management in Semantic File Systems
IJERA Editor
Study on potential capabilities of a nodb system
Study on potential capabilities of a nodb system
ijitjournal
Methodology for Optimizing Storage on Cloud Using Authorized De-Duplication –...
Methodology for Optimizing Storage on Cloud Using Authorized De-Duplication –...
IRJET Journal
Improving availability and reducing redundancy using deduplication of cloud s...
Improving availability and reducing redundancy using deduplication of cloud s...
dhanarajp
1771 1775
1771 1775
Editor IJARCET
Google File System
Google File System
vivatechijri
Flevy.com - Feasibility Study for an eMail Archiving solution
Flevy.com - Feasibility Study for an eMail Archiving solution
David Tracy
Recommended
BFC: High-Performance Distributed Big-File Cloud Storage Based On Key-Value S...
BFC: High-Performance Distributed Big-File Cloud Storage Based On Key-Value S...
dbpublications
Dynamic Metadata Management in Semantic File Systems
Dynamic Metadata Management in Semantic File Systems
IJERA Editor
Study on potential capabilities of a nodb system
Study on potential capabilities of a nodb system
ijitjournal
Methodology for Optimizing Storage on Cloud Using Authorized De-Duplication –...
Methodology for Optimizing Storage on Cloud Using Authorized De-Duplication –...
IRJET Journal
Improving availability and reducing redundancy using deduplication of cloud s...
Improving availability and reducing redundancy using deduplication of cloud s...
dhanarajp
1771 1775
1771 1775
Editor IJARCET
Google File System
Google File System
vivatechijri
Flevy.com - Feasibility Study for an eMail Archiving solution
Flevy.com - Feasibility Study for an eMail Archiving solution
David Tracy
ANALYSIS OF ATTACK TECHNIQUES ON CLOUD BASED DATA DEDUPLICATION TECHNIQUES
ANALYSIS OF ATTACK TECHNIQUES ON CLOUD BASED DATA DEDUPLICATION TECHNIQUES
neirew J
ANALYSIS STUDY ON CACHING AND REPLICA PLACEMENT ALGORITHM FOR CONTENT DISTRIB...
ANALYSIS STUDY ON CACHING AND REPLICA PLACEMENT ALGORITHM FOR CONTENT DISTRIB...
ijp2p
Dr.Hadoop- an infinite scalable metadata management for Hadoop-How the baby e...
Dr.Hadoop- an infinite scalable metadata management for Hadoop-How the baby e...
Dipayan Dev
IRJET- Cross User Bigdata Deduplication
IRJET- Cross User Bigdata Deduplication
IRJET Journal
X.500 More Than a Global Directory
X.500 More Than a Global Directory
lurdhu agnes
Design of file system architecture with cluster
Design of file system architecture with cluster
eSAT Publishing House
EVALUATE DATABASE COMPRESSION PERFORMANCE AND PARALLEL BACKUP
EVALUATE DATABASE COMPRESSION PERFORMANCE AND PARALLEL BACKUP
ijdms
Authorized Duplicate Check Scheme
Authorized Duplicate Check Scheme
IRJET Journal
A New Architecture for Group Replication in Data Grid
A New Architecture for Group Replication in Data Grid
Editor IJCATR
iaetsd Controlling data deuplication in cloud storage
iaetsd Controlling data deuplication in cloud storage
Iaetsd Iaetsd
Data mining
Data mining
sweetysweety8
Approved TPA along with Integrity Verification in Cloud
Approved TPA along with Integrity Verification in Cloud
Editor IJCATR
Secure Distributed Deduplication Systems with Improved Reliability
Secure Distributed Deduplication Systems with Improved Reliability
1crore projects
SiDe Enabled Reliable Replica Optimization
SiDe Enabled Reliable Replica Optimization
IJCSIS Research Publications
Secure distributed deduplication systems
Secure distributed deduplication systems
Pvrtechnologies Nellore
IRJET- Cloud based Deduplication using Middleware Approach
IRJET- Cloud based Deduplication using Middleware Approach
IRJET Journal
An Enhanced Cloud Backed Frugal File System
An Enhanced Cloud Backed Frugal File System
IRJET Journal
IRJET- An Integrity Auditing &Data Dedupe withEffective Bandwidth in Cloud St...
IRJET- An Integrity Auditing &Data Dedupe withEffective Bandwidth in Cloud St...
IRJET Journal
IRJET- A Study of Comparatively Analysis for HDFS and Google File System ...
IRJET- A Study of Comparatively Analysis for HDFS and Google File System ...
IRJET Journal
Fota Delta Size Reduction Using FIle Similarity Algorithms
Fota Delta Size Reduction Using FIle Similarity Algorithms
Shivansh Gaur
A Strategy for Improving the Performance of Small Files in Openstack Swift
A Strategy for Improving the Performance of Small Files in Openstack Swift
Editor IJCATR
IRJET- A Novel Approach to Process Small HDFS Files with Apache Spark
IRJET- A Novel Approach to Process Small HDFS Files with Apache Spark
IRJET Journal
More Related Content
What's hot
ANALYSIS OF ATTACK TECHNIQUES ON CLOUD BASED DATA DEDUPLICATION TECHNIQUES
ANALYSIS OF ATTACK TECHNIQUES ON CLOUD BASED DATA DEDUPLICATION TECHNIQUES
neirew J
ANALYSIS STUDY ON CACHING AND REPLICA PLACEMENT ALGORITHM FOR CONTENT DISTRIB...
ANALYSIS STUDY ON CACHING AND REPLICA PLACEMENT ALGORITHM FOR CONTENT DISTRIB...
ijp2p
Dr.Hadoop- an infinite scalable metadata management for Hadoop-How the baby e...
Dr.Hadoop- an infinite scalable metadata management for Hadoop-How the baby e...
Dipayan Dev
IRJET- Cross User Bigdata Deduplication
IRJET- Cross User Bigdata Deduplication
IRJET Journal
X.500 More Than a Global Directory
X.500 More Than a Global Directory
lurdhu agnes
Design of file system architecture with cluster
Design of file system architecture with cluster
eSAT Publishing House
EVALUATE DATABASE COMPRESSION PERFORMANCE AND PARALLEL BACKUP
EVALUATE DATABASE COMPRESSION PERFORMANCE AND PARALLEL BACKUP
ijdms
Authorized Duplicate Check Scheme
Authorized Duplicate Check Scheme
IRJET Journal
A New Architecture for Group Replication in Data Grid
A New Architecture for Group Replication in Data Grid
Editor IJCATR
iaetsd Controlling data deuplication in cloud storage
iaetsd Controlling data deuplication in cloud storage
Iaetsd Iaetsd
Data mining
Data mining
sweetysweety8
Approved TPA along with Integrity Verification in Cloud
Approved TPA along with Integrity Verification in Cloud
Editor IJCATR
Secure Distributed Deduplication Systems with Improved Reliability
Secure Distributed Deduplication Systems with Improved Reliability
1crore projects
SiDe Enabled Reliable Replica Optimization
SiDe Enabled Reliable Replica Optimization
IJCSIS Research Publications
Secure distributed deduplication systems
Secure distributed deduplication systems
Pvrtechnologies Nellore
IRJET- Cloud based Deduplication using Middleware Approach
IRJET- Cloud based Deduplication using Middleware Approach
IRJET Journal
An Enhanced Cloud Backed Frugal File System
An Enhanced Cloud Backed Frugal File System
IRJET Journal
What's hot
(17)
ANALYSIS OF ATTACK TECHNIQUES ON CLOUD BASED DATA DEDUPLICATION TECHNIQUES
ANALYSIS OF ATTACK TECHNIQUES ON CLOUD BASED DATA DEDUPLICATION TECHNIQUES
ANALYSIS STUDY ON CACHING AND REPLICA PLACEMENT ALGORITHM FOR CONTENT DISTRIB...
ANALYSIS STUDY ON CACHING AND REPLICA PLACEMENT ALGORITHM FOR CONTENT DISTRIB...
Dr.Hadoop- an infinite scalable metadata management for Hadoop-How the baby e...
Dr.Hadoop- an infinite scalable metadata management for Hadoop-How the baby e...
IRJET- Cross User Bigdata Deduplication
IRJET- Cross User Bigdata Deduplication
X.500 More Than a Global Directory
X.500 More Than a Global Directory
Design of file system architecture with cluster
Design of file system architecture with cluster
EVALUATE DATABASE COMPRESSION PERFORMANCE AND PARALLEL BACKUP
EVALUATE DATABASE COMPRESSION PERFORMANCE AND PARALLEL BACKUP
Authorized Duplicate Check Scheme
Authorized Duplicate Check Scheme
A New Architecture for Group Replication in Data Grid
A New Architecture for Group Replication in Data Grid
iaetsd Controlling data deuplication in cloud storage
iaetsd Controlling data deuplication in cloud storage
Data mining
Data mining
Approved TPA along with Integrity Verification in Cloud
Approved TPA along with Integrity Verification in Cloud
Secure Distributed Deduplication Systems with Improved Reliability
Secure Distributed Deduplication Systems with Improved Reliability
SiDe Enabled Reliable Replica Optimization
SiDe Enabled Reliable Replica Optimization
Secure distributed deduplication systems
Secure distributed deduplication systems
IRJET- Cloud based Deduplication using Middleware Approach
IRJET- Cloud based Deduplication using Middleware Approach
An Enhanced Cloud Backed Frugal File System
An Enhanced Cloud Backed Frugal File System
Similar to An Efficient Approach to Manage Small Files in Distributed File Systems
IRJET- An Integrity Auditing &Data Dedupe withEffective Bandwidth in Cloud St...
IRJET- An Integrity Auditing &Data Dedupe withEffective Bandwidth in Cloud St...
IRJET Journal
IRJET- A Study of Comparatively Analysis for HDFS and Google File System ...
IRJET- A Study of Comparatively Analysis for HDFS and Google File System ...
IRJET Journal
Fota Delta Size Reduction Using FIle Similarity Algorithms
Fota Delta Size Reduction Using FIle Similarity Algorithms
Shivansh Gaur
A Strategy for Improving the Performance of Small Files in Openstack Swift
A Strategy for Improving the Performance of Small Files in Openstack Swift
Editor IJCATR
IRJET- A Novel Approach to Process Small HDFS Files with Apache Spark
IRJET- A Novel Approach to Process Small HDFS Files with Apache Spark
IRJET Journal
A Survey on Different File Handling Mechanisms in HDFS
A Survey on Different File Handling Mechanisms in HDFS
IRJET Journal
CollaborativeDatasetBuilding
CollaborativeDatasetBuilding
Armaan Bindra
File systems versus a dbms
File systems versus a dbms
RituBhargava7
191
191
vivatechijri
H017144148
H017144148
IOSR Journals
Comparative Analysis, Security Aspects & Optimization of Workload in Gfs Base...
Comparative Analysis, Security Aspects & Optimization of Workload in Gfs Base...
IOSR Journals
Data Deduplication: Venti and its improvements
Data Deduplication: Venti and its improvements
Umair Amjad
E018142329
E018142329
IOSR Journals
1771 1775
1771 1775
Editor IJARCET
A cloud environment for backup and data storage
A cloud environment for backup and data storage
IGEEKS TECHNOLOGIES
Cloud Storage System like Dropbox
Cloud Storage System like Dropbox
IRJET Journal
Performance Improvement of Heterogeneous Hadoop Cluster using Ranking Algorithm
Performance Improvement of Heterogeneous Hadoop Cluster using Ranking Algorithm
IRJET Journal
A cloud enviroment for backup and data storage
A cloud enviroment for backup and data storage
IGEEKS TECHNOLOGIES
[IJET-V1I6P11] Authors: A.Stenila, M. Kavitha, S.Alonshia
[IJET-V1I6P11] Authors: A.Stenila, M. Kavitha, S.Alonshia
IJET - International Journal of Engineering and Techniques
Dos unit 4
Dos unit 4
JebasheelaSJ
Similar to An Efficient Approach to Manage Small Files in Distributed File Systems
(20)
IRJET- An Integrity Auditing &Data Dedupe withEffective Bandwidth in Cloud St...
IRJET- An Integrity Auditing &Data Dedupe withEffective Bandwidth in Cloud St...
IRJET- A Study of Comparatively Analysis for HDFS and Google File System ...
IRJET- A Study of Comparatively Analysis for HDFS and Google File System ...
Fota Delta Size Reduction Using FIle Similarity Algorithms
Fota Delta Size Reduction Using FIle Similarity Algorithms
A Strategy for Improving the Performance of Small Files in Openstack Swift
A Strategy for Improving the Performance of Small Files in Openstack Swift
IRJET- A Novel Approach to Process Small HDFS Files with Apache Spark
IRJET- A Novel Approach to Process Small HDFS Files with Apache Spark
A Survey on Different File Handling Mechanisms in HDFS
A Survey on Different File Handling Mechanisms in HDFS
CollaborativeDatasetBuilding
CollaborativeDatasetBuilding
File systems versus a dbms
File systems versus a dbms
191
191
H017144148
H017144148
Comparative Analysis, Security Aspects & Optimization of Workload in Gfs Base...
Comparative Analysis, Security Aspects & Optimization of Workload in Gfs Base...
Data Deduplication: Venti and its improvements
Data Deduplication: Venti and its improvements
E018142329
E018142329
1771 1775
1771 1775
A cloud environment for backup and data storage
A cloud environment for backup and data storage
Cloud Storage System like Dropbox
Cloud Storage System like Dropbox
Performance Improvement of Heterogeneous Hadoop Cluster using Ranking Algorithm
Performance Improvement of Heterogeneous Hadoop Cluster using Ranking Algorithm
A cloud enviroment for backup and data storage
A cloud enviroment for backup and data storage
[IJET-V1I6P11] Authors: A.Stenila, M. Kavitha, S.Alonshia
[IJET-V1I6P11] Authors: A.Stenila, M. Kavitha, S.Alonshia
Dos unit 4
Dos unit 4
More from IRJET Journal
TUNNELING IN HIMALAYAS WITH NATM METHOD: A SPECIAL REFERENCES TO SUNGAL TUNNE...
TUNNELING IN HIMALAYAS WITH NATM METHOD: A SPECIAL REFERENCES TO SUNGAL TUNNE...
IRJET Journal
STUDY THE EFFECT OF RESPONSE REDUCTION FACTOR ON RC FRAMED STRUCTURE
STUDY THE EFFECT OF RESPONSE REDUCTION FACTOR ON RC FRAMED STRUCTURE
IRJET Journal
A COMPARATIVE ANALYSIS OF RCC ELEMENT OF SLAB WITH STARK STEEL (HYSD STEEL) A...
A COMPARATIVE ANALYSIS OF RCC ELEMENT OF SLAB WITH STARK STEEL (HYSD STEEL) A...
IRJET Journal
Effect of Camber and Angles of Attack on Airfoil Characteristics
Effect of Camber and Angles of Attack on Airfoil Characteristics
IRJET Journal
A Review on the Progress and Challenges of Aluminum-Based Metal Matrix Compos...
A Review on the Progress and Challenges of Aluminum-Based Metal Matrix Compos...
IRJET Journal
Dynamic Urban Transit Optimization: A Graph Neural Network Approach for Real-...
Dynamic Urban Transit Optimization: A Graph Neural Network Approach for Real-...
IRJET Journal
Structural Analysis and Design of Multi-Storey Symmetric and Asymmetric Shape...
Structural Analysis and Design of Multi-Storey Symmetric and Asymmetric Shape...
IRJET Journal
A Review of “Seismic Response of RC Structures Having Plan and Vertical Irreg...
A Review of “Seismic Response of RC Structures Having Plan and Vertical Irreg...
IRJET Journal
A REVIEW ON MACHINE LEARNING IN ADAS
A REVIEW ON MACHINE LEARNING IN ADAS
IRJET Journal
Long Term Trend Analysis of Precipitation and Temperature for Asosa district,...
Long Term Trend Analysis of Precipitation and Temperature for Asosa district,...
IRJET Journal
P.E.B. Framed Structure Design and Analysis Using STAAD Pro
P.E.B. Framed Structure Design and Analysis Using STAAD Pro
IRJET Journal
A Review on Innovative Fiber Integration for Enhanced Reinforcement of Concre...
A Review on Innovative Fiber Integration for Enhanced Reinforcement of Concre...
IRJET Journal
Survey Paper on Cloud-Based Secured Healthcare System
Survey Paper on Cloud-Based Secured Healthcare System
IRJET Journal
Review on studies and research on widening of existing concrete bridges
Review on studies and research on widening of existing concrete bridges
IRJET Journal
React based fullstack edtech web application
React based fullstack edtech web application
IRJET Journal
A Comprehensive Review of Integrating IoT and Blockchain Technologies in the ...
A Comprehensive Review of Integrating IoT and Blockchain Technologies in the ...
IRJET Journal
A REVIEW ON THE PERFORMANCE OF COCONUT FIBRE REINFORCED CONCRETE.
A REVIEW ON THE PERFORMANCE OF COCONUT FIBRE REINFORCED CONCRETE.
IRJET Journal
Optimizing Business Management Process Workflows: The Dynamic Influence of Mi...
Optimizing Business Management Process Workflows: The Dynamic Influence of Mi...
IRJET Journal
Multistoried and Multi Bay Steel Building Frame by using Seismic Design
Multistoried and Multi Bay Steel Building Frame by using Seismic Design
IRJET Journal
Cost Optimization of Construction Using Plastic Waste as a Sustainable Constr...
Cost Optimization of Construction Using Plastic Waste as a Sustainable Constr...
IRJET Journal
More from IRJET Journal
(20)
TUNNELING IN HIMALAYAS WITH NATM METHOD: A SPECIAL REFERENCES TO SUNGAL TUNNE...
TUNNELING IN HIMALAYAS WITH NATM METHOD: A SPECIAL REFERENCES TO SUNGAL TUNNE...
STUDY THE EFFECT OF RESPONSE REDUCTION FACTOR ON RC FRAMED STRUCTURE
STUDY THE EFFECT OF RESPONSE REDUCTION FACTOR ON RC FRAMED STRUCTURE
A COMPARATIVE ANALYSIS OF RCC ELEMENT OF SLAB WITH STARK STEEL (HYSD STEEL) A...
A COMPARATIVE ANALYSIS OF RCC ELEMENT OF SLAB WITH STARK STEEL (HYSD STEEL) A...
Effect of Camber and Angles of Attack on Airfoil Characteristics
Effect of Camber and Angles of Attack on Airfoil Characteristics
A Review on the Progress and Challenges of Aluminum-Based Metal Matrix Compos...
A Review on the Progress and Challenges of Aluminum-Based Metal Matrix Compos...
Dynamic Urban Transit Optimization: A Graph Neural Network Approach for Real-...
Dynamic Urban Transit Optimization: A Graph Neural Network Approach for Real-...
Structural Analysis and Design of Multi-Storey Symmetric and Asymmetric Shape...
Structural Analysis and Design of Multi-Storey Symmetric and Asymmetric Shape...
A Review of “Seismic Response of RC Structures Having Plan and Vertical Irreg...
A Review of “Seismic Response of RC Structures Having Plan and Vertical Irreg...
A REVIEW ON MACHINE LEARNING IN ADAS
A REVIEW ON MACHINE LEARNING IN ADAS
Long Term Trend Analysis of Precipitation and Temperature for Asosa district,...
Long Term Trend Analysis of Precipitation and Temperature for Asosa district,...
P.E.B. Framed Structure Design and Analysis Using STAAD Pro
P.E.B. Framed Structure Design and Analysis Using STAAD Pro
A Review on Innovative Fiber Integration for Enhanced Reinforcement of Concre...
A Review on Innovative Fiber Integration for Enhanced Reinforcement of Concre...
Survey Paper on Cloud-Based Secured Healthcare System
Survey Paper on Cloud-Based Secured Healthcare System
Review on studies and research on widening of existing concrete bridges
Review on studies and research on widening of existing concrete bridges
React based fullstack edtech web application
React based fullstack edtech web application
A Comprehensive Review of Integrating IoT and Blockchain Technologies in the ...
A Comprehensive Review of Integrating IoT and Blockchain Technologies in the ...
A REVIEW ON THE PERFORMANCE OF COCONUT FIBRE REINFORCED CONCRETE.
A REVIEW ON THE PERFORMANCE OF COCONUT FIBRE REINFORCED CONCRETE.
Optimizing Business Management Process Workflows: The Dynamic Influence of Mi...
Optimizing Business Management Process Workflows: The Dynamic Influence of Mi...
Multistoried and Multi Bay Steel Building Frame by using Seismic Design
Multistoried and Multi Bay Steel Building Frame by using Seismic Design
Cost Optimization of Construction Using Plastic Waste as a Sustainable Constr...
Cost Optimization of Construction Using Plastic Waste as a Sustainable Constr...
Recently uploaded
Churning of Butter, Factors affecting .
Churning of Butter, Factors affecting .
Satyam Kumar
Concrete Mix Design - IS 10262-2019 - .pptx
Concrete Mix Design - IS 10262-2019 - .pptx
KartikeyaDwivedi3
HARMONY IN THE HUMAN BEING - Unit-II UHV-2
HARMONY IN THE HUMAN BEING - Unit-II UHV-2
RajaP95
HARMONY IN THE NATURE AND EXISTENCE - Unit-IV
HARMONY IN THE NATURE AND EXISTENCE - Unit-IV
RajaP95
What are the advantages and disadvantages of membrane structures.pptx
What are the advantages and disadvantages of membrane structures.pptx
wendy cai
Call Us -/9953056974- Call Girls In Vikaspuri-/- Delhi NCR
Call Us -/9953056974- Call Girls In Vikaspuri-/- Delhi NCR
9953056974 Low Rate Call Girls In Saket, Delhi NCR
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
Asst.prof M.Gokilavani
Past, Present and Future of Generative AI
Past, Present and Future of Generative AI
abhishek36461
POWER SYSTEMS-1 Complete notes examples
POWER SYSTEMS-1 Complete notes examples
Dr. Gudipudi Nageswara Rao
main PPT.pptx of girls hostel security using rfid
main PPT.pptx of girls hostel security using rfid
NikhilNagaraju
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130
Suhani Kapoor
Introduction-To-Agricultural-Surveillance-Rover.pptx
Introduction-To-Agricultural-Surveillance-Rover.pptx
k795866
Gfe Mayur Vihar Call Girls Service WhatsApp -> 9999965857 Available 24x7 ^ De...
Gfe Mayur Vihar Call Girls Service WhatsApp -> 9999965857 Available 24x7 ^ De...
srsj9000
chaitra-1.pptx fake news detection using machine learning
chaitra-1.pptx fake news detection using machine learning
misbanausheenparvam
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
Asst.prof M.Gokilavani
Artificial-Intelligence-in-Electronics (K).pptx
Artificial-Intelligence-in-Electronics (K).pptx
britheesh05
SPICE PARK APR2024 ( 6,793 SPICE Models )
SPICE PARK APR2024 ( 6,793 SPICE Models )
Tsuyoshi Horigome
Biology for Computer Engineers Course Handout.pptx
Biology for Computer Engineers Course Handout.pptx
DeepakSakkari2
complete construction, environmental and economics information of biomass com...
complete construction, environmental and economics information of biomass com...
asadnawaz62
young call girls in Green Park🔝 9953056974 🔝 escort Service
young call girls in Green Park🔝 9953056974 🔝 escort Service
9953056974 Low Rate Call Girls In Saket, Delhi NCR
Recently uploaded
(20)
Churning of Butter, Factors affecting .
Churning of Butter, Factors affecting .
Concrete Mix Design - IS 10262-2019 - .pptx
Concrete Mix Design - IS 10262-2019 - .pptx
HARMONY IN THE HUMAN BEING - Unit-II UHV-2
HARMONY IN THE HUMAN BEING - Unit-II UHV-2
HARMONY IN THE NATURE AND EXISTENCE - Unit-IV
HARMONY IN THE NATURE AND EXISTENCE - Unit-IV
What are the advantages and disadvantages of membrane structures.pptx
What are the advantages and disadvantages of membrane structures.pptx
Call Us -/9953056974- Call Girls In Vikaspuri-/- Delhi NCR
Call Us -/9953056974- Call Girls In Vikaspuri-/- Delhi NCR
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
Past, Present and Future of Generative AI
Past, Present and Future of Generative AI
POWER SYSTEMS-1 Complete notes examples
POWER SYSTEMS-1 Complete notes examples
main PPT.pptx of girls hostel security using rfid
main PPT.pptx of girls hostel security using rfid
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130
Introduction-To-Agricultural-Surveillance-Rover.pptx
Introduction-To-Agricultural-Surveillance-Rover.pptx
Gfe Mayur Vihar Call Girls Service WhatsApp -> 9999965857 Available 24x7 ^ De...
Gfe Mayur Vihar Call Girls Service WhatsApp -> 9999965857 Available 24x7 ^ De...
chaitra-1.pptx fake news detection using machine learning
chaitra-1.pptx fake news detection using machine learning
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
Artificial-Intelligence-in-Electronics (K).pptx
Artificial-Intelligence-in-Electronics (K).pptx
SPICE PARK APR2024 ( 6,793 SPICE Models )
SPICE PARK APR2024 ( 6,793 SPICE Models )
Biology for Computer Engineers Course Handout.pptx
Biology for Computer Engineers Course Handout.pptx
complete construction, environmental and economics information of biomass com...
complete construction, environmental and economics information of biomass com...
young call girls in Green Park🔝 9953056974 🔝 escort Service
young call girls in Green Park🔝 9953056974 🔝 escort Service
An Efficient Approach to Manage Small Files in Distributed File Systems
1.
International Research Journal
of Engineering and Technology (IRJET) e-ISSN: 2395 -0056 Volume: 04 Issue: 03 | Mar -2017 www.irjet.net p-ISSN: 2395-0072 © 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 2311 An Efficient Approach to Manage Small Files in Distributed File Systems Aakash Patil, Ganesh Sagare, Kunal Saraf (BE in Computer Engineering, Sandip Institute of Engineering and Management, Nashik.) Prof. Sujit. A. Ahirrao Assistant Professor, Department of Computer Engineering ---------------------------------------------------------------------***--------------------------------------------------------------------- Abstract: Nowadays, to manage excessive number of small files is became a challenge in Distributed File System. Currently, the combined block storage technique is used to store the files this technique is used in existing system such as Extfs and Xfs. This technique is liable to inefficiency when accessing files randomly. We present the proposed system to manage small files which is based on simple metadata and storage architecture. Our system focuses on replacing the existing system drawbacks in Data servers that used to store excessive number of small files and retrieval of files in a better way. We designed new metadata structure which will decrease the size of original metadata that will help to increase the speed of file accessing. Keywords: Information System , Information Storage And Retrieval. Indexing Methods, Content Analysis Computing Methodologies, Documents Processing, Various types of files. 1.Introduction: We know that Metadata consist of data related data that means in file system metadata contains the information which is helpful to search the files in file systems for eg. Address of the file, size of the file, modified date of updated information etc. Nowadays, Everyone is using social networking and e-commerce websites for communication and purchasing purpose by considering the usage of the websites which required to store the data which is small in size then there is the difficulty in storing and retrieving the files which are smaller in size and the number of this files are bulk because of many users are frequently uploading or modifying the data in the storage space. So, the managing this small files is became a problem in distributed file system becauseofthe metadatageneratedbythefilesisbiggerinsize. In some cases the files are rarely modified or updated and the size of this file is in between 1kb’s to 10kb’ssuchaspictures,textetc.uploaded on social networking and e-commerce websites in daily or timely basis. Distributed file system is based on storing and accessing filesbased on simple client-server architecture. In distributed file system all data is copied and placed on the differentdataserversandtheinformation about the data is stored in which are then connected in network. A client or user searches the file using metadata server other than the using the actual locationofthat filethesameprocessisusedinexisting system, client request the file which is stored in a distributed file system by using two phases. 1.Client sends the query containing about the data needed to the metadata server and gets the IP address of data server which stores the target file. 2.In next phase connection between data server and user is established and granted for fetching the data file. Why we are shrinking the size of metadata ? In our proposed system the main reason behind shrinking the size of metadata is, in DFS when we are storing the file,thesizeofitsmetadata is big in size because of it contains every attributes as discussed earlier. Because of these the accessing speed of a particular file takes more time. In our system the metadata will contain only two things that are size of the file and physical address of that file so that accessing speed can be increased.
2.
International Research Journal
of Engineering and Technology (IRJET) e-ISSN: 2395 -0056 Volume: 04 Issue: 03 | Mar -2017 www.irjet.net p-ISSN: 2395-0072 © 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 2312 2.System Architecture: Fig 1. System Architecture The system architecture defines the flow of accessing the files from the Distributed File System. Clients are the actual users who query the required file to obtain the contents of data. Metadata server contains the metadata (contains file attributes) of the files. Access File is the file which is requested by the client. The component determine hardware configuration detects the system hardware configuration of the client side system that will decide the client can access or cant access the data from the distributed file system. Classifiers classifies the data types in three ways: DBS(Divided Block Storage) CBS(Combined Block Storage) NDS(NoSQL Database System) In existing system the classifiers are used to locate the files in three different locations as shown in Fig 1. DBS contains the large files, CBS contains the small files and NDS contains the byte level files so because of the three approachthe timecomplexityisincreasedsoinproposed system we are combining these three techniques in a single storage architecture that will help to increase the performance of the accessing time and reduces the time complexity. 3.Literature Survey: Granrt Mackey,Saba Sehrish,Jung Wanvg (Granrt Mackey, 2009): In thispaperitisgivenabouttoimprovemetadatamanagementforsmall files in HDFS. This scheme is based on the assumption that each client is assigned quota in file system for the SPACE as well AS NUMBER OF FILES. the compression method "harballing" provides by hadoop is used. Qinqin He,Zhanhuai Li,Bo Wang,Huifeng Wang,Jian Sun: (Qinqin He, 2011) In this paper the scientist has given about how to enhance system's performance and how to optimize system under the different configuration the future work. Randolph Y Wang,Thomas E Anderson: (Randolph Y Wang, 1993) In 1993 generation of file system an inadequate in facing challenges of wide area networks and massive storage. XFS is a prototype file system developed to explore the issues brought about by these technology advances. It organizes hosts into a hierarchical structure so, locally within the cluster of workstation can be better exploited. XFS achieve better performance and ability then current generation network file system runs in wide area.. S. Anjanadevi,D. Vijaykumar. Dr. K. G.Shrinivasan: (S. Anjanadevi, 2014) Cloud computing is an emerging computing model wherein the tasks are associated to software, combination of connection and service accessed over network. Xian Tao, Liang Alei (Xian Tao, 2014): small file access management based on GlusterFS is a strategy to optimize small files reading and writing performance on traditional distributed file system. Tao Wang, Shilong Yao, Lian Xiong, Xin gu (Tao Wang, 2015): HDFS,DFS are adopted to support cloud storage and are designed for optimizing large file access but unfortunately the problem of massive small files is neglected and seriously restricts the performanceofDFS. To improve and even solve the small files problem in this research user task access is defined. The co-relation among the access task, Client 1 Client 2 Client 3 Metadata Server Acces s File Determine H/W Configurati on Classifier DBS CBS NDS
3.
International Research Journal
of Engineering and Technology (IRJET) e-ISSN: 2395 -0056 Volume: 04 Issue: 03 | Mar -2017 www.irjet.net p-ISSN: 2395-0072 © 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 2313 application and access fie are constructed by improving PLSA and research object is transformed from file level to task level Songling Fu, Liagang He, Chenlin Huang and Keni Li: (Songling Fu, 2015)Theprocessingofmassivenumberofsmallfilesischallengeinthe design of distributed file system currently the block-storage is used it causes inefficiency when accessing small files. iflatLFS is used to manage small files which are based on metadata scheme and flat storage architecture. 4.Mathematical Model: Figure 2 shows the Mathematical model Where, Let the system is decided by, S= {D,CS,AF,MS,C} D: Data (Text,Image,Video) CS: Client Search: Request for required data such as text, images, video audio Retrieve the data from server. MS: Metadata Server : Queries locally for id of data block IP of all of data server Retrieve id of data block IP address of data server to client. AF: Access File: Files which are requested by client such as text, multimedia files C: Classifier: Classifies the file into different data blocks (Combined block, divided block, No SQL block) DB: Database: Contains different type of data blocks (Combined block, divided block, No SQL block) Fig 2. Mathematical Model 5.Methodologies/Algorithm: Reading local metadata to retrieve the logical address of the target file data in the corresponding data block file. This phase includes three steps: Phase1: 1. T1:ReadIFInode: Reading the inode of index file. 2. T2:ReadIFData: Reading index data from the index file. 3. TQueryLA: Querying the corresponding index item from the index data to get the logical address of the file. Phase 2: Reading file data. This phase includes 4 steps: T3: ReadDBFInode:Reading the inode of the data block file.
4.
International Research Journal
of Engineering and Technology (IRJET) e-ISSN: 2395 -0056 Volume: 04 Issue: 03 | Mar -2017 www.irjet.net p-ISSN: 2395-0072 © 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 2314 T4: ReadAddressBlock: Reading the corresponding address block from the disk if the logical address is beyond the size of 12 disk blocks. TQueryPA: Querying the physical address of the target file data from the inode or the address block. T5:AccessData: Accessing file data using a disk operation. 6.Performance Evaluation: Firstly, we evaluate the performance of our implementation in a typical DFS environment on a generic system, which has a 2.33 GHz Intel processor with four cores and 4 MB L2 cache, 4 GB of physical memory and a 1 TB SATA disk. Since the objective of the experiments is to evaluate how well the implementation works the data storage layout in the data servers and the accessing patterns to these stored data were generated in the experiments to DFS context. It can be seen from Figure that implemented system delivers the higher performance than the ordinaryandgenerallyusedfileaccessing that is used in sending and receiving the files from distributed file system. Figure also show that the performance of our implementation based DFS is better than that of traditional DFS in all ratios. In the experiments, the performance of traditional DFS. This result suggests that implemented project can always deliverbetterperformancethan traditional DFS. Fig 3. Comparison of performance Nowdays, there are three types of large-scale distributeddatastoragesystems:thedivided-block-storageDFSes,thecombined-block-storage DFSes and the NoSQL database systems. The divided-block-storage DFSes, such as GFS and HDFS,areusuallyusedtostorebigfiles.Butthese DFSes cannot deliver the ideal performance when handling small files. The main aim of designing combined-block-storage DFSes,istosolve the problem of accessing massive numbers of small files efficiently. Furthermore, the NoSQL database systems are mainly designed for storing the data of tiny size. From the Figure 3 our system can improve the performance of accessing massive numbers of small files with the KB-level size in the combined block storage DFSes. For the files with more than kb level or bigger size the another file storage system such as Divided Block Storage can achieve better performance. 7.Conclusion: When developing efficient distributed file systems, one of the challenges is to optimize the storage and access of massive numbers of small files for Internet based applications. Previous work mainly focuses on reducing the problems in traditional filessystems,whichgeneratetoo much metadata and causes lack of file access performance on data servers. We focus on optimizing the performance of data servers in accessing massive numbers of small files and present a proposed system which directly accesses raw disks and adopts a simple metadata scheme and a flat storage architecture to manage massive numbers of small files. New metadata generated by our system consume only a fraction of total space used by the original metadata based on traditional file systems. In this, each file access needs only one disk operationexcept when updating files, which rarely happens. Thus the performance of data servers and the whole DFS can be improved greatly. This paper finally proposes a hybrid storage system to integrate different storage systems, each of which represents a better solution for different ranges of data sizes.
5.
International Research Journal
of Engineering and Technology (IRJET) e-ISSN: 2395 -0056 Volume: 04 Issue: 03 | Mar -2017 www.irjet.net p-ISSN: 2395-0072 © 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 2315 8. References: [1].Grant Mackey, Saba Sehrish, Jun Wang, (Granrt Mackey, 2009) “Improving metadata managementforsmallfilesinHDFS,”inProc.IEEE Int. Conf. Cluster Computer. Workshops, New Orleans, LA, USA, Sep. 2009, pp. [2].Qinqin He (Qinqin He, 2011) Department of Computer Science Northwestern Polytechnic University, Xi’an China luluhe8848@hotmail.com Research On Cloud Storage environment File System Performance Optimization. [3].Randolph Y. Wang and Thomas E. Anderson { rywang,tea} @cs.berkeley.edu Computer Science Division University of California Berkeley, CA 94720 (Randolph Y Wang, 1993) XFS: A Wide Area Mass Storage File System [4]. S. Anjanadevi, D. Vijayakumar, Dr. K .G. Srinivasagan (S. Anjanadevi, 2014) PG Scholar, Assistant Professor, Professor & Head Department of Computer Science and Engineering – PG National Engineering College (Autonomous), Kovilpatti, India An Efficient Dynamic Indexing andMetadata Based Storage in Cloud Environment [5]. Xie Tao, Liang Ale (Xian Tao, 2014) Small File Access Optimization Based on GlusterFS i School of Software Engineering Shanghai Jiao Tong University Shanghai, China Foxterran@163.com, liangalei@sjtu.edu.cn [6]. Tao Wang, Shihong Yao, Zhengquan Xu*, Lian Xiong, Xin Gu, Xiping Yang State Key Laboratory for Information Engineering in Surveying, Mapping and Remote Sensin Wuhan University Wuhan, China wangtao.mac@gmail.com (Tao Wang, 2015) An effective strategy for improving small file problem in distributed file system. [7]. Songling Fu, Liagang He, Chenlin Huang and Keni Li: (Songling Fu, 2015) The processing of massive number of small files is challenge in the design of distributed file system.
Download now