SlideShare a Scribd company logo
1 of 6
PAGE: A Partition Aware Engine for Parallel Graph
Computation
ABSTRACT:
Graph partition quality affects the overall performance of parallel graph
computation systems. The quality of a graph partition is measured by the balance
factor and edge cut ratio. A balanced graph partition with small edge cut ratio is
generally preferred since it reduces the expensive network communication cost.
However, according to an empirical study on Giraph, the performance over well
partitioned graph might be even two times worse than simple random partitions.
This is because these systems only optimize for the simple partition strategies and
cannot efficiently handle the increasing workload of local message processing
when a high quality graph partition is used. In this paper, we propose a novel
partition aware graph computation engine named PAGE, which equips a new
message processor and a dynamic concurrency control model. The new message
processor concurrently processes local and remote messages in a unified way. The
dynamic model adaptively adjusts the concurrency of the processor based on the
online statistics. The experimental evaluation demonstrates the superiority of
PAGE over the graph partitions with various qualities.
EXISTING SYSTEM:
 A lot of parallel graph computation systems have been introduced, e.g.,
Pregel, Giraph, GPS and Graph-Lab. These systems follow the vertex centric
programming model. The graph algorithms in them are split into several
supersteps by synchronization barriers.
 In a super step, each active vertex simultaneously updates its status based on
the neighbors’ messages from previous superstep, and then sends the new
status as a message to its neighbors. With the limited workers (computing
nodes) in practice, a worker usually stores a subgraph, not a vertex, at local,
and sequentially executes the local active vertices. The computations of
these workers are in parallel.
DISADVANTAGES OF EXISTING SYSTEM:
 Lots of existing parallel graph systems are unaware of such effect of the
underlying partitioned subgraphs, and ignore the increasing workload of
local message processing when the quality of partition scheme is improved.
Therefore, these systems handle the local messages and remote messages
unequally and only optimize the processing of remote messages.
 Though there is a simple extension of centralized message buffer used to
process local and remote incoming messages all together, the existing graph
systems still cannot effectively utilize the benefit of high quality graph
partitions.
PROPOSED SYSTEM:
 In this paper, we present a novel graph computation engine, partition aware
graph computation engine (PAGE). To efficiently support computation tasks
with different partitioning qualities, we develop some unique components in
this new framework.
 First, in PAGE’s worker, communication module is extended with a new
dual concurrent message processor. The message processor concurrently
handles both local and remote incoming messages in a unified way, thus
accelerating the message processing. Furthermore, the concurrency of the
message processoris tunable according to the online statistics of the system.
 Second, a partition aware module is added in each worker to monitor the
partition related characters and adjust the concurrency of the message
processoradaptively to fit the online workload.
 To fulfill the goal of estimating a reasonable concurrency for the dual
concurrent message processor, we introduce the Dynamic Concurrency
Control Model (DCCM). Since the message processing pipeline satisfied the
producer consumer model, several heuristic rules are proposed by
considering the producer-consumer constraints. With the help of DCCM,
PAGE provides sufficient message process units to handle current workload
and each message process unit has similar workload. Finally, PAGE can
adaptively accept various qualities of the integrated graph partition.
ADVANTAGES OF PROPOSED SYSTEM:
 We propose the problem that existing graph computation systems cannot
efficiently exploit the benefit of high quality graph partitions.
 We design a new partition aware graph computation engine, called PAGE. It
can effectively harness the partition information to guide parallel processing
resource allocation, and improve the computation performance.
 We introduce a dual concurrent message processor. The message processor
concurrently processes incoming messages in a unified way and is the
cornerstone in PAGE.
 We present a dynamic concurrency control model. The model estimates
concurrency for dual concurrent message processor by satisfying the
producer consumer constraints. The model always generate proper
configurations for PAGE when the graph applications or underlying graph
partitions change.
 We implement a prototype of PAGE and test it with real-world graphs and
various graph algorithms.
 The results clearly demonstrate that PAGE performs efficiently over various
graph partitioning qualities.
SYSTEM ARCHITECTURE:
SYSTEM REQUIREMENTS:
HARDWARE REQUIREMENTS:
 System : Pentium IV 2.4 GHz.
 Hard Disk : 40 GB.
 Floppy Drive : 1.44 Mb.
 Monitor : 15 VGA Colour.
 Mouse : Logitech.
 Ram : 512 Mb.
SOFTWARE REQUIREMENTS:
 Operating system : Windows XP/7.
 Coding Language : ASP.net, C#.net
 Tool : Visual Studio 2010
 Database : SQL SERVER 2008
REFERENCE:
Yingxia Shao, Bin Cui, Senior Member, IEEE, and Lin Ma, “PAGE: A Partition
Aware Engine for Parallel Graph Computation”, IEEE TRANSACTIONS ON
KNOWLEDGE AND DATA ENGINEERING, VOL. 27, NO. 2, FEBRUARY
2015.

More Related Content

What's hot

Hadoop MapReduce joins
Hadoop MapReduce joinsHadoop MapReduce joins
Hadoop MapReduce joinsShalish VJ
 
Pack prediction based cloud bandwidth and cost reduction system
Pack prediction based cloud bandwidth and cost reduction systemPack prediction based cloud bandwidth and cost reduction system
Pack prediction based cloud bandwidth and cost reduction systemIEEEFINALYEARPROJECTS
 
Repartition join in mapreduce
Repartition join in mapreduceRepartition join in mapreduce
Repartition join in mapreduceUday Vakalapudi
 
Introduction to map reduce s. jency jayastina II MSC COMPUTER SCIENCE BON SEC...
Introduction to map reduce s. jency jayastina II MSC COMPUTER SCIENCE BON SEC...Introduction to map reduce s. jency jayastina II MSC COMPUTER SCIENCE BON SEC...
Introduction to map reduce s. jency jayastina II MSC COMPUTER SCIENCE BON SEC...jencyjayastina
 
Novel Scheduling Algorithms for Efficient Deployment of Map Reduce Applicatio...
Novel Scheduling Algorithms for Efficient Deployment of Map Reduce Applicatio...Novel Scheduling Algorithms for Efficient Deployment of Map Reduce Applicatio...
Novel Scheduling Algorithms for Efficient Deployment of Map Reduce Applicatio...IRJET Journal
 
Hadoop World 2011: Next Generation Apache Hadoop MapReduce - Mohadev Konar, H...
Hadoop World 2011: Next Generation Apache Hadoop MapReduce - Mohadev Konar, H...Hadoop World 2011: Next Generation Apache Hadoop MapReduce - Mohadev Konar, H...
Hadoop World 2011: Next Generation Apache Hadoop MapReduce - Mohadev Konar, H...Cloudera, Inc.
 
Join Algorithms in MapReduce
Join Algorithms in MapReduceJoin Algorithms in MapReduce
Join Algorithms in MapReduceShrihari Rathod
 
Raminder kaur presentation_two
Raminder kaur presentation_twoRaminder kaur presentation_two
Raminder kaur presentation_tworamikaurraminder
 
Pack prediction based cloud bandwidth and cost reduction system
Pack prediction based cloud bandwidth and cost reduction systemPack prediction based cloud bandwidth and cost reduction system
Pack prediction based cloud bandwidth and cost reduction systemJPINFOTECH JAYAPRAKASH
 
Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...
Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...
Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...Cloudera, Inc.
 
Hadoop, mapreduce and yarn networks
Hadoop, mapreduce and yarn networksHadoop, mapreduce and yarn networks
Hadoop, mapreduce and yarn networksHariniA7
 
Introducing MapReduce Programming Framework
Introducing MapReduce Programming FrameworkIntroducing MapReduce Programming Framework
Introducing MapReduce Programming FrameworkSamuel Yee
 
Relational Algebra and MapReduce
Relational Algebra and MapReduceRelational Algebra and MapReduce
Relational Algebra and MapReducePietro Michiardi
 
Sawmill - Integrating R and Large Data Clouds
Sawmill - Integrating R and Large Data CloudsSawmill - Integrating R and Large Data Clouds
Sawmill - Integrating R and Large Data CloudsRobert Grossman
 
Mapreduce script
Mapreduce scriptMapreduce script
Mapreduce scriptHaripritha
 
Map reduce presentation
Map reduce presentationMap reduce presentation
Map reduce presentationAhmad El Tawil
 

What's hot (18)

Hadoop MapReduce joins
Hadoop MapReduce joinsHadoop MapReduce joins
Hadoop MapReduce joins
 
Hadoop Mapreduce joins
Hadoop Mapreduce joinsHadoop Mapreduce joins
Hadoop Mapreduce joins
 
Pack prediction based cloud bandwidth and cost reduction system
Pack prediction based cloud bandwidth and cost reduction systemPack prediction based cloud bandwidth and cost reduction system
Pack prediction based cloud bandwidth and cost reduction system
 
Repartition join in mapreduce
Repartition join in mapreduceRepartition join in mapreduce
Repartition join in mapreduce
 
Introduction to map reduce s. jency jayastina II MSC COMPUTER SCIENCE BON SEC...
Introduction to map reduce s. jency jayastina II MSC COMPUTER SCIENCE BON SEC...Introduction to map reduce s. jency jayastina II MSC COMPUTER SCIENCE BON SEC...
Introduction to map reduce s. jency jayastina II MSC COMPUTER SCIENCE BON SEC...
 
Novel Scheduling Algorithms for Efficient Deployment of Map Reduce Applicatio...
Novel Scheduling Algorithms for Efficient Deployment of Map Reduce Applicatio...Novel Scheduling Algorithms for Efficient Deployment of Map Reduce Applicatio...
Novel Scheduling Algorithms for Efficient Deployment of Map Reduce Applicatio...
 
Hadoop World 2011: Next Generation Apache Hadoop MapReduce - Mohadev Konar, H...
Hadoop World 2011: Next Generation Apache Hadoop MapReduce - Mohadev Konar, H...Hadoop World 2011: Next Generation Apache Hadoop MapReduce - Mohadev Konar, H...
Hadoop World 2011: Next Generation Apache Hadoop MapReduce - Mohadev Konar, H...
 
Join Algorithms in MapReduce
Join Algorithms in MapReduceJoin Algorithms in MapReduce
Join Algorithms in MapReduce
 
Raminder kaur presentation_two
Raminder kaur presentation_twoRaminder kaur presentation_two
Raminder kaur presentation_two
 
Pack prediction based cloud bandwidth and cost reduction system
Pack prediction based cloud bandwidth and cost reduction systemPack prediction based cloud bandwidth and cost reduction system
Pack prediction based cloud bandwidth and cost reduction system
 
Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...
Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...
Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...
 
Hadoop, mapreduce and yarn networks
Hadoop, mapreduce and yarn networksHadoop, mapreduce and yarn networks
Hadoop, mapreduce and yarn networks
 
Introducing MapReduce Programming Framework
Introducing MapReduce Programming FrameworkIntroducing MapReduce Programming Framework
Introducing MapReduce Programming Framework
 
Relational Algebra and MapReduce
Relational Algebra and MapReduceRelational Algebra and MapReduce
Relational Algebra and MapReduce
 
MapReduce in Cloud Computing
MapReduce in Cloud ComputingMapReduce in Cloud Computing
MapReduce in Cloud Computing
 
Sawmill - Integrating R and Large Data Clouds
Sawmill - Integrating R and Large Data CloudsSawmill - Integrating R and Large Data Clouds
Sawmill - Integrating R and Large Data Clouds
 
Mapreduce script
Mapreduce scriptMapreduce script
Mapreduce script
 
Map reduce presentation
Map reduce presentationMap reduce presentation
Map reduce presentation
 

Viewers also liked

Caso integrador diseño de materiales
Caso integrador diseño de materialesCaso integrador diseño de materiales
Caso integrador diseño de materiales27096473
 
Prefixes of number and quality
Prefixes of number and qualityPrefixes of number and quality
Prefixes of number and qualityArgee Abadines
 
Lahan Untuk Tebu Jadi Perkebunan Sawit
Lahan Untuk Tebu Jadi Perkebunan Sawit		Lahan Untuk Tebu Jadi Perkebunan Sawit
Lahan Untuk Tebu Jadi Perkebunan Sawit RNI Holding
 
Reabsorción dentaria
Reabsorción dentariaReabsorción dentaria
Reabsorción dentariadentometric
 
Memoratorium Konsesi Sawit Ciptakan Disiplin
Memoratorium Konsesi Sawit Ciptakan DisiplinMemoratorium Konsesi Sawit Ciptakan Disiplin
Memoratorium Konsesi Sawit Ciptakan DisiplinRNI Holding
 
Importancia de diseñar, implementar y evaluar un Objeto Virtual de Aprendizaj...
Importancia de diseñar, implementar y evaluar un Objeto Virtual de Aprendizaj...Importancia de diseñar, implementar y evaluar un Objeto Virtual de Aprendizaj...
Importancia de diseñar, implementar y evaluar un Objeto Virtual de Aprendizaj...Karen Lunazco
 
Voltage & current division
Voltage & current divisionVoltage & current division
Voltage & current divisionmanu chaturvedi
 
Literature survey in biotechnology
Literature survey in biotechnologyLiterature survey in biotechnology
Literature survey in biotechnologyjyothi ramachandra
 
Software Educativo y Plataforma Educativa
Software Educativo y Plataforma Educativa Software Educativo y Plataforma Educativa
Software Educativo y Plataforma Educativa waxel666
 
Tec enf manejo conservación instrumental quirúrgico - CICAT-SALUD
Tec enf manejo conservación instrumental quirúrgico - CICAT-SALUDTec enf manejo conservación instrumental quirúrgico - CICAT-SALUD
Tec enf manejo conservación instrumental quirúrgico - CICAT-SALUDCICAT SALUD
 

Viewers also liked (17)

09 bio 01
09 bio 0109 bio 01
09 bio 01
 
Caso integrador diseño de materiales
Caso integrador diseño de materialesCaso integrador diseño de materiales
Caso integrador diseño de materiales
 
Prefixes of number and quality
Prefixes of number and qualityPrefixes of number and quality
Prefixes of number and quality
 
Lahan Untuk Tebu Jadi Perkebunan Sawit
Lahan Untuk Tebu Jadi Perkebunan Sawit		Lahan Untuk Tebu Jadi Perkebunan Sawit
Lahan Untuk Tebu Jadi Perkebunan Sawit
 
Redaccion de textos
Redaccion de textosRedaccion de textos
Redaccion de textos
 
Reabsorción dentaria
Reabsorción dentariaReabsorción dentaria
Reabsorción dentaria
 
Memoratorium Konsesi Sawit Ciptakan Disiplin
Memoratorium Konsesi Sawit Ciptakan DisiplinMemoratorium Konsesi Sawit Ciptakan Disiplin
Memoratorium Konsesi Sawit Ciptakan Disiplin
 
Importancia de diseñar, implementar y evaluar un Objeto Virtual de Aprendizaj...
Importancia de diseñar, implementar y evaluar un Objeto Virtual de Aprendizaj...Importancia de diseñar, implementar y evaluar un Objeto Virtual de Aprendizaj...
Importancia de diseñar, implementar y evaluar un Objeto Virtual de Aprendizaj...
 
Voltage & current division
Voltage & current divisionVoltage & current division
Voltage & current division
 
Literature survey in biotechnology
Literature survey in biotechnologyLiterature survey in biotechnology
Literature survey in biotechnology
 
Día de Pi
Día de PiDía de Pi
Día de Pi
 
Modelos
ModelosModelos
Modelos
 
Software Educativo y Plataforma Educativa
Software Educativo y Plataforma Educativa Software Educativo y Plataforma Educativa
Software Educativo y Plataforma Educativa
 
Las curaciones
Las curacionesLas curaciones
Las curaciones
 
Evaluación de implantes
Evaluación de implantesEvaluación de implantes
Evaluación de implantes
 
Tec enf manejo conservación instrumental quirúrgico - CICAT-SALUD
Tec enf manejo conservación instrumental quirúrgico - CICAT-SALUDTec enf manejo conservación instrumental quirúrgico - CICAT-SALUD
Tec enf manejo conservación instrumental quirúrgico - CICAT-SALUD
 
Erotismo na arte aula3
Erotismo na arte aula3Erotismo na arte aula3
Erotismo na arte aula3
 

Similar to Page a partition aware engine

PAGE: A PARTITION AWARE ENGINE FOR PARALLEL GRAPH COMPUTATION
PAGE: A PARTITION AWARE ENGINE FOR PARALLEL GRAPH COMPUTATIONPAGE: A PARTITION AWARE ENGINE FOR PARALLEL GRAPH COMPUTATION
PAGE: A PARTITION AWARE ENGINE FOR PARALLEL GRAPH COMPUTATIONNexgen Technology
 
Page a partition aware engine
Page a partition aware enginePage a partition aware engine
Page a partition aware enginenexgentech15
 
Automatic Compilation Of MATLAB Programs For Synergistic Execution On Heterog...
Automatic Compilation Of MATLAB Programs For Synergistic Execution On Heterog...Automatic Compilation Of MATLAB Programs For Synergistic Execution On Heterog...
Automatic Compilation Of MATLAB Programs For Synergistic Execution On Heterog...Sara Alvarez
 
Page a partition aware engine for parallel graph computation
Page a partition aware engine for parallel graph computationPage a partition aware engine for parallel graph computation
Page a partition aware engine for parallel graph computationPvrtechnologies Nellore
 
NETWORK-AWARE DATA PREFETCHING OPTIMIZATION OF COMPUTATIONS IN A HETEROGENEOU...
NETWORK-AWARE DATA PREFETCHING OPTIMIZATION OF COMPUTATIONS IN A HETEROGENEOU...NETWORK-AWARE DATA PREFETCHING OPTIMIZATION OF COMPUTATIONS IN A HETEROGENEOU...
NETWORK-AWARE DATA PREFETCHING OPTIMIZATION OF COMPUTATIONS IN A HETEROGENEOU...IJCNCJournal
 
ON TRAFFIC-AWARE PARTITION AND AGGREGATION IN MAPREDUCE FOR BIG DATA APPLICAT...
ON TRAFFIC-AWARE PARTITION AND AGGREGATION IN MAPREDUCE FOR BIG DATA APPLICAT...ON TRAFFIC-AWARE PARTITION AND AGGREGATION IN MAPREDUCE FOR BIG DATA APPLICAT...
ON TRAFFIC-AWARE PARTITION AND AGGREGATION IN MAPREDUCE FOR BIG DATA APPLICAT...I3E Technologies
 
GRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEM
GRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEMGRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEM
GRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEMIJCSEA Journal
 
Load rebalancing for distributed file systems in clouds
Load rebalancing for distributed file systems in cloudsLoad rebalancing for distributed file systems in clouds
Load rebalancing for distributed file systems in cloudsJPINFOTECH JAYAPRAKASH
 
JPM1402 An Efficient Parallel Approach for Sclera Vein Recognition
JPM1402  An Efficient Parallel Approach for Sclera Vein RecognitionJPM1402  An Efficient Parallel Approach for Sclera Vein Recognition
JPM1402 An Efficient Parallel Approach for Sclera Vein Recognitionchennaijp
 
IRJET - Evaluating and Comparing the Two Variation with Current Scheduling Al...
IRJET - Evaluating and Comparing the Two Variation with Current Scheduling Al...IRJET - Evaluating and Comparing the Two Variation with Current Scheduling Al...
IRJET - Evaluating and Comparing the Two Variation with Current Scheduling Al...IRJET Journal
 
The Concept of Load Balancing Server in Secured and Intelligent Network
The Concept of Load Balancing Server in Secured and Intelligent NetworkThe Concept of Load Balancing Server in Secured and Intelligent Network
The Concept of Load Balancing Server in Secured and Intelligent NetworkIJAEMSJORNAL
 
Scaling Application on High Performance Computing Clusters and Analysis of th...
Scaling Application on High Performance Computing Clusters and Analysis of th...Scaling Application on High Performance Computing Clusters and Analysis of th...
Scaling Application on High Performance Computing Clusters and Analysis of th...Rusif Eyvazli
 
Hybrid Scheduling Algorithm for Efficient Load Balancing In Cloud Computing
Hybrid Scheduling Algorithm for Efficient Load Balancing In Cloud ComputingHybrid Scheduling Algorithm for Efficient Load Balancing In Cloud Computing
Hybrid Scheduling Algorithm for Efficient Load Balancing In Cloud ComputingEswar Publications
 
A New Approach for Parallel Region Growing Algorithm in Image Segmentation u...
A New Approach for Parallel Region Growing Algorithm in Image Segmentation u...A New Approach for Parallel Region Growing Algorithm in Image Segmentation u...
A New Approach for Parallel Region Growing Algorithm in Image Segmentation u...maneesh boddu
 

Similar to Page a partition aware engine (20)

PAGE: A PARTITION AWARE ENGINE FOR PARALLEL GRAPH COMPUTATION
PAGE: A PARTITION AWARE ENGINE FOR PARALLEL GRAPH COMPUTATIONPAGE: A PARTITION AWARE ENGINE FOR PARALLEL GRAPH COMPUTATION
PAGE: A PARTITION AWARE ENGINE FOR PARALLEL GRAPH COMPUTATION
 
Page a partition aware engine
Page a partition aware enginePage a partition aware engine
Page a partition aware engine
 
Automatic Compilation Of MATLAB Programs For Synergistic Execution On Heterog...
Automatic Compilation Of MATLAB Programs For Synergistic Execution On Heterog...Automatic Compilation Of MATLAB Programs For Synergistic Execution On Heterog...
Automatic Compilation Of MATLAB Programs For Synergistic Execution On Heterog...
 
International Journal of Engineering Inventions (IJEI)
International Journal of Engineering Inventions (IJEI)International Journal of Engineering Inventions (IJEI)
International Journal of Engineering Inventions (IJEI)
 
Page a partition aware engine for parallel graph computation
Page a partition aware engine for parallel graph computationPage a partition aware engine for parallel graph computation
Page a partition aware engine for parallel graph computation
 
NETWORK-AWARE DATA PREFETCHING OPTIMIZATION OF COMPUTATIONS IN A HETEROGENEOU...
NETWORK-AWARE DATA PREFETCHING OPTIMIZATION OF COMPUTATIONS IN A HETEROGENEOU...NETWORK-AWARE DATA PREFETCHING OPTIMIZATION OF COMPUTATIONS IN A HETEROGENEOU...
NETWORK-AWARE DATA PREFETCHING OPTIMIZATION OF COMPUTATIONS IN A HETEROGENEOU...
 
ON TRAFFIC-AWARE PARTITION AND AGGREGATION IN MAPREDUCE FOR BIG DATA APPLICAT...
ON TRAFFIC-AWARE PARTITION AND AGGREGATION IN MAPREDUCE FOR BIG DATA APPLICAT...ON TRAFFIC-AWARE PARTITION AND AGGREGATION IN MAPREDUCE FOR BIG DATA APPLICAT...
ON TRAFFIC-AWARE PARTITION AND AGGREGATION IN MAPREDUCE FOR BIG DATA APPLICAT...
 
GRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEM
GRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEMGRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEM
GRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEM
 
Load rebalancing for distributed file systems in clouds
Load rebalancing for distributed file systems in cloudsLoad rebalancing for distributed file systems in clouds
Load rebalancing for distributed file systems in clouds
 
1844 1849
1844 18491844 1849
1844 1849
 
1844 1849
1844 18491844 1849
1844 1849
 
JPM1402 An Efficient Parallel Approach for Sclera Vein Recognition
JPM1402  An Efficient Parallel Approach for Sclera Vein RecognitionJPM1402  An Efficient Parallel Approach for Sclera Vein Recognition
JPM1402 An Efficient Parallel Approach for Sclera Vein Recognition
 
IRJET - Evaluating and Comparing the Two Variation with Current Scheduling Al...
IRJET - Evaluating and Comparing the Two Variation with Current Scheduling Al...IRJET - Evaluating and Comparing the Two Variation with Current Scheduling Al...
IRJET - Evaluating and Comparing the Two Variation with Current Scheduling Al...
 
E031201032036
E031201032036E031201032036
E031201032036
 
The Concept of Load Balancing Server in Secured and Intelligent Network
The Concept of Load Balancing Server in Secured and Intelligent NetworkThe Concept of Load Balancing Server in Secured and Intelligent Network
The Concept of Load Balancing Server in Secured and Intelligent Network
 
Scaling Application on High Performance Computing Clusters and Analysis of th...
Scaling Application on High Performance Computing Clusters and Analysis of th...Scaling Application on High Performance Computing Clusters and Analysis of th...
Scaling Application on High Performance Computing Clusters and Analysis of th...
 
journal for research
journal for researchjournal for research
journal for research
 
Hybrid Scheduling Algorithm for Efficient Load Balancing In Cloud Computing
Hybrid Scheduling Algorithm for Efficient Load Balancing In Cloud ComputingHybrid Scheduling Algorithm for Efficient Load Balancing In Cloud Computing
Hybrid Scheduling Algorithm for Efficient Load Balancing In Cloud Computing
 
2 mapreduce-model-principles
2 mapreduce-model-principles2 mapreduce-model-principles
2 mapreduce-model-principles
 
A New Approach for Parallel Region Growing Algorithm in Image Segmentation u...
A New Approach for Parallel Region Growing Algorithm in Image Segmentation u...A New Approach for Parallel Region Growing Algorithm in Image Segmentation u...
A New Approach for Parallel Region Growing Algorithm in Image Segmentation u...
 

More from jpstudcorner

Variable length signature for near-duplicate
Variable length signature for near-duplicateVariable length signature for near-duplicate
Variable length signature for near-duplicatejpstudcorner
 
Robust representation and recognition of facial
Robust representation and recognition of facialRobust representation and recognition of facial
Robust representation and recognition of facialjpstudcorner
 
Revealing the trace of high quality jpeg
Revealing the trace of high quality jpegRevealing the trace of high quality jpeg
Revealing the trace of high quality jpegjpstudcorner
 
Revealing the trace of high quality jpeg
Revealing the trace of high quality jpegRevealing the trace of high quality jpeg
Revealing the trace of high quality jpegjpstudcorner
 
Pareto depth for multiple-query image retrieval
Pareto depth for multiple-query image retrievalPareto depth for multiple-query image retrieval
Pareto depth for multiple-query image retrievaljpstudcorner
 
Multifocus image fusion based on nsct
Multifocus image fusion based on nsctMultifocus image fusion based on nsct
Multifocus image fusion based on nsctjpstudcorner
 
Image super resolution based on
Image super resolution based onImage super resolution based on
Image super resolution based onjpstudcorner
 
Fractal analysis for reduced reference
Fractal analysis for reduced referenceFractal analysis for reduced reference
Fractal analysis for reduced referencejpstudcorner
 
Face sketch synthesis via sparse representation based greedy search
Face sketch synthesis via sparse representation based greedy searchFace sketch synthesis via sparse representation based greedy search
Face sketch synthesis via sparse representation based greedy searchjpstudcorner
 
Face recognition across non uniform motion
Face recognition across non uniform motionFace recognition across non uniform motion
Face recognition across non uniform motionjpstudcorner
 
Combining left and right palmprint images for
Combining left and right palmprint images forCombining left and right palmprint images for
Combining left and right palmprint images forjpstudcorner
 
A probabilistic approach for color correction
A probabilistic approach for color correctionA probabilistic approach for color correction
A probabilistic approach for color correctionjpstudcorner
 
A no reference texture regularity metric
A no reference texture regularity metricA no reference texture regularity metric
A no reference texture regularity metricjpstudcorner
 
A feature enriched completely blind image
A feature enriched completely blind imageA feature enriched completely blind image
A feature enriched completely blind imagejpstudcorner
 
Sel csp a framework to facilitate
Sel csp a framework to facilitateSel csp a framework to facilitate
Sel csp a framework to facilitatejpstudcorner
 
Query aware determinization of uncertain
Query aware determinization of uncertainQuery aware determinization of uncertain
Query aware determinization of uncertainjpstudcorner
 
Psmpa patient self controllable
Psmpa patient self controllablePsmpa patient self controllable
Psmpa patient self controllablejpstudcorner
 
Privacy preserving and truthful detection
Privacy preserving and truthful detectionPrivacy preserving and truthful detection
Privacy preserving and truthful detectionjpstudcorner
 
Privacy policy inference of user uploaded
Privacy policy inference of user uploadedPrivacy policy inference of user uploaded
Privacy policy inference of user uploadedjpstudcorner
 
Optimal configuration of network
Optimal configuration of networkOptimal configuration of network
Optimal configuration of networkjpstudcorner
 

More from jpstudcorner (20)

Variable length signature for near-duplicate
Variable length signature for near-duplicateVariable length signature for near-duplicate
Variable length signature for near-duplicate
 
Robust representation and recognition of facial
Robust representation and recognition of facialRobust representation and recognition of facial
Robust representation and recognition of facial
 
Revealing the trace of high quality jpeg
Revealing the trace of high quality jpegRevealing the trace of high quality jpeg
Revealing the trace of high quality jpeg
 
Revealing the trace of high quality jpeg
Revealing the trace of high quality jpegRevealing the trace of high quality jpeg
Revealing the trace of high quality jpeg
 
Pareto depth for multiple-query image retrieval
Pareto depth for multiple-query image retrievalPareto depth for multiple-query image retrieval
Pareto depth for multiple-query image retrieval
 
Multifocus image fusion based on nsct
Multifocus image fusion based on nsctMultifocus image fusion based on nsct
Multifocus image fusion based on nsct
 
Image super resolution based on
Image super resolution based onImage super resolution based on
Image super resolution based on
 
Fractal analysis for reduced reference
Fractal analysis for reduced referenceFractal analysis for reduced reference
Fractal analysis for reduced reference
 
Face sketch synthesis via sparse representation based greedy search
Face sketch synthesis via sparse representation based greedy searchFace sketch synthesis via sparse representation based greedy search
Face sketch synthesis via sparse representation based greedy search
 
Face recognition across non uniform motion
Face recognition across non uniform motionFace recognition across non uniform motion
Face recognition across non uniform motion
 
Combining left and right palmprint images for
Combining left and right palmprint images forCombining left and right palmprint images for
Combining left and right palmprint images for
 
A probabilistic approach for color correction
A probabilistic approach for color correctionA probabilistic approach for color correction
A probabilistic approach for color correction
 
A no reference texture regularity metric
A no reference texture regularity metricA no reference texture regularity metric
A no reference texture regularity metric
 
A feature enriched completely blind image
A feature enriched completely blind imageA feature enriched completely blind image
A feature enriched completely blind image
 
Sel csp a framework to facilitate
Sel csp a framework to facilitateSel csp a framework to facilitate
Sel csp a framework to facilitate
 
Query aware determinization of uncertain
Query aware determinization of uncertainQuery aware determinization of uncertain
Query aware determinization of uncertain
 
Psmpa patient self controllable
Psmpa patient self controllablePsmpa patient self controllable
Psmpa patient self controllable
 
Privacy preserving and truthful detection
Privacy preserving and truthful detectionPrivacy preserving and truthful detection
Privacy preserving and truthful detection
 
Privacy policy inference of user uploaded
Privacy policy inference of user uploadedPrivacy policy inference of user uploaded
Privacy policy inference of user uploaded
 
Optimal configuration of network
Optimal configuration of networkOptimal configuration of network
Optimal configuration of network
 

Recently uploaded

Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...
Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...
Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...Christo Ananth
 
Introduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptxIntroduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptxupamatechverse
 
Processing & Properties of Floor and Wall Tiles.pptx
Processing & Properties of Floor and Wall Tiles.pptxProcessing & Properties of Floor and Wall Tiles.pptx
Processing & Properties of Floor and Wall Tiles.pptxpranjaldaimarysona
 
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur EscortsHigh Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escortsranjana rawat
 
College Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service NashikCollege Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service NashikCall Girls in Nagpur High Profile
 
(SHREYA) Chakan Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Esc...
(SHREYA) Chakan Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Esc...(SHREYA) Chakan Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Esc...
(SHREYA) Chakan Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Esc...ranjana rawat
 
Software Development Life Cycle By Team Orange (Dept. of Pharmacy)
Software Development Life Cycle By  Team Orange (Dept. of Pharmacy)Software Development Life Cycle By  Team Orange (Dept. of Pharmacy)
Software Development Life Cycle By Team Orange (Dept. of Pharmacy)Suman Mia
 
(PRIYA) Rajgurunagar Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(PRIYA) Rajgurunagar Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(PRIYA) Rajgurunagar Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(PRIYA) Rajgurunagar Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...ranjana rawat
 
Introduction and different types of Ethernet.pptx
Introduction and different types of Ethernet.pptxIntroduction and different types of Ethernet.pptx
Introduction and different types of Ethernet.pptxupamatechverse
 
Call Girls Delhi {Jodhpur} 9711199012 high profile service
Call Girls Delhi {Jodhpur} 9711199012 high profile serviceCall Girls Delhi {Jodhpur} 9711199012 high profile service
Call Girls Delhi {Jodhpur} 9711199012 high profile servicerehmti665
 
chaitra-1.pptx fake news detection using machine learning
chaitra-1.pptx  fake news detection using machine learningchaitra-1.pptx  fake news detection using machine learning
chaitra-1.pptx fake news detection using machine learningmisbanausheenparvam
 
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...Soham Mondal
 
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...ranjana rawat
 
Architect Hassan Khalil Portfolio for 2024
Architect Hassan Khalil Portfolio for 2024Architect Hassan Khalil Portfolio for 2024
Architect Hassan Khalil Portfolio for 2024hassan khalil
 
Microscopic Analysis of Ceramic Materials.pptx
Microscopic Analysis of Ceramic Materials.pptxMicroscopic Analysis of Ceramic Materials.pptx
Microscopic Analysis of Ceramic Materials.pptxpurnimasatapathy1234
 
Extrusion Processes and Their Limitations
Extrusion Processes and Their LimitationsExtrusion Processes and Their Limitations
Extrusion Processes and Their Limitations120cr0395
 
GDSC ASEB Gen AI study jams presentation
GDSC ASEB Gen AI study jams presentationGDSC ASEB Gen AI study jams presentation
GDSC ASEB Gen AI study jams presentationGDSCAESB
 
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur Escorts
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur EscortsCall Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur Escorts
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur EscortsCall Girls in Nagpur High Profile
 
main PPT.pptx of girls hostel security using rfid
main PPT.pptx of girls hostel security using rfidmain PPT.pptx of girls hostel security using rfid
main PPT.pptx of girls hostel security using rfidNikhilNagaraju
 

Recently uploaded (20)

Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...
Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...
Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...
 
Introduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptxIntroduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptx
 
Processing & Properties of Floor and Wall Tiles.pptx
Processing & Properties of Floor and Wall Tiles.pptxProcessing & Properties of Floor and Wall Tiles.pptx
Processing & Properties of Floor and Wall Tiles.pptx
 
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur EscortsHigh Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
 
College Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service NashikCollege Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
 
(SHREYA) Chakan Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Esc...
(SHREYA) Chakan Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Esc...(SHREYA) Chakan Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Esc...
(SHREYA) Chakan Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Esc...
 
Software Development Life Cycle By Team Orange (Dept. of Pharmacy)
Software Development Life Cycle By  Team Orange (Dept. of Pharmacy)Software Development Life Cycle By  Team Orange (Dept. of Pharmacy)
Software Development Life Cycle By Team Orange (Dept. of Pharmacy)
 
(PRIYA) Rajgurunagar Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(PRIYA) Rajgurunagar Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(PRIYA) Rajgurunagar Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(PRIYA) Rajgurunagar Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
 
Introduction and different types of Ethernet.pptx
Introduction and different types of Ethernet.pptxIntroduction and different types of Ethernet.pptx
Introduction and different types of Ethernet.pptx
 
Call Girls Delhi {Jodhpur} 9711199012 high profile service
Call Girls Delhi {Jodhpur} 9711199012 high profile serviceCall Girls Delhi {Jodhpur} 9711199012 high profile service
Call Girls Delhi {Jodhpur} 9711199012 high profile service
 
chaitra-1.pptx fake news detection using machine learning
chaitra-1.pptx  fake news detection using machine learningchaitra-1.pptx  fake news detection using machine learning
chaitra-1.pptx fake news detection using machine learning
 
DJARUM4D - SLOT GACOR ONLINE | SLOT DEMO ONLINE
DJARUM4D - SLOT GACOR ONLINE | SLOT DEMO ONLINEDJARUM4D - SLOT GACOR ONLINE | SLOT DEMO ONLINE
DJARUM4D - SLOT GACOR ONLINE | SLOT DEMO ONLINE
 
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
 
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
 
Architect Hassan Khalil Portfolio for 2024
Architect Hassan Khalil Portfolio for 2024Architect Hassan Khalil Portfolio for 2024
Architect Hassan Khalil Portfolio for 2024
 
Microscopic Analysis of Ceramic Materials.pptx
Microscopic Analysis of Ceramic Materials.pptxMicroscopic Analysis of Ceramic Materials.pptx
Microscopic Analysis of Ceramic Materials.pptx
 
Extrusion Processes and Their Limitations
Extrusion Processes and Their LimitationsExtrusion Processes and Their Limitations
Extrusion Processes and Their Limitations
 
GDSC ASEB Gen AI study jams presentation
GDSC ASEB Gen AI study jams presentationGDSC ASEB Gen AI study jams presentation
GDSC ASEB Gen AI study jams presentation
 
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur Escorts
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur EscortsCall Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur Escorts
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur Escorts
 
main PPT.pptx of girls hostel security using rfid
main PPT.pptx of girls hostel security using rfidmain PPT.pptx of girls hostel security using rfid
main PPT.pptx of girls hostel security using rfid
 

Page a partition aware engine

  • 1. PAGE: A Partition Aware Engine for Parallel Graph Computation ABSTRACT: Graph partition quality affects the overall performance of parallel graph computation systems. The quality of a graph partition is measured by the balance factor and edge cut ratio. A balanced graph partition with small edge cut ratio is generally preferred since it reduces the expensive network communication cost. However, according to an empirical study on Giraph, the performance over well partitioned graph might be even two times worse than simple random partitions. This is because these systems only optimize for the simple partition strategies and cannot efficiently handle the increasing workload of local message processing when a high quality graph partition is used. In this paper, we propose a novel partition aware graph computation engine named PAGE, which equips a new message processor and a dynamic concurrency control model. The new message processor concurrently processes local and remote messages in a unified way. The dynamic model adaptively adjusts the concurrency of the processor based on the online statistics. The experimental evaluation demonstrates the superiority of PAGE over the graph partitions with various qualities.
  • 2. EXISTING SYSTEM:  A lot of parallel graph computation systems have been introduced, e.g., Pregel, Giraph, GPS and Graph-Lab. These systems follow the vertex centric programming model. The graph algorithms in them are split into several supersteps by synchronization barriers.  In a super step, each active vertex simultaneously updates its status based on the neighbors’ messages from previous superstep, and then sends the new status as a message to its neighbors. With the limited workers (computing nodes) in practice, a worker usually stores a subgraph, not a vertex, at local, and sequentially executes the local active vertices. The computations of these workers are in parallel. DISADVANTAGES OF EXISTING SYSTEM:  Lots of existing parallel graph systems are unaware of such effect of the underlying partitioned subgraphs, and ignore the increasing workload of local message processing when the quality of partition scheme is improved. Therefore, these systems handle the local messages and remote messages unequally and only optimize the processing of remote messages.
  • 3.  Though there is a simple extension of centralized message buffer used to process local and remote incoming messages all together, the existing graph systems still cannot effectively utilize the benefit of high quality graph partitions. PROPOSED SYSTEM:  In this paper, we present a novel graph computation engine, partition aware graph computation engine (PAGE). To efficiently support computation tasks with different partitioning qualities, we develop some unique components in this new framework.  First, in PAGE’s worker, communication module is extended with a new dual concurrent message processor. The message processor concurrently handles both local and remote incoming messages in a unified way, thus accelerating the message processing. Furthermore, the concurrency of the message processoris tunable according to the online statistics of the system.  Second, a partition aware module is added in each worker to monitor the partition related characters and adjust the concurrency of the message processoradaptively to fit the online workload.  To fulfill the goal of estimating a reasonable concurrency for the dual concurrent message processor, we introduce the Dynamic Concurrency Control Model (DCCM). Since the message processing pipeline satisfied the
  • 4. producer consumer model, several heuristic rules are proposed by considering the producer-consumer constraints. With the help of DCCM, PAGE provides sufficient message process units to handle current workload and each message process unit has similar workload. Finally, PAGE can adaptively accept various qualities of the integrated graph partition. ADVANTAGES OF PROPOSED SYSTEM:  We propose the problem that existing graph computation systems cannot efficiently exploit the benefit of high quality graph partitions.  We design a new partition aware graph computation engine, called PAGE. It can effectively harness the partition information to guide parallel processing resource allocation, and improve the computation performance.  We introduce a dual concurrent message processor. The message processor concurrently processes incoming messages in a unified way and is the cornerstone in PAGE.  We present a dynamic concurrency control model. The model estimates concurrency for dual concurrent message processor by satisfying the producer consumer constraints. The model always generate proper configurations for PAGE when the graph applications or underlying graph partitions change.
  • 5.  We implement a prototype of PAGE and test it with real-world graphs and various graph algorithms.  The results clearly demonstrate that PAGE performs efficiently over various graph partitioning qualities. SYSTEM ARCHITECTURE: SYSTEM REQUIREMENTS: HARDWARE REQUIREMENTS:  System : Pentium IV 2.4 GHz.  Hard Disk : 40 GB.
  • 6.  Floppy Drive : 1.44 Mb.  Monitor : 15 VGA Colour.  Mouse : Logitech.  Ram : 512 Mb. SOFTWARE REQUIREMENTS:  Operating system : Windows XP/7.  Coding Language : ASP.net, C#.net  Tool : Visual Studio 2010  Database : SQL SERVER 2008 REFERENCE: Yingxia Shao, Bin Cui, Senior Member, IEEE, and Lin Ma, “PAGE: A Partition Aware Engine for Parallel Graph Computation”, IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 27, NO. 2, FEBRUARY 2015.