SlideShare a Scribd company logo
1 of 6
PAGE: A Partition Aware Engine for Parallel Graph
Computation
ABSTRACT:
Graph partition quality affects the overall performance of parallel graph
computation systems. The quality of a graph partition is measured by the balance
factor and edge cut ratio. A balanced graph partition with small edge cut ratio is
generally preferred since it reduces the expensive network communication cost.
However, according to an empirical study on Giraph, the performance over well
partitioned graph might be even two times worse than simple random partitions.
This is because these systems only optimize for the simple partition strategies and
cannot efficiently handle the increasing workload of local message processing
when a high quality graph partition is used. In this paper, we propose a novel
partition aware graph computation engine named PAGE, which equips a new
message processor and a dynamic concurrency control model. The new message
processor concurrently processes local and remote messages in a unified way. The
dynamic model adaptively adjusts the concurrency of the processor based on the
online statistics. The experimental evaluation demonstrates the superiority of
PAGE over the graph partitions with various qualities.
EXISTING SYSTEM:
īļ A lot of parallel graph computation systems have been introduced, e.g.,
Pregel, Giraph, GPS and Graph-Lab. These systems follow the vertex centric
programming model. The graph algorithms in them are split into several
supersteps by synchronization barriers.
īļ In a super step, each active vertex simultaneously updates its status based on
the neighbors’ messages from previous superstep, and then sends the new
status as a message to its neighbors. With the limited workers (computing
nodes) in practice, a worker usually stores a subgraph, not a vertex, at local,
and sequentially executes the local active vertices. The computations of
these workers are in parallel.
DISADVANTAGES OF EXISTING SYSTEM:
īļ Lots of existing parallel graph systems are unaware of such effect of the
underlying partitioned subgraphs, and ignore the increasing workload of
local message processing when the quality of partition scheme is improved.
Therefore, these systems handle the local messages and remote messages
unequally and only optimize the processing of remote messages.
īļ Though there is a simple extension of centralized message buffer used to
process local and remote incoming messages all together, the existing graph
systems still cannot effectively utilize the benefit of high quality graph
partitions.
PROPOSED SYSTEM:
īļ In this paper, we present a novel graph computation engine, partition aware
graph computation engine (PAGE). To efficiently support computation tasks
with different partitioning qualities, we develop some unique components in
this new framework.
īļ First, in PAGE’s worker, communication module is extended with a new
dual concurrent message processor. The message processor concurrently
handles both local and remote incoming messages in a unified way, thus
accelerating the message processing. Furthermore, the concurrency of the
message processoris tunable according to the online statistics of the system.
īļ Second, a partition aware module is added in each worker to monitor the
partition related characters and adjust the concurrency of the message
processoradaptively to fit the online workload.
īļ To fulfill the goal of estimating a reasonable concurrency for the dual
concurrent message processor, we introduce the Dynamic Concurrency
Control Model (DCCM). Since the message processing pipeline satisfied the
producer consumer model, several heuristic rules are proposed by
considering the producer-consumer constraints. With the help of DCCM,
PAGE provides sufficient message process units to handle current workload
and each message process unit has similar workload. Finally, PAGE can
adaptively accept various qualities of the integrated graph partition.
ADVANTAGES OF PROPOSED SYSTEM:
īļ We propose the problem that existing graph computation systems cannot
efficiently exploit the benefit of high quality graph partitions.
īļ We design a new partition aware graph computation engine, called PAGE. It
can effectively harness the partition information to guide parallel processing
resource allocation, and improve the computation performance.
īļ We introduce a dual concurrent message processor. The message processor
concurrently processes incoming messages in a unified way and is the
cornerstone in PAGE.
īļ We present a dynamic concurrency control model. The model estimates
concurrency for dual concurrent message processor by satisfying the
producer consumer constraints. The model always generate proper
configurations for PAGE when the graph applications or underlying graph
partitions change.
īļ We implement a prototype of PAGE and test it with real-world graphs and
various graph algorithms.
īļ The results clearly demonstrate that PAGE performs efficiently over various
graph partitioning qualities.
SYSTEM ARCHITECTURE:
SYSTEM REQUIREMENTS:
HARDWARE REQUIREMENTS:
īƒ˜ System : Pentium IV 2.4 GHz.
īƒ˜ Hard Disk : 40 GB.
īƒ˜ Floppy Drive : 1.44 Mb.
īƒ˜ Monitor : 15 VGA Colour.
īƒ˜ Mouse : Logitech.
īƒ˜ Ram : 512 Mb.
SOFTWARE REQUIREMENTS:
īƒ˜ Operating system : Windows XP/7.
īƒ˜ Coding Language : ASP.net, C#.net
īƒ˜ Tool : Visual Studio 2010
īƒ˜ Database : SQL SERVER 2008
REFERENCE:
Yingxia Shao, Bin Cui, Senior Member, IEEE, and Lin Ma, “PAGE: A Partition
Aware Engine for Parallel Graph Computation”, IEEE TRANSACTIONS ON
KNOWLEDGE AND DATA ENGINEERING, VOL. 27, NO. 2, FEBRUARY
2015.

More Related Content

What's hot

Hadoop MapReduce joins
Hadoop MapReduce joinsHadoop MapReduce joins
Hadoop MapReduce joinsShalish VJ
 
Hadoop Mapreduce joins
Hadoop Mapreduce joinsHadoop Mapreduce joins
Hadoop Mapreduce joinsUday Vakalapudi
 
Pack prediction based cloud bandwidth and cost reduction system
Pack prediction based cloud bandwidth and cost reduction systemPack prediction based cloud bandwidth and cost reduction system
Pack prediction based cloud bandwidth and cost reduction systemIEEEFINALYEARPROJECTS
 
Repartition join in mapreduce
Repartition join in mapreduceRepartition join in mapreduce
Repartition join in mapreduceUday Vakalapudi
 
Introduction to map reduce s. jency jayastina II MSC COMPUTER SCIENCE BON SEC...
Introduction to map reduce s. jency jayastina II MSC COMPUTER SCIENCE BON SEC...Introduction to map reduce s. jency jayastina II MSC COMPUTER SCIENCE BON SEC...
Introduction to map reduce s. jency jayastina II MSC COMPUTER SCIENCE BON SEC...jencyjayastina
 
Novel Scheduling Algorithms for Efficient Deployment of Map Reduce Applicatio...
Novel Scheduling Algorithms for Efficient Deployment of Map Reduce Applicatio...Novel Scheduling Algorithms for Efficient Deployment of Map Reduce Applicatio...
Novel Scheduling Algorithms for Efficient Deployment of Map Reduce Applicatio...IRJET Journal
 
Hadoop World 2011: Next Generation Apache Hadoop MapReduce - Mohadev Konar, H...
Hadoop World 2011: Next Generation Apache Hadoop MapReduce - Mohadev Konar, H...Hadoop World 2011: Next Generation Apache Hadoop MapReduce - Mohadev Konar, H...
Hadoop World 2011: Next Generation Apache Hadoop MapReduce - Mohadev Konar, H...Cloudera, Inc.
 
Join Algorithms in MapReduce
Join Algorithms in MapReduceJoin Algorithms in MapReduce
Join Algorithms in MapReduceShrihari Rathod
 
Raminder kaur presentation_two
Raminder kaur presentation_twoRaminder kaur presentation_two
Raminder kaur presentation_tworamikaurraminder
 
Pack prediction based cloud bandwidth and cost reduction system
Pack prediction based cloud bandwidth and cost reduction systemPack prediction based cloud bandwidth and cost reduction system
Pack prediction based cloud bandwidth and cost reduction systemJPINFOTECH JAYAPRAKASH
 
Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...
Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...
Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...Cloudera, Inc.
 
Hadoop, mapreduce and yarn networks
Hadoop, mapreduce and yarn networksHadoop, mapreduce and yarn networks
Hadoop, mapreduce and yarn networksHariniA7
 
Introducing MapReduce Programming Framework
Introducing MapReduce Programming FrameworkIntroducing MapReduce Programming Framework
Introducing MapReduce Programming FrameworkSamuel Yee
 
Relational Algebra and MapReduce
Relational Algebra and MapReduceRelational Algebra and MapReduce
Relational Algebra and MapReducePietro Michiardi
 
MapReduce in Cloud Computing
MapReduce in Cloud ComputingMapReduce in Cloud Computing
MapReduce in Cloud ComputingMohammad Mustaqeem
 
Sawmill - Integrating R and Large Data Clouds
Sawmill - Integrating R and Large Data CloudsSawmill - Integrating R and Large Data Clouds
Sawmill - Integrating R and Large Data CloudsRobert Grossman
 
Mapreduce script
Mapreduce scriptMapreduce script
Mapreduce scriptHaripritha
 
Map reduce presentation
Map reduce presentationMap reduce presentation
Map reduce presentationAhmad El Tawil
 

What's hot (18)

Hadoop MapReduce joins
Hadoop MapReduce joinsHadoop MapReduce joins
Hadoop MapReduce joins
 
Hadoop Mapreduce joins
Hadoop Mapreduce joinsHadoop Mapreduce joins
Hadoop Mapreduce joins
 
Pack prediction based cloud bandwidth and cost reduction system
Pack prediction based cloud bandwidth and cost reduction systemPack prediction based cloud bandwidth and cost reduction system
Pack prediction based cloud bandwidth and cost reduction system
 
Repartition join in mapreduce
Repartition join in mapreduceRepartition join in mapreduce
Repartition join in mapreduce
 
Introduction to map reduce s. jency jayastina II MSC COMPUTER SCIENCE BON SEC...
Introduction to map reduce s. jency jayastina II MSC COMPUTER SCIENCE BON SEC...Introduction to map reduce s. jency jayastina II MSC COMPUTER SCIENCE BON SEC...
Introduction to map reduce s. jency jayastina II MSC COMPUTER SCIENCE BON SEC...
 
Novel Scheduling Algorithms for Efficient Deployment of Map Reduce Applicatio...
Novel Scheduling Algorithms for Efficient Deployment of Map Reduce Applicatio...Novel Scheduling Algorithms for Efficient Deployment of Map Reduce Applicatio...
Novel Scheduling Algorithms for Efficient Deployment of Map Reduce Applicatio...
 
Hadoop World 2011: Next Generation Apache Hadoop MapReduce - Mohadev Konar, H...
Hadoop World 2011: Next Generation Apache Hadoop MapReduce - Mohadev Konar, H...Hadoop World 2011: Next Generation Apache Hadoop MapReduce - Mohadev Konar, H...
Hadoop World 2011: Next Generation Apache Hadoop MapReduce - Mohadev Konar, H...
 
Join Algorithms in MapReduce
Join Algorithms in MapReduceJoin Algorithms in MapReduce
Join Algorithms in MapReduce
 
Raminder kaur presentation_two
Raminder kaur presentation_twoRaminder kaur presentation_two
Raminder kaur presentation_two
 
Pack prediction based cloud bandwidth and cost reduction system
Pack prediction based cloud bandwidth and cost reduction systemPack prediction based cloud bandwidth and cost reduction system
Pack prediction based cloud bandwidth and cost reduction system
 
Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...
Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...
Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...
 
Hadoop, mapreduce and yarn networks
Hadoop, mapreduce and yarn networksHadoop, mapreduce and yarn networks
Hadoop, mapreduce and yarn networks
 
Introducing MapReduce Programming Framework
Introducing MapReduce Programming FrameworkIntroducing MapReduce Programming Framework
Introducing MapReduce Programming Framework
 
Relational Algebra and MapReduce
Relational Algebra and MapReduceRelational Algebra and MapReduce
Relational Algebra and MapReduce
 
MapReduce in Cloud Computing
MapReduce in Cloud ComputingMapReduce in Cloud Computing
MapReduce in Cloud Computing
 
Sawmill - Integrating R and Large Data Clouds
Sawmill - Integrating R and Large Data CloudsSawmill - Integrating R and Large Data Clouds
Sawmill - Integrating R and Large Data Clouds
 
Mapreduce script
Mapreduce scriptMapreduce script
Mapreduce script
 
Map reduce presentation
Map reduce presentationMap reduce presentation
Map reduce presentation
 

Similar to IEEE Projects 2015 | Page a partition aware engine for parallel graph computation

PAGE: A PARTITION AWARE ENGINE FOR PARALLEL GRAPH COMPUTATION
PAGE: A PARTITION AWARE ENGINE FOR PARALLEL GRAPH COMPUTATIONPAGE: A PARTITION AWARE ENGINE FOR PARALLEL GRAPH COMPUTATION
PAGE: A PARTITION AWARE ENGINE FOR PARALLEL GRAPH COMPUTATIONNexgen Technology
 
Page a partition aware engine
Page a partition aware enginePage a partition aware engine
Page a partition aware enginenexgentech15
 
Automatic Compilation Of MATLAB Programs For Synergistic Execution On Heterog...
Automatic Compilation Of MATLAB Programs For Synergistic Execution On Heterog...Automatic Compilation Of MATLAB Programs For Synergistic Execution On Heterog...
Automatic Compilation Of MATLAB Programs For Synergistic Execution On Heterog...Sara Alvarez
 
Page a partition aware engine for parallel graph computation
Page a partition aware engine for parallel graph computationPage a partition aware engine for parallel graph computation
Page a partition aware engine for parallel graph computationPvrtechnologies Nellore
 
NETWORK-AWARE DATA PREFETCHING OPTIMIZATION OF COMPUTATIONS IN A HETEROGENEOU...
NETWORK-AWARE DATA PREFETCHING OPTIMIZATION OF COMPUTATIONS IN A HETEROGENEOU...NETWORK-AWARE DATA PREFETCHING OPTIMIZATION OF COMPUTATIONS IN A HETEROGENEOU...
NETWORK-AWARE DATA PREFETCHING OPTIMIZATION OF COMPUTATIONS IN A HETEROGENEOU...IJCNCJournal
 
ON TRAFFIC-AWARE PARTITION AND AGGREGATION IN MAPREDUCE FOR BIG DATA APPLICAT...
ON TRAFFIC-AWARE PARTITION AND AGGREGATION IN MAPREDUCE FOR BIG DATA APPLICAT...ON TRAFFIC-AWARE PARTITION AND AGGREGATION IN MAPREDUCE FOR BIG DATA APPLICAT...
ON TRAFFIC-AWARE PARTITION AND AGGREGATION IN MAPREDUCE FOR BIG DATA APPLICAT...I3E Technologies
 
GRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEM
GRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEMGRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEM
GRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEMIJCSEA Journal
 
Load rebalancing for distributed file systems in clouds
Load rebalancing for distributed file systems in cloudsLoad rebalancing for distributed file systems in clouds
Load rebalancing for distributed file systems in cloudsJPINFOTECH JAYAPRAKASH
 
JPM1402 An Efficient Parallel Approach for Sclera Vein Recognition
JPM1402  An Efficient Parallel Approach for Sclera Vein RecognitionJPM1402  An Efficient Parallel Approach for Sclera Vein Recognition
JPM1402 An Efficient Parallel Approach for Sclera Vein Recognitionchennaijp
 
IRJET - Evaluating and Comparing the Two Variation with Current Scheduling Al...
IRJET - Evaluating and Comparing the Two Variation with Current Scheduling Al...IRJET - Evaluating and Comparing the Two Variation with Current Scheduling Al...
IRJET - Evaluating and Comparing the Two Variation with Current Scheduling Al...IRJET Journal
 
E031201032036
E031201032036E031201032036
E031201032036ijceronline
 
The Concept of Load Balancing Server in Secured and Intelligent Network
The Concept of Load Balancing Server in Secured and Intelligent NetworkThe Concept of Load Balancing Server in Secured and Intelligent Network
The Concept of Load Balancing Server in Secured and Intelligent NetworkIJAEMSJORNAL
 
Scaling Application on High Performance Computing Clusters and Analysis of th...
Scaling Application on High Performance Computing Clusters and Analysis of th...Scaling Application on High Performance Computing Clusters and Analysis of th...
Scaling Application on High Performance Computing Clusters and Analysis of th...Rusif Eyvazli
 
journal for research
journal for researchjournal for research
journal for researchchaitanya451336
 
Hybrid Scheduling Algorithm for Efficient Load Balancing In Cloud Computing
Hybrid Scheduling Algorithm for Efficient Load Balancing In Cloud ComputingHybrid Scheduling Algorithm for Efficient Load Balancing In Cloud Computing
Hybrid Scheduling Algorithm for Efficient Load Balancing In Cloud ComputingEswar Publications
 
A New Approach for Parallel Region Growing Algorithm in Image Segmentation u...
A New Approach for Parallel Region Growing Algorithm in Image Segmentation u...A New Approach for Parallel Region Growing Algorithm in Image Segmentation u...
A New Approach for Parallel Region Growing Algorithm in Image Segmentation u...maneesh boddu
 

Similar to IEEE Projects 2015 | Page a partition aware engine for parallel graph computation (20)

PAGE: A PARTITION AWARE ENGINE FOR PARALLEL GRAPH COMPUTATION
PAGE: A PARTITION AWARE ENGINE FOR PARALLEL GRAPH COMPUTATIONPAGE: A PARTITION AWARE ENGINE FOR PARALLEL GRAPH COMPUTATION
PAGE: A PARTITION AWARE ENGINE FOR PARALLEL GRAPH COMPUTATION
 
Page a partition aware engine
Page a partition aware enginePage a partition aware engine
Page a partition aware engine
 
Automatic Compilation Of MATLAB Programs For Synergistic Execution On Heterog...
Automatic Compilation Of MATLAB Programs For Synergistic Execution On Heterog...Automatic Compilation Of MATLAB Programs For Synergistic Execution On Heterog...
Automatic Compilation Of MATLAB Programs For Synergistic Execution On Heterog...
 
International Journal of Engineering Inventions (IJEI)
International Journal of Engineering Inventions (IJEI)International Journal of Engineering Inventions (IJEI)
International Journal of Engineering Inventions (IJEI)
 
Page a partition aware engine for parallel graph computation
Page a partition aware engine for parallel graph computationPage a partition aware engine for parallel graph computation
Page a partition aware engine for parallel graph computation
 
NETWORK-AWARE DATA PREFETCHING OPTIMIZATION OF COMPUTATIONS IN A HETEROGENEOU...
NETWORK-AWARE DATA PREFETCHING OPTIMIZATION OF COMPUTATIONS IN A HETEROGENEOU...NETWORK-AWARE DATA PREFETCHING OPTIMIZATION OF COMPUTATIONS IN A HETEROGENEOU...
NETWORK-AWARE DATA PREFETCHING OPTIMIZATION OF COMPUTATIONS IN A HETEROGENEOU...
 
ON TRAFFIC-AWARE PARTITION AND AGGREGATION IN MAPREDUCE FOR BIG DATA APPLICAT...
ON TRAFFIC-AWARE PARTITION AND AGGREGATION IN MAPREDUCE FOR BIG DATA APPLICAT...ON TRAFFIC-AWARE PARTITION AND AGGREGATION IN MAPREDUCE FOR BIG DATA APPLICAT...
ON TRAFFIC-AWARE PARTITION AND AGGREGATION IN MAPREDUCE FOR BIG DATA APPLICAT...
 
GRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEM
GRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEMGRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEM
GRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEM
 
Load rebalancing for distributed file systems in clouds
Load rebalancing for distributed file systems in cloudsLoad rebalancing for distributed file systems in clouds
Load rebalancing for distributed file systems in clouds
 
1844 1849
1844 18491844 1849
1844 1849
 
1844 1849
1844 18491844 1849
1844 1849
 
JPM1402 An Efficient Parallel Approach for Sclera Vein Recognition
JPM1402  An Efficient Parallel Approach for Sclera Vein RecognitionJPM1402  An Efficient Parallel Approach for Sclera Vein Recognition
JPM1402 An Efficient Parallel Approach for Sclera Vein Recognition
 
IRJET - Evaluating and Comparing the Two Variation with Current Scheduling Al...
IRJET - Evaluating and Comparing the Two Variation with Current Scheduling Al...IRJET - Evaluating and Comparing the Two Variation with Current Scheduling Al...
IRJET - Evaluating and Comparing the Two Variation with Current Scheduling Al...
 
E031201032036
E031201032036E031201032036
E031201032036
 
The Concept of Load Balancing Server in Secured and Intelligent Network
The Concept of Load Balancing Server in Secured and Intelligent NetworkThe Concept of Load Balancing Server in Secured and Intelligent Network
The Concept of Load Balancing Server in Secured and Intelligent Network
 
Scaling Application on High Performance Computing Clusters and Analysis of th...
Scaling Application on High Performance Computing Clusters and Analysis of th...Scaling Application on High Performance Computing Clusters and Analysis of th...
Scaling Application on High Performance Computing Clusters and Analysis of th...
 
journal for research
journal for researchjournal for research
journal for research
 
Hybrid Scheduling Algorithm for Efficient Load Balancing In Cloud Computing
Hybrid Scheduling Algorithm for Efficient Load Balancing In Cloud ComputingHybrid Scheduling Algorithm for Efficient Load Balancing In Cloud Computing
Hybrid Scheduling Algorithm for Efficient Load Balancing In Cloud Computing
 
2 mapreduce-model-principles
2 mapreduce-model-principles2 mapreduce-model-principles
2 mapreduce-model-principles
 
A New Approach for Parallel Region Growing Algorithm in Image Segmentation u...
A New Approach for Parallel Region Growing Algorithm in Image Segmentation u...A New Approach for Parallel Region Growing Algorithm in Image Segmentation u...
A New Approach for Parallel Region Growing Algorithm in Image Segmentation u...
 

Recently uploaded

9548086042 for call girls in Indira Nagar with room service
9548086042  for call girls in Indira Nagar  with room service9548086042  for call girls in Indira Nagar  with room service
9548086042 for call girls in Indira Nagar with room servicediscovermytutordmt
 
mini mental status format.docx
mini    mental       status     format.docxmini    mental       status     format.docx
mini mental status format.docxPoojaSen20
 
APM Welcome, APM North West Network Conference, Synergies Across Sectors
APM Welcome, APM North West Network Conference, Synergies Across SectorsAPM Welcome, APM North West Network Conference, Synergies Across Sectors
APM Welcome, APM North West Network Conference, Synergies Across SectorsAssociation for Project Management
 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsTechSoup
 
Activity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfActivity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfciinovamais
 
The byproduct of sericulture in different industries.pptx
The byproduct of sericulture in different industries.pptxThe byproduct of sericulture in different industries.pptx
The byproduct of sericulture in different industries.pptxShobhayan Kirtania
 
Russian Call Girls in Andheri Airport Mumbai WhatsApp 9167673311 💞 Full Nigh...
Russian Call Girls in Andheri Airport Mumbai WhatsApp  9167673311 💞 Full Nigh...Russian Call Girls in Andheri Airport Mumbai WhatsApp  9167673311 💞 Full Nigh...
Russian Call Girls in Andheri Airport Mumbai WhatsApp 9167673311 💞 Full Nigh...Pooja Nehwal
 
Measures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SDMeasures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SDThiyagu K
 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introductionMaksud Ahmed
 
Beyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactBeyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactPECB
 
Advance Mobile Application Development class 07
Advance Mobile Application Development class 07Advance Mobile Application Development class 07
Advance Mobile Application Development class 07Dr. Mazin Mohamed alkathiri
 
Measures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and ModeMeasures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and ModeThiyagu K
 
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Sapana Sha
 
Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104misteraugie
 
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...EduSkills OECD
 
Organic Name Reactions for the students and aspirants of Chemistry12th.pptx
Organic Name Reactions  for the students and aspirants of Chemistry12th.pptxOrganic Name Reactions  for the students and aspirants of Chemistry12th.pptx
Organic Name Reactions for the students and aspirants of Chemistry12th.pptxVS Mahajan Coaching Centre
 
Sports & Fitness Value Added Course FY..
Sports & Fitness Value Added Course FY..Sports & Fitness Value Added Course FY..
Sports & Fitness Value Added Course FY..Disha Kariya
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Krashi Coaching
 

Recently uploaded (20)

9548086042 for call girls in Indira Nagar with room service
9548086042  for call girls in Indira Nagar  with room service9548086042  for call girls in Indira Nagar  with room service
9548086042 for call girls in Indira Nagar with room service
 
mini mental status format.docx
mini    mental       status     format.docxmini    mental       status     format.docx
mini mental status format.docx
 
APM Welcome, APM North West Network Conference, Synergies Across Sectors
APM Welcome, APM North West Network Conference, Synergies Across SectorsAPM Welcome, APM North West Network Conference, Synergies Across Sectors
APM Welcome, APM North West Network Conference, Synergies Across Sectors
 
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The Basics
 
Activity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfActivity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdf
 
The byproduct of sericulture in different industries.pptx
The byproduct of sericulture in different industries.pptxThe byproduct of sericulture in different industries.pptx
The byproduct of sericulture in different industries.pptx
 
Russian Call Girls in Andheri Airport Mumbai WhatsApp 9167673311 💞 Full Nigh...
Russian Call Girls in Andheri Airport Mumbai WhatsApp  9167673311 💞 Full Nigh...Russian Call Girls in Andheri Airport Mumbai WhatsApp  9167673311 💞 Full Nigh...
Russian Call Girls in Andheri Airport Mumbai WhatsApp 9167673311 💞 Full Nigh...
 
Measures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SDMeasures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SD
 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introduction
 
Beyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactBeyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global Impact
 
Advance Mobile Application Development class 07
Advance Mobile Application Development class 07Advance Mobile Application Development class 07
Advance Mobile Application Development class 07
 
Measures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and ModeMeasures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and Mode
 
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
 
Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104
 
Mattingly "AI & Prompt Design: The Basics of Prompt Design"
Mattingly "AI & Prompt Design: The Basics of Prompt Design"Mattingly "AI & Prompt Design: The Basics of Prompt Design"
Mattingly "AI & Prompt Design: The Basics of Prompt Design"
 
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
 
Organic Name Reactions for the students and aspirants of Chemistry12th.pptx
Organic Name Reactions  for the students and aspirants of Chemistry12th.pptxOrganic Name Reactions  for the students and aspirants of Chemistry12th.pptx
Organic Name Reactions for the students and aspirants of Chemistry12th.pptx
 
Sports & Fitness Value Added Course FY..
Sports & Fitness Value Added Course FY..Sports & Fitness Value Added Course FY..
Sports & Fitness Value Added Course FY..
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
 

IEEE Projects 2015 | Page a partition aware engine for parallel graph computation

  • 1. PAGE: A Partition Aware Engine for Parallel Graph Computation
  • 2. ABSTRACT: Graph partition quality affects the overall performance of parallel graph computation systems. The quality of a graph partition is measured by the balance factor and edge cut ratio. A balanced graph partition with small edge cut ratio is generally preferred since it reduces the expensive network communication cost. However, according to an empirical study on Giraph, the performance over well partitioned graph might be even two times worse than simple random partitions. This is because these systems only optimize for the simple partition strategies and cannot efficiently handle the increasing workload of local message processing when a high quality graph partition is used. In this paper, we propose a novel partition aware graph computation engine named PAGE, which equips a new message processor and a dynamic concurrency control model. The new message processor concurrently processes local and remote messages in a unified way. The dynamic model adaptively adjusts the concurrency of the processor based on the online statistics. The experimental evaluation demonstrates the superiority of PAGE over the graph partitions with various qualities. EXISTING SYSTEM: īļ A lot of parallel graph computation systems have been introduced, e.g., Pregel, Giraph, GPS and Graph-Lab. These systems follow the vertex centric programming model. The graph algorithms in them are split into several supersteps by synchronization barriers.
  • 3. īļ In a super step, each active vertex simultaneously updates its status based on the neighbors’ messages from previous superstep, and then sends the new status as a message to its neighbors. With the limited workers (computing nodes) in practice, a worker usually stores a subgraph, not a vertex, at local, and sequentially executes the local active vertices. The computations of these workers are in parallel. DISADVANTAGES OF EXISTING SYSTEM: īļ Lots of existing parallel graph systems are unaware of such effect of the underlying partitioned subgraphs, and ignore the increasing workload of local message processing when the quality of partition scheme is improved. Therefore, these systems handle the local messages and remote messages unequally and only optimize the processing of remote messages. īļ Though there is a simple extension of centralized message buffer used to process local and remote incoming messages all together, the existing graph systems still cannot effectively utilize the benefit of high quality graph partitions. PROPOSED SYSTEM: īļ In this paper, we present a novel graph computation engine, partition aware graph computation engine (PAGE). To efficiently support computation tasks with different partitioning qualities, we develop some unique components in this new framework.
  • 4. īļ First, in PAGE’s worker, communication module is extended with a new dual concurrent message processor. The message processor concurrently handles both local and remote incoming messages in a unified way, thus accelerating the message processing. Furthermore, the concurrency of the message processoris tunable according to the online statistics of the system. īļ Second, a partition aware module is added in each worker to monitor the partition related characters and adjust the concurrency of the message processoradaptively to fit the online workload. īļ To fulfill the goal of estimating a reasonable concurrency for the dual concurrent message processor, we introduce the Dynamic Concurrency Control Model (DCCM). Since the message processing pipeline satisfied the producer consumer model, several heuristic rules are proposed by considering the producer-consumer constraints. With the help of DCCM, PAGE provides sufficient message process units to handle current workload and each message process unit has similar workload. Finally, PAGE can adaptively accept various qualities of the integrated graph partition. ADVANTAGES OF PROPOSED SYSTEM: īļ We propose the problem that existing graph computation systems cannot efficiently exploit the benefit of high quality graph partitions. īļ We design a new partition aware graph computation engine, called PAGE. It can effectively harness the partition information to guide parallel processing resource allocation, and improve the computation performance.
  • 5. īļ We introduce a dual concurrent message processor. The message processor concurrently processes incoming messages in a unified way and is the cornerstone in PAGE. īļ We present a dynamic concurrency control model. The model estimates concurrency for dual concurrent message processor by satisfying the producer consumer constraints. The model always generate proper configurations for PAGE when the graph applications or underlying graph partitions change. īļ We implement a prototype of PAGE and test it with real-world graphs and various graph algorithms. īļ The results clearly demonstrate that PAGE performs efficiently over various graph partitioning qualities. SYSTEM ARCHITECTURE: SYSTEM REQUIREMENTS:
  • 6. HARDWARE REQUIREMENTS: īƒ˜ System : Pentium IV 2.4 GHz. īƒ˜ Hard Disk : 40 GB. īƒ˜ Floppy Drive : 1.44 Mb. īƒ˜ Monitor : 15 VGA Colour. īƒ˜ Mouse : Logitech. īƒ˜ Ram : 512 Mb. SOFTWARE REQUIREMENTS: īƒ˜ Operating system : Windows XP/7. īƒ˜ Coding Language : ASP.net, C#.net īƒ˜ Tool : Visual Studio 2010 īƒ˜ Database : SQL SERVER 2008 REFERENCE: Yingxia Shao, Bin Cui, Senior Member, IEEE, and Lin Ma, “PAGE: A Partition Aware Engine for Parallel Graph Computation”, IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 27, NO. 2, FEBRUARY 2015.