• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Scaling Online Social Networks: extended SPAR using Gossip Learning
 

Scaling Online Social Networks: extended SPAR using Gossip Learning

on

  • 641 views

 

Statistics

Views

Total Views
641
Views on SlideShare
641
Embed Views
0

Actions

Likes
0
Downloads
5
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Scaling Online Social Networks: extended SPAR using Gossip Learning Scaling Online Social Networks: extended SPAR using Gossip Learning Presentation Transcript

    • Motivation Algorithms Our Contribution Evaluation ConclusionScaling Online Social Networks: extended SPAR using Gossip Learning Presented by: Muhammad Anis uddin Nasir Coworker: Maria Stylianou Supervised by: Sarunas Girdzijauskas KTH Royal Institute of Technology December 5, 2012 Muhammad Anis uddin Nasir Scaling Online Social Networks 1/24
    • Motivation Algorithms Our Contribution Evaluation Conclusion1 Motivation2 Algorithms SPAR JA-BE-JA3 Our Contribution Challenges Proposed Algorithm4 Evaluation Datasets Implementation Results5 Conclusion Muhammad Anis uddin Nasir Scaling Online Social Networks 2/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionOnline Social Networks Muhammad Anis uddin Nasir Scaling Online Social Networks 3/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionScalability Hardware Scalability Muhammad Anis uddin Nasir Scaling Online Social Networks 4/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionScalability Hardware Scalability Application Scalability Muhammad Anis uddin Nasir Scaling Online Social Networks 4/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionScaling Vertical Scaling Muhammad Anis uddin Nasir Scaling Online Social Networks 5/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionScaling Vertical Scaling Full Replication Muhammad Anis uddin Nasir Scaling Online Social Networks 5/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionScaling Vertical Scaling Full Replication Data Locality Muhammad Anis uddin Nasir Scaling Online Social Networks 5/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionScaling Vertical Scaling Full Replication Data Locality High Cost Muhammad Anis uddin Nasir Scaling Online Social Networks 5/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionScaling Vertical Scaling Full Replication Data Locality High Cost Horizontal Scaling Muhammad Anis uddin Nasir Scaling Online Social Networks 5/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionScaling Vertical Scaling Full Replication Data Locality High Cost Horizontal Scaling Sharding Muhammad Anis uddin Nasir Scaling Online Social Networks 5/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionScaling Vertical Scaling Full Replication Data Locality High Cost Horizontal Scaling Sharding Disjoint Data Muhammad Anis uddin Nasir Scaling Online Social Networks 5/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionScaling Vertical Scaling Full Replication Data Locality High Cost Horizontal Scaling Sharding Disjoint Data Partitioning OSNs Muhammad Anis uddin Nasir Scaling Online Social Networks 5/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation Conclusion1 Motivation2 Algorithms SPAR JA-BE-JA3 Our Contribution Challenges Proposed Algorithm4 Evaluation Datasets Implementation Results5 Conclusion Muhammad Anis uddin Nasir Scaling Online Social Networks 6/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionFeatures Local Semantics Muhammad Anis uddin Nasir Scaling Online Social Networks 7/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionFeatures Local Semantics Load Balancing Muhammad Anis uddin Nasir Scaling Online Social Networks 7/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionFeatures Local Semantics Load Balancing Fault Tolerant Muhammad Anis uddin Nasir Scaling Online Social Networks 7/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionFeatures Local Semantics Load Balancing Fault Tolerant Dynamic Muhammad Anis uddin Nasir Scaling Online Social Networks 7/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionFeatures Local Semantics Load Balancing Fault Tolerant Dynamic Low Replication Overhead Muhammad Anis uddin Nasir Scaling Online Social Networks 7/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionArchitecture Partition Manager Muhammad Anis uddin Nasir Scaling Online Social Networks 8/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionArchitecture Partition Manager Directory Service Muhammad Anis uddin Nasir Scaling Online Social Networks 8/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionArchitecture Partition Manager Directory Service Local Directory Service Muhammad Anis uddin Nasir Scaling Online Social Networks 8/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionArchitecture Partition Manager Directory Service Local Directory Service Replication Manager Muhammad Anis uddin Nasir Scaling Online Social Networks 8/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionSPAR Algorithm Heuristic Muhammad Anis uddin Nasir Scaling Online Social Networks 9/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionSPAR Algorithm Heuristic Greedy Optimization Muhammad Anis uddin Nasir Scaling Online Social Networks 9/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionSPAR Algorithm Heuristic Greedy Optimization Local Search Muhammad Anis uddin Nasir Scaling Online Social Networks 9/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionSPAR Algorithm Heuristic Greedy Optimization Local Search Node Add/Remove Muhammad Anis uddin Nasir Scaling Online Social Networks 9/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionSPAR Algorithm Heuristic Greedy Optimization Local Search Node Add/Remove Edge Add/Remove Muhammad Anis uddin Nasir Scaling Online Social Networks 9/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionSPAR Algorithm Heuristic Greedy Optimization Local Search Node Add/Remove Edge Add/Remove 3 Configurations Muhammad Anis uddin Nasir Scaling Online Social Networks 9/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionSPAR Algorithm Heuristic Greedy Optimization Local Search Node Add/Remove Edge Add/Remove 3 Configurations Server Add/Remove Muhammad Anis uddin Nasir Scaling Online Social Networks 9/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionSPAR Algorithm Heuristic Greedy Optimization Local Search Node Add/Remove Edge Add/Remove 3 Configurations Server Add/Remove Redistribution Muhammad Anis uddin Nasir Scaling Online Social Networks 9/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionSPAR Algorithm Heuristic Greedy Optimization Local Search Node Add/Remove Edge Add/Remove 3 Configurations Server Add/Remove Redistribution Let it fill Muhammad Anis uddin Nasir Scaling Online Social Networks 9/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionSPAR Algorithm Heuristic Greedy Optimization Local Search Node Add/Remove Edge Add/Remove 3 Configurations Server Add/Remove Redistribution Let it fill Muhammad Anis uddin Nasir Scaling Online Social Networks 9/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionSPAR Algorithm Heuristic Greedy Optimization Local Search Node Add/Remove Edge Add/Remove 3 Configurations Server Add/Remove Redistribution Let it fill Muhammad Anis uddin Nasir Scaling Online Social Networks 9/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionSPAR Algorithm Heuristic Greedy Optimization Local Search Node Add/Remove Edge Add/Remove 3 Configurations Server Add/Remove Redistribution Let it fill Muhammad Anis uddin Nasir Scaling Online Social Networks 9/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionSPAR Algorithm Heuristic Greedy Optimization Local Search Node Add/Remove Edge Add/Remove 3 Configurations Server Add/Remove Redistribution Let it fill Muhammad Anis uddin Nasir Scaling Online Social Networks 9/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionFeatures Distributed Partitioning Muhammad Anis uddin Nasir Scaling Online Social Networks 10/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionFeatures Distributed Partitioning k-way Partitioning Muhammad Anis uddin Nasir Scaling Online Social Networks 10/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionFeatures Distributed Partitioning k-way Partitioning Load Balancing Muhammad Anis uddin Nasir Scaling Online Social Networks 10/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionFeatures Distributed Partitioning k-way Partitioning Load Balancing Low Inter-communication Overhead Muhammad Anis uddin Nasir Scaling Online Social Networks 10/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionFeatures Distributed Partitioning k-way Partitioning Load Balancing Low Inter-communication Overhead Local Search Muhammad Anis uddin Nasir Scaling Online Social Networks 10/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionOverview Sampling Policies Muhammad Anis uddin Nasir Scaling Online Social Networks 11/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionOverview Sampling Policies Local Muhammad Anis uddin Nasir Scaling Online Social Networks 11/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionOverview Sampling Policies Local Random Muhammad Anis uddin Nasir Scaling Online Social Networks 11/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionOverview Sampling Policies Local Random Hybrid Muhammad Anis uddin Nasir Scaling Online Social Networks 11/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionOverview Sampling Policies Local Random Hybrid Swapping Policies Muhammad Anis uddin Nasir Scaling Online Social Networks 11/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionOverview Sampling Policies Local Random Hybrid Swapping Policies Energy Function Muhammad Anis uddin Nasir Scaling Online Social Networks 11/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionOverview Sampling Policies Local Random Hybrid Swapping Policies Energy Function Simulated Annealing Muhammad Anis uddin Nasir Scaling Online Social Networks 11/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionOverview Sampling Policies Local Random Hybrid Swapping Policies Energy Function Simulated Annealing Algorithm Muhammad Anis uddin Nasir Scaling Online Social Networks 11/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionOverview Sampling Policies Local Random Hybrid Swapping Policies Energy Function Simulated Annealing Algorithm Hybrid Sampling Muhammad Anis uddin Nasir Scaling Online Social Networks 11/24
    • Motivation Algorithms SPAR Our Contribution JA-BE-JA Evaluation ConclusionOverview Sampling Policies Local Random Hybrid Swapping Policies Energy Function Simulated Annealing Algorithm Hybrid Sampling Simulated Annealing Muhammad Anis uddin Nasir Scaling Online Social Networks 11/24
    • Motivation Algorithms Challenges Our Contribution Proposed Algorithm Evaluation Conclusion1 Motivation2 Algorithms SPAR JA-BE-JA3 Our Contribution Challenges Proposed Algorithm4 Evaluation Datasets Implementation Results5 Conclusion Muhammad Anis uddin Nasir Scaling Online Social Networks 12/24
    • Motivation Algorithms Challenges Our Contribution Proposed Algorithm Evaluation ConclusionChallenges Global View Muhammad Anis uddin Nasir Scaling Online Social Networks 13/24
    • Motivation Algorithms Challenges Our Contribution Proposed Algorithm Evaluation ConclusionChallenges Global View Partition Manager Muhammad Anis uddin Nasir Scaling Online Social Networks 13/24
    • Motivation Algorithms Challenges Our Contribution Proposed Algorithm Evaluation ConclusionChallenges Global View Partition Manager Replication Overhead Muhammad Anis uddin Nasir Scaling Online Social Networks 13/24
    • Motivation Algorithms Challenges Our Contribution Proposed Algorithm Evaluation ConclusionChallenges Global View Partition Manager Replication Overhead Load Balancing Muhammad Anis uddin Nasir Scaling Online Social Networks 13/24
    • Motivation Algorithms Challenges Our Contribution Proposed Algorithm Evaluation ConclusionProposed Algorithm SPAR + JA-BE-JA Muhammad Anis uddin Nasir Scaling Online Social Networks 14/24
    • Motivation Algorithms Challenges Our Contribution Proposed Algorithm Evaluation ConclusionProposed Algorithm SPAR + JA-BE-JA Gossip Learning Muhammad Anis uddin Nasir Scaling Online Social Networks 14/24
    • Motivation Algorithms Challenges Our Contribution Proposed Algorithm Evaluation ConclusionProposed Algorithm SPAR + JA-BE-JA Gossip Learning Simulated Annealing Muhammad Anis uddin Nasir Scaling Online Social Networks 14/24
    • Motivation Algorithms Challenges Our Contribution Proposed Algorithm Evaluation ConclusionProposed Algorithm SPAR + JA-BE-JA Gossip Learning Simulated Annealing Optimal Replication Muhammad Anis uddin Nasir Scaling Online Social Networks 14/24
    • Motivation Algorithms Datasets Our Contribution Implementation Evaluation Results Conclusion1 Motivation2 Algorithms SPAR JA-BE-JA3 Our Contribution Challenges Proposed Algorithm4 Evaluation Datasets Implementation Results5 Conclusion Muhammad Anis uddin Nasir Scaling Online Social Networks 15/24
    • Motivation Algorithms Datasets Our Contribution Implementation Evaluation Results ConclusionDatasets Synthetic Graphs 0 http://snap.stanford.edu/data/ Muhammad Anis uddin Nasir Scaling Online Social Networks 16/24
    • Motivation Algorithms Datasets Our Contribution Implementation Evaluation Results ConclusionDatasets Synthetic Graphs Facebook Graphs 0 http://snap.stanford.edu/data/ Muhammad Anis uddin Nasir Scaling Online Social Networks 16/24
    • Motivation Algorithms Datasets Our Contribution Implementation Evaluation Results ConclusionDatasets Synthetic Graphs Randomized Facebook Graphs 0 https://gephi.org/ Muhammad Anis uddin Nasir Scaling Online Social Networks 17/24
    • Motivation Algorithms Datasets Our Contribution Implementation Evaluation Results ConclusionDatasets Synthetic Graphs Randomized Clustered Facebook Graphs 0 https://gephi.org/ Muhammad Anis uddin Nasir Scaling Online Social Networks 17/24
    • Motivation Algorithms Datasets Our Contribution Implementation Evaluation Results ConclusionDatasets Synthetic Graphs Randomized Clustered Highly Clustered Facebook Graphs 0 https://gephi.org/ Muhammad Anis uddin Nasir Scaling Online Social Networks 17/24
    • Motivation Algorithms Datasets Our Contribution Implementation Evaluation Results ConclusionDatasets Synthetic Graphs Randomized Clustered Highly Clustered Facebook Graphs 150 nodes, 3386 edges 0 https://gephi.org/ Muhammad Anis uddin Nasir Scaling Online Social Networks 17/24
    • Motivation Algorithms Datasets Our Contribution Implementation Evaluation Results ConclusionDatasets Synthetic Graphs Randomized Clustered Highly Clustered Facebook Graphs 150 nodes, 3386 edges 224 nodes, 6384 edges 0 https://gephi.org/ Muhammad Anis uddin Nasir Scaling Online Social Networks 17/24
    • Motivation Algorithms Datasets Our Contribution Implementation Evaluation Results ConclusionDatasets Synthetic Graphs Randomized Clustered Highly Clustered Facebook Graphs 150 nodes, 3386 edges 224 nodes, 6384 edges 786 nodes, 60050 edges 0 https://gephi.org/ Muhammad Anis uddin Nasir Scaling Online Social Networks 17/24
    • Motivation Algorithms Datasets Our Contribution Implementation Evaluation Results ConclusionImplementation SPAR Proposed Algorithm Metric Replication Overhead Muhammad Anis uddin Nasir Scaling Online Social Networks 18/24
    • Motivation Algorithms Datasets Our Contribution Implementation Evaluation Results ConclusionEvaluation of Replication Overhead Replication Factor Fault tolerance replicas reduce replication overhead Proposed Algorithm performs better than SPAR 0 replication overhead = number of replicas/number of users Muhammad Anis uddin Nasir Scaling Online Social Networks 19/24
    • Motivation Algorithms Datasets Our Contribution Implementation Evaluation Results ConclusionEvaluation of Replication Overhead Number of Servers Less Replication overhead in the case of proposed algorithm Proposed Algorithm performs better in the case of high clusterization 0 replication overhead = number of replicas/number of users Muhammad Anis uddin Nasir Scaling Online Social Networks 20/24
    • Motivation Algorithms Datasets Our Contribution Implementation Evaluation Results ConclusionEvaluation of Replication Overhead Number of Servers Less Replication overhead in the case of proposed algorithm Proposed Algorithm performs better in case of high clusterization 0 replication overhead = number of replicas/number of users Muhammad Anis uddin Nasir Scaling Online Social Networks 21/24
    • Motivation Algorithms Our Contribution Evaluation Conclusion1 Motivation2 Algorithms SPAR JA-BE-JA3 Our Contribution Challenges Proposed Algorithm4 Evaluation Datasets Implementation Results5 Conclusion Muhammad Anis uddin Nasir Scaling Online Social Networks 22/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionConclusion Distributed social-based partitioning Muhammad Anis uddin Nasir Scaling Online Social Networks 23/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionConclusion Distributed social-based partitioning Local Semantics Muhammad Anis uddin Nasir Scaling Online Social Networks 23/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionConclusion Distributed social-based partitioning Local Semantics Reduced Replication overhead compared to SPAR Muhammad Anis uddin Nasir Scaling Online Social Networks 23/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionConclusion Distributed social-based partitioning Local Semantics Reduced Replication overhead compared to SPAR Better load balancing using k-way partitioning Muhammad Anis uddin Nasir Scaling Online Social Networks 23/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionConclusion Distributed social-based partitioning Local Semantics Reduced Replication overhead compared to SPAR Better load balancing using k-way partitioning Transparent Scaling Muhammad Anis uddin Nasir Scaling Online Social Networks 23/24
    • Motivation Algorithms Our Contribution Evaluation ConclusionScaling Online Social Networks: extended SPAR using Gossip Learning Presented by: Muhammad Anis uddin Nasir Coworker: Maria Stylianou Supervised by: Sarunas Girdzijauskas KTH Royal Institute of Technology December 5, 2012 Muhammad Anis uddin Nasir Scaling Online Social Networks 24/24