Pregel

2,063 views

Published on

  • Be the first to comment

  • Be the first to like this

Pregel

  1. 1. PREGELA  Systems  for  Large  Scale  Graph  Processing by Iuliia Proskurnia ber 2012 ovem 6 th of M
  2. 2. GRAPHS  ARE  EVERYWHERE Graphs  Examples 2
  3. 3. GRAPHS  ARE  EVERYWHERE Algorithms 3
  4. 4. OutlineMo#va#onBasic  Concepts  in  designImplementa#on  DetailsEvalua#onConclusions 4
  5. 5. MOTIVATION Large  Graph  Processing No Such System Exist Single Machine Algorithms Parallel Solution 5
  6. 6. MAPREDUCE  LIKE  SOLUTION MapReduce  is  great  :) 6
  7. 7. MAPREDUCE  LIKE  SOLUTION MapReduce  is  great  :) BUT 6
  8. 8. MAPREDUCE  LIKE  SOLUTION MapReduce  is  great  :) BUT Pregel 6
  9. 9. Inspira@onValiant’s  Bulk  Synchronous   Parallel  Model 7
  10. 10. BASIC  DESIGN  CONCEPTS 8
  11. 11. VERTEX-­‐CENTRIC  APPROACH Uses  supersteps  for  computa@on Send/Receive Messages Change the state Modify topology VERTEX Termination? 9
  12. 12. MAXIMUM  VALUE  EXAMPLE Chicken  Chicken • Dotted Arrows - messages • Grey Nodes - InActive 10
  13. 13. API        DETAILS Combiners 11
  14. 14. API        DETAILS Aggregators 12
  15. 15. IMPLEMENTATION  DETAILS 13
  16. 16. IMPLEMENTATION Master  is  chosen 14
  17. 17. IMPLEMENTATION Master  is  chosen Cluster Management System’s name service 14
  18. 18. IMPLEMENTATION Par@@on hash(VertexID) mod R hash(VertexID) mod R hash(VertexID) mod R 15
  19. 19. IMPLEMENTATION Reading  the  input 16
  20. 20. IMPLEMENTATION Reading  the  input GFS,BigTable 16
  21. 21. IMPLEMENTATION SuperStep 17
  22. 22. IMPLEMENTATION SuperStep 17
  23. 23. IMPLEMENTATION SuperStep Terminationif (all VoteToHalt) { terminate();} 17
  24. 24. IMPLEMENTATION Saving  the  results Save graph state 18
  25. 25. FAULT-­‐TOLERANCE CheckPoin@ng.  Chicken  Chicken. 19
  26. 26. FAULT-­‐TOLERANCE CheckPoin@ng.  Chicken  Chicken. 19
  27. 27. EVALUATION 20
  28. 28. EVALUATION Number  of  Worker  Tasks 300 PCs Multicore Billion Vertices Binary Tree 21
  29. 29. EVALUATION Number  of  Ver@ces 300 PCs Multicore Tree with 127 average node degree 22
  30. 30. ConclusionVertex-­‐Centric  ApproachComputa#on  over  SuperStepsUsability  and  ScalabilityFault  Tolerance  with  checkpointsPerformance  -­‐  almost  linear  to  the  size  of  the  graph 23

×