∙ To design and implement a generalized distributed architecture which can solve a typical set of problems ∙ To use this architecture and compute all connected subgraphs of a given graph

1. distributed graph algorithms
Generalized Architecture For Some Graph Problems
Abhilash Kumar and Saurav Kumar
November 10, 2015
Indian Institute of Technology Kanpur

2. problem statement

3. problem statement
∙ Compute all connected sub-graphs of a given graph,
in a distributed environment
2

4. problem statement
∙ Compute all connected sub-graphs of a given graph,
in a distributed environment
∙ Develop a generalized architecture to solve similar
graph problems
2

5. motivation

6. motivation
∙ Exponential number of connected sub-graphs of a
given graph
4

7. motivation
∙ Exponential number of connected sub-graphs of a
given graph
∙ Necessity to build distributed systems which utilize
the worldwide plethora of distributed resources
4

8. approach

9. approach
Insights
∙ Connected sub-graphs exhibit sub-structure
6

10. approach
Insights
∙ Connected sub-graphs exhibit sub-structure
∙ Extend smaller sub-graphs by adding an outgoing edge to
generate larger sub-graphs
6

11. approach
Insights
∙ Connected sub-graphs exhibit sub-structure
∙ Extend smaller sub-graphs by adding an outgoing edge to
generate larger sub-graphs
∙ Base cases are sub-graphs represented by all the edges of the
graph
6

12. approach
Algorithm to compute all connected sub-graphs
∙ Initialize:
7

13. approach
Algorithm to compute all connected sub-graphs
∙ Initialize:
∙ Queue Q
7

14. approach
Algorithm to compute all connected sub-graphs
∙ Initialize:
∙ Queue Q
∙ For each edge in G
7

15. approach
Algorithm to compute all connected sub-graphs
∙ Initialize:
∙ Queue Q
∙ For each edge in G
∙ Create a sub-graph G’ representing the edge
7

16. approach
Algorithm to compute all connected sub-graphs
∙ Initialize:
∙ Queue Q
∙ For each edge in G
∙ Create a sub-graph G’ representing the edge
∙ Push G’ to Q
7

17. approach
Algorithm to compute all connected sub-graphs
∙ Initialize:
∙ Queue Q
∙ For each edge in G
∙ Create a sub-graph G’ representing the edge
∙ Push G’ to Q
∙ Process:
7

18. approach
Algorithm to compute all connected sub-graphs
∙ Initialize:
∙ Queue Q
∙ For each edge in G
∙ Create a sub-graph G’ representing the edge
∙ Push G’ to Q
∙ Process:
∙ while Q is not empty
7

19. approach
Algorithm to compute all connected sub-graphs
∙ Initialize:
∙ Queue Q
∙ For each edge in G
∙ Create a sub-graph G’ representing the edge
∙ Push G’ to Q
∙ Process:
∙ while Q is not empty
∙ G = Q.pop()
7

20. approach
Algorithm to compute all connected sub-graphs
∙ Initialize:
∙ Queue Q
∙ For each edge in G
∙ Create a sub-graph G’ representing the edge
∙ Push G’ to Q
∙ Process:
∙ while Q is not empty
∙ G = Q.pop()
∙ Save G
7

21. approach
Algorithm to compute all connected sub-graphs
∙ Initialize:
∙ Queue Q
∙ For each edge in G
∙ Create a sub-graph G’ representing the edge
∙ Push G’ to Q
∙ Process:
∙ while Q is not empty
∙ G = Q.pop()
∙ Save G
∙ For each outgoing edge E of G
G’ = G U E
if G’ has not been seen yet
Push G’ to Q
7

22. approach
Algorithm to compute all connected sub-graphs
∙ Initialize:
∙ Queue Q
∙ For each edge in G
∙ Create a sub-graph G’ representing the edge
∙ Push G’ to Q
∙ Process:
∙ while Q is not empty
∙ G = Q.pop()
∙ Save G
∙ For each outgoing edge E of G
G’ = G U E
if G’ has not been seen yet
Push G’ to Q
7

23. approach
Algorithm to compute all connected sub-graphs
∙ Initialize:
∙ Queue Q
∙ For each edge in G
∙ Create a sub-graph G’ representing the edge
∙ Push G’ to Q
∙ Process:
∙ while Q is not empty
∙ G = Q.pop()
∙ Save G
∙ For each outgoing edge E of G
G’ = G U E
if G’ has not been seen yet
Push G’ to Q
7

24. approach
Figure: Generating initial sub-graphs from a given graph
8

25. approach
Figure: Extending a sub-graph to generate new sub-graphs
9

26. approach
Figure: Consider only unique sub-graphs generated for further processing
10

27. architecture

28. architecture
Master-Slave Architecture
∙ Commonly used approach for parallel and distributed
applications
12

29. architecture
Master-Slave Architecture
∙ Commonly used approach for parallel and distributed
applications
∙ Message passing to communicate over TCP
12

30. architecture
Master-Slave Architecture
∙ Commonly used approach for parallel and distributed
applications
∙ Message passing to communicate over TCP
∙ Master assigns tasks to slaves and finally collects the results
12

31. architecture
Master-Slave Architecture
∙ Commonly used approach for parallel and distributed
applications
∙ Message passing to communicate over TCP
∙ Master assigns tasks to slaves and finally collects the results
∙ A Task object represents a sub-graph which contains all
necessary information to process that sub-graph
12

32. architecture
Master-Slave Architecture
∙ Commonly used approach for parallel and distributed
applications
∙ Message passing to communicate over TCP
∙ Master assigns tasks to slaves and finally collects the results
∙ A Task object represents a sub-graph which contains all
necessary information to process that sub-graph
∙ A slave may request a task from other slaves when its task
queue is empty and processing ends when all task queues are
empty
12

33. architecture
Task, Queue and Bloom filter
∙ A task has these information:
13

34. architecture
Task, Queue and Bloom filter
∙ A task has these information:
∙ A list of vertices that are already in the sub-graph
13

35. architecture
Task, Queue and Bloom filter
∙ A task has these information:
∙ A list of vertices that are already in the sub-graph
∙ A list of edges that can be extended in the next step
13

36. architecture
Task, Queue and Bloom filter
∙ A task has these information:
∙ A list of vertices that are already in the sub-graph
∙ A list of edges that can be extended in the next step
∙ Task Queue
13

37. architecture
Task, Queue and Bloom filter
∙ A task has these information:
∙ A list of vertices that are already in the sub-graph
∙ A list of edges that can be extended in the next step
∙ Task Queue
∙ Each slave has a task queue
13

38. architecture
Task, Queue and Bloom filter
∙ A task has these information:
∙ A list of vertices that are already in the sub-graph
∙ A list of edges that can be extended in the next step
∙ Task Queue
∙ Each slave has a task queue
∙ Slave picks up a task from its task queue and processes it
13

39. architecture
Task, Queue and Bloom filter
∙ A task has these information:
∙ A list of vertices that are already in the sub-graph
∙ A list of edges that can be extended in the next step
∙ Task Queue
∙ Each slave has a task queue
∙ Slave picks up a task from its task queue and processes it
∙ Newly generated unique tasks are pushed into the task queue
13

40. architecture
Task, Queue and Bloom filter
∙ A task has these information:
∙ A list of vertices that are already in the sub-graph
∙ A list of edges that can be extended in the next step
∙ Task Queue
∙ Each slave has a task queue
∙ Slave picks up a task from its task queue and processes it
∙ Newly generated unique tasks are pushed into the task queue
∙ Bloom filter
13

41. architecture
Task, Queue and Bloom filter
∙ A task has these information:
∙ A list of vertices that are already in the sub-graph
∙ A list of edges that can be extended in the next step
∙ Task Queue
∙ Each slave has a task queue
∙ Slave picks up a task from its task queue and processes it
∙ Newly generated unique tasks are pushed into the task queue
∙ Bloom filter
∙ We use Bloom filter to check uniqueness of the newly generated
tasks (i.e. sub-graphs)
13

42. architecture
Task, Queue and Bloom filter
∙ A task has these information:
∙ A list of vertices that are already in the sub-graph
∙ A list of edges that can be extended in the next step
∙ Task Queue
∙ Each slave has a task queue
∙ Slave picks up a task from its task queue and processes it
∙ Newly generated unique tasks are pushed into the task queue
∙ Bloom filter
∙ We use Bloom filter to check uniqueness of the newly generated
tasks (i.e. sub-graphs)
∙ Bloom filter is also distributed so that none of the servers get
loaded
13

43. architecture
Bloom Filter Vs Hashing
∙ Used bloom filter because its very space efficient
14

44. architecture
Bloom Filter Vs Hashing
∙ Used bloom filter because its very space efficient
∙ Space required to get error probability of p is
−n × ln p
(ln 2)2
bits
14

45. architecture
Bloom Filter Vs Hashing
∙ Used bloom filter because its very space efficient
∙ Space required to get error probability of p is
−n × ln p
(ln 2)2
bits
∙ Error probability can be reduced with very little extra space
14

46. architecture
Bloom Filter Vs Hashing
∙ Used bloom filter because its very space efficient
∙ Space required to get error probability of p is
−n × ln p
(ln 2)2
bits
∙ Error probability can be reduced with very little extra space
∙ Hashing can be used to make the algorithm deterministic
14

47. architecture
Bloom Filter Vs Hashing
∙ Used bloom filter because its very space efficient
∙ Space required to get error probability of p is
−n × ln p
(ln 2)2
bits
∙ Error probability can be reduced with very little extra space
∙ Hashing can be used to make the algorithm deterministic
∙ Bloom filter can also be parallelized whereas Hashing cannot be.
14

48. architecture
How to use this architecture?
∙ Two functions required: initialize and process
15

49. architecture
How to use this architecture?
∙ Two functions required: initialize and process
∙ Initialize generates initial tasks. Master randomly assigns these
tasks to the slaves.
15

50. architecture
How to use this architecture?
∙ Two functions required: initialize and process
∙ Initialize generates initial tasks. Master randomly assigns these
tasks to the slaves.
∙ Process defines a procedure that will generate new tasks from a
given task (extend sub-graph in our case)
15

51. architecture
How to use this architecture?
∙ Two functions required: initialize and process
∙ Initialize generates initial tasks. Master randomly assigns these
tasks to the slaves.
∙ Process defines a procedure that will generate new tasks from a
given task (extend sub-graph in our case)
15

52. architecture
How to use this architecture?
∙ Two functions required: initialize and process
∙ Initialize generates initial tasks. Master randomly assigns these
tasks to the slaves.
∙ Process defines a procedure that will generate new tasks from a
given task (extend sub-graph in our case)
15

53. architecture
Fitting the connected sub-graph problem
∙ Initialize creates all the tasks (sub-graphs) with one edge.
16

54. architecture
Fitting the connected sub-graph problem
∙ Initialize creates all the tasks (sub-graphs) with one edge.
∙ Process takes a connected sub-graph and extends it by adding
all extend-able edges, one at a time
16

55. simulation

56. simulation
Simulation for testing
∙ Used 2 machines, say H and L.
18

57. simulation
Simulation for testing
∙ Used 2 machines, say H and L.
∙ H: 24 core, 200 GB, Xeon E5645 @ 2.40GHz
18

58. simulation
Simulation for testing
∙ Used 2 machines, say H and L.
∙ H: 24 core, 200 GB, Xeon E5645 @ 2.40GHz
∙ L: 4 core, 8 GB, i5-3230M CPU @ 2.60GHz
18

59. simulation
Simulation for testing
∙ Used 2 machines, say H and L.
∙ H: 24 core, 200 GB, Xeon E5645 @ 2.40GHz
∙ L: 4 core, 8 GB, i5-3230M CPU @ 2.60GHz
∙ Opened multiple ports (6 on H, 2 on L) to mimic 8 slave servers.
18

60. simulation
Simulation for testing
∙ Used various combinations of number of slaves on H and L
19

61. simulation
Simulation for testing
∙ Used various combinations of number of slaves on H and L
∙ Used 2 tree graphs G(14, 13) and G(16, 15): easy to match results
19

62. simulation
Simulation for testing
∙ Used various combinations of number of slaves on H and L
∙ Used 2 tree graphs G(14, 13) and G(16, 15): easy to match results
∙ Collected data for number of tasks processed by each slave and
number of hash-check queries made by each slave.
19

63. simulation
Simulation for testing
∙ Used various combinations of number of slaves on H and L
∙ Used 2 tree graphs G(14, 13) and G(16, 15): easy to match results
∙ Collected data for number of tasks processed by each slave and
number of hash-check queries made by each slave.
∙ Collected total running time data for both graphs, including the
cases of network fault.
19

64. results

65. results
Figure: Number of hash check queries vs number of slaves for G(14, 13)
21

66. results
Figure: Distribution of number of tasks processed by slaves for G(14, 13)
22

67. results
Figure: Distribution of number of tasks processed by slaves for G(14, 13)
23

68. results
Figure: Distribution of number of tasks processed by slaves for G(14, 13)
24

69. results
Figure: Number of hash check queries vs number of slaves for G(16, 15)
25

70. results
Figure: Distribution of number of tasks processed by slaves for G(16, 15)
26

71. results
Figure: Distribution of number of tasks processed by slaves for G(16, 15)
27

72. results
Figure: Distribution of number of tasks processed by slaves for G(16, 15)
28

73. results
Actual Running Time
∙ Network faults happened, specially due to fewer physical
machines
29

74. results
Actual Running Time
∙ Network faults happened, specially due to fewer physical
machines
∙ The architecture recovers from these faults, but a lot of time is
consumed
29

75. results
Actual Running Time
∙ Network faults happened, specially due to fewer physical
machines
∙ The architecture recovers from these faults, but a lot of time is
consumed
∙ For G(14, 13), running time ranged from 15s to 91s
29

76. results
Actual Running Time
∙ Network faults happened, specially due to fewer physical
machines
∙ The architecture recovers from these faults, but a lot of time is
consumed
∙ For G(14, 13), running time ranged from 15s to 91s
∙ For G(15, 14), running time ranged from 255s to 447s
29

77. results
Actual Running Time
∙ Network faults happened, specially due to fewer physical
machines
∙ The architecture recovers from these faults, but a lot of time is
consumed
∙ For G(14, 13), running time ranged from 15s to 91s
∙ For G(15, 14), running time ranged from 255s to 447s
∙ These are the cases when process function doesn’t do
additional computation per subgraph.
29

78. results
Figure: Running time when process does addition computation(10ms)
30

79. advantages

80. advantages
Advantages
∙ Highly scalable
32

81. advantages
Advantages
∙ Highly scalable
∙ More slaves can be added easily
32

82. advantages
Advantages
∙ Highly scalable
∙ More slaves can be added easily
∙ Performance increases with number of slaves
32

83. advantages
Advantages
∙ Highly scalable
∙ More slaves can be added easily
∙ Performance increases with number of slaves
∙ Even distribution of tasks: efficient machines process more tasks
32

84. advantages
Advantages
∙ Highly scalable
∙ More slaves can be added easily
∙ Performance increases with number of slaves
∙ Even distribution of tasks: efficient machines process more tasks
∙ Architecture is very reusable
32

85. advantages
Advantages
∙ Highly scalable
∙ More slaves can be added easily
∙ Performance increases with number of slaves
∙ Even distribution of tasks: efficient machines process more tasks
∙ Architecture is very reusable
∙ Many other problems can be solved using this architecture
32

86. advantages
Advantages
∙ Highly scalable
∙ More slaves can be added easily
∙ Performance increases with number of slaves
∙ Even distribution of tasks: efficient machines process more tasks
∙ Architecture is very reusable
∙ Many other problems can be solved using this architecture
∙ Only need to provide 2 functions: initialize and process
32

87. advantages
Advantages
∙ Highly scalable
∙ More slaves can be added easily
∙ Performance increases with number of slaves
∙ Even distribution of tasks: efficient machines process more tasks
∙ Architecture is very reusable
∙ Many other problems can be solved using this architecture
∙ Only need to provide 2 functions: initialize and process
∙ Network fault tolerant
32

88. advantages
Other problems that can be solved using this
paradigm
∙ Generating all cliques, paths, cycles, sub-trees, spanning
sub-trees
33

89. advantages
Other problems that can be solved using this
paradigm
∙ Generating all cliques, paths, cycles, sub-trees, spanning
sub-trees
∙ Can also solve few classical NP problems like finding all
maximal cliques and TSP
33

90. future works

91. future works
Further improvements
∙ Implement parallelized bloom filter
35

92. future works
Further improvements
∙ Implement parallelized bloom filter
∙ Parallely solving tasks in a slave (on powerful servers)
35

93. future works
Further improvements
∙ Implement parallelized bloom filter
∙ Parallely solving tasks in a slave (on powerful servers)
∙ Handle slave/master failures
35

94. future works
Further improvements
∙ Implement parallelized bloom filter
∙ Parallely solving tasks in a slave (on powerful servers)
∙ Handle slave/master failures
∙ Using file I/O to store task queue for large problems
35

95. future works
Further improvements
∙ Implement parallelized bloom filter
∙ Parallely solving tasks in a slave (on powerful servers)
∙ Handle slave/master failures
∙ Using file I/O to store task queue for large problems
∙ Exploring this paradigm to solve other problems
35

96. conclusion

97. conclusion
Conclusion
∙ The algorithm is very efficient, total computation is not greater
than m * T, where T is the minimum computation required to
find all sub-graphs and m is number of edges.
37

98. conclusion
Conclusion
∙ The algorithm is very efficient, total computation is not greater
than m * T, where T is the minimum computation required to
find all sub-graphs and m is number of edges.
∙ In practice time complexity is c*T where c is much smaller.
Bound on c can be improved to min(m, log T).
37

99. conclusion
Conclusion
∙ The algorithm is very efficient, total computation is not greater
than m * T, where T is the minimum computation required to
find all sub-graphs and m is number of edges.
∙ In practice time complexity is c*T where c is much smaller.
Bound on c can be improved to min(m, log T).
∙ As we are interested in finding all connected sub-graph, T better
not be very large.
37

100. conclusion
Conclusion
∙ The algorithm is very efficient, total computation is not greater
than m * T, where T is the minimum computation required to
find all sub-graphs and m is number of edges.
∙ In practice time complexity is c*T where c is much smaller.
Bound on c can be improved to min(m, log T).
∙ As we are interested in finding all connected sub-graph, T better
not be very large.
∙ The architecture help us solve this problem in much scalable
manner and significantly reduces the time of computation
provided good infrastructure and better implementation.
37

101. Questions?
Implementation of the algorithm and the architecture available at
github.com/abhilak/DGA
Slides created using Beamer(mtheme) and plot.ly on ShareLaTeX
38

102. Thank You
39