The document provides an overview of graph-tool, a powerful and efficient network analysis tool for Python that utilizes fast algorithms and visualization capabilities. It outlines the process of creating, visualizing, and analyzing graphs, and discusses various centrality measures and algorithms applicable in network analysis. Additionally, it emphasizes the need for defining problems graphically, parsing raw data, and employing filtering and visualization techniques to derive insights.

1. Graph-Tool
The Efficient Network
AnalyzingTool for Python
Mosky

2. Graph-Tool
in Practice
Mosky

3. MOSKY
3

4. MOSKY
• Python Charmer at Pinkoi
3

5. MOSKY
• Python Charmer at Pinkoi
• An author of the Python packages:
• MoSQL, Clime, Uniout, ZIPCodeTW, …
3

6. MOSKY
• Python Charmer at Pinkoi
• An author of the Python packages:
• MoSQL, Clime, Uniout, ZIPCodeTW, …
• A speaker of the conferences
• 2014: PyCon APAC, OSDC; 2013: PyCon APAC, PyConTW, COSCUP, …
3

7. MOSKY
• Python Charmer at Pinkoi
• An author of the Python packages:
• MoSQL, Clime, Uniout, ZIPCodeTW, …
• A speaker of the conferences
• 2014: PyCon APAC, OSDC; 2013: PyCon APAC, PyConTW, COSCUP, …
• A Python instructor
3

8. MOSKY
• Python Charmer at Pinkoi
• An author of the Python packages:
• MoSQL, Clime, Uniout, ZIPCodeTW, …
• A speaker of the conferences
• 2014: PyCon APAC, OSDC; 2013: PyCon APAC, PyConTW, COSCUP, …
• A Python instructor
• mosky.tw
3

9. OUTLINE
4

10. OUTLINE
• Introduction
4

11. OUTLINE
• Introduction
• Create Graph
4

12. OUTLINE
• Introduction
• Create Graph
• Visualize Graph
4

13. OUTLINE
• Introduction
• Create Graph
• Visualize Graph
• Analyze Graph
4

14. OUTLINE
• Introduction
• Create Graph
• Visualize Graph
• Analyze Graph
• Conclusion
4

15. INTRODUCTION

16. GRAPH-TOOL
6

17. GRAPH-TOOL
• It's for analyzing graph.
6

18. GRAPH-TOOL
• It's for analyzing graph.
• Fast. It bases on
Boost Graph in C++.
6

19. GRAPH-TOOL
• It's for analyzing graph.
• Fast. It bases on
Boost Graph in C++.
• Powerful visualization
6

20. GRAPH-TOOL
• It's for analyzing graph.
• Fast. It bases on
Boost Graph in C++.
• Powerful visualization
• Lot of useful algorithms
6

21. GET GRAPH-TOOL
7

22. GET GRAPH-TOOL
• Super easy on Debian / Ubuntu
• http://graph-tool.skewed.de/download#debian
7

23. GET GRAPH-TOOL
• Super easy on Debian / Ubuntu
• http://graph-tool.skewed.de/download#debian
• Super hard on Mac
• http://graph-tool.skewed.de/download#macos
• Install the dependencies by homebrew and pip.
Then compile it from source.
• Note it may take you 3~4 hours. I warned you!
7

24. CREATE GRAPH

25. BEFORE STARTING
9

26. BEFORE STARTING
• Define your problem.
9

27. BEFORE STARTING
• Define your problem.
• Convert it into a graphic form.
9

28. BEFORE STARTING
• Define your problem.
• Convert it into a graphic form.
• Parse raw data.
9

29. MY PROBLEM
10

30. MY PROBLEM
• To improve the duration of an online marketplace.
10

31. MY PROBLEM
• To improve the duration of an online marketplace.
• What's product browsing flow that users prefer?
10

32. IN GRAPHIC FORM
11
What Weight
Vertex Product Count
Edge
Directed
Browsing
Count

33. PARSING
12

34. PARSING
• Regular expression
• Filter garbages.
12

35. PARSING
• Regular expression
• Filter garbages.
• Sorting
12

36. PARSING
• Regular expression
• Filter garbages.
• Sorting
• Pickle
• HIGHEST_PROTOCOL
• Use tuple to save space/time.
• Save into serial files.
12

37. VERTEX AND EDGE
import graph_tool.all as gt
!
g = gt.Graph()
v1 = g.add_vertex()
v2 = g.add_vertex()
e = g.add_edge(v1, v2)
13

38. PROPERTY
v_count_p = g.new_vertex_property('int')
!
# store it in our graph, optionally
g.vp['count'] = v_count_p
14

39. FASTER IMPORT
from graph_tool import Graph
15

40. COUNTING
name_v_map = {}
for name in names:
v = name_v_map.get(name)
if v is None:
v = g.add_vertex()
v_count_p[v] = 0
name_v_map[name] = v
v_count_p[v] += 1
16

41. VISUALIZE GRAPH

42. THE SIMPLEST
gt.graph_draw(
g,
output_path = 'output.pdf',
)
!
gt.graph_draw(
g,
output_path = 'output.png',
)
18

43. 19

44. USE CONSTANTS
SIZE = 400
V_SIZE = SIZE / 20.
E_PWIDTH = V_SIZE / 4.
gt.graph_draw(
…
output_size = (SIZE, SIZE),
vertex_size = V_SIZE,
edge_pen_width = E_PWDITH,
)
20

45. 21

46. USE PROP_TO_SIZE
v_size_p = gt.prop_to_size(
v_count_p,
MI_V_SIZE,
MA_V_SIZE,
)
…
gt.graph_draw(
…
vertex_size = v_size_p,
edge_pen_width = e_pwidth_p,
)
22

47. 23

48. USE FILL_COLOR
gt.graph_draw(
…
vertex_fill_color = v_size_p,
)
24

49. 25

50. ANALYZE GRAPH

51. CHOOSE AN ALGORITHM
27

52. CHOOSE AN ALGORITHM
• Search algorithms
• BFS search …
27

53. CHOOSE AN ALGORITHM
• Search algorithms
• BFS search …
• Assessing graph topology
• shortest path …
27

54. CHOOSE AN ALGORITHM
• Search algorithms
• BFS search …
• Assessing graph topology
• shortest path …
• Centrality measures
• pagerank, betweenness, closeness …
27

55. 28

56. • Maximum flow algorithms
28

57. • Maximum flow algorithms
• Community structures
28

58. • Maximum flow algorithms
• Community structures
• Clustering coefficients
28

59. CENTRALITY MEASURES
29

60. CENTRALITY MEASURES
• Degree centrality
• the number of links incident upon a node
• the immediate risk of taking a node out
29

61. CENTRALITY MEASURES
• Degree centrality
• the number of links incident upon a node
• the immediate risk of taking a node out
• Closeness centrality
• sum of a node's distances to all other nodes
• the cost to spread information to all other nodes
29

62. 30

63. • Betweenness centrality
• the number of times a node acts as a bridge
• the control of a node on the communication
between other nodes
30

64. • Betweenness centrality
• the number of times a node acts as a bridge
• the control of a node on the communication
between other nodes
• Eigenvector centrality
• the influence of a node in a network
• Google's PageRank is a variant of the Eigenvector
centrality measure
30

65. MY CHOICE
31

66. MY CHOICE
• Centrality measures - Closeness centrality
31

67. MY CHOICE
• Centrality measures - Closeness centrality
• Get the products are easier to all other products.
31

68. CALCULATE CLOSENESS
!
!
e_icount_p = g.new_edge_property('int')
e_icount_p.a = e_count_p.a.max()-e_count_p.a
!
v_cl_p = closeness(g, weight=e_icount_p)
!
import numpy as np
v_cl_p.a = np.nan_to_num(v_cl_p.a)
32

69. DRAW CLOSENESS
v_cl_size_p = gt.prop_to_size(
v_cl_p,
MI_V_SIZE,
MA_V_SIZE,
)
…
gt.graph_draw(
…
vertex_fill_color = v_cl_size_p,
)
33

70. 34

71. ONTHE FLY FILTERING
!
v_pck_p = g.new_vertex_property('bool')
v_pck_p.a = v_count_p.a > v_count_p.a.mean()
!
g.set_vertex_filter(v_pck_p)
# g.set_vertex_filter(None) # unset
35

72. 36

73. TOP N
t10_idxs = v_count_p.a.argsort()[-10:][::-1]
!
t1_idx = t10_idxs[0]
t1_v = g.vertex(t1_idx)
t1_name = v_name_p[t1_v]
t1_count = v_count_p[t1_v]
37

74. SFDF LAYOUT
gt.graph_draw(
…
pos = gt.sfdp_layout(g),
)
38

75. 39

76. gt.graph_draw(
…
pos = gt.sfdp_layout(
g, eweight=e_count_p
),
)
!
gt.graph_draw(
…
pos = gt.sfdp_layout(
g,
eweight=e_count_p, vweight=v_count_p
),
)
40

77. 41

78. 42

79. 43

80. FR LAYOUT
gt.graph_draw(
…
pos = gt.fruchterman_reingold_layout(g),
)
!
gt.graph_draw(
…
pos = gt.fruchterman_reingold_layout(
g, weight=e_count_p
),
)
44

81. 45

82. 46

83. 47

84. ARF LAYOUT
gt.graph_draw(
…
pos = gt.arf_layout(g),
)
!
gt.graph_draw(
…
pos = gt.arf_layout(
g, weight=e_count_p
),
)
48

85. 49

86. 50

87. 51

88. MY GRAPH

89. 53

90. CONCLUSION

91. 55
CONCLUSION

92. • Define problem in graphic form.
55
CONCLUSION

93. • Define problem in graphic form.
• Parse raw data.
• Watch out!
Your data will bite you. →
55
CONCLUSION

94. • Define problem in graphic form.
• Parse raw data.
• Watch out!
Your data will bite you. →
• Visualize to understand.
55
CONCLUSION

95. • Define problem in graphic form.
• Parse raw data.
• Watch out!
Your data will bite you. →
• Visualize to understand.
• Choose a proper algorithms.
55
CONCLUSION

96. • Define problem in graphic form.
• Parse raw data.
• Watch out!
Your data will bite you. →
• Visualize to understand.
• Choose a proper algorithms.
• Filter data which interest you.
55
CONCLUSION

97. • Define problem in graphic form.
• Parse raw data.
• Watch out!
Your data will bite you. →
• Visualize to understand.
• Choose a proper algorithms.
• Filter data which interest you.
• Visualize again to convince.
55
CONCLUSION

98. • Define problem in graphic form.
• Parse raw data.
• Watch out!
Your data will bite you. →
• Visualize to understand.
• Choose a proper algorithms.
• Filter data which interest you.
• Visualize again to convince.
• mosky.tw
55
CONCLUSION

99. DEMO

100. COSCUP 2014
2014.07.19 - 2014.07.20 | Academia Sinica,Taipei,Taiwan

101. LINKS
• Quick start using graph-tool
http://graph-tool.skewed.de/static/doc/quickstart.html
• Learn more about Graph object
http://graph-tool.skewed.de/static/doc/graph_tool.html
• The possible property value types
http://graph-tool.skewed.de/static/doc/
graph_tool.html#graph_tool.PropertyMap
58

102. • Graph drawing and layout
http://graph-tool.skewed.de/static/doc/draw.html
• Available subpackages - Graph-Tool
http://graph-tool.skewed.de/static/doc/
graph_tool.html#available-subpackages
• Centrality - Wiki
http://en.wikipedia.org/wiki/Centrality
• NumPy Reference
http://docs.scipy.org/doc/numpy/reference/
59