The Pathology of Graph Databases

1. The Pathology of Graph Databases Marko A. Rodriguez Graph Systems Architect http://markorodriguez.com http://twitter.com/twarko http://tinkerpop.com TinkerPop WindyCityDB - Chicago, Illinois – June 25, 2011

2. Abstract There is nothing more fascinating and utterly mind- bending than traversing a graph. Those who succumb to this data processing pattern euphorically suffer from graph pathology. This is a case study of the Graph Addict. Gremlin G = (V, E) What is presented is as of Gremlin 1.1 (Released July 15, 2011)

3. What you are about to see may disturb you. It has infected Neo4j, OrientDB, DEX, RDF Sail, TinkerGraph, and ReXster. reXster ...it will infect others.

4. ? ? ?

5. ~$ cd /tmp/graphdb

6. graphdb$ gremlin

7. ,,,/ (o o) -----o00o(_)-o00o----- gremlin>

8. The Basics

9. Gremlin is a Groovy DSL and as such, has native JVM access. 'Gremlin is a Groovy DSL and as such, has native JVM access.'

10. 3 1+2

11. gremlin 'gremlin'

12. emlin 'gremlin'.substring(2)

13. [1,2,3,4,5] list = [1,2,3,4,5]

14. key1=value key2=123 map = [key1:'value',key2:123]

15. <modem sound/> socket = new Socket('127.0.0.1',8182)

16. true TextAnalysis.isMean('You are that which is not good.')

17. Graph Traversing

18. /tmp/graphdb g = new Neo4jGraph('/tmp/graphdb')

19. /tmp/graphdb g

20. 1 g.v(1)

21. name=marko 1 g.v(1).name

22. 1 g.v(1)

23. city=santafe 1 g.v(1).city

24. 1 g.v(1)

25. name=marko city=santafe 1 g.v(1).map()

26. 1 g.v(1)

27. created 1 knows knows g.v(1).outE

29. null created since=2010 1 knows since=2005 knows g.v(1).outE.since

31. 2 created 1 knows 3 knows 4 g.v(1).outE.inV

32. 2 name=rexster created 1 knows 3 name=stephen knows 4 name=peter g.v(1).outE.inV.name

33. /tmp/graphdb g

34. 1 g.v(1)

36. created 1 knows knows g.v(1).outE[[label:'knows']]

37. 1 g.v(1)

39. created 1 knows knows g.v(1).outE.filter{it.label=='knows'}

40. 2 g.v(1).outE.filter{it.label=='knows'}.count()

42. created 1 knows knows g.v(1).outE.filter{it.label=='knows' & since > 2006}

43. [OutEdgesPipe, LabelFilterPipe(NOT_EQUAL,knows), InVertexPipe] g.v(1).outE[[label:'knows']].inV.toString()

44. 1 g.v(1)

45. 1 knows knows g.v(1).outE('knows')

46. 1 knows 3 knows 4 g.v(1).outE('knows').inV

47. [OutEdgesPipe(knows), InVertexPipe] g.v(1).outE('knows').inV.toString()

48. 1 g.v(1)

49. 1 knows 3 knows 4 g.v(1).out('knows')

50. 1 knows 3 name=stephen knows 4 name=peter g.v(1).out('knows').name

51. 1 knows 3 name=stephen knows 4 name=peter g.v(1).out('knows').name.filter{it.startsWith('st')}

52. 1 knows 3 name=stephen knows 4 name=peter g.v(1).out('knows').name.filter{it.matches('p.{4}')}

53. 1 knows 3 name=stephen knows 4 name=peter g.v(1).out('knows').name.filter{it.matches('p.{4}') & false}

54. [OutPipe(knows)] g.v(1).out('knows').toString()

55. 1 g.v(1)

56. o o! o oH W 1 g.v(1).filter{new Random().nextBoolean()}

57. 1 g.v(1)

58. t! A lrigh 1 g.v(1).filter{new Random().nextBoolean()}

59. 1 g.v(1)

60. Doh! 1 g.v(1).filter{new Random().nextBoolean()}

65. Graph Inference

66. 1 g.v(1)

67. 2 created 1 g.v(1).out('created')

68. 2 name=rexster created 1 g.v(1).out('created').name

69. 1 g.v(1)

71. 2 created created 1 3 g.v(1).out('created').in('created')

72. 2 created created 1 3 g.v(1).out('created').in('created').except([g.v(1)])

73. 2 created created 1 3 name=stephen g.v(1).out('created').in('created').except([g.v(1)]).name

74. 2 created created 1 3 name=stephen g.v(1).out('created').in('created').except([g.v(1)]).name.back(1)

75. 2 created created 1 collaborator 3 name=stephen g.v(1).out('created').in('created').except([g.v(1)]).name.back(1). sideEffect{g.addEdge(g.v(1),it,'collaborator')}

76. 2 created created 1 collaborator 3 name=stephen g.v(1).out('created').in('created').except([g.v(1)]).name.back(1). sideEffect{g.addEdge(g.v(1),it,'collaborator')}.filter{false}

77. Graph Backtracking and Looping

78. 1 g.v(1)

80. 5 imports 2 imports 6 imports created 7 1 g.v(1).out('created').out('imports')

81. 5 name=blueprints imports 2 imports 6 name=pipes imports created 7 name=gremlin 1 g.v(1).out('created').out('imports').name

82. 5 name=blueprints imports 2 imports 6 name=pipes imports created 7 name=gremlin 1 g.v(1).out('created').out('imports').name.back(1)

83. 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').out('imports').name.back(1).out('imports')

84. 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').out('imports').name.back(1).out('imports')

85. name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').out('imports').name.back(1).out('imports').name

86. name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').out('imports').name.back(1).out('imports').name .back(1)

87. name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').out('imports').name.back(1).out('imports').name .back(1).out('imports')

88. name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').out('imports').name.back(1).out('imports').name .back(1).out('imports').out('imports')

89. name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').out('imports').name.back(1).out('imports').name .back(1).out('imports').out('imports').out('imports')

90. name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1

91. name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').out('imports').loop(1){it.loops < 5}

97. name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 2 g.v(1).out('created').out('imports').loop(1){it.loops < 5}

101. m m = [:]

102. m name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1

103. m name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').out('import').groupCount(m).loop(2){true}

106. v[5]=1 m v[6]=1 v[7]=1 name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').out('import').groupCount(m).loop(2){true}

107. v[5]=1 m v[6]=1 v[7]=1 name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').out('import').groupCount(m).loop(2){true}

108. v[5]=3 v[6]=2 m v[7]=1 v[8]=1 name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').out('import').groupCount(m).loop(2){true}

114. m m = [:]

115. m name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1

116. m name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').out('import').name.groupCount(m).back(2).loop(2){true}

120. blueprints=1 m pipes=1 gremlin=1 name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').out('import').name.groupCount(m).back(2).loop(2){true}

124. blueprints=3 pipes=2 m gremlin=1 neo4j=1 name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').out('import').name.groupCount(m).back(2).loop(2){true}

135. Graph Paths

136. 1 g.v(1)

137. 1 g.v(1).as('x')

138. 9 10 child child 1 g.v(1).as('x').in('child')

139. 9 10 child child 1 g.v(1).as('x').in('child').loop('x')

140. child child 9 10 child child 1 g.v(1).as('x').in('child').loop('x'){it.object.in.count() != 0}

141. child child child child 9 10 child child 1 g.v(1).as('x').in('child').loop('x'){it.object.in.count() != 0}

142. child child child child child child 9 10 child child 1 g.v(1).as('x').in('child').loop('x'){it.object.in.count() != 0}

143. 100,000,000 100,000,001 child child ... ... child child child child child child 9 10 child child 1 g.v(1).as('x').in('child').loop('x'){it.object.in.count() != 0}

144. 100,000,000 100,000,001 child child ... ... child child child child child child 9 10 child child [v[1], v[9], ..., v[100,000,000]] [v[1], v[10], ..., v[100,000,001]] ... 1 g.v(1).as('x').in('child').loop('x'){it.object.in.count() != 0}.paths

145. name=angelina name=brad 100,000,000 100,000,001 child child ... ... child child child child child child 9 10 child child 1 g.v(1).as('x').in('child').loop('x'){it.object.in.count() != 0}.name

146. name=angelina name=brad 100,000,000 100,000,001 child child ... ... child child child child child child 9 10 child child [v[1], v[9], ..., v[100,000,000], angelina] [v[1], v[10], ..., v[100,000,001], brad] ... 1 g.v(1).as('x').in('child').loop('x'){it.object.in.count() != 0}.name.paths

147. 1 g.v(1)

148. child child 1 g.v(1).inE('child')

149. thinks="You suck." thinks="You're great." child child 1 g.v(1).inE('child').filter{TextAnalysis.isMean(it.thinks)}

150. thinks="You suck." child 1 g.v(1).inE('child').filter{TextAnalysis.isMean(it.thinks)}

151. 9 thinks="You suck." child 1 g.v(1).inE('child').filter{TextAnalysis.isMean(it.thinks)}.outV

152. irl." ag w like thro ou ks="Y thin thinks="No pony!" thi n ks ="Y ou 're fat ." thinks="Get a real job." 9 thinks="You suck." child 1 g.v(1).inE('child').filter{TextAnalysis.isMean(it.thinks)}.outV.loop(3){true}

153. irl." ag w like thro ou ks="Y thin thinks="No pony!" thi n ks ="Y ou 're fat ." thinks="Get a real job." 9 thinks="You suck." child 1 g.v(1).inE('child').filter{TextAnalysis.isMean(it.thinks)}.outV.loop(3){true}

154. Graph Pattern Matching

156. SELECT ?x ?y WHERE { 1 created ?x . ?x imports ?y } name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1

159. SELECT ?x ?y WHERE { ?x ?y 1 created ?x . v[2] v[5] v[2] v[6] ?x imports ?y v[2] v[7] } name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1

161. t name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 t = new Table()

162. t name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1)

163. t name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created')

164. t name=neo4j 8 imports 5 name=blueprints imports imports x imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').as('x')

165. t name=neo4j 8 imports 5 name=blueprints imports imports x imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').as('x').out('imports')

166. t name=neo4j 8 imports y 5 name=blueprints imports y imports x imports 2 imports 6 name=pipes imports y imports created 7 name=gremlin 1 g.v(1).out('created').as('x').out('imports').as('y')

167. x y v[2] v[5] t v[2] v[6] v[2] v[7] name=neo4j 8 imports y 5 name=blueprints imports y imports x imports 2 imports 6 name=pipes imports y imports created 7 name=gremlin 1 g.v(1).out('created').as('x').out('imports').as('y').table(t)

168. x y v[2] v[5] t v[2] v[6] v[2] v[7] name=neo4j 8 imports y 5 name=blueprints v[5] imports y imports x imports 2 imports 6 name=pipes imports y imports created 7 name=gremlin 1 t.get(0,1)

169. x y v[2] v[5] t v[2] v[6] v[2] v[7] name=neo4j 8 imports y 5 name=blueprints v[2] imports y imports x imports 2 imports 6 name=pipes imports y imports created 7 name=gremlin 1 t.get(0,'x')

170. x y v[2] v[5] t v[2] v[6] v[2] v[7] name=neo4j 8 imports y 5 name=blueprints v[5] v[6] imports v[7] y imports x imports 2 imports 6 name=pipes imports y imports created 7 name=gremlin 1 t.getColumn('y')

171. x y v[2] v[5] t v[2] v[6] v[2] v[7] name=neo4j 8 imports y 5 name=blueprints [x, y] imports y imports x imports 2 imports 6 name=pipes imports y imports created 7 name=gremlin 1 t.getColumnNames()

172. t name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 t = new Table()

173. t name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1)

174. t name=neo4j 8 imports 5 name=blueprints imports imports imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created')

175. t name=neo4j 8 imports 5 name=blueprints imports imports x imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').as('x')

176. t name=neo4j 8 imports 5 name=blueprints imports imports x imports 2 imports 6 name=pipes imports imports created 7 name=gremlin 1 g.v(1).out('created').as('x').out('imports')

177. t name=neo4j 8 imports y 5 name=blueprints imports y imports x imports 2 imports 6 name=pipes imports y imports created 7 name=gremlin 1 g.v(1).out('created').as('x').out('imports').as('y')

178. x y 2 blueprints t 2 pipes 2 gremlin name=neo4j 8 imports y 5 name=blueprints imports y imports x imports 2 imports 6 name=pipes imports y imports created 7 name=gremlin 1 g.v(1).out('created').as('x').out('imports').as('y').table(t){it.id}{it.name}

179. g.clear()

180. Graph Construction

181. 1 a = g.addVertex()

182. name=pierre 1 a.name = 'pierre'

183. name=pierre city=brussels 1 a.city = 'brussels'

184. name=stephen city=dc 2 name=pierre city=brussels 1 b = g.addVertex([name:'stephen',city:'dc'])

185. name=rexster logo= reXster . 3 name=stephen city=dc 2 name=pierre city=brussels 1 c = g.addVertex([name:'rexster',logo:[a12f04b312bc...]])

186. name=rexster logo= reXster . 3 since=2011 created name=stephen city=dc 2 name=pierre city=brussels 1 g.addEdge(b,c,'created',[since:2011])

187. name=rexster logo= reXster . 3 since=2011 created name=stephen since=2011 city=dc reviews 2 name=pierre city=brussels 1 g.addEdge(a,c,'reviews',[since:2011])

188. name=gremlin logo= Gremlin G = (V, E) . 4 name=rexster imports logo= reXster . 3 since=2011 created name=stephen since=2011 city=dc reviews 2 name=pierre city=brussels 1 g.addEdge(c,g.addVertex([name:'gremlin',logo:[ab1346f47...]),'imports')

189. name=pipes logo= Pipes. 5 name=gremlin logo= Gremlin G = (V, E) . imports 4 name=rexster imports logo= reXster . 3 since=2011 created name=stephen since=2011 city=dc reviews 2 name=pierre city=brussels 1 g.addEdge(c,g.addVertex([name:'pipes',logo:[12cbef46...]]),'imports')

190. name=blueprints name=pipes logo= Blueprints . logo= Pipes. 6 5 name=gremlin logo= Gremlin imports . G = (V, E) imports 4 name=rexster imports logo= reXster . 3 since=2011 created name=stephen since=2011 city=dc reviews 2 name=pierre city=brussels 1 g.addEdge(c,g.addVertex([name:'blueprints',logo:[f13cd1467...]]),'imports')

191. 6 5 4 3 2 1 g.V

192. 6 g.V.count()

193. blueprints pipes gremlin rexster stephen pierre g.V.name

194. 6 5 4 3 2 1 g.V

195. Blueprints Pipes Gremlin G = (V, E) reXster g.V.logo

196. 4 g.V.logo.count()

197. name=blueprints name=pipes logo= Blueprints . logo= Pipes . name=gremlin logo= Gremlin G = (V, E) . name=rexster logo= reXster . name=stephen city=dc name=pierre city=brussels g.V.transform{it.map()}

198. 6 g.V.transform{it.map()}.count()

199. 7 g.V.transform{it.map()}.count() + 1

200. 8 g.V.transform{it.map()}.count() + 1 + 1

201. 4 (g.V.transform{it.map()}.count() + 1 + 1) / 2

202. 4 is a number. ((g.V.transform{it.map()}.count() + 1 + 1) / 2) + ' is a number.'

203. 14 (((g.V.transform{it.map()}.count() + 1 + 1) / 2) + ' is a number.').length()

204. imports imports imports created reviews g.E

205. since=2011 since=2011 g.E.transform{it.map()}

206. g.clear()

207. Graph Ranking

208. 7 friend 6 friend friend 8 5 friend friend friend friend friend friend 1 4 friend friend 2 friend 3 g.loadGraphML('loopy-lou-and-the-loops.xml')

209. m 7 friend 6 friend friend 8 5 friend friend friend friend friend friend 1 4 friend friend 2 friend 3 m = [:]

210. m 7 friend 6 friend friend c 0 8 5 friend friend friend friend friend friend 1 4 friend friend 2 friend 3 c = 0

211. m 7 friend 6 friend friend c 0 8 5 friend friend friend friend friend friend 1 4 friend friend 2 friend 3 g.v(1).out.groupCount(m).loop(2){c++ < 35}.filter{false}

212. m 7 friend 6 friend friend c 0 8 5 friend friend friend friend friend friend 1 4 friend friend 2 friend 3 g.v(1).out.groupCount(m).loop(2){c++ < 35}.filter{false}

213. v[2]=1 m v[6]=1 7 friend 6 friend friend c 0 8 5 friend friend friend friend friend friend 1 4 friend friend 2 friend 3 g.v(1).out.groupCount(m).loop(2){c++ < 35}.filter{false}

214. v[2]=1 m v[6]=1 7 friend 6 friend friend c 2 8 5 friend friend friend friend friend friend 1 4 friend friend 2 friend 3 g.v(1).out.groupCount(m).loop(2){c++ < 35}.filter{false}

215. v[2]=1 v[3]=1 m v[6]=2 v[7]=1 7 friend 6 friend friend c 2 8 5 friend friend friend friend friend friend 1 4 friend friend 2 friend 3 g.v(1).out.groupCount(m).loop(2){c++ < 35}.filter{false}

216. v[2]=1 v[3]=1 m v[6]=2 v[7]=1 7 friend 6 friend friend c 5 8 5 friend friend friend friend friend friend 1 4 friend friend 2 friend 3 g.v(1).out.groupCount(m).loop(2){c++ < 35}.filter{false}

217. v[1]=1 v[2]=1 v[3]=1 m v[4]=1 v[6]=2 v[7]=2 v[8]=1 7 friend 6 friend friend c 5 8 5 friend friend friend friend friend friend 1 4 friend friend 2 friend 3 g.v(1).out.groupCount(m).loop(2){c++ < 35}.filter{false}

218. v[1]=1 v[2]=1 v[3]=1 m v[4]=1 v[6]=2 v[7]=2 v[8]=1 7 friend 6 friend friend c 9 8 5 friend friend friend friend friend friend 1 4 friend friend 2 friend 3 g.v(1).out.groupCount(m).loop(2){c++ < 35}.filter{false}

219. v[1]=3 v[2]=2 v[3]=1 v[4]=1 m v[5]=1 v[6]=3 v[7]=2 v[8]=2 7 friend 6 friend friend c 9 8 5 friend friend friend friend friend friend 1 4 friend friend 2 friend 3 g.v(1).out.groupCount(m).loop(2){c++ < 35}.filter{false}

226. v[1]=5 v[2]=5 v[3]=5 v[4]=3 m v[5]=1 v[6]=10 v[7]=7 v[8]=3 7 friend 6 friend friend c 39 8 5 friend friend friend friend friend friend 1 4 friend friend 2 friend 3 println 'do you understand the concept of centrality?'

227. g.clear()

228. IDE Development reXster Command Line REPL REST Server

229. Via any JVM language (JSR 223) Neo4j Server

230. http://gremlin.tinkerpop.com

The Pathology of Graph Databases

You are reading a preview.

The Pathology of Graph Databases

More Related Content

Viewers also liked

More from Marko Rodriguez

Featured

Related Books

Related Audiobooks