Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Gremlin 101.3 On Your FM Dial

1,826 views

Published on

The Gremlin traversal machine is composed of three components: a graph, a traversal, and a set of traversers. Learn how these components interact to enable distributed, vendor-agnostic, OLTP/OLAP-based graph computing.

This talk was presented live at DataStax's Support Summit in Carmel, CA (April 2017) and Engineering Summit in Las Vegas, NV (May 2017).

Published in: Technology
  • Be the first to comment

Gremlin 101.3 On Your FM Dial

  1. 1. Gremlin 101.3 on your FM Dial Marko A. Rodriguez
  2. 2. Who are person 1’s friends?
  3. 3. SELECT p2.* FROM person p1 INNER JOIN knows k ON k.out = p1.id INNER JOIN person p2 ON p2.id = k.in WHERE p1.id = 1 person 1 knows 1 2 1 4 person 2 4 p2kp1
  4. 4. g.V(1).out(‘knows’)
  5. 5. g.V(1).out(‘knows’)
  6. 6. g.V(1).out(‘knows’) 1
  7. 7. g.V(1).out(‘knows’) 4 2
  8. 8. g.V(1).out(‘knows’)
  9. 9. g.V(1).out(‘knows’)
  10. 10. g.V(1).out(‘knows’)
  11. 11. g.V(1).out(‘knows’)
  12. 12. G ⟵μ t∈T ψ⟶Ψ Rodriguez, M.A., “The Gremlin Graph Traversal Machine and Language,” ACM Proceedings of the 15th Symposium on Database Programming Languages, 2015. http://arxiv.org/abs/1508.03843
  13. 13. G ⟵μ t∈T ψ⟶Ψ Graph Traversers Traversal
  14. 14. G ⟵μ t∈T ψ⟶Ψ Graph Traversers Traversalobject steptraverser
  15. 15. g.V(1).out(‘knows’)Ψ G μ ψ tT Graph Traversers Traversal Vertices,Edges,Properties Locations,Bulk,Sack,Path,Loops Steps,SideEffects
  16. 16. Ψ The Traversal
  17. 17. g.V(1).out(‘knows’)
  18. 18. g.V(1).out(‘knows’) GraphStep VertexStep compiles to
  19. 19. g.V(1).out(‘knows’) GraphStep VertexStep GraphStep VertexStep1 2 4 compiles to executes as
  20. 20. g.V(1).out(‘knows’) GraphStep VertexStep GraphStep VertexStep1 2 4 GraphStep VertexStep1 2 4 compiles to executes as executes as
  21. 21. g.V(1).out(‘knows’) ψ GraphStep VertexStep1 2 4 →Graph
  22. 22. g.V(1).out(‘knows’) ψ GraphStep VertexStep1 2 4 Graph→Vertex
  23. 23. g.V(1).out(‘knows’) ψ ψ GraphStep VertexStep1 2 4 Vertex→Vertex
  24. 24. out : V→V g.V(1).out(‘knows’) knows V : G→V1
  25. 25. out : V→V* g.V(1).out(‘knows’) knows V : G→V*1
  26. 26. out : V→V* g.V(1).out(‘knows’) knows V : G→V*1
  27. 27. out : V→V* V ( )1 g.V(1).out(‘knows’) knows V : G→V*1
  28. 28. out : V→V* g.V(1).out(‘knows’) knows V : G→V*1 1
  29. 29. out : V→V* g.V(1).out(‘knows’) knows V : G→V*1 out ( )1knows
  30. 30. out : V→V* g.V(1).out(‘knows’) knows V : G→V*1 2 4
  31. 31. g.V(1).out(‘knows’) V ( )1out ( ) =knows 2 4
  32. 32. g.V(1).out(‘knows’).where(out(‘created’).has(‘name’,’lop’)).values(‘name’) What are the names of the people that person 1 knows who created LinkedProcess (lop)?
  33. 33. g.V(1).out(‘knows’). where(out(‘created’).has(‘name’,’lop’)) values(‘name’) What are the names of the people that person 1 knows who created LinkedProcess (lop)?
  34. 34. g.V(1).out(‘knows’). where(out(‘created’).has(‘name’,’lop’)) values(‘name’) global scope local scope locally-global scope
  35. 35. g.V(1).out(‘knows’). where(out(‘created’).has(‘name’,’lop’)) values(‘name’) global scope
  36. 36. g.V(1).out(‘knows’). where(out(‘created’).has(‘name’,’lop’)) values(‘name’) global scope
  37. 37. g.V(1).out(‘knows’). where(out(‘created’).has(‘name’,’lop’)) values(‘name’) i’m waiting… I hope the vertex I reference created lop! local scope
  38. 38. i’m still waiting… g.V(1).out(‘knows’). where(out(‘created’).has(‘name’,’lop’)) values(‘name’) 33% com plete. locally-global scope
  39. 39. i’m still waiting some more… g.V(1).out(‘knows’). where(out(‘created’).has(‘name’,’lop’)) values(‘name’) uh oh! bummer! locally-global scope
  40. 40. g.V(1).out(‘knows’). where(out(‘created’).has(‘name’,’lop’)) values(‘name’) damn! my turn yet? local scope
  41. 41. g.V(1).out(‘knows’). where(out(‘created’).has(‘name’,’lop’)) values(‘name’)my turn finally! local scope
  42. 42. g.V(1).out(‘knows’). where(out(‘created’).has(‘name’,’lop’)) values(‘name’) prospermy child. locally-global scope
  43. 43. g.V(1).out(‘knows’). where(out(‘created’).has(‘name’,’lop’)) values(‘name’)one successful offspring is all I need. locally-global scope
  44. 44. g.V(1).out(‘knows’). where(out(‘created’).has(‘name’,’lop’)) values(‘name’) woohoo! live and learn :( booyea! locally-global scope
  45. 45. g.V(1).out(‘knows’). where(out(‘created’).has(‘name’,’lop’)) values(‘name’) I reference a vertex that created lop! local scope
  46. 46. g.V(1).out(‘knows’). where(out(‘created’).has(‘name’,’lop’)) values(‘name’) global scope
  47. 47. g.V(1).out(‘knows’). where(out(‘created’).has(‘name’,’lop’)) values(‘name’) I reference a result. global scope
  48. 48. Gremlin Language Traversal g.V(1).out(‘knows’). where(out(‘created’).has(‘name’,’lop’)) values(‘name’) traversal = op((object | traversal)*)*
  49. 49. [[V,1][out,knows] [where,[[out,created],[has,name,lop]]] [values,name]] Gremlin Bytecode Traversal traversal = [[op,(object | traversal)*]*]
  50. 50. Gremlin Machine Traversal GraphStep VertexStep WhereStep VertexStep HasStep PropertiesStep traversal = step[(object | traversal)*]*
  51. 51. machine code machine code g.V().has('name','marko'). repeat(outE().identity().inV()).times(2). name.groupCount() [[V],[has,name,eq(marko)], [repeat,[[outE],[identity],[inV]]],[times,2], [values,name],[groupCount]] [GraphStep,HasStep(name,eq(marko)), RepeatStep([VertexStep(out,edges),IdentityStep, EdgeVertexStep(in)],2), PropertiesStep(values,name),GroupCountStep] translate compile optimize execute languagebytecode [ProviderGraphStep(name,eq(marko)), VertexStep(out,vertices),NoOpBarrierStep, VertexStep(out,vertices),NoOpBarrierStep, PropertiesStep(values,name),GroupCountStep]
  52. 52. “FilterStep” Rodriguez, M.A., “A Gremlin Implementation of the Gremlin Traversal Machine,” DataStax Engineering Blog, October 2016. https://www.datastax.com/dev/blog/a-gremlin-implementation-of-the-gremlin-traversal-machine
  53. 53. “FilterStep”
  54. 54. “FilterStep”
  55. 55. “FilterStep”
  56. 56. “FilterStep”
  57. 57. “FilterStep”
  58. 58. “FilterStep”
  59. 59. “FilterStep”
  60. 60. “MapStep”
  61. 61. “MapStep”
  62. 62. “MapStep”
  63. 63. “MapStep”
  64. 64. “MapStep”
  65. 65. “MapStep”
  66. 66. “MapStep”
  67. 67. “MapStep”
  68. 68. Chained and Nested Steps
  69. 69. Map: one-to-one FlatMap: one-to-many Filter: one-to-one.or.none SideEffect: one-to[?]-one Reducer: many-to-one Barrier: many-to-many Step Types
  70. 70. 1 label() person Map
  71. 71. 1 out(‘knows’) label() 1 2 4 person MapFlatMap
  72. 72. 1 out(‘knows’) label() 1 2 4 person MapFlatMap Filter 1 hasLabel(‘android’)
  73. 73. 1 out(‘knows’) label() 1 2 4 person MapFlatMap Filter 1 hasLabel(‘android’) SideEffect 1 store(‘x’) 1 x=[ ]1
  74. 74. 1 out(‘knows’) label() 1 2 4 person MapFlatMap Filter 1 hasLabel(‘android’) SideEffect 1 store(‘x’) 1 x=[ ]1 Barrier 1 1 barrier() 1 2
  75. 75. 1 out(‘knows’) label() 1 2 4 person MapFlatMap Filter 1 hasLabel(‘android’) SideEffect 1 store(‘x’) 1 x=[ ]1 Barrier 1 1 barrier() 1 2 Reducer 2 4 count() 2
  76. 76. g.V().has(‘name’,’gremlin’)
  77. 77. g.V().has(‘name’,’gremlin’). out(‘bought’).aggregate(‘stash’) stash
  78. 78. g.V().has(‘name’,’gremlin’). out(‘bought’).aggregate(‘stash’). in(‘bought’) stash
  79. 79. g.V().has(‘name’,’gremlin’). out(‘bought’).aggregate(‘stash’). in(‘bought’).has(‘name’,neq(‘gremlin’)) stash
  80. 80. g.V().has(‘name’,’gremlin’). out(‘bought’).aggregate(‘stash’). in(‘bought’).has(‘name’,neq(‘gremlin’)). out(‘bought’) stash Failure Is An Option
  81. 81. g.V().has(‘name’,’gremlin’). out(‘bought’).aggregate(‘stash’). in(‘bought’).has(‘name’,neq(‘gremlin’)). out(‘bought’).not(within(‘stash’)) stash Failure Is An Option
  82. 82. Failure Is An Option g.V().has(‘name’,’gremlin’). out(‘bought’).aggregate(‘stash’). in(‘bought’).has(‘name’,neq(‘gremlin’)). out(‘bought’).not(within(‘stash’)). groupCount() stash Failure Is An Option 3 1 1
  83. 83. Failure Is An Option g.V().has(‘name’,’gremlin’). out(‘bought’).aggregate(‘stash’). in(‘bought’).has(‘name’,neq(‘gremlin’)). out(‘bought’).not(within(‘stash’)). groupCount(). order(local).by(values,decr) stash Failure Is An Option 3 1 1
  84. 84. Failure Is An Option g.V().has(‘name’,’gremlin’). out(‘bought’).aggregate(‘stash’). in(‘bought’).has(‘name’,neq(‘gremlin’)). out(‘bought’).not(within(‘stash’)). groupCount(). order(local).by(values,decr). select(keys).unfold().limit(1) stash Failure Is An Option 3 1 1
  85. 85. repeat().until() until().repeat() choose().option() choose() as() select() do-while while-do if-then-else switch var set var get Turing Completeness
  86. 86. Every modern programming language supports function chaining and nesting. g.V(1).out(“knows”) g.V(1).out(‘knows’) g.V(1).out(‘knows’)
  87. 87. Quiz
  88. 88. id() out(‘created’) values(‘name’) count() where(‘a’,eq(‘b’)) property(‘name’,’marko’) label() has(‘age’) groupCount() groupCount(’m’) barrier() repeat(out()).times(2)
  89. 89. id() out(‘created’) values(‘name’) count() where(‘a’,eq(‘b’)) property(‘name’,’marko’) label() has(‘age’) groupCount() groupCount(’m’) barrier() repeat(out()).times(2) flatmap: vertex vertex*
  90. 90. id() out(‘created’) values(‘name’) count() where(‘a’,eq(‘b’)) property(‘name’,’marko’) label() has(‘age’) groupCount() groupCount(’m’) barrier() repeat(out()).times(2) flatmap: vertex vertex* reducer: object* long map: element object flatmap: vertex vertex*
  91. 91. id() out(‘created’) values(‘name’) count() where(‘a’,eq(‘b’)) property(‘name’,’marko’) label() has(‘age’) groupCount() groupCount(’m’) barrier() repeat(out()).times(2) flatmap: vertex vertex* reducer: object* long map: element object map: element string reducer: object* map<object,long> filter: object object side-effect: object , ( ), object flatmap: vertex vertex*
  92. 92. id() out(‘created’) values(‘name’) count() where(‘a’,eq(‘b’)) property(‘name’,’marko’) label() has(‘age’) groupCount() groupCount(’m’) barrier() repeat(out()).times(2) flatmap: vertex vertex* reducer: object* long map: element object map: element string reducer: object* map<object,long> filter: object object filter: element element side-effect: object , ( ), object flatmap: vertex vertex* side-effect: element ) element barrier: object* object* flatmap: element string*
  93. 93. The traversal guides the traverser down the long and winding graph. Ψ G T
  94. 94. T The Traversers
  95. 95. G ⟵μ t∈T ψ⟶Ψ Graph Traversers Traversal
  96. 96. PathBulk G ⟵μ t∈T ψ⟶Ψ Traversers β,Δ,ς,ι Sack Loops
  97. 97. g.V().both().id() 1 2 4 knows 3 5 6 created created knows created created β
  98. 98. g.V().both().id() 1 2 4 knows 3 5 6 created created knows created created ψ
  99. 99. g.V().both().id() 1 2 4 knows 3 5 6 created created knows created created ψ
  100. 100. 1 2 4 knows 3 5 6 created created knows created created g.V().both().id() ψ
  101. 101. 1 2 4 knows 3 5 6 created created knows created created g.V().both().id() ψ 2 1 11 5 4 4 4 3 33 6
  102. 102. 1 2 4 knows 3 5 6 created created knows created created g.V().both().id() ψ 2 1 11 5 4 4 4 3 33 6 ==>2 ==>5 ==>1 ==>1 ==>1 ==>4 ==>4 ==>4 ==>3 ==>3 ==>3 ==>6
  103. 103. 1 2 4 knows 3 5 6 created created knows created created g.V().both().id() ψ 2 1 11 5 4 4 4 3 33 6 id( )5id( )2 id( )1 id( )4 id( )4 id( )1 id( )1 id( )4 id( )3 id( )3 id( )3 id( )6
  104. 104. Its time to bulk up. Rodriguez, M.A., “The Beauty of Bulking,” Gremlin-Users Mailing List, June 2015. https://twitter.com/twarko/status/606513003090092035
  105. 105. 1 2 4 knows 3 5 6 created created knows created created g.V().both().id() ψ
  106. 106. 1 2 4 knows 3 5 6 created created knows created created 1 3 1 3 3 1 g.V().both().id() ψ
  107. 107. 1 2 4 knows 3 5 6 created created knows created created 1 3 1 3 3 1 g.V().both().id() ψ 2 1 5 4 3 6
  108. 108. 1 2 4 knows 3 5 6 created created knows created created 1 3 1 3 3 1 g.V().both().id() ψ ==>2 ==>5 ==>1 ==>1 ==>1 ==>4 ==>4 ==>4 ==>3 ==>3 ==>3 ==>61 4 3 6 2 5
  109. 109. 1 2 4 knows 3 5 6 created created knows created created 1 3 1 3 3 1 g.V().both().id() ψ 1 4 3 6 id( )5id( )2 id( )1 id( )4 id( )3 id( )61/2 as many function calls with bulking! ballin’. 2 5
  110. 110. gremlin> g.V().both().id().explain() ==>Traversal Explanation ======================================================================================================================= Original Traversal [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep] ConnectiveStrategy [D] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep] MatchPredicateStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep] FilterRankingStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep] InlineFilterStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep] RepeatUnrollStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep] PathRetractionStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep] IncidentToAdjacentStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep] AdjacentToIncidentStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep] RangeByIsCountStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep] LazyBarrierStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(2500), IdStep] TinkerGraphCountStrategy [P] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(2500), IdStep] TinkerGraphStepStrategy [P] [TinkerGraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(2500), IdStep] ProfileStrategy [F] [TinkerGraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(2500), IdStep] StandardVerificationStrategy [V] [TinkerGraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(2500), IdStep] Final Traversal [TinkerGraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(2500), IdStep] g.V().both().id() g.V().both().barrier().id() optimizes to
  111. 111. g.V(1).out(‘knows’).out(‘created’).path() 1 2 4 knows 3 5 6 created created knows created created ψ [ ]1Δ=
  112. 112. g.V(1).out(‘knows’).out(‘created’).path() 1 2 4 knows 3 5 6 created created knows created created ψ [ ]1 2 [ ]1 4
  113. 113. g.V(1).out(‘knows’).out(‘created’).path() 1 2 4 knows 3 5 6 created created knows created created ψ [ ]1 4 5 [ ]1 4 3
  114. 114. g.V(1).out(‘knows’).out(‘created’).path() 1 2 4 knows 3 5 6 created created knows created created ψ [ ]1 4 5 [ ]1 4 3
  115. 115. Its time to get crazy with locally scoped traversers.
  116. 116. g.V(1).out(‘knows’).out(‘created’).limit(1). path().by(inE().count()) 1 2 4 knows 3 5 6 created created knows created created ψ [ ]1 4 5 [ ]1 4 3
  117. 117. g.V(1).out(‘knows’).out(‘created’).limit(1). path().by(inE().count()) 1 2 4 knows 3 5 6 created created knows created created ψ [ ]1 4 5
  118. 118. g.V(1).out(‘knows’).out(‘created’).limit(1). path().by(inE().count()) 1 2 4 knows 3 5 6 created created knows created created ψ [ ]1 4 5
  119. 119. g.V(1).out(‘knows’).out(‘created’).limit(1). path().by(inE().count()) 1 2 4 knows 3 5 6 created created knows created created ψ [ ]1 4 5
  120. 120. g.V(1).out(‘knows’).out(‘created’).limit(1). path().by(inE().count()) 1 2 4 knows 3 5 6 created created knows created created ψ [ ]1 4 5 g.V(4).inE().count() g.V(1).inE().count() g.V(5).inE().count()
  121. 121. g.V(1).out(‘knows’).out(‘created’).limit(1). path().by(inE().count()) 1 2 4 knows 3 5 6 created created knows created created ψ [ ]1 4 5
  122. 122. g.V(1).out(‘knows’).out(‘created’).limit(1). path().by(inE().count()) 1 2 4 knows 3 5 6 created created knows created created ψ [ ]1 4 5 0 1 1
  123. 123. g.V(1).out(‘knows’).out(‘created’).limit(1). path().by(inE().count()) 1 2 4 knows 3 5 6 created created knows created created ψ [ ]1 4 5 0 1 1
  124. 124. [ ] g.V(1).out(‘knows’).out(‘created’).limit(1). path().by(inE().count()) 1 2 4 knows 3 5 6 created created knows created created ψ 0 1 1
  125. 125. g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 ς
  126. 126. g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 ψ
  127. 127. g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 ψ 1
  128. 128. g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 ψ 1
  129. 129. g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 ψ 1 ι=0
  130. 130. g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 ψ 1ι=0 1 ι=0
  131. 131. g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 1.0ι=0 0.5 ι=0 ψ
  132. 132. g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 ψ 1.0ι=0 0.5 ι=0
  133. 133. g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 ψ 1.0ι=1 0.5 ι=1
  134. 134. g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 1.0ι=1 0.5 ι=1 ψ ? ?
  135. 135. g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 1.0ι=1 0.5 ι=1 ψ
  136. 136. g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 1.0 ι=1 ψ 1.0 ι=1
  137. 137. g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 0.4 ι=1 ψ 1.0 ι=1
  138. 138. ψ g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 0.4 ι=11.0 ι=1
  139. 139. ψ g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 0.4 ι=21.0 ι=2
  140. 140. g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 0.4 ι=21.0 ι=2 ψ ! !
  141. 141. g.withSack(1).V(1). repeat(outE().sack(mult).by(‘weight’).inV()). times(2).sack() 1 2 4 3 5 6 0.5 0.4 1.0 0.4 1.0 0.2 0.4 1.0 ψ 1.0 0.4
  142. 142. machine 2 machine 3machine 1 μ ψ The Gremlin traversal machine is a distributed virtual machine — traversers can independently move around the graph (across machines) and through the traversal. ψt ψt+1 μt+1 μt
  143. 143. Quiz
  144. 144. 2 2 Barrier 2 3 To Bulk or Not To Bulk
  145. 145. 2 2 Barrier 2 3 2 5 2 3 Barrier 2 3 To Bulk or Not To Bulk
  146. 146. 5 2 2 Barrier 2 3 2 5 2 3 Barrier 2 3 2 3 2 3 [ ]1 4 5 5 3 [ ]1 2 5 1 Barrier To Bulk or Not To Bulk
  147. 147. 5 2 2 Barrier 2 3 2 5 2 3 Barrier 2 3 2 3 2 3 [ ]1 4 5 5 3 [ ]1 2 5 1 Barrier 5 [ ]1 4 5 5 3 [ ]1 2 5 1 4 1 4 3 72 Barrier withSack(?,sum) To Bulk or Not To Bulk
  148. 148. 5 2 2 Barrier 2 3 2 5 2 3 Barrier 2 3 2 3 2 3 [ ]1 4 5 5 3 [ ]1 2 5 1 Barrier 5 [ ]1 4 5 5 3 [ ]1 2 5 1 4 1 4 3 72 Barrier 4 4 9 withSack(?,sum) 2 2 Barrier 2 3 ι=1 ι=2 To Bulk or Not To Bulk
  149. 149. 5 2 2 Barrier 2 3 2 5 2 3 Barrier 2 3 2 3 2 3 [ ]1 4 5 5 3 [ ]1 2 5 1 Barrier 5 [ ]1 4 5 5 3 [ ]1 2 5 1 4 1 4 3 72 Barrier 4 4 9 withSack(?,sum) 2 2 Barrier 2 3 ι=1 ι=2 2 2 2 3 ι=1 ι=2 To Bulk or Not To Bulk 3 Barrier 3
  150. 150. 5 2 2 Barrier 2 3 2 5 2 3 Barrier 2 3 2 3 2 3 [ ]1 4 5 5 3 [ ]1 2 5 1 Barrier 5 [ ]1 4 5 5 3 [ ]1 2 5 1 4 1 4 3 72 Barrier 4 4 9 withSack(?,sum) 2 2 Barrier 2 3 ι=1 ι=2 2 2 2 3 ι=1 ι=2 To Bulk or Not To Bulk 3 Barrier 3 3 3
  151. 151. The traverser is an atomic unit of computing — moving through both the traversal and the graph.
  152. 152. G The Graph
  153. 153. Lattice Scale-Free Random Rodriguez, M.A., “Graphoendodonticology,” DataStax Blog, March 2017. https://www.datastax.com/2017/03/graphoendodonticology
  154. 154. Lattice Scale-Free Random Diamond Jackson Pollock TV Fuzz
  155. 155. g.V().groupCount().by(both().count()) Lattice Scale-Free Random0.00.20.40.6 degree frequency 0 1 2 3 4 0 5 10 15 20 25 30 0.00.20.4 degree frequency many vertices have few edges few vertices have many edges 2 4 6 8 10 0.050.100.15 degree frequency gaussian distribution
  156. 156. Lattice Scale-Free Random g.V().repeat(both()).times(x).count() 2 4 6 8 10 0.0e+001.0e+102.0e+103.0e+10 path length pathcount scale-free random lattice
  157. 157. Scale-Free ! "I’m cool.
  158. 158. Scale-Free ! "
  159. 159. Scale-Free ! "
  160. 160. Scale-Free ! " Why am I holding a bomb?
  161. 161. Scale-Free hub! ! "
  162. 162. Scale-Free explosion! !
  163. 163. Scale-Free
  164. 164. Scale-Free ! ! ! ! !
  165. 165. Scale-Free ! ! ! ! ! diiiiiiiamn.
  166. 166. Facebook Subgraph |V| = 4039 |E| = 88234 Aliev, A., Rodriguez, M.A., “Reducing Computational Complexity with Correlate Traversals,” DataStax Blog, April 2017. https://www.datastax.com/2017/04/reducing-computational-complexity-with-correlate-traversals
  167. 167. ! Yo. Facebook Subgraph |V| = 4039 |E| = 88234
  168. 168. ! Facebook Subgraph |V| = 4039 |E| = 88234
  169. 169. So you want to be a graph surgeon? * The same traversal on different graphs can have wildly different performance characteristics. * Traversals on scale-free/natural graphs can easily explode due to “hubs” (super nodes). * Use Gremlin to study the graph’s statistics before using Gremlin to query the graph. * Build your traversals up piecewise and use profile() to find bottlenecks.
  170. 170. knows knows watchedwatched watched watched watched watched watched ⭐⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐⭐⭐
  171. 171. knows knows watchedwatched watched watched watched watched watched ⭐⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐⭐⭐ " g.V(1).count() 1 Sup.
  172. 172. knows knows watchedwatched watched watched watched watched watched ⭐⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐⭐⭐ g.V(1).out().count() 9
  173. 173. knows knows watchedwatched watched watched watched watched watched ⭐⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐⭐⭐ g.V(1).out(‘watched’).count() 7
  174. 174. knows knows watchedwatched watched watched watched watched watched ⭐⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐⭐⭐ g.V(1).outE(‘watched’)
  175. 175. knows knows watchedwatched watched watched watched watched watched ⭐⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐⭐⭐ g.V(1).outE(‘watched’).has(‘stars’,gte(3))
  176. 176. knows knows watchedwatched watched watched watched watched watched ⭐⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐⭐⭐ g.V(1).outE(‘watched’).has(‘stars’,gte(3))
  177. 177. knows knows watchedwatched watched watched watched watched watched ⭐⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ ⭐⭐⭐ g.V(1).outE(‘watched’).has(‘stars’,gte(3)).inV().count() 3
  178. 178. ! I’ll be more selective this time. Facebook Subgraph |V| = 4039 |E| = 88234
  179. 179. Facebook Subgraph |V| = 4039 |E| = 88234
  180. 180. Quiz
  181. 181. Controlling Traverser Populations out(‘label’) outE().has(‘key’,value).inV() local(out().coin(0.5)) local(outE().sample(10).by(‘weight’).inV()) out() local(out().limit(10))
  182. 182. Controlling Traverser Populations out(‘label’) outE().has(‘key’,value).inV() local(out().coin(0.5)) local(outE().sample(10).by(‘weight’).inV()) out() only edges in one direction local(out().limit(10))
  183. 183. Controlling Traverser Populations out(‘label’) outE().has(‘key’,value).inV() local(out().coin(0.5)) local(outE().sample(10).by(‘weight’).inV()) only edges w/ label out() only edges in one direction local(out().limit(10))
  184. 184. Controlling Traverser Populations out(‘label’) outE().has(‘key’,value).inV() local(out().coin(0.5)) local(outE().sample(10).by(‘weight’).inV()) only edges w/ label only edges w/ property condition out() only edges in one direction local(out().limit(10))
  185. 185. Controlling Traverser Populations out(‘label’) outE().has(‘key’,value).inV() local(out().coin(0.5)) local(outE().sample(10).by(‘weight’).inV()) only edges w/ label only edges w/ property condition out() only edges in one direction local(out().limit(10)) only the first 10 edges
  186. 186. Controlling Traverser Populations out(‘label’) outE().has(‘key’,value).inV() local(out().coin(0.5)) local(outE().sample(10).by(‘weight’).inV()) only edges w/ label only edges w/ property condition only 50% of edges out() only edges in one direction local(out().limit(10)) only the first 10 edges
  187. 187. Controlling Traverser Populations out(‘label’) outE().has(‘key’,value).inV() local(out().coin(0.5)) local(outE().sample(10).by(‘weight’).inV()) only edges w/ label only edges w/ property condition only 50% of edges only 10 edges biased by weight out() only edges in one direction local(out().limit(10)) only the first 10 edges
  188. 188. The computational complexity of the traversal is a function of the structural complexity of the graph.
  189. 189. G ⟵μ t∈T ψ⟶Ψ Graph Traversers Traversal
  190. 190. G ⟵μ t∈T ψ⟶Ψ Graph Traversers Traversal CPUs ProgramData
  191. 191. The show is over. Please tip your waitress and drive home safe! traverse

×