Gremlin 101.3 On Your FM Dial

Marko Rodriguez
Marko RodriguezFounder at RReduX, Inc.
Gremlin 101.3
on your
FM Dial
Marko A. Rodriguez
Who are person 1’s friends?
SELECT p2.*
FROM person p1
INNER JOIN knows k
ON k.out = p1.id
INNER JOIN person p2
ON p2.id = k.in
WHERE p1.id = 1
person
1
knows
1 2
1 4
person
2
4
p2kp1
g.V(1).out(‘knows’)
g.V(1).out(‘knows’)
g.V(1).out(‘knows’)
1
g.V(1).out(‘knows’)
4
2
g.V(1).out(‘knows’)
g.V(1).out(‘knows’)
g.V(1).out(‘knows’)
g.V(1).out(‘knows’)
G ⟵μ t∈T ψ⟶Ψ
Rodriguez, M.A., “The Gremlin Graph Traversal Machine and Language,” ACM
Proceedings of the 15th Symposium on Database Programming Languages, 2015.
http://arxiv.org/abs/1508.03843
G ⟵μ t∈T ψ⟶Ψ
Graph
Traversers
Traversal
G ⟵μ t∈T ψ⟶Ψ
Graph
Traversers
Traversalobject steptraverser
g.V(1).out(‘knows’)Ψ
G
μ
ψ
tT
Graph
Traversers
Traversal
Vertices,Edges,Properties
Locations,Bulk,Sack,Path,Loops
Steps,SideEffects
Ψ
The Traversal
g.V(1).out(‘knows’)
g.V(1).out(‘knows’)
GraphStep VertexStep
compiles to
g.V(1).out(‘knows’)
GraphStep VertexStep
GraphStep VertexStep1 2 4
compiles to
executes as
g.V(1).out(‘knows’)
GraphStep VertexStep
GraphStep VertexStep1 2 4
GraphStep VertexStep1 2 4
compiles to
executes as
executes as
g.V(1).out(‘knows’)
ψ
GraphStep VertexStep1 2 4
→Graph
g.V(1).out(‘knows’)
ψ
GraphStep VertexStep1 2 4
Graph→Vertex
g.V(1).out(‘knows’)
ψ ψ
GraphStep VertexStep1 2 4
Vertex→Vertex
out : V→V
g.V(1).out(‘knows’)
knows
V : G→V1
out : V→V*
g.V(1).out(‘knows’)
knows
V : G→V*1
out : V→V*
g.V(1).out(‘knows’)
knows
V : G→V*1
out : V→V*
V ( )1
g.V(1).out(‘knows’)
knows
V : G→V*1
out : V→V*
g.V(1).out(‘knows’)
knows
V : G→V*1
1
out : V→V*
g.V(1).out(‘knows’)
knows
V : G→V*1
out ( )1knows
out : V→V*
g.V(1).out(‘knows’)
knows
V : G→V*1
2 4
g.V(1).out(‘knows’)
V ( )1out ( ) =knows 2 4
g.V(1).out(‘knows’).where(out(‘created’).has(‘name’,’lop’)).values(‘name’)
What are the names of the people that person
1 knows who created LinkedProcess (lop)?
g.V(1).out(‘knows’).
where(out(‘created’).has(‘name’,’lop’))
values(‘name’)
What are the names of the people that person
1 knows who created LinkedProcess (lop)?
g.V(1).out(‘knows’).
where(out(‘created’).has(‘name’,’lop’))
values(‘name’)
global scope
local scope
locally-global scope
g.V(1).out(‘knows’).
where(out(‘created’).has(‘name’,’lop’))
values(‘name’)
global scope
g.V(1).out(‘knows’).
where(out(‘created’).has(‘name’,’lop’))
values(‘name’)
global scope
g.V(1).out(‘knows’).
where(out(‘created’).has(‘name’,’lop’))
values(‘name’)
i’m waiting…
I hope the vertex I
reference created lop!
local scope
i’m still waiting…
g.V(1).out(‘knows’).
where(out(‘created’).has(‘name’,’lop’))
values(‘name’)
33%
com
plete.
locally-global scope
i’m still waiting some more…
g.V(1).out(‘knows’).
where(out(‘created’).has(‘name’,’lop’))
values(‘name’)
uh
oh! bummer!
locally-global scope
g.V(1).out(‘knows’).
where(out(‘created’).has(‘name’,’lop’))
values(‘name’)
damn!
my turn yet?
local scope
g.V(1).out(‘knows’).
where(out(‘created’).has(‘name’,’lop’))
values(‘name’)my turn finally!
local scope
g.V(1).out(‘knows’).
where(out(‘created’).has(‘name’,’lop’))
values(‘name’)
prospermy child.
locally-global scope
g.V(1).out(‘knows’).
where(out(‘created’).has(‘name’,’lop’))
values(‘name’)one successful
offspring is all I need.
locally-global scope
g.V(1).out(‘knows’).
where(out(‘created’).has(‘name’,’lop’))
values(‘name’)
woohoo!
live and learn :(
booyea!
locally-global scope
g.V(1).out(‘knows’).
where(out(‘created’).has(‘name’,’lop’))
values(‘name’)
I reference a vertex
that created lop!
local scope
g.V(1).out(‘knows’).
where(out(‘created’).has(‘name’,’lop’))
values(‘name’)
global scope
g.V(1).out(‘knows’).
where(out(‘created’).has(‘name’,’lop’))
values(‘name’)
I
reference
a
result.
global scope
Gremlin Language Traversal
g.V(1).out(‘knows’).
where(out(‘created’).has(‘name’,’lop’))
values(‘name’)
traversal = op((object | traversal)*)*
[[V,1][out,knows]
[where,[[out,created],[has,name,lop]]]
[values,name]]
Gremlin Bytecode Traversal
traversal = [[op,(object | traversal)*]*]
Gremlin Machine Traversal
GraphStep VertexStep
WhereStep VertexStep HasStep
PropertiesStep
traversal = step[(object | traversal)*]*
machine
code
machine
code
g.V().has('name','marko').
repeat(outE().identity().inV()).times(2).
name.groupCount()
[[V],[has,name,eq(marko)],
[repeat,[[outE],[identity],[inV]]],[times,2],
[values,name],[groupCount]]
[GraphStep,HasStep(name,eq(marko)),
RepeatStep([VertexStep(out,edges),IdentityStep,
EdgeVertexStep(in)],2),
PropertiesStep(values,name),GroupCountStep]
translate
compile
optimize
execute
languagebytecode
[ProviderGraphStep(name,eq(marko)),
VertexStep(out,vertices),NoOpBarrierStep,
VertexStep(out,vertices),NoOpBarrierStep,
PropertiesStep(values,name),GroupCountStep]
“FilterStep”
Rodriguez, M.A., “A Gremlin Implementation of the Gremlin Traversal Machine,”
DataStax Engineering Blog, October 2016.
https://www.datastax.com/dev/blog/a-gremlin-implementation-of-the-gremlin-traversal-machine
“FilterStep”
“FilterStep”
“FilterStep”
“FilterStep”
“FilterStep”
“FilterStep”
“FilterStep”
“MapStep”
“MapStep”
“MapStep”
“MapStep”
“MapStep”
“MapStep”
“MapStep”
“MapStep”
Chained and Nested Steps
Map: one-to-one
FlatMap: one-to-many
Filter: one-to-one.or.none
SideEffect: one-to[?]-one
Reducer: many-to-one
Barrier: many-to-many
Step Types
1
label()
person
Map
1
out(‘knows’)
label()
1 2 4
person
MapFlatMap
1
out(‘knows’)
label()
1 2 4
person
MapFlatMap
Filter
1 hasLabel(‘android’)
1
out(‘knows’)
label()
1 2 4
person
MapFlatMap
Filter
1 hasLabel(‘android’)
SideEffect
1 store(‘x’) 1
x=[ ]1
1
out(‘knows’)
label()
1 2 4
person
MapFlatMap
Filter
1 hasLabel(‘android’)
SideEffect
1 store(‘x’) 1
x=[ ]1
Barrier
1 1
barrier()
1
2
1
out(‘knows’)
label()
1 2 4
person
MapFlatMap
Filter
1 hasLabel(‘android’)
SideEffect
1 store(‘x’) 1
x=[ ]1
Barrier
1 1
barrier()
1
2
Reducer
2 4
count()
2
g.V().has(‘name’,’gremlin’)
g.V().has(‘name’,’gremlin’).
out(‘bought’).aggregate(‘stash’)
stash
g.V().has(‘name’,’gremlin’).
out(‘bought’).aggregate(‘stash’).
in(‘bought’)
stash
g.V().has(‘name’,’gremlin’).
out(‘bought’).aggregate(‘stash’).
in(‘bought’).has(‘name’,neq(‘gremlin’))
stash
g.V().has(‘name’,’gremlin’).
out(‘bought’).aggregate(‘stash’).
in(‘bought’).has(‘name’,neq(‘gremlin’)).
out(‘bought’)
stash
Failure
Is
An
Option
g.V().has(‘name’,’gremlin’).
out(‘bought’).aggregate(‘stash’).
in(‘bought’).has(‘name’,neq(‘gremlin’)).
out(‘bought’).not(within(‘stash’))
stash
Failure
Is
An
Option
Failure
Is
An
Option
g.V().has(‘name’,’gremlin’).
out(‘bought’).aggregate(‘stash’).
in(‘bought’).has(‘name’,neq(‘gremlin’)).
out(‘bought’).not(within(‘stash’)).
groupCount()
stash
Failure
Is
An
Option
3
1
1
Failure
Is
An
Option
g.V().has(‘name’,’gremlin’).
out(‘bought’).aggregate(‘stash’).
in(‘bought’).has(‘name’,neq(‘gremlin’)).
out(‘bought’).not(within(‘stash’)).
groupCount().
order(local).by(values,decr)
stash
Failure
Is
An
Option
3
1
1
Failure
Is
An
Option
g.V().has(‘name’,’gremlin’).
out(‘bought’).aggregate(‘stash’).
in(‘bought’).has(‘name’,neq(‘gremlin’)).
out(‘bought’).not(within(‘stash’)).
groupCount().
order(local).by(values,decr).
select(keys).unfold().limit(1)
stash
Failure
Is
An
Option
3
1
1
repeat().until()
until().repeat()
choose().option()
choose()
as()
select()
do-while
while-do
if-then-else
switch
var set
var get
Turing Completeness
Every modern programming language
supports function chaining and nesting.
g.V(1).out(“knows”)
g.V(1).out(‘knows’)
g.V(1).out(‘knows’)
Quiz
id()
out(‘created’)
values(‘name’)
count()
where(‘a’,eq(‘b’))
property(‘name’,’marko’)
label()
has(‘age’)
groupCount()
groupCount(’m’)
barrier()
repeat(out()).times(2)
id()
out(‘created’)
values(‘name’)
count()
where(‘a’,eq(‘b’))
property(‘name’,’marko’)
label()
has(‘age’)
groupCount()
groupCount(’m’)
barrier()
repeat(out()).times(2)
flatmap: vertex vertex*
id()
out(‘created’)
values(‘name’)
count()
where(‘a’,eq(‘b’))
property(‘name’,’marko’)
label()
has(‘age’)
groupCount()
groupCount(’m’)
barrier()
repeat(out()).times(2)
flatmap: vertex vertex*
reducer: object* long
map: element object
flatmap: vertex vertex*
id()
out(‘created’)
values(‘name’)
count()
where(‘a’,eq(‘b’))
property(‘name’,’marko’)
label()
has(‘age’)
groupCount()
groupCount(’m’)
barrier()
repeat(out()).times(2)
flatmap: vertex vertex*
reducer: object* long
map: element object
map: element string
reducer: object* map<object,long>
filter: object object
side-effect: object , ( ), object
flatmap: vertex vertex*
id()
out(‘created’)
values(‘name’)
count()
where(‘a’,eq(‘b’))
property(‘name’,’marko’)
label()
has(‘age’)
groupCount()
groupCount(’m’)
barrier()
repeat(out()).times(2)
flatmap: vertex vertex*
reducer: object* long
map: element object
map: element string
reducer: object* map<object,long>
filter: object object
filter: element element
side-effect: object , ( ), object
flatmap: vertex vertex*
side-effect: element ) element
barrier: object* object* flatmap: element string*
The traversal guides the traverser down the
long and winding graph.
Ψ
G
T
T
The Traversers
G ⟵μ t∈T ψ⟶Ψ
Graph
Traversers
Traversal
PathBulk
G ⟵μ t∈T ψ⟶Ψ
Traversers
β,Δ,ς,ι
Sack Loops
g.V().both().id()
1
2
4
knows
3
5
6
created
created
knows created
created
β
g.V().both().id()
1
2
4
knows
3
5
6
created
created
knows created
created
ψ
g.V().both().id()
1
2
4
knows
3
5
6
created
created
knows created
created
ψ
1
2
4
knows
3
5
6
created
created
knows created
created
g.V().both().id()
ψ
1
2
4
knows
3
5
6
created
created
knows created
created
g.V().both().id()
ψ
2
1
11
5
4
4
4
3
33 6
1
2
4
knows
3
5
6
created
created
knows created
created
g.V().both().id()
ψ
2
1
11
5
4
4
4
3
33 6
==>2
==>5
==>1
==>1
==>1
==>4
==>4
==>4
==>3
==>3
==>3
==>6
1
2
4
knows
3
5
6
created
created
knows created
created
g.V().both().id()
ψ
2
1
11
5
4
4
4
3
33 6
id( )5id( )2
id( )1
id( )4 id( )4
id( )1
id( )1 id( )4
id( )3 id( )3
id( )3
id( )6
Its time to bulk up.
Rodriguez, M.A., “The Beauty of Bulking,” Gremlin-Users Mailing List, June 2015.
https://twitter.com/twarko/status/606513003090092035
1
2
4
knows
3
5
6
created
created
knows created
created
g.V().both().id()
ψ
1
2
4
knows
3
5
6
created
created
knows created
created
1
3
1
3
3 1
g.V().both().id()
ψ
1
2
4
knows
3
5
6
created
created
knows created
created
1
3
1
3
3 1
g.V().both().id()
ψ
2
1
5
4
3 6
1
2
4
knows
3
5
6
created
created
knows created
created
1
3
1
3
3 1
g.V().both().id()
ψ ==>2
==>5
==>1
==>1
==>1
==>4
==>4
==>4
==>3
==>3
==>3
==>61
4
3 6
2 5
1
2
4
knows
3
5
6
created
created
knows created
created
1
3
1
3
3 1
g.V().both().id()
ψ
1
4
3 6
id( )5id( )2
id( )1 id( )4
id( )3 id( )61/2 as many function
calls with bulking!
ballin’.
2 5
gremlin> g.V().both().id().explain()
==>Traversal Explanation
=======================================================================================================================
Original Traversal [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep]
ConnectiveStrategy [D] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep]
MatchPredicateStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep]
FilterRankingStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep]
InlineFilterStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep]
RepeatUnrollStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep]
PathRetractionStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep]
IncidentToAdjacentStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep]
AdjacentToIncidentStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep]
RangeByIsCountStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), IdStep]
LazyBarrierStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(2500), IdStep]
TinkerGraphCountStrategy [P] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(2500), IdStep]
TinkerGraphStepStrategy [P] [TinkerGraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(2500), IdStep]
ProfileStrategy [F] [TinkerGraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(2500), IdStep]
StandardVerificationStrategy [V] [TinkerGraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(2500), IdStep]
Final Traversal [TinkerGraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(2500), IdStep]
g.V().both().id()
g.V().both().barrier().id()
optimizes to
g.V(1).out(‘knows’).out(‘created’).path()
1
2
4
knows
3
5
6
created
created
knows created
created
ψ
[ ]1Δ=
g.V(1).out(‘knows’).out(‘created’).path()
1
2
4
knows
3
5
6
created
created
knows created
created
ψ
[ ]1 2
[ ]1 4
g.V(1).out(‘knows’).out(‘created’).path()
1
2
4
knows
3
5
6
created
created
knows created
created
ψ
[ ]1 4 5
[ ]1 4 3
g.V(1).out(‘knows’).out(‘created’).path()
1
2
4
knows
3
5
6
created
created
knows created
created
ψ
[ ]1 4 5
[ ]1 4 3
Its time to get crazy with
locally scoped traversers.
g.V(1).out(‘knows’).out(‘created’).limit(1).
path().by(inE().count())
1
2
4
knows
3
5
6
created
created
knows created
created
ψ
[ ]1 4 5
[ ]1 4 3
g.V(1).out(‘knows’).out(‘created’).limit(1).
path().by(inE().count())
1
2
4
knows
3
5
6
created
created
knows created
created
ψ
[ ]1 4 5
g.V(1).out(‘knows’).out(‘created’).limit(1).
path().by(inE().count())
1
2
4
knows
3
5
6
created
created
knows created
created
ψ
[ ]1 4 5
g.V(1).out(‘knows’).out(‘created’).limit(1).
path().by(inE().count())
1
2
4
knows
3
5
6
created
created
knows created
created
ψ
[ ]1 4 5
g.V(1).out(‘knows’).out(‘created’).limit(1).
path().by(inE().count())
1
2
4
knows
3
5
6
created
created
knows created
created
ψ
[ ]1 4 5
g.V(4).inE().count()
g.V(1).inE().count()
g.V(5).inE().count()
g.V(1).out(‘knows’).out(‘created’).limit(1).
path().by(inE().count())
1
2
4
knows
3
5
6
created
created
knows created
created
ψ
[ ]1 4 5
g.V(1).out(‘knows’).out(‘created’).limit(1).
path().by(inE().count())
1
2
4
knows
3
5
6
created
created
knows created
created
ψ
[ ]1 4 5
0
1
1
g.V(1).out(‘knows’).out(‘created’).limit(1).
path().by(inE().count())
1
2
4
knows
3
5
6
created
created
knows created
created
ψ
[ ]1 4 5
0
1
1
[ ]
g.V(1).out(‘knows’).out(‘created’).limit(1).
path().by(inE().count())
1
2
4
knows
3
5
6
created
created
knows created
created
ψ
0 1 1
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
ς
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
ψ
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
ψ
1
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
ψ
1
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
ψ
1
ι=0
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
ψ
1ι=0
1
ι=0
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
1.0ι=0
0.5
ι=0
ψ
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
ψ
1.0ι=0
0.5
ι=0
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
ψ
1.0ι=1
0.5
ι=1
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
1.0ι=1
0.5
ι=1
ψ
?
?
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
1.0ι=1
0.5
ι=1
ψ
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
1.0
ι=1
ψ
1.0
ι=1
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
0.4
ι=1
ψ
1.0
ι=1
ψ
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
0.4
ι=11.0
ι=1
ψ
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
0.4
ι=21.0
ι=2
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
0.4
ι=21.0
ι=2
ψ
!
!
g.withSack(1).V(1).
repeat(outE().sack(mult).by(‘weight’).inV()).
times(2).sack()
1
2
4
3
5
6
0.5
0.4
1.0 0.4
1.0
0.2
0.4
1.0
ψ
1.0
0.4
machine 2 machine 3machine 1
μ
ψ
The Gremlin traversal machine is a distributed virtual
machine — traversers can independently move around the
graph (across machines) and through the traversal.
ψt
ψt+1
μt+1
μt
Quiz
2 2 Barrier
2 3
To Bulk or Not To Bulk
2 2 Barrier
2 3
2
5
2 3 Barrier
2 3
To Bulk or Not To Bulk
5
2 2 Barrier
2 3
2
5
2 3 Barrier
2 3
2 3
2 3
[ ]1 4 5
5
3
[ ]1 2 5
1
Barrier
To Bulk or Not To Bulk
5
2 2 Barrier
2 3
2
5
2 3 Barrier
2 3
2 3
2 3
[ ]1 4 5
5
3
[ ]1 2 5
1
Barrier 5
[ ]1 4 5
5
3
[ ]1 2 5
1
4
1
4
3 72
Barrier
withSack(?,sum)
To Bulk or Not To Bulk
5
2 2 Barrier
2 3
2
5
2 3 Barrier
2 3
2 3
2 3
[ ]1 4 5
5
3
[ ]1 2 5
1
Barrier 5
[ ]1 4 5
5
3
[ ]1 2 5
1
4
1
4
3 72
Barrier
4
4 9
withSack(?,sum)
2 2 Barrier
2 3
ι=1 ι=2
To Bulk or Not To Bulk
5
2 2 Barrier
2 3
2
5
2 3 Barrier
2 3
2 3
2 3
[ ]1 4 5
5
3
[ ]1 2 5
1
Barrier 5
[ ]1 4 5
5
3
[ ]1 2 5
1
4
1
4
3 72
Barrier
4
4 9
withSack(?,sum)
2 2 Barrier
2 3
ι=1 ι=2
2 2
2 3
ι=1 ι=2
To Bulk or Not To Bulk
3 Barrier
3
5
2 2 Barrier
2 3
2
5
2 3 Barrier
2 3
2 3
2 3
[ ]1 4 5
5
3
[ ]1 2 5
1
Barrier 5
[ ]1 4 5
5
3
[ ]1 2 5
1
4
1
4
3 72
Barrier
4
4 9
withSack(?,sum)
2 2 Barrier
2 3
ι=1 ι=2
2 2
2 3
ι=1 ι=2
To Bulk or Not To Bulk
3 Barrier
3
3
3
The traverser is an atomic unit of computing —
moving through both the traversal and the graph.
G
The Graph
Lattice Scale-Free Random
Rodriguez, M.A., “Graphoendodonticology,” DataStax Blog, March 2017.
https://www.datastax.com/2017/03/graphoendodonticology
Lattice Scale-Free Random
Diamond Jackson Pollock TV Fuzz
g.V().groupCount().by(both().count())
Lattice Scale-Free Random0.00.20.40.6
degree
frequency
0 1 2 3 4 0 5 10 15 20 25 30
0.00.20.4
degree
frequency
many vertices have
few edges
few vertices have
many edges
2 4 6 8 10
0.050.100.15
degree
frequency
gaussian
distribution
Lattice Scale-Free Random
g.V().repeat(both()).times(x).count()
2 4 6 8 10
0.0e+001.0e+102.0e+103.0e+10
path length
pathcount
scale-free
random
lattice
Scale-Free
!
"I’m cool.
Scale-Free
!
"
Scale-Free
!
"
Scale-Free
!
"
Why am I
holding a bomb?
Scale-Free
hub!
!
"
Scale-Free
explosion!
!
Scale-Free
Scale-Free
!
!
!
!
!
Scale-Free
!
! !
!
!
diiiiiiiamn.
Facebook Subgraph
|V| = 4039
|E| = 88234
Aliev, A., Rodriguez, M.A., “Reducing
Computational Complexity with Correlate
Traversals,” DataStax Blog, April 2017.
https://www.datastax.com/2017/04/reducing-computational-complexity-with-correlate-traversals
!
Yo.
Facebook Subgraph
|V| = 4039
|E| = 88234
!
Facebook Subgraph
|V| = 4039
|E| = 88234
So you want to be a graph surgeon?
* The same traversal on different graphs can have
wildly different performance characteristics.
* Traversals on scale-free/natural
graphs can easily explode due to
“hubs” (super nodes).
* Use Gremlin to study the
graph’s statistics before
using Gremlin to query the
graph.
* Build your traversals up
piecewise and use profile()
to find bottlenecks.
knows knows
watchedwatched
watched
watched
watched
watched
watched
⭐⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐⭐⭐
knows knows
watchedwatched
watched
watched
watched
watched
watched
⭐⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐⭐⭐
"
g.V(1).count()
1
Sup.
knows knows
watchedwatched
watched
watched
watched
watched
watched
⭐⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐⭐⭐
g.V(1).out().count()
9
knows knows
watchedwatched
watched
watched
watched
watched
watched
⭐⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐⭐⭐
g.V(1).out(‘watched’).count()
7
knows knows
watchedwatched
watched
watched
watched
watched
watched
⭐⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐⭐⭐
g.V(1).outE(‘watched’)
knows knows
watchedwatched
watched
watched
watched
watched
watched
⭐⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐⭐⭐
g.V(1).outE(‘watched’).has(‘stars’,gte(3))
knows knows
watchedwatched
watched
watched
watched
watched
watched
⭐⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐⭐⭐
g.V(1).outE(‘watched’).has(‘stars’,gte(3))
knows knows
watchedwatched
watched
watched
watched
watched
watched
⭐⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐
⭐⭐⭐
g.V(1).outE(‘watched’).has(‘stars’,gte(3)).inV().count()
3
!
I’ll be more
selective this time.
Facebook Subgraph
|V| = 4039
|E| = 88234
Facebook Subgraph
|V| = 4039
|E| = 88234
Quiz
Controlling Traverser Populations
out(‘label’)
outE().has(‘key’,value).inV()
local(out().coin(0.5))
local(outE().sample(10).by(‘weight’).inV())
out()
local(out().limit(10))
Controlling Traverser Populations
out(‘label’)
outE().has(‘key’,value).inV()
local(out().coin(0.5))
local(outE().sample(10).by(‘weight’).inV())
out()
only edges in one direction
local(out().limit(10))
Controlling Traverser Populations
out(‘label’)
outE().has(‘key’,value).inV()
local(out().coin(0.5))
local(outE().sample(10).by(‘weight’).inV())
only edges w/ label
out()
only edges in one direction
local(out().limit(10))
Controlling Traverser Populations
out(‘label’)
outE().has(‘key’,value).inV()
local(out().coin(0.5))
local(outE().sample(10).by(‘weight’).inV())
only edges w/ label
only edges w/ property condition
out()
only edges in one direction
local(out().limit(10))
Controlling Traverser Populations
out(‘label’)
outE().has(‘key’,value).inV()
local(out().coin(0.5))
local(outE().sample(10).by(‘weight’).inV())
only edges w/ label
only edges w/ property condition
out()
only edges in one direction
local(out().limit(10))
only the first 10 edges
Controlling Traverser Populations
out(‘label’)
outE().has(‘key’,value).inV()
local(out().coin(0.5))
local(outE().sample(10).by(‘weight’).inV())
only edges w/ label
only edges w/ property condition
only 50% of edges
out()
only edges in one direction
local(out().limit(10))
only the first 10 edges
Controlling Traverser Populations
out(‘label’)
outE().has(‘key’,value).inV()
local(out().coin(0.5))
local(outE().sample(10).by(‘weight’).inV())
only edges w/ label
only edges w/ property condition
only 50% of edges
only 10 edges biased by weight
out()
only edges in one direction
local(out().limit(10))
only the first 10 edges
The computational complexity of the traversal is a
function of the structural complexity of the graph.
G ⟵μ t∈T ψ⟶Ψ
Graph
Traversers
Traversal
G ⟵μ t∈T ψ⟶Ψ
Graph
Traversers
Traversal
CPUs
ProgramData
The show is over.
Please tip your waitress and drive home safe!
traverse
1 of 191

More Related Content

Viewers also liked(11)

Gremlin's Graph Traversal MachineryGremlin's Graph Traversal Machinery
Gremlin's Graph Traversal Machinery
Marko Rodriguez8.3K views
Resource Aware Scheduling in Apache StormResource Aware Scheduling in Apache Storm
Resource Aware Scheduling in Apache Storm
DataWorks Summit/Hadoop Summit93.1K views
Realtime Analytics with Storm and HadoopRealtime Analytics with Storm and Hadoop
Realtime Analytics with Storm and Hadoop
DataWorks Summit238.1K views
Apache REEF - stdlib for big dataApache REEF - stdlib for big data
Apache REEF - stdlib for big data
Sergiy Matusevych30.4K views
Yahoo compares Storm and SparkYahoo compares Storm and Spark
Yahoo compares Storm and Spark
Chicago Hadoop Users Group198.4K views

Similar to Gremlin 101.3 On Your FM Dial

Introduction to goIntroduction to go
Introduction to goJaehue Jang
2.9K views45 slides
Golang勉強会Golang勉強会
Golang勉強会Shin Sekaryo
725 views32 slides
GroovyGroovy
GroovyZen Urban
1.1K views35 slides

Similar to Gremlin 101.3 On Your FM Dial(20)

Introduction to goIntroduction to go
Introduction to go
Jaehue Jang2.9K views
Golang勉強会Golang勉強会
Golang勉強会
Shin Sekaryo725 views
GroovyGroovy
Groovy
Zen Urban1.1K views
Clojure for Java developers - StockholmClojure for Java developers - Stockholm
Clojure for Java developers - Stockholm
Jan Kronquist1.7K views
golang_getting_started.pptxgolang_getting_started.pptx
golang_getting_started.pptx
Guy Komari99 views
Clojure Small IntroClojure Small Intro
Clojure Small Intro
John Vlachoyiannis603 views
A Prettier PrinterA Prettier Printer
A Prettier Printer
jlongster262 views
A CTF Hackers ToolboxA CTF Hackers Toolbox
A CTF Hackers Toolbox
Stefan 5.8K views
AST Transformations at JFokusAST Transformations at JFokus
AST Transformations at JFokus
HamletDRC945 views
The Curious Clojurist - Neal Ford (Thoughtworks)The Curious Clojurist - Neal Ford (Thoughtworks)
The Curious Clojurist - Neal Ford (Thoughtworks)
jaxLondonConference2.4K views
Five Languages in a MomentFive Languages in a Moment
Five Languages in a Moment
Sergio Gil787 views
Start Writing GroovyStart Writing Groovy
Start Writing Groovy
Evgeny Goldin1K views
(first '(Clojure.))(first '(Clojure.))
(first '(Clojure.))
niklal458 views

More from Marko Rodriguez(20)

Gremlin 101.3 On Your FM Dial