2011-11-01 | 10:40 AM - 11:40 AM Doctor Who is the world’s longest running science-fiction TV series. Battling daleks, cybermen and sontarans, and always accompanied by his trusted human companions, the last Timelord has saved earth from destruction more times than you’ve cursed Maven. Neo4j is the world’s leading open source graph database. Designed to interrogate densely connected data with lightning speed, it lets you traverse millions of nodes in a fraction of the time it takes to run a multi-join SQLquery. When these two meet, the result is an entertaining introduction to the complex history of a complex hero, and a rapid survey of the elegant APIs of a delightfully simple graph database. Armed only with a data store packed full of geeky Doctor Who facts, by the end of this session we’ll have you answering questions of the Doctor Who universe like a die-hard fan.

1. An
Introduc+on
to
Doctor
Who
(and
Neo4j)
@iansrobinson
#neo4j

2. Neo4j
is
a
Graph
Database
#neo4j

3. Property
Graph
#neo4j

4. Property
Graph
#neo4j

5. Neo4j
#neo4j

6. #neo4j

7. 32
billion
nodes
32
billion
rela+onships
64
billion
proper+es

8. #neo4j

9. Community
Advanced
Enterprise

10. stole
companion
from
loves
loves
appeared
enemy
in
companion
appeared
in
appeared
in
enemy
enemy
appeared
appeared
A
Good
Man
in
in
Goes
to
War
Victory
of
the
Daleks
appeared
in

11. GraphDatabaseService db = new EmbeddedGraphDatabase("/data/drwho");

12. GraphDatabaseService db = new EmbeddedGraphDatabase("/data/drwho");
Node theDoctor = db.createNode();
theDoctor.setProperty("name", "The Doctor");
Node daleks = db.createNode();
daleks.setProperty("name", "Daleks");
Node cybermen = db.createNode();
cybermen.setProperty("name", "Cybermen");

13. GraphDatabaseService db = new EmbeddedGraphDatabase("/data/drwho");
Node theDoctor = db.createNode();
enemy
theDoctor.setProperty("name", "The Doctor");
Node daleks = db.createNode();
daleks.setProperty("name", "Daleks");
Node cybermen = db.createNode();
cybermen.setProperty("name", "Cybermen");
enemy
theDoctor.createRelationshipTo(daleks,
DynamicRelationshipType.withName("enemy"));
theDoctor.createRelationshipTo(cybermen,
DynamicRelationshipType.withName("enemy"));

14. GraphDatabaseService db = new EmbeddedGraphDatabase("/data/drwho");
Transaction tx = db.beginTx();
try {
Node theDoctor = db.createNode();
enemy
theDoctor.setProperty("name", "The Doctor");
Node daleks = db.createNode();
daleks.setProperty("name", "Daleks");
Node cybermen = db.createNode();
cybermen.setProperty("name", "Cybermen");
enemy
theDoctor.createRelationshipTo(daleks,
DynamicRelationshipType.withName("enemy"));
theDoctor.createRelationshipTo(cybermen,
DynamicRelationshipType.withName("enemy"));
tx.success();
} finally {
tx.finish();
}

15. How
do
I
query
the
data?
hOp://opfm.jpl.nasa.gov/
hOp://news.xinhuanet.com
#neo4j

16. Dalek
Props
hOp://www.dalek6388.co.uk/
#neo4j

17. species:Dalek
APPEARED_IN
!tle:Power
of
the
Daleks
USED_IN
props:Daleks
MEMBER_OF
MEMBER_OF
name:Dalek
Six-­‐5
name:Dalek
1
name:Dalek
2
name:Dalek
7
type:shoulders
type:skirt

18. name:Dalek
1
name:Dalek
2
name:Dalek
7
name:Dalek
Six-­‐5
ORIGINAL_PROP
ORIGINAL_PROP
ORIGINAL_PROP
ORIGINAL_PROP
name:Dalek
6
name:Dalek
5

19. !tle:The
!tle:Power
of
!tle:The
Dalek
the
Daleks
Daleks
Invasion
of
Earth
name:
name:Dalek
1
name:Dalek
2
name:Dalek
7
Dalek
Six-­‐5
name:Dalek
Two-­‐1
name:Dalek
One-­‐5
name:Dalek
Six-­‐7

20. Supply
Chain
Traceability
#neo4j

21. Traversal
Framework
• Visits
(and
returns)
nodes
based
on
traversal
descrip+on
• Powerful;
can
customize:
– Rela+onships
followed
– Branch
selec+on
policy
– Node/rela+onship
uniqueness
constraints
– Path
evaluators
#neo4j

22. Node theDaleks = database.index().forNodes("species").get("name", "Dalek").getSingle();
Traverser traverser = Traversal.description()
.depthFirst()
.relationships(Rels.APPEARED_IN, Direction.OUTGOING)
.relationships(Rels.USED_IN, Direction.INCOMING)
.relationships(Rels.MEMBER_OF, Direction.INCOMING)
.relationships(Rels.COMPOSED_OF, Direction.OUTGOING)
.relationships(Rels.ORIGINAL_PROP, Direction.OUTGOING)
.evaluator(new Evaluator() {
@Override
public Evaluation evaluate(Path path) {
if (path.lastRelationship() != null && path.lastRelationship().isType(Rels.ORIGINAL_PROP)){
return Evaluation.INCLUDE_AND_PRUNE;
}
return Evaluation.EXCLUDE_AND_CONTINUE;
}
})
.uniqueness(Uniqueness.NONE)
.traverse(theDaleks); Description

23. Node theDaleks = database.index().forNodes("species").get("name", "Dalek").getSingle();
Traverser traverser = Traversal.description()
.depthFirst()
.relationships(Rels.APPEARED_IN, Direction.OUTGOING)
.relationships(Rels.USED_IN, Direction.INCOMING)
.relationships(Rels.MEMBER_OF, Direction.INCOMING)
.relationships(Rels.COMPOSED_OF, Direction.OUTGOING)
.relationships(Rels.ORIGINAL_PROP, Direction.OUTGOING)
.evaluator(new Evaluator() {
@Override
public Evaluation evaluate(Path path) {
if (path.lastRelationship() != null && path.lastRelationship().isType(Rels.ORIGINAL_PROP)){
return Evaluation.INCLUDE_AND_PRUNE;
}
return Evaluation.EXCLUDE_AND_CONTINUE;
}
})
.uniqueness(Uniqueness.NONE)
.traverse(theDaleks); Index Lookup

24. Node theDaleks = database.index().forNodes("species").get("name", "Dalek").getSingle();
Traverser traverser = Traversal.description()
.depthFirst()
.relationships(Rels.APPEARED_IN, Direction.OUTGOING)
.relationships(Rels.USED_IN, Direction.INCOMING)
.relationships(Rels.MEMBER_OF, Direction.INCOMING)
.relationships(Rels.COMPOSED_OF, Direction.OUTGOING)
.relationships(Rels.ORIGINAL_PROP, Direction.OUTGOING)
.evaluator(new Evaluator() {
@Override
public Evaluation evaluate(Path path) {
if (path.lastRelationship() != null && path.lastRelationship().isType(Rels.ORIGINAL_PROP)){
return Evaluation.INCLUDE_AND_PRUNE;
}
return Evaluation.EXCLUDE_AND_CONTINUE;
}
})
.uniqueness(Uniqueness.NONE)
.traverse(theDaleks); Branching

25. Node theDaleks = database.index().forNodes("species").get("name", "Dalek").getSingle();
Traverser traverser = Traversal.description()
.depthFirst()
.relationships(Rels.APPEARED_IN, Direction.OUTGOING)
.relationships(Rels.USED_IN, Direction.INCOMING)
.relationships(Rels.MEMBER_OF, Direction.INCOMING)
.relationships(Rels.COMPOSED_OF, Direction.OUTGOING)
.relationships(Rels.ORIGINAL_PROP, Direction.OUTGOING)
.evaluator(new Evaluator() {
@Override
public Evaluation evaluate(Path path) {
if (path.lastRelationship() != null && path.lastRelationship().isType(Rels.ORIGINAL_PROP)){
return Evaluation.INCLUDE_AND_PRUNE;
}
return Evaluation.EXCLUDE_AND_CONTINUE;
}
})
.uniqueness(Uniqueness.NONE)
.traverse(theDaleks); Relationships

26. Node theDaleks = database.index().forNodes("species").get("name", "Dalek").getSingle();
Traverser traverser = Traversal.description()
.depthFirst()
.relationships(Rels.APPEARED_IN, Direction.OUTGOING)
.relationships(Rels.USED_IN, Direction.INCOMING)
.relationships(Rels.MEMBER_OF, Direction.INCOMING)
.relationships(Rels.COMPOSED_OF, Direction.OUTGOING)
.relationships(Rels.ORIGINAL_PROP, Direction.OUTGOING)
.evaluator(new Evaluator() {
@Override
public Evaluation evaluate(Path path) {
if (path.lastRelationship() != null && path.lastRelationship().isType(Rels.ORIGINAL_PROP)){
return Evaluation.INCLUDE_AND_PRUNE;
}
return Evaluation.EXCLUDE_AND_CONTINUE;
}
})
.uniqueness(Uniqueness.NONE)
.traverse(theDaleks); Evaluator

27. Node theDaleks = database.index().forNodes("species").get("name", "Dalek").getSingle();
Traverser traverser = Traversal.description()
.depthFirst()
.relationships(Rels.APPEARED_IN, Direction.OUTGOING)
.relationships(Rels.USED_IN, Direction.INCOMING)
.relationships(Rels.MEMBER_OF, Direction.INCOMING)
.relationships(Rels.COMPOSED_OF, Direction.OUTGOING)
.relationships(Rels.ORIGINAL_PROP, Direction.OUTGOING)
.evaluator(new Evaluator() {
@Override
public Evaluation evaluate(Path path) {
if (path.lastRelationship() != null && path.lastRelationship().isType(Rels.ORIGINAL_PROP)){
return Evaluation.INCLUDE_AND_PRUNE;
}
return Evaluation.EXCLUDE_AND_CONTINUE;
}
})
.uniqueness(Uniqueness.NONE)
.traverse(theDaleks); Uniqueness

28. Node theDaleks = database.index().forNodes("species").get("name", "Dalek").getSingle();
Traverser traverser = Traversal.description()
.depthFirst()
.relationships(Rels.APPEARED_IN, Direction.OUTGOING)
.relationships(Rels.USED_IN, Direction.INCOMING)
.relationships(Rels.MEMBER_OF, Direction.INCOMING)
.relationships(Rels.COMPOSED_OF, Direction.OUTGOING)
.relationships(Rels.ORIGINAL_PROP, Direction.OUTGOING)
.evaluator(new Evaluator() {
@Override
public Evaluation evaluate(Path path) {
if (path.lastRelationship() != null && path.lastRelationship().isType(Rels.ORIGINAL_PROP)){
return Evaluation.INCLUDE_AND_PRUNE;
}
return Evaluation.EXCLUDE_AND_CONTINUE;
}
})
.uniqueness(Uniqueness.NONE)
.traverse(theDaleks); Traverse

29. Iterable<Path>
path.startNode() path.endNode()
+tle:Power
of
species:Dalek
the
Daleks
props:Daleks
name:Dalek
7
type:skirt
name:Dalek
7
APPEARED_IN
USED_IN
MEMBER_OF
COMPOSED_OF
ORIGINAL_PROP
+tle:Power
of
species:Daleks
the
Daleks
props:Daleks
name:Dalek
7
type:shoulder
name:Dalek
7
APPEARED_IN
USED_IN
MEMBER_OF
COMPOSED_OF
ORIGINAL_PROP
+tle:Power
of
species:Daleks
the
Daleks
props:Daleks
name:Dalek
Six-­‐5
type:skirt
name:Dalek
5
APPEARED_IN
USED_IN
MEMBER_OF
COMPOSED_OF
ORIGINAL_PROP

30. Cypher
• Declara+ve
graph
pa8ern
matching
language
– Humane
regular
expressions
for
graphs
• New
in
1.4
– Con+nuing,
rapid
feature
development
• Supports
queries
– Including
aggrega+on,
ordering
and
limits
– Muta+ng
opera+ons
coming
in
1.6
#neo4j

31. Cypher Query
start
daleks=node:species(species='Dalek')
match
daleks-­‐[:APPEARED_IN]-­‐>episode<-­‐[:USED_IN]-­‐
()<-­‐[:MEMBER_OF]-­‐()-­‐[:COMPOSED_OF]-­‐>
part-­‐[:ORIGINAL_PROP]-­‐>originalprop
return
originalprop.name,
part.type,
count(episode.Qtle)
order
by
count(episode.Qtle)
desc
limit
1

32. Index Lookup
start
daleks=node:species(species='Dalek')
match
daleks-­‐[:APPEARED_IN]-­‐>episode<-­‐[:USED_IN]-­‐
()<-­‐[:MEMBER_OF]-­‐()-­‐[:COMPOSED_OF]-­‐>
part-­‐[:ORIGINAL_PROP]-­‐>originalprop
return
originalprop.name,
part.type,
count(episode.Qtle)
order
by
count(episode.Qtle)
desc
limit
1

33. Match Nodes & Relationships
start
daleks=node:species(species='Dalek')
match
daleks-­‐[:APPEARED_IN]-­‐>episode<-­‐[:USED_IN]-­‐
()<-­‐[:MEMBER_OF]-­‐()-­‐[:COMPOSED_OF]-­‐>
part-­‐[:ORIGINAL_PROP]-­‐>originalprop
return
originalprop.name,
part.type,
count(episode.Qtle)
order
by
count(episode.Qtle)
desc
limit
1

34. match
daleks-­‐[:APPEARED_IN]-­‐>episode<-­‐[:USED_IN]-­‐
()<-­‐[:MEMBER_OF]-­‐()-­‐[:COMPOSED_OF]-­‐>
part-­‐[:ORIGINAL_PROP]-­‐>originalprop
APPEARED_IN
daleks
episode
USED_IN
MEMBER_OF
COMPOSED_OF
ORIGINAL_PROP
part
originalprop

35. Return Values
start
daleks=node:species(species='Dalek')
match
daleks-­‐[:APPEARED_IN]-­‐>episode<-­‐[:USED_IN]-­‐
()<-­‐[:MEMBER_OF]-­‐()-­‐[:COMPOSED_OF]-­‐>
part-­‐[:ORIGINAL_PROP]-­‐>originalprop
return
originalprop.name,
part.type,
count(episode.Qtle)
order
by
count(episode.Qtle)
desc
limit
1

36. In Code
CypherParser
parser
=
new
CypherParser();
ExecuQonEngine
engine
=
new
ExecuQonEngine(universe.getDatabase());
String
cql
=
"start
daleks=node:species(species='Dalek')"
+
"
match
daleks-­‐[:APPEARED_IN]-­‐>episode<-­‐[:USED_IN]-­‐()<-­‐[:MEMBER_OF]-­‐()"
+
"-­‐[:COMPOSED_OF]-­‐>part-­‐[:ORIGINAL_PROP]-­‐>originalprop"
+
"
return
originalprop.name,
part.type,
count(episode.Qtle)"
+
"
order
by
count(episode.Qtle)
desc
limit
1";
Query
query
=
parser.parse(cql);
ExecuQonResult
result
=
engine.execute(query);
String
originalProp
=
result.javaColumnAs("originalprop.name").next().toString();
String
part
=
result.javaColumnAs("part.type").next().toString();
int
episodeCount
=
(Integer)
result.javaColumnAs("count(episode.Qtle)").next();

37. Setup
CypherParser
parser
=
new
CypherParser();
ExecuQonEngine
engine
=
new
ExecuQonEngine(universe.getDatabase());
String
cql
=
"start
daleks=node:species(species='Dalek')"
+
"
match
daleks-­‐[:APPEARED_IN]-­‐>episode<-­‐[:USED_IN]-­‐()<-­‐[:MEMBER_OF]-­‐()"
+
"-­‐[:COMPOSED_OF]-­‐>part-­‐[:ORIGINAL_PROP]-­‐>originalprop"
+
"
return
originalprop.name,
part.type,
count(episode.Qtle)"
+
"
order
by
count(episode.Qtle)
desc
limit
1";
Query
query
=
parser.parse(cql);
ExecuQonResult
result
=
engine.execute(query);
String
originalProp
=
result.javaColumnAs("originalprop.name").next().toString();
String
part
=
result.javaColumnAs("part.type").next().toString();
int
episodeCount
=
(Integer)
result.javaColumnAs("count(episode.Qtle)").next();

38. Parse & Execute
CypherParser
parser
=
new
CypherParser();
ExecuQonEngine
engine
=
new
ExecuQonEngine(universe.getDatabase());
String
cql
=
"start
daleks=node:species(species='Dalek')"
+
"
match
daleks-­‐[:APPEARED_IN]-­‐>episode<-­‐[:USED_IN]-­‐()<-­‐[:MEMBER_OF]-­‐()"
+
"-­‐[:COMPOSED_OF]-­‐>part-­‐[:ORIGINAL_PROP]-­‐>originalprop"
+
"
return
originalprop.name,
part.type,
count(episode.Qtle)"
+
"
order
by
count(episode.Qtle)
desc
limit
1";
Query
query
=
parser.parse(cql);
ExecuQonResult
result
=
engine.execute(query);
String
originalProp
=
result.javaColumnAs("originalprop.name").next().toString();
String
part
=
result.javaColumnAs("part.type").next().toString();
int
episodeCount
=
(Integer)
result.javaColumnAs("count(episode.Qtle)").next();

39. Results
CypherParser
parser
=
new
CypherParser();
ExecuQonEngine
engine
=
new
ExecuQonEngine(universe.getDatabase());
String
cql
=
"start
daleks=node:species(species='Dalek')"
+
"
match
daleks-­‐[:APPEARED_IN]-­‐>episode<-­‐[:USED_IN]-­‐()<-­‐[:MEMBER_OF]-­‐()"
+
"-­‐[:COMPOSED_OF]-­‐>part-­‐[:ORIGINAL_PROP]-­‐>originalprop"
+
"
return
originalprop.name,
part.type,
count(episode.Qtle)"
+
"
order
by
count(episode.Qtle)
desc
limit
1";
Query
query
=
parser.parse(cql);
ExecuQonResult
result
=
engine.execute(query);
String
originalProp
=
result.javaColumnAs("originalprop.name").next().toString();
String
part
=
result.javaColumnAs("part.type").next().toString();
int
episodeCount
=
(Integer)
result.javaColumnAs("count(episode.Qtle)").next();

40. In Webadmin

41. The
Hardest
Working
Prop
Part
Dalek
One’s
shoulders
hOp://www.dalek6388.co.uk/
#neo4j

42. Download
hOp://neo4j.org/download/
hOp://www.springsource.org/
spring-­‐data/neo4j
#neo4j

43. Tutorial
hOps://github.com/jimwebber/neo4j-­‐tutorial
#neo4j

44. Dematerialize…
@iansrobinson
ian.robinson@neotechnology.com
#neo4j

45. 1. Subtract:
Select
rectangle
then
3
black
triangles
2. Union:
All

46. 1. Subtract:
Select
rectangle
then
3
black
triangles
2. Union:
All

47. 1. Subtract:
Select
rectangle
then
3
black
triangles
2. Union:
All

48. match
(daleks)-­‐[:APPEARED_IN]-­‐>(episode)<-­‐[:USED_IN]-­‐
()<-­‐[:MEMBER_OF]-­‐()-­‐[:COMPOSED_OF]-­‐>
(part)-­‐[:ORIGINAL_PROP]-­‐>(originalprop)
APPEARED_IN
daleks
episode
USED_IN
MEMBER_OF
COMPOSED_OF
ORIGINAL_PROP
part
originalprop