Graphs are brilliant. They're really well studied and have some amazing properties, especially for predictive analytics. But then there's AI: it's super-cool and also has some amazing abilities to guess the future given the past. So, which are we meant to choose? I'd argue we should use both. In this talk we'll see how graphs give us a framework for contextualizing the world around us. We'll explore how simple rules from graph theory can evolve our model to show how it might be in the future. But that's not all, we'll also see how we can take our graphs and feed them into our ML pipelines for better scores than simple row-wise data using graph neural nets. And we'll see how to learn on graphs directly with graph convolutional networks. Finally, we'll close the loop by asking our machine learning to tell us about other queries we should be running against the input graph to find patterns in data we don’t even know are valuable today.

1. 10/07/2019
1
Graphs for AI and ML
Dr. Jim Webber
Chief Scientist, Neo4j
@jimwebber
● Some no-BS definitions
● Graphs and an accidental Skynet
● Graph theory
● Contemporary graph ML
● The future of graph AI
Overview

2. 10/07/2019
2
● ML - Machine Learning
○ Finding functions from historical data to guide future
interactions within a given domain
● AI - Artificial Intelligence
● The property of a system that it appears intelligent to its
users
● Often, but not always, using ML techniques
● Or ML implementations that can be cheaply retrained to
address neighbouring domains
A Bluffer’s Guide to AI-cronyms
● Predictive analytics
● Use past data to predict the future
● General purpose AI
● ML with transfer learning such that learned experiences in
one domain can be applied elsewhere
● Human-like AI
Often conflated with

3. 10/07/2019
3
ML all the things
What we do today

4. 10/07/2019
4
Extract all the features!
• What do we do? Turn it to
vectors and pump it through a
classification or regression
model
• That’s actually not a bad
thing
• But we can do so much before
we even get to ML…
• … if we have graph data
Credit: Graph Algorithms, Holder and Needham, O’Reilly 2019

5. 10/07/2019
5
http://www.bbc.co.uk/london/travel/downloads/tube_map.html
• Nodes with optional properties and optional labels
• Named, directed relationships with optional properties
• Relationships have exactly one start and end node
• Which may be the same node
Labeled Property graph model

6. 10/07/2019
6
stole
from
loves
loves
enemy
enemy
A Good
Man Goes
to War
appeared
in
appeared
in
appeared
in
appeared
in
Victory of
the Daleks
appeared
in
appeared
in
companion
companion
enemy
stole
from
loves
loves
enemy
enemy
A Good
Man Goes
to War
appeared
in
appeared
in
appeared
in
appeared
in
Victory of
the Daleks
appeared
in
appeared
in
companion
companion
enemy
planet
prop
species
species
species
character
character
character
episode
episode

7. 10/07/2019
7
R..ted R..ted
R..ted R..ted
R..ted
R..ted
R..ted
R..ted
R..ted
R..ted
R..ted
R
..ted
R..ted
R..ted
R..ted

8. 10/07/2019
8
Fearless querying
MATCH path = (:author {name:’Jim Webber’}
-[*]->(:character {name:’The Doctor’})
RETURN path
OR
MATCH (me:author {name:’Jim Webber’},
(doc:character {name:’The Doctor’}),
path = shortestPath((me)-[*]->(doc))
RETURN path
regeneration
character
actor
character
actor
landlord
author
tenant
flat
book
author
wrote
wrote
owns
sublet
played
seriesin
in
played
played
character
played

9. 10/07/2019
9
Take a step back
We can be smarter about this

10. 10/07/2019
10
Realtime Predictive Analytics
(circa 2008)
+ +
=

11. 10/07/2019
11
Not AI, but extremely effective
Credit: https://medium.com/basecs/breaking-down-breadth-first-search-cebe696709d9
Credit:
https://www.networkworld.com/article/3211410
/lan-wan/the-10-most-powerful-companies-in-
enterprise-networking.html

12. 10/07/2019
12
Toolkit matures into
proper database
• Cypher and Neo4j server
make real time graph
analytical patterns simple to
apply
• Amazing and humane to
implement
Firstname:
Mickey
Surname: Smith
DoB: 19781006
SKU: 5e175641
Product:
Badgers
Nadgers Ale
SKU: 2555f258
Product:
Peewee Pilsner
Category: beer
SKU: 49d102bc
Product: Baby
Dry Nights
Category:
nappies
Category: baby Category:
alcoholic
drinks
SKU: 49d102bc
Product: XBox
360
Category:
consumer
electronics
Category:
console
BOUGHT
BOUGHTBOUGHT
BOUGHT
MEMBER_OF
MEMBER_OF
MEMBER_OFMEMBER_OF
MEMBER_OFMEMBER_OF
MEMBER_OF

13. 10/07/2019
13
Firstname: *
Surname: *
DoB: 1996 > x
> 1972
Category: beerCategory:
nappies
BOUGHT
BOUGHT
BOUGHTCategory: game
console
Young fathers pattern

14. 10/07/2019
14
Firstname: *
Surname: *
DoB: 1996 > x
> 1972
Category: beerCategory:
nappies
BOUGHT
BOUGHT
!BOUGHTCategory: game
console
Business opportunity

15. 10/07/2019
15
(beer)(nappies)
(console)
(daddy)
<-[:MEMBER_OF]-
()
<-[:BOUGHT]-
-[:MEMBER_OF]->
()
-[:BOUGHT]->
<-[:MEMBER_OF]-
<-[:BOUGHT]-()
(d)-[:BOUGHT]->()-[:MEMBER_OF]->(n)
(d)-[:BOUGHT]->()-[:MEMBER_OF]->(b)
(d)-[:BOUGHT]->()-[:MEMBER_OF]->(c)
Flatten the graph

16. 10/07/2019
16
(d:Person)-[:BOUGHT]->()-[:MEMBER_OF]->(n:Category)
(d:Person)-[:BOUGHT]->()-[:MEMBER_OF]->(b:Category)
(d:Person)-[:BOUGHT]->()-[:MEMBER_OF]->(c:Category)
Include any labels
MATCH (d:Person)-[:BOUGHT]->()-[:MEMBER_OF]->(n:Category),
(d:Person)-[:BOUGHT]->()-[:MEMBER_OF]->(b:Category)
Add a MATCH clause

17. 10/07/2019
17
MATCH (d:Person)-[:BOUGHT]->()-[:MEMBER_OF]->(n:Category),
(d:Person)-[:BOUGHT]->()-[:MEMBER_OF]->(b:Category),
(c:Category)
WHERE NOT((d)-[:BOUGHT]->()-[:MEMBER_OF]->(c))
Constrain the Pattern
MATCH (d:Person)-[:BOUGHT]->()-[:MEMBER_OF]->(n:Category),
(d:Person)-[:BOUGHT]->()-[:MEMBER_OF]->(b:Category),
(c:Category)
WHERE n.category = "nappies" AND
b.category = "beer" AND
c.category = "console" AND
NOT((d)-[:BOUGHT]->()-[:MEMBER_OF]->(c))
Add property constraints

18. 10/07/2019
18
MATCH (d:Person)-[:BOUGHT]->()-[:MEMBER_OF]->(n:Category),
(d:Person)-[:BOUGHT]->()-[:MEMBER_OF]->(b:Category),
(c:Category)
WHERE n.category = "nappies" AND
b.category = "beer" AND
c.category = "console" AND
NOT((d)-[:BOUGHT]->()-[:MEMBER_OF]->(c))
RETURN DISTINCT d AS daddy
Profit!
==> +---------------------------------------------+
==> | daddy |
==> +---------------------------------------------+
==> | Node[15]{name:"Rory Williams",dob:19880121} |
==> +---------------------------------------------+
==> 1 row
==> 0 ms
==>
neo4j-sh (0)$
Results

19. 10/07/2019
19
Which sushi restaurants
in NYC do my friends
like?
Facebook Graph Search
See http://maxdemarzi.com/

20. 10/07/2019
20
Graph Structure
Simple Query, Intelligent Results
MATCH (:Person {name: 'Jim'})
-[:IS_FRIEND_OF]->(:Person)
-[:LIKES]->(restaurant:Restaurant)
-[:LOCATED_IN]->(:Place {location: 'New York'}),
(restaurant)-[:SERVES]->(:Cuisine {cuisine: 'Sushi'})
RETURN restaurant

21. 10/07/2019
21
Search structure
Graph Theory
• Rich knowledge of how graphs
operate in many domains
• Off the shelf algorithms to
process those graphs for
information, insight,
predictions
• Low barrier to entry
• Amazingly powerful

22. 10/07/2019
22
Triadic Closure
name: Kyle
name: Stan name: Kenny
FRIENDFRIEND

23. 10/07/2019
23
Triadic Closure
name: Kyle
name: Stan name: Kenny
FRIEND
FRIEND
name: Kyle
name: Stan name: Kenny
FRIENDFRIEND
FRIEND
Structural Balance
name:
Cartman
name: Craig name: Tweek
ENEMYFRIEND

24. 10/07/2019
24
Structural Balance
name:
Cartman
name: Craig name: Tweek
ENEMYFRIEND
name:
Cartman
name: Craig name: Tweek
ENEMYFRIEND
FRIEND
Structural Balance
name:
Cartman
name: Craig name: Tweek
ENEMYFRIEND
name:
Cartman
name: Craig name: Tweek
ENEMYFRIEND
ENEMY

25. 10/07/2019
25
Structural Balance
name: Kyle
name: Stan name: Kenny
FRIENDFRIEND
name: Kyle
name: Stan name: Kenny
FRIENDFRIEND
FRIEND

26. 10/07/2019
26
Structural Balance is a key
predictive technique
And it’s domain-agnostic

27. 10/07/2019
27
Allies and Enemies
UK
GermanyFrance
Russia Italy
Austria
Allies and Enemies
UK
GermanyFrance
Russia Italy
Austria

28. 10/07/2019
28
Allies and Enemies
UK
GermanyFrance
Russia Italy
Austria
Allies and Enemies
UK
GermanyFrance
Russia Italy
Austria

29. 10/07/2019
29
Allies and Enemies
UK
GermanyFrance
Russia Italy
Austria
Allies and Enemies
UK
GermanyFrance
Russia Italy
Austria

30. 10/07/2019
30
Predicting WWI
[Easley and Kleinberg]

31. 10/07/2019
31
It if a node has strong relationships to two neighbours, then these
neighbours must have at least a weak relationship between them.
[Wikipedia]
Strong Triadic Closure
Triadic Closure
(weak relationship)
name: Kenny
name: Stan name: Cartman
FRIENDFRIEND

32. 10/07/2019
32
Triadic Closure
(weak relationship)
name: Kenny
name: Stan name: Cartman
FRIENDFRIEND
name: Kenny
name: Stan name: Cartman
FRIENDFRIEND
FRIEND 50%
• Relationships can have “strength” as well as intent
• Think: weighting on a relationship in a property graph
• Weak links play another super-important structural role in graph
theory
• They bridge neighbourhoods
Weak relationships

33. 10/07/2019
33
Local Bridges
FRIEND
name: Kenny
name: Stanname: Kyle
FRIENDFRIEND
FRIEND
name: Sally
name: Bebename: Wendy
FRIENDFRIEND
FRIEND 50%
name:
Cartman
FRIEND
FRIEND
50%ENEMY
“If a node A in a network satisfies the Strong Triadic Closure Property
and is involved in at least two strong relationships, then any local
bridge it is involved in must be a weak relationship.”
[Easley and Kleinberg]
Local Bridge Property

34. 10/07/2019
34
University Karate Club
• (NP) Hard problem
• Repeatedly remove the spanning links between dense regions
• Or recursively merge nodes into ever larger “subgraph” nodes
• Choose your algorithm carefully – some are better than others
for a given domain
• Can use to (almost exactly) predict the
break up of the karate club!
Graph Partitioning

35. 10/07/2019
35
University Karate Clubs
(predicted by Graph Theory)
9
University Karate Clubs
(what actually happened!)

36. 10/07/2019
36
• Label Propagation
• Union Find / Weakly Connected Components
• Strongly Connected Components
• Triangle-Count / Clustering Coefficient
ClusteringCentrality
• PageRank
• Betweenness
• Closeness
• Degree
Path Finding
• Breadth-first search
• Depth-first search
• Single-source shortest
path
• All-pairs shortest path
• Minimum weight spanning
tree
Graph Algorithms in Neo4j

37. 10/07/2019
37
Amazing Native Graph Performance
Credit: https://reezocar.blob.core.windows.net/blog/2015/09/k2000.jpg

38. 10/07/2019
38
Find and stop spammers
Extract graph structure over time
Not message content!
(Fakhraei et al, KDD 2015)
Learning to stop bad guys
Result: find and classify 70% spammers with 90% accuracy
Much of modern graph ML is still about turning graphs to vectors
Graph2Vec and friends
Highly complementary techniques
Mixing structural data and features gives better results
Better data into the model, better results out
But we don’t have to always vectorize graphs...
Graph ML

39. 10/07/2019
39
Knowledge Graphs
• Semantic domain knowledge for
inference and understanding
• E.g. eBay Google Assistant
• What’s the next best question to
ask when a potential customer
says they want a bag?
• Price? Function? Colour?
• Depends on context! Demographic,
history, user journey.
• Richly connected data makes the
system seem intelligent
• But it’s “just” data and algorithms in
reality
Graph Convolutional
Neural Networks
A general architecture for
predicting node and
relationship attributes in
graphs.
(Kipf and Welling, ICLR 2017)
Credit: Andrew Docherty (CSIRO), YowData 2017
https://www.youtube.com/watch?v=Gmxz41L70Fg

40. 10/07/2019
40
Graph Networks for
Structured Causal Models
• Position paper from Google,
MIT, Edinburgh
• Structured representations
and computations (graphs)
are key
• Goal: generalize beyond direct
experience
• Like human infants can
https://arxiv.org/pdf/1806.01261.pdf
credit: @markhneedham

41. 10/07/2019
41
Thanks for listening
Dr. Jim Webber
Chief Scientist, Neo4j
@jimwebber