@joerg_schad #Graph_ML
Graph Powered Machine Learning
Graph ML is the future of ML
2
Gartner Top 10 Data and Analytics Trends for 2021
In fact, as many as 50% of Gartner client inquiries around the topic of AI
involve a discussion around the use of graph technology.
“You can make better predictions utilizing relationships
within the data than you can from just the data alone.”
— Dr. James Fowler (UC San Diego)
DeepMind ETA Prediction
Jörg Schad, PhD
● Previous
○ Suki.ai
○ Mesosphere
○ PhD Distributed
DB Systems
● @joerg_schad
@joerg_schad
Agenda ML Infrastructure &
Metadata
Graphs
Graph Database
Graph Analytics
Graph Embeddings I
Graphs Neural Networks
What is Graph ML?
5
Graph Query
‣ Who can introduce me to x?
Graph Analytics
‣ Who is the most connected persons?
Graph ML 
‣ Predict potential
connections?
‣ Who is likely to churn?
What is Graph ML?
6
Graph Query
‣ Who can introduce me to x?
Graph Analytics
‣ Who is the most connected persons?
Graph ML 
‣ Predict potential
connections?
‣ Who is likely to churn?
What is Graph ML?
7
Graph Query
‣ Who can introduce me to x?
Graph Analytics
‣ Who is the most connected persons?
Graph ML 
‣ Predict potential
connections?
‣ Who is likely to churn?
What problems can we solve?
Graph Analytics
Answer questions from
Graph
- Community
Detection
- Recommendations
- Centrality
- Path Finding
- Fraud Detection
- Permission
Management
- ...
8
Graph Embeddings and
Graph Neural Networks
Learning Graphs
- Node/Link Classification
- Link Prediction
- Classification of Graphs
- ...
ML vs Graph ML
Default ML assumption
Independent and identically distributed data
Graphs
Homophily - neighbours are similar
Structural Equivalent nodes
Heterophily - Neighbours are different
9
Graph Database to Graph ML
10
Graph Queries
Identify an explicit
pattern
E.g., Find common
connections
between two
people at LinkedIn
Graph Algorithms
Function beyond
select/filter
E.g., Find shortest
path between two
cities
Graph Analytics
Get insight from
Graphs
E.g., Identify
subcommunities in
my Graph
Graph ML
Train ML Models
based on Graphs
E.g., Graph
Convolutional
Networks or Graph
Embeddings as
input to TensorFlow
Different options
Cora Citation Dataset Graph Query
- Who cited paper x?
Graph Algorithm/Analytics
- Most cited paper
- Sub Communities
Graph ML
- Predict reviewers
- Predict missing citations
- Predict paper labels (or other features)
11
Graph ML: Even more Options
Cora Citation Dataset Task: Predict Paper Label
- Label Propagation
- Graph Convolutional Network
- Embeddings and trad. ML
12
13
Collaborative Filtering
https://blog.dgraph.io/post/recommendation/
https://grouplens.org/datasets/movielens/100k/
14
What should I watch next...
https://www.independent.co.uk/arts-entertainment/films/features/films-best-wat
ch-coronavirus-isolation-quarantine-movies-classic-greatest-essential-list-a939
4006.html
15
User Movie
Rates
16
User Movie
Rates
How to find movies I like?
Collaborative Filtering
“Find highly rated movies, by people
who also like movies I rated highly”
1. Find movies I rated with 5 stars
2. Find users who also rated these
movies also with 5 stars
3. Find additional movies also
rated 5 stars by those users
Agenda ML Infrastructure &
Metadata
Graphs
Graph Database
Graph Analytics
Graph Embeddings I
Graphs Neural Networks
What are Embeddings?
18
Word embedding is the collective name for a set of language
modeling and feature learning techniques in natural language
processing (NLP) where words or phrases from the
vocabulary are mapped to vectors of real numbers.
Wikipedia
https://towardsdatascience.com/creating-word-embeddings-coding-the-word2vec-algorithm-in-python-using-deep-learning-b337d0ba17a8
Node2Vec
19
https://arxiv.org/pdf/1607.00653.pdf
https://towardsdatascience.com/graph-embeddings-the-summary-cc6075aba007
Agenda ML Infrastructure &
Metadata
Graphs
Graph Database
Graph Analytics
Graph Embeddings I
Graphs Neural Networks
21
https://www.youtube.com/watch?v=uF53xsT7mjc
http://snap.stanford.edu/graphsage/
https://cs.stanford.edu/people/jure/pubs/graphsage-nips17.pdf 22
Two problems with early GNNs
- assume fixed graph
size/structure
- Unknown graphs
- Changing structure
- Consider entire graph
during training
- Scalability
- ≠ mini-batches in NN
23
https://dsgiitr.com/blogs/gat/
https://arxiv.org/pdf/1710.10903.pdf
Deep Walk vs Node2Vec vs GCN vs Graph Sage
24
Deep Walk Node2Vec GCN Graph Sage GAT
Thanks for listening!
Where to go next?
- Stanford CS224W: Machine Learning with Graphs (+ videos)
- Tomas Kipf Blog
- Graph Representation Learning Repository
- Graph Representation Learning Book
- https://towardsdatascience.com/graph-deep-learning/home
Feedback
Your feedback is important to us.
Don’t forget to rate and review the sessions.

Graph-Powered Machine Learning

  • 1.
  • 2.
    Graph ML isthe future of ML 2 Gartner Top 10 Data and Analytics Trends for 2021 In fact, as many as 50% of Gartner client inquiries around the topic of AI involve a discussion around the use of graph technology. “You can make better predictions utilizing relationships within the data than you can from just the data alone.” — Dr. James Fowler (UC San Diego) DeepMind ETA Prediction
  • 3.
    Jörg Schad, PhD ●Previous ○ Suki.ai ○ Mesosphere ○ PhD Distributed DB Systems ● @joerg_schad @joerg_schad
  • 4.
    Agenda ML Infrastructure& Metadata Graphs Graph Database Graph Analytics Graph Embeddings I Graphs Neural Networks
  • 5.
    What is GraphML? 5 Graph Query ‣ Who can introduce me to x? Graph Analytics ‣ Who is the most connected persons? Graph ML  ‣ Predict potential connections? ‣ Who is likely to churn?
  • 6.
    What is GraphML? 6 Graph Query ‣ Who can introduce me to x? Graph Analytics ‣ Who is the most connected persons? Graph ML  ‣ Predict potential connections? ‣ Who is likely to churn?
  • 7.
    What is GraphML? 7 Graph Query ‣ Who can introduce me to x? Graph Analytics ‣ Who is the most connected persons? Graph ML  ‣ Predict potential connections? ‣ Who is likely to churn?
  • 8.
    What problems canwe solve? Graph Analytics Answer questions from Graph - Community Detection - Recommendations - Centrality - Path Finding - Fraud Detection - Permission Management - ... 8 Graph Embeddings and Graph Neural Networks Learning Graphs - Node/Link Classification - Link Prediction - Classification of Graphs - ...
  • 9.
    ML vs GraphML Default ML assumption Independent and identically distributed data Graphs Homophily - neighbours are similar Structural Equivalent nodes Heterophily - Neighbours are different 9
  • 10.
    Graph Database toGraph ML 10 Graph Queries Identify an explicit pattern E.g., Find common connections between two people at LinkedIn Graph Algorithms Function beyond select/filter E.g., Find shortest path between two cities Graph Analytics Get insight from Graphs E.g., Identify subcommunities in my Graph Graph ML Train ML Models based on Graphs E.g., Graph Convolutional Networks or Graph Embeddings as input to TensorFlow
  • 11.
    Different options Cora CitationDataset Graph Query - Who cited paper x? Graph Algorithm/Analytics - Most cited paper - Sub Communities Graph ML - Predict reviewers - Predict missing citations - Predict paper labels (or other features) 11
  • 12.
    Graph ML: Evenmore Options Cora Citation Dataset Task: Predict Paper Label - Label Propagation - Graph Convolutional Network - Embeddings and trad. ML 12
  • 13.
  • 14.
    14 What should Iwatch next... https://www.independent.co.uk/arts-entertainment/films/features/films-best-wat ch-coronavirus-isolation-quarantine-movies-classic-greatest-essential-list-a939 4006.html
  • 15.
  • 16.
    16 User Movie Rates How tofind movies I like? Collaborative Filtering “Find highly rated movies, by people who also like movies I rated highly” 1. Find movies I rated with 5 stars 2. Find users who also rated these movies also with 5 stars 3. Find additional movies also rated 5 stars by those users
  • 17.
    Agenda ML Infrastructure& Metadata Graphs Graph Database Graph Analytics Graph Embeddings I Graphs Neural Networks
  • 18.
    What are Embeddings? 18 Wordembedding is the collective name for a set of language modeling and feature learning techniques in natural language processing (NLP) where words or phrases from the vocabulary are mapped to vectors of real numbers. Wikipedia https://towardsdatascience.com/creating-word-embeddings-coding-the-word2vec-algorithm-in-python-using-deep-learning-b337d0ba17a8
  • 19.
  • 20.
    Agenda ML Infrastructure& Metadata Graphs Graph Database Graph Analytics Graph Embeddings I Graphs Neural Networks
  • 21.
  • 22.
    http://snap.stanford.edu/graphsage/ https://cs.stanford.edu/people/jure/pubs/graphsage-nips17.pdf 22 Two problemswith early GNNs - assume fixed graph size/structure - Unknown graphs - Changing structure - Consider entire graph during training - Scalability - ≠ mini-batches in NN
  • 23.
  • 24.
    Deep Walk vsNode2Vec vs GCN vs Graph Sage 24 Deep Walk Node2Vec GCN Graph Sage GAT
  • 25.
    Thanks for listening! Whereto go next? - Stanford CS224W: Machine Learning with Graphs (+ videos) - Tomas Kipf Blog - Graph Representation Learning Repository - Graph Representation Learning Book - https://towardsdatascience.com/graph-deep-learning/home
  • 26.
    Feedback Your feedback isimportant to us. Don’t forget to rate and review the sessions.