Many powerful Machine Learning algorithms are based on graphs, e.g., Page Rank (Pregel), Recommendation Engines (collaborative filtering), text summarization, and other NLP tasks. Also, the recent developments with Graph Neural Networks connect the worlds of Graphs and Machine Learning even further. Considering data pre-processing and feature engineering which are both vital tasks in Machine Learning Pipelines extends this relationship across the entire ecosystem. In this session, we will investigate the entire range of Graphs and Machine Learning with many practical exercises.

1. @joerg_schad #Graph_ML
Graph Powered Machine Learning

2. Graph ML is the future of ML
2
Gartner Top 10 Data and Analytics Trends for 2021
In fact, as many as 50% of Gartner client inquiries around the topic of AI
involve a discussion around the use of graph technology.
“You can make better predictions utilizing relationships
within the data than you can from just the data alone.”
— Dr. James Fowler (UC San Diego)
DeepMind ETA Prediction

3. Jörg Schad, PhD
● Previous
○ Suki.ai
○ Mesosphere
○ PhD Distributed
DB Systems
● @joerg_schad
@joerg_schad

4. Agenda ML Infrastructure &
Metadata
Graphs
Graph Database
Graph Analytics
Graph Embeddings I
Graphs Neural Networks

5. What is Graph ML?
5
Graph Query
‣ Who can introduce me to x?
Graph Analytics
‣ Who is the most connected persons?
Graph ML
‣ Predict potential
connections?
‣ Who is likely to churn?

6. What is Graph ML?
6
Graph Query
‣ Who can introduce me to x?
Graph Analytics
‣ Who is the most connected persons?
Graph ML
‣ Predict potential
connections?
‣ Who is likely to churn?

7. What is Graph ML?
7
Graph Query
‣ Who can introduce me to x?
Graph Analytics
‣ Who is the most connected persons?
Graph ML
‣ Predict potential
connections?
‣ Who is likely to churn?

8. What problems can we solve?
Graph Analytics
Answer questions from
Graph
- Community
Detection
- Recommendations
- Centrality
- Path Finding
- Fraud Detection
- Permission
Management
- ...
8
Graph Embeddings and
Graph Neural Networks
Learning Graphs
- Node/Link Classification
- Link Prediction
- Classification of Graphs
- ...

9. ML vs Graph ML
Default ML assumption
Independent and identically distributed data
Graphs
Homophily - neighbours are similar
Structural Equivalent nodes
Heterophily - Neighbours are different
9

10. Graph Database to Graph ML
10
Graph Queries
Identify an explicit
pattern
E.g., Find common
connections
between two
people at LinkedIn
Graph Algorithms
Function beyond
select/filter
E.g., Find shortest
path between two
cities
Graph Analytics
Get insight from
Graphs
E.g., Identify
subcommunities in
my Graph
Graph ML
Train ML Models
based on Graphs
E.g., Graph
Convolutional
Networks or Graph
Embeddings as
input to TensorFlow

11. Different options
Cora Citation Dataset Graph Query
- Who cited paper x?
Graph Algorithm/Analytics
- Most cited paper
- Sub Communities
Graph ML
- Predict reviewers
- Predict missing citations
- Predict paper labels (or other features)
11

12. Graph ML: Even more Options
Cora Citation Dataset Task: Predict Paper Label
- Label Propagation
- Graph Convolutional Network
- Embeddings and trad. ML
12

13. 13
Collaborative Filtering
https://blog.dgraph.io/post/recommendation/
https://grouplens.org/datasets/movielens/100k/

14. 14
What should I watch next...
https://www.independent.co.uk/arts-entertainment/films/features/films-best-wat
ch-coronavirus-isolation-quarantine-movies-classic-greatest-essential-list-a939
4006.html

15. 15
User Movie
Rates

16. 16
User Movie
Rates
How to find movies I like?
Collaborative Filtering
“Find highly rated movies, by people
who also like movies I rated highly”
1. Find movies I rated with 5 stars
2. Find users who also rated these
movies also with 5 stars
3. Find additional movies also
rated 5 stars by those users

17. Agenda ML Infrastructure &
Metadata
Graphs
Graph Database
Graph Analytics
Graph Embeddings I
Graphs Neural Networks

18. What are Embeddings?
18
Word embedding is the collective name for a set of language
modeling and feature learning techniques in natural language
processing (NLP) where words or phrases from the
vocabulary are mapped to vectors of real numbers.
Wikipedia
https://towardsdatascience.com/creating-word-embeddings-coding-the-word2vec-algorithm-in-python-using-deep-learning-b337d0ba17a8

19. Node2Vec
19
https://arxiv.org/pdf/1607.00653.pdf
https://towardsdatascience.com/graph-embeddings-the-summary-cc6075aba007

20. Agenda ML Infrastructure &
Metadata
Graphs
Graph Database
Graph Analytics
Graph Embeddings I
Graphs Neural Networks

21. 21
https://www.youtube.com/watch?v=uF53xsT7mjc

22. http://snap.stanford.edu/graphsage/
https://cs.stanford.edu/people/jure/pubs/graphsage-nips17.pdf 22
Two problems with early GNNs
- assume fixed graph
size/structure
- Unknown graphs
- Changing structure
- Consider entire graph
during training
- Scalability
- ≠ mini-batches in NN

23. 23
https://dsgiitr.com/blogs/gat/
https://arxiv.org/pdf/1710.10903.pdf

24. Deep Walk vs Node2Vec vs GCN vs Graph Sage
24
Deep Walk Node2Vec GCN Graph Sage GAT

25. Thanks for listening!
Where to go next?
- Stanford CS224W: Machine Learning with Graphs (+ videos)
- Tomas Kipf Blog
- Graph Representation Learning Repository
- Graph Representation Learning Book
- https://towardsdatascience.com/graph-deep-learning/home

26. Feedback
Your feedback is important to us.
Don’t forget to rate and review the sessions.