NS-CUK Seminar: H.B.Kim, Review on "Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Ranking", ICLR 2018

1. Ho-Beom Kim
Network Science Lab
Dept. of Mathematics
The Catholic University of Korea
E-mail: hobeom2001@catholic.ac.kr
2023 / 07 / 17
BOJCHEVSKI, Aleksandar; GÜNNEMANN, Stephan.

2. 2
Introduction
Problem Statements
• All existing graph embedding approaches represent each node by a single point in a low-dimensional
continuous vector space.
• We do not have information about the uncertainty of that representation
• Node for which the different sources of information are conflicting with each other
• Such discrepancy should be reflected in the uncertainty of its embedding.

3. 3
Introduction
Contributions
• They introduce a novel embedding approach that represents nodes as Gaussian distributions
• Each node becomes a full distribution rather than a single point
• They capture uncertainty about its representation
• They propose a novel unsupervised personalized ranking formulation to learn the embeddings
• The distances between their embeddings naturally leads to their ranking formulation.
• Graph2Gauss is inductive, which is a significant benefit over existing methods that are inherently
transductive and do not naturally generalize to unseen nodes.
1. They embed nodes as Gaussian distributions allowing us to capture uncertainty
2. Their unsupervised personalized ranking formulation exploits the natural ordering of the nodes
capturing the network structure at multiple scales
3. They propose an inductive method that generalizes to unseen nodes and is applicable to different
types of graphs: plain/attributed, directed/undirected

4. 4
Related Work
Related Work
• DeepWalk
• Node2vec
• Skip-Gram
• LINE
• SDNE
• GraRep
• Tri-Party Deep Network Representation (TRIDNR)
• CENE
• Text-Associated DeepWalk (TADW)
• GraphSAGE
• GCN
• GAE
• Knowledge graph

5. 5
Methodology
Architecture

6. 6
Methodology
Network Structure Representation Via Personalized Ranking

7. 7
Methodology
Learning Via Energy-Based Loss

8. 8
Methodology
Sampling Strategy

9. 9
Experiments
Baseline & Dataasets
• Baseline
• Graph2Gauss
• Graph2Gauss_oh
• TRIDNR
• TADW
• GAE
• Node2vec
• Logistic Regression
• Datasets
• Cora
• CoRA-ML
• DBLP
• PUBMBED

10. 10
Experiments
Link Predictin Performance For Real-World Datasets with L = 128

11. 11
Experiments
Link Prediction Performance for different embedding sizes
Percentages of training edges on Cora-ML

12. 12
Experiments
Classification performance comparison

13. 13
Experiments
Node-Anchored Sampling Strategy

14. 14
Experiments
The benefit of modeling the uncertainty of the nodes

15. 15
Methodology
Inductive link prediction performance

16. 16
Experiments
2D visualization of the embeddings on theCora-ML dataset

17. 17
Conclusion
Conclusion & Future work
• They proposed Graph2Gauss – the first unsupervised approach that represents nodes in attributed
graphs as Gaussian distributions and is therefore able to capture uncertainty.
• Graph2Gauss leverages the natural ordering of the nodes w.r.t. their neighborhoods via a personalized
ranking formulation.
• As future work they aim to study personalized rankings beyond the ones imposed by the shortest path
distance