Shift-Robust Node Classification via Graph Adversarial Clustering Neurips 2022

1. Joo-Ho Lee
School of Computer Science and Information Engineering,
The Catholic University of Korea
E-mail: jooho414@gmail.com
2023-09-11

2. 1
Introduction
Problem Statement
• open-set shift: there are emerging new classes in the graph (e.g., new COVID variant in community transmission)
• close-set shift: time-augmented test graph has substantially different class distributions (e.g., time-evolving
Twitter and Academia graphs)

3. 2
Introduction
Problem Statement
• In machine learning, domain adaption helps model generalize to target domain when there is data shift between
source and target domain
• For open-set shift, Out-of-Distribution (OOD) detection methods calibrate supervised classifiers to detect
samples belonging to other domains (e.g., different datasets) or unseen classes
• More general close-set shift is observed in time-evolving (dynamic) graph, where training graph can be viewed
as a non-IID sub-graph of test graph

4. 3
Introduction
Problem Statement
• To the light of this issue, the most well-known approach is domain invariant representation learning, namely,
indistinguishable feature representations between source and target domain
• However, the limitation of these methods is obvious when class-conditional distributions of input features change

5. 4
Introduction
Problem Statement
• Despite the popularity and importance of GNNs, only a few attempts aim to make GNNs robust of either open-
set shift or close-set shift
• All previous work do not incorporate graph-specific designs and still suffer from the aforementioned conditional
shift

6. 5
Introduction
Problem Statement
• In this paper, they propose a unified domain adaption framework for shift-robust node classification, where
unlabeled target (testing) graph is used to facilitate model generalization
• A major reason behind the ineffectiveness of existing domain adaption techniques for GNN is the lack of
modeling classes in target data
• Without modeling class distribution on target, it is impossible to mitigate the conditional shift when it happens
• Motivated by graph homophily, they propose to use graph clustering to identify latent classes by breaking less
frequent edges (possibly heterophily) between potential clusters

7. 6
Introduction
Contribution
• This is the first domain adaption framework considers the joint target probability Pr 𝑥𝑖
𝑡
, 𝑦𝑖
𝑡
on graph
• They present a domain adaption framework for two kinds of most common data shift in graph structured data
• Their experiments show that the shift-robust classifier trained in this way can detect more than 70% open-set
samples on three widely used benchmarks

8. 7
Methodology
Architecture

9. 8
Methodology
Shift-Robust Node Classification

10. 9
Methodology
Graph Adversarial Clustering
• GNN output
• Cluster

11. 10
Methodology
Graph Adversarial Clustering
• real-valued cluster assignment matrix

12. 11
Methodology
Graph Adversarial Clustering
• Adversarial regularization term on clustering

13. 12
Methodology
Model Optimization
• Overall loss (𝛼 = 1 for semi-supervised 𝛼 = 0 for full-supervised source data)
• Overall adversarial clustering

14. 13
Methodology
Model Optimization

15. 14
Experiment
Datasets

16. 15
Experiment
Graph Regression

17. 16
Experiment
Graph Classification

18. 17
Experiment
Graph Classification

19. 18
Conclusion
• In this paper, they proposed a general framework, SRNC, to enable graph neural networks for distributional shift
• Different from existing work on either open-set shift or close-set shift, SRNC works for both scenario with a novel
graph adversarial clustering component
• Accordingly, SRNC employ a latent variable model (i.e. cluster GNN) to model the latent structure and allows the
clustering GNN to improve the generalization of shift-robust classification GNN
• Future work includes applying the model to applications such as medical diagnosis and molecular dynamics,
where robustness towards distribution shifts are in critical need