Uncovering the Structural Fairness in Graph Contrastive Learning

1. Van Thuy Hoang
Network Science Lab
Dept. of Artificial Intelligence
The Catholic University of Korea
E-mail: hoangvanthuy90@gmail.com
2023-11-26
Ruijia Wang; NeurIPS 23

2. 2
Graph Convolutional Networks (GCNs)
 Generate node embeddings based on local network neighborhoods
 Nodes have embeddings at each layer, repeating combine messages
from their neighbor using neural networks

3. 3
phenomenon
 Node degrees of real-world graphs often follow a long-tailed distribution

4. 4
phenomenon
 Graph Contrastive Learning (GCL)
 GCL integrates the power of GCN and contrastive learning.
 GCL relieves GCN from annotations, and displays SOTA performance in
many tasks

5. 5
Contributions
 The first to discover that GCL methods exhibit more structural fairness than
GCN
 Theoretically validate the reason for structural fairness in GCL is that GCL
stimulates intracommunity concentration and inter-community scatter
 Propose a method GRADE to further improve structural fairness by enriching
the neighborhood of tail nodes while purifying neighbors of head nodes.
 Comprehensive experiments demonstrate that our GRADE outperforms
baselines on multiple benchmark datasets and enhances the fairness to degree
bias.

6. 6
The Proposed Model
 Aim to increase intra-community edges while decreasing inter-community
edges
Tail nodes:
Interpolate the ego network of the
anchor tail node with that of a similar
node
Head nodes:
Purify the neighborhood by similarity-
based sampling.

7. 7
Graph Augmentation
 Topology Augmentation
 The similarity matrix based on cosine similarity of representations
 1) For any tail node v_tail, we sample a node v_sample
 2) The similarity
 Feature Augmentation
 randomly sample a mask

8. 8
Optimization Objective
 Node representations h_i and o_i from different graph augmentations form the
positive pair.
 Node representations of other nodes in graph augmentations are regarded as
negative pairs.
 The overall objective to be maximized is the average of all positive pairs

9. 9
Experiments
 Datasets
 1) citation networks including Cora and Citeseer
 2) social networks Photo and Computer [23] from Amazon
 Evaluation Protocol :
 state-of-the-art GCL models

10. 10
Experiments
 Quantitative results (%) on node classification. (bold: best; em dash: out-of-
memory)

11. 11
Experiments
 Quantitative results (%) on fairness analysis

12. 12
Conclusion
 Structural unfairness for node representation learning
 Node representations obtained by GCL methods are fairer to degree bias than
those learned by GCN, and explore the underlying cause of this phenomenon.
 Based on our theoretical analysis, we further propose a novel GCL model
targeting degree bias.
 A limitation of GRADE is its heuristic design