H.B.Kim, Review on "DeeperGCN: ALL You Need to Train Deeper GCN", arXiv e-prints 2019

1. Ho-Beom Kim
Network Science Lab
Dept. of Mathematics
The Catholic University of Korea
E-mail: hobeom2001@catholic.ac.kr
2023 / 09 / 11
LI, Guohao, et al.
arXiv preprint

2. 2
Introduction
Problem Statements
• The rise of availability of non-Euclidean data has recently shed interest into the topic of GCNs
• Recent works suggest that different aggregations have different impact depending on the task.
• To complement the efficacy of aggregation function in GCNs, recent work borrowed concepts from 2D
CNNs to train very deep GCNs.

3. 3
Introduction
Contributions
• They analyze the performance of GCNs on large scale graphs.
• They propose a novel Generalized Aggregation Function suited for graph convolutions to enhance the
effect of aggregation functions
• Their proposed work presents a suite of techniques to train GCNs on diverse large scale datasets
• They show how combining their generalized aggregations, modified skip connection, and graph
normalization, can achieve SOTA performance in most OGB benchmarks.

4. 4
Methodology
GCNs for Learning Graph Representation
• 𝑚𝑣𝑢
𝑙
= 𝜌 𝑙
ℎ𝑣
𝑙
, ℎ𝑢
𝑙
, ℎ𝑒𝑣𝑢
𝑙
, 𝑢 ∈ 𝑁 𝑣
• 𝑚𝑣
𝑙
= 𝜁 𝑙
𝑚𝑣𝑢
𝑙
| 𝑢 ∈ 𝑁(𝑣)
• ℎ𝑣
𝑙
= 𝜙 𝑙
(ℎ𝑣
𝑙
, 𝑚𝑣
𝑙
)

5. 5
Methodology
GENeralized Message Aggregator
• 𝑚𝑣𝑢
𝑙
= 𝜌 𝑙
ℎ𝑣
𝑙
, ℎ𝑢
𝑙
, ℎ𝑒𝑣𝑢
𝑙
= ReLU ℎ𝑢
𝑙
+ 𝕝 ℎ𝑒𝑣𝑢
𝑙
∙ ℎ𝑒𝑣𝑢
𝑙
+ 𝜖, 𝑢 ∈ 𝑁(𝑣)
Message Normalization
• ℎ𝑣
𝑙+1
= 𝜙 𝑙
ℎ𝑣
𝑙
, 𝑚𝑣
𝑙
= 𝑀𝐿𝑃 ℎ𝑣
𝑙
+ 𝑠 ∙ ℎ𝑣
𝑙
2
∙
𝑚𝑣
𝑙
𝑚𝑣
𝑙
2

6. 6
Experiments
Experimental Setup
• PlainGCN
• ResGCN
• ResGCN+
• ResGEN
• DyResGEN

7. 7
Experiments
Ablation study on residual connections, model depth and aggregators on ogbn-proteins

8. 8
Experiments
Ablation study on ResGEN-SoftMax_Agg with different B values on ogbn-proteins

9. 9
Experiments
Ablation study on ResGEN-PowerMean_Agg with different p values on ogbn-proteins

10. 10
Experiments
Ablation study on DyResGEN on ogbn-proteins

11. 11
Experiments
Comparisons with SOTA

12. 12
Conclusions
Conclusions
• They propose a differentiable generalized message aggregation function, which defines a family of
permutation invariant functions.
• The definition of such a generalized aggregation function provides a new view to the design of aggregation
functions in GCNs
• They introduce a new variant of residual connections and message normalization layers
• They show the effectiveness of training their proposed deep GCN models, whereby they set a new SOTA
on four datasets of the challenging Open Graph Benchmark