NS-CUK Seminar: S.T.Nguyen, Review on "Hierarchical Graph Convolutional Networks for Semi-supervised Node Classification", IJCAI 2019

1. Nguyen Thanh Sang
Network Science Lab
Dept. of Artificial Intelligence
The Catholic University of Korea
E-mail: sang.ngt99@gmail.com
2023-04-07

2. 1
 Paper
 Introduction
 Problem
 Contributions
 Framework
 Experiment
 Conclusion

3. 2
Hierarchical graph structure
• Hierarchical structure is pervasive across complex networks with examples spanning from
neuroscience, economics, social organizations, urban systems, communications,
pharmaceuticals and biology, particularly metabolic and gene networks.

4. 3
Problems
 Many embedding methods can be used in the node classification task by converting the graph
structure into sequences by performing random walks on the graph and computing co-
occurrence statistics.
=> unsupervised algorithms and cannot perform node classification tasks in an end-to-end
applications.

5. 4
Problems
 Graph convolutional networks (GCNs) generates node embedding by combining information
from neighborhoods.
 lack the “graph pooling” mechanism, which restricts the scale of the receptive field.
 difficulty in obtaining adequate global information.
 adding too many convolutional layers will result in the output features over-smoothed and
make them indistinguishable

6. 5
Problems
 Some recent methods try to get the global information through deeper models.
 they are either unsupervised models or need many training examples.
 they are still not capable of solving the semi-supervised node classification task directly.

7. 6
Contributions
• First work to design a deep hierarchical model for the semi-supervised node classification task
which consists of more layers with larger receptive fields.
 obtain more global information through the coarsening and refining procedures.
• Applying deep architectures and the pooling mechanism into classification tasks.
• The proposed model outperforms other state-of-the-art approaches and gains a considerable
improvement over other approaches with very few labeled samples provided for each class.

8. 7
Overview
• For each coarsening layer, the GCN is conducted to learn node representations.
• A coarsening operation is performed to aggregate structurally similar nodes into hyper-nodes.
 each hyper-node represents a local structure of the original graph, which can facilitate exploiting global
structures on the graph.
• Following coarsening layers, a symmetric graph refining layers is applied to restore the original graph
structure for node classification tasks.
 comprehensively capture nodes’ information from local to global perspectives, leading to better node
representations.

9. 8
Graph Convolutional Networks
• Update node representation by using neighbor features.

10. 9
Graph Coarsening Layer
 Two steps:
• Structural equivalence grouping (SEG). If two nodes
share the same set of neighbors, they are considered to
be structurally equivalent.
 assign these two nodes to be a hyper-node.
• Structural similarity grouping (SSG). Then, we
calculate the structural similarity between the
unmarked node pairs.
=> form a new hyper-node and mark the two nodes by
largest structural similarity. The largest structural similarity
is defined by comparing the normalized connection
strength:

11. 10
Graph Coarsening Layer
• The hidden node embedding matrix is determined as:
• Update adjacency matrix:
 The hidden representation is fed into the next layer as
input.
 The resulting node embedding to generate in each
coarsening layer will then be of lower resolution.

12. 11
Graph Refining Layer
• To restore the original topological structure of the graph and further facilitate node
classification
 stacking the same numbers of graph refining layers as coarsening layers.
• Each refining layer contains two steps:
o generating node embedding vectors
o restoring node representations.
• A residual connections between the two corresponding coarsening and refining layers.

13. 12
Node Weight Embedding and Multiple Channels
• Multi-channel mechanisms help explore
features in different subspaces and H-GCN
employs multiple channels on GCN to
obtain rich information jointly at each layer

14. 13
The Output Layer
• Softmax classifier:
• The loss function is defined as the cross-entropy of predictions over the labeled nodes:

15. 14
Datasets
• Four widely-used datasets including three citation networks and one knowledge graph.

16. 15
Experiment
• The proposed method consistently outperforms
other state-of-the-art methods, which verify the
effectiveness of the proposed coarsening and
refining mechanisms.
• DeepWalk cannot model the attribute information,
which heavily restricts its performance.
• The proposed H-GCN manages to capture global
information through different levels of
convolutional layers and achieves the best results
among all four datasets.

17. 16
Impact of Scale of Training Data
• The proposed method outperforms other baselines
in all cases.
• With the number of labeled data decreasing, it
obtains a more considerable margin over these
baseline algorithms.
• The proposed H-GCN with increased receptive
fields is well-suited when training data is extremely
scarce and thereby is of significant practical values.

18. 17
Coarsening, refining layers and embedding weights
• The proposed H-GCN has better performance
compared to H-GCN without coarsening mechanisms
on all datasets.
=> The coarsening and refining mechanisms contribute
to performance improvements since they can obtain
global information with larger receptive fields.
• Model with node weight embeddings performs
better,
 the necessity to add this embedding vector in the
node embeddings.

19. 18
Comparing with number of coarsening layers and channels
• Since fewer labeled nodes are supplied on NELL than
others, deeper layers and larger receptive fields are needed.
• Adding too many coarsening layers, the performance drops
due to overfitting.
• The performance improves with the number of channels
increasing until four channels => help capture accurate
node features.
• Too many channels will inevitably introduce redundant
parameters to the model, leading to overfitting as well.

20. 19
Conclusions
• A novel hierarchical graph convolutional networks for the semi-supervised node
classification task.
• The H-GCN model consists of coarsening layers and symmetric refining layers.
• By grouping structurally similar nodes to hyper-nodes, this model can get a larger receptive
field and enable sufficient information propagation.
• Compared with other previous work, H-GCN is deeper and can fully utilize both local and
global information.
• The has achieved substantial gains over them in the case that labeled data is extremely
scarce.

21. 20