NS - CUK Seminar: V.T.Hoang, Review on "Structured self-attention architecture for graph-level representation learning", 2020
Van Thuy Hoang
Dept. of Artificial Intelligence,
The Catholic University of Korea
X. Fan, M. Gong and Y. Xie et al. / Pattern Recognition
100 (2020) 107084
a Structured Self-attention Architecture for graph level representation
based on a GNN variant
The proposed architecture’s readout can be incorporated into any
existing node-level GNNs and provide effective features for graph-
Compared with the pooling readout, the proposed architecture
shows its superior performance.
The main limitation of GNN schemes currently used for graphlevel
representation is the lack of effective utilization of graph
The contribution of each node to the output representation in the
pooling method is consistent.
Graph Attention Networks (GATs) introduce the selfattention
mechanism to node-level classification of graph structure
Considering the graph-level readout, GAT layer cannot be directly
used to aggregate node representations due to inconsistent output
Graph neural networks
For graph classification, the readout function ϒ aggregates node
features to obtain the entire graph’s representation g in the form of
Graph attention network
Graph attention network (GAT) incorporates the attention mechanism
into the graph aggregate function.
The representations of each node are computed by a selfattention
Graph isomorphism network.
Graph isomorphism network (GIN) aims to develop a simple
architecture that is as powerful as the Weisfeiler–Lehman graph
Two types of self-attention for graph
Node-focused self-attention for node-level output
Graph-focused self-attention for graph-level output
Structured self-attention architecture
Two types of self-attention propagation for graph.
Node-focused self-attention aims to generate the node-level
representation vectors by aggregating neighbor nodes and graph-
level self-attention focuses on aggregating nodes from the whole
The overview of Structured Self-attention Architecture
The graph-level output is generated by the node-focused and graph-
The layer-focused self-attention generates the final graph-level
representation by aggregating layer-wise representations.
T represents the number of layers.
a Structured Self-attention Architecture for graph-level representation
a scaled dot-product self-attention for node-level representation
learning and then introduces a graph-focused self-attention to
generate graph-level representation
The scaled dot-product self-attention is faster and space-efficient and
the graph-focused self-attention tends to focus on the most
influential part of graph nodes
a layer-focused self-attention which aggregates layer-wise graphlevel