Advertisement

May. 26, 2023•0 likes## 0 likes

•16 views## views

Be the first to like this

Show More

Total views

0

On Slideshare

0

From embeds

0

Number of embeds

0

Download to read offline

Report

Technology

NS - CUK Seminar: V.T.Hoang, Review on "Structured self-attention architecture for graph-level representation learning", 2020

ssuser4b1f48Follow

Advertisement

Advertisement

Advertisement

Colloquium.pptxMythili680896

Gnn overviewLouis (Yufeng) Wang

Deep learning for 3 d point clouds presentationVijaylaxmiNagurkar

DDGK: Learning Graph Representations for Deep Divergence Graph Kernelsivaderivader

220206 transformer interpretability beyond attention visualizationtaeseon ryu

NS-CUK Joint Journal Club : S.T.Nguyen, Review on "Graph Neural Networks for ...ssuser4b1f48

- Van Thuy Hoang Dept. of Artificial Intelligence, The Catholic University of Korea hoangvanthuy90@gmail.com X. Fan, M. Gong and Y. Xie et al. / Pattern Recognition 100 (2020) 107084
- 2 Contributions a Structured Self-attention Architecture for graph level representation based on a GNN variant The proposed architecture’s readout can be incorporated into any existing node-level GNNs and provide effective features for graph- level representation. Compared with the pooling readout, the proposed architecture shows its superior performance.
- 3 Problem The main limitation of GNN schemes currently used for graphlevel representation is the lack of effective utilization of graph representation information. The contribution of each node to the output representation in the pooling method is consistent.
- 4 Problem Graph Attention Networks (GATs) introduce the selfattention mechanism to node-level classification of graph structure Considering the graph-level readout, GAT layer cannot be directly used to aggregate node representations due to inconsistent output targets.
- 5 Graph neural networks For graph classification, the readout function ϒ aggregates node features to obtain the entire graph’s representation g in the form of
- 6 Graph attention network Graph attention network (GAT) incorporates the attention mechanism into the graph aggregate function. The representations of each node are computed by a selfattention strategy
- 7 Graph isomorphism network. Graph isomorphism network (GIN) aims to develop a simple architecture that is as powerful as the Weisfeiler–Lehman graph isomorphism test.
- 8 Mathematical model Two types of self-attention for graph Node-focused self-attention for node-level output Graph-focused self-attention for graph-level output Structured self-attention architecture
- 9 Two types of self-attention propagation for graph. Node-focused self-attention aims to generate the node-level representation vectors by aggregating neighbor nodes and graph- level self-attention focuses on aggregating nodes from the whole graph.
- 10 The overview of Structured Self-attention Architecture The graph-level output is generated by the node-focused and graph- focused self-attention. The layer-focused self-attention generates the final graph-level representation by aggregating layer-wise representations. T represents the number of layers.
- 11 Node-focused self-attention for node-level output node representations obtain more local information as the number of layers increases.
- 12 Graph-focused self-attention for graph-level output Graph-level output aims to aggregate node features to obtain the entire graph’s representation.
- 13 Structured self-attention architecture Then graph-focused self-attention is used to obtain graph-level
- 14 Optimized structured self-attention architecture. ZZZ
- 15 Comparison and discussion Benchmarks MUTAG PTC PROTEINS NCI1 REDDIT-B REDDIT-M5K IMDB-B IMDB-M COLLAB
- 16 Comparison of the 10-fold cross validation ZZZ
- 17 Conclusion a Structured Self-attention Architecture for graph-level representation a scaled dot-product self-attention for node-level representation learning and then introduces a graph-focused self-attention to generate graph-level representation The scaled dot-product self-attention is faster and space-efficient and the graph-focused self-attention tends to focus on the most influential part of graph nodes a layer-focused self-attention which aggregates layer-wise graphlevel representations

Advertisement