NS-CUK Seminar:V.T.Hoang, Review on "GRPE: Relative Positional Enco...

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Van Thuy Hoang
Dept. of Artificial Intelligence,
The Catholic University of Korea
hoangvanthuy90@gmail.com
Park et al., IC...

2
 Encode absolution position in the sequence of nodes
 Encode relative position with another node using bias
terms
 pr...

3
Problems
 Explicit representations of position in graph convolutional networks
are lost, thus incorporating graph struc...

4
Problems
 Introduce two sets of learnable positional encoding vectors to
represent spatial relation or edge between two...

5
BACKGROUND
 The self-attention module computes query q, key k, and value v with
independent linear transformations:
 T...

6
GRAPH WITH TRANSFORMER
 Graphormer adopted two additional terms on the self-attention
module to encode graph informatio...

7
The proposed Graph Relative Positional Encoding
 Left shows an example of how GRPE process relative relation
between no...

8
NODE-AWARE ATTENTION
 two terms to encode graph on the attention map with two newly
proposed encodings
 The 1st term:
...

9
Problems
 Two terms consider node-spatial relation and node-edge relation,
but Graphormer did not consider the interact...

10
GRAPH-ENCODED VALUE
 to encode a graph to the hidden features of self-attention, when
values are weighted summed with ...

11
EXPERIMENT
 VIRTUAL NODE
 The role of a virtual node is similar to special tokens such as a
classification token

12
EXPERIMENT
 Results on ZINC

13
EXPERIMENT
 Results on MolHIV
 Results on MolPCBA

14
Problems
 Effects of components of GRPE on ZINC datasets (The lower the
better)
 Empirically, sharing the encodings d...

NS-CUK Seminar:V.T.Hoang, Review on "GRPE: Relative Positional Encoding for Graph Transformer", ICLR 2022

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

2024 Trend Updates: What Really Wor... by Search Engine Jou... 1011447 views
Storytelling For The Web: Integrate... by Chiara Aliotta 919244 views
Artificial Intelligence, Data and C... by OECD Directorate ... 820562 views
How to Leverage AI to Boost Employe... by SocialHRCamp 342424 views
2024 State of Marketing Report – by... by Marius Sescu 195380 views
Everything You Need To Know About C... by Expeed Software 218594 views

View on Slideshare

1 of 15 Ad

View on Slideshare

1 of 15 Ad