1) The document proposes a graph-based method using graph convolutional networks to address challenges in sequential recommendation, such as extracting implicit preferences from long behavior sequences and adapting to changing user preferences over time. 2) It constructs an interest graph from user behaviors and designs an attentive graph convolutional network and dynamic pooling technique to aggregate implicit signals into explicit preferences. 3) Experimental results on two large-scale datasets show the proposed method significantly outperforms state-of-the-art sequential recommendation methods.

1. Ho-Beom Kim
Network Science Lab
Dept. of Mathematics
The Catholic University of Korea
E-mail: hobeom2001@catholic.ac.kr
2023 / 07 / 27
CHANG, Jianxin, et al
ACM SIGIR

2. 2
Introduction
Problem Statements
• The aforementioned works commonly concentrate more on user behaviors of recent times
→ They are not capable of fully mining older behavior-sequences to accurately estimate their current
interests.
• There two major challenges in sequential recommendation that have not been well-addressed so far
1. User behaviors in long sequences reflect implicit and noisy preference signals
2. User preferences are always drifting over time due to their diversity
 To address these two challenges, they propose a graph-based method with graph convolutional networks
to extract implicit preference signals
 Sequence  Graph
 Design a attentive graph convolutional network
 Dynamic pooling technique

3. 3
Introduction
Contributions
• They approach sequential recommendation from a new perspective by taking into consideration the
implicit-signal behaviors and fast-changing preferences
• They Propose to aggregate implicit signals into explicit ones from user behaviors by designing graph
neural network-based models on constructed item-item interest graphs. Then they design dynamic-pooling
for filtering and reserving activated core preferences for recommendation
• They conduct extensive experiments on two large-scale datasets collected from real-world applications.
The experimental results show significant performance improvements compared with the state-of-the-art
methods of sequential recommendation. Further studies also verify that our method can model long
behavioral sequences effectively and efficiently

4. 4
Methodology
SURGE model

5. 5
Methodology
Interest Graph Construction (Raw graph consturction)
• 𝒢 = {𝒱, ℰ, 𝐴}
• Aim : Learn the adjacency matrix A
• By representing each user’s interaction history as a graph,
it is easier to distinguish core and peripheral interests
• The core interest node has a higher degree than the
peripheral interest node
• The higher frequency of similar interest results in a denser
and larger subgraph.

6. 6
Methodology
Interest Graph Construction (Node similarity metric learning)
• The graph learning problem  node similarity metric
learning
• They adopt weighted cosine similarity
• 𝑀𝑖𝑗 = cos(𝑤 ⊙ ℎ𝑖, 𝑤 ⊙ ℎ𝑗)
• The learned graph structure changes continuously with the
update of item embeddings
• The similarity metric function  The multi-head metric
fucntion
→ Increase the expressive power and stabilize the
learning process
• 𝑀𝑖𝑗
𝛿
= cos 𝑤𝛿 ⊙ ℎ𝑖, 𝑤𝛿 ⊙ ℎ𝑗 , 𝑀𝑖𝑗 =
1
𝛿 𝛿=1
𝜙
𝑀𝑖𝑗
𝛿

7. 7
Methodology
Interest Graph Construction (Graph sparsification via 𝜀- sparseness)
• They extract the symmetric sparse non-negative adjacency
matrix A from M by considering only the node pair with the
most vital connection.
• They adopt a relative ranking strategy of the entire graph
• They mask off those elements in M that are smaller than a
non-negative threshold, which is obtained by ranking the
metric value in M
• 𝐴𝑖𝑗
1, ≥ 𝑅𝑎𝑛𝑘𝜖𝑛2 𝑀 ;
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒;

8. 8
Methodology
Interest-fusion Graph Convolutional Layer (Interest fusion via graph attentive convolution)
• They propose a cluster- and query-aware graph attentive
convolutional layer
• ℎ1, ℎ2, … , ℎ𝑛 , ℎ𝑖 ∈ ℝ𝑑
: The input
• ℎ1
′
, ℎ2
′
, … , ℎ𝑛
′
, ℎ𝑖 ∈ ℝ𝑑′
: The new node embedding matrix
• ℎ𝑖
′
= 𝜎(𝑊
𝑎 ∙ 𝐴𝑔𝑔𝑟𝑒𝑔𝑎𝑡𝑒 𝐸𝑖𝑗 ∗ ℎ𝑗|𝑗 ∈ 𝑁𝑖 + ℎ𝑖)
• ℎ𝑖
′
= ||𝛿=1
𝜙
𝜎 𝑊
𝑎
𝛿 ∙ 𝐴𝑔𝑔𝑟𝑒𝑔𝑎𝑡𝑒 𝐸𝑖𝑗
𝛿
∗ ℎ𝑗|𝑗 ∈ 𝑁𝑖 + ℎ𝑖

9. 9
Methodology
Interest-fusion Graph Convolutional Layer(Cluster- and query-aware attention)
• They propose a cluster and query-aware attention
mechanism
• They uses the attention coefficients to redistribute weights
on edge information in the process of message passing
• 𝛼𝑖 = 𝐴𝑡𝑡𝑒𝑛𝑡𝑖𝑜𝑛𝑐 𝑊
𝑐ℎ𝑖 ℎ𝑖𝑐
𝑊
𝑐ℎ𝑖 ⊙ ℎ𝑖𝑐
• 𝛽𝑗 = 𝐴𝑡𝑡𝑒𝑛𝑡𝑖𝑜𝑛𝑞 𝑊
𝑞ℎ𝑗 ℎ𝑡 𝑊
𝑞ℎ𝑗 ⊙ ℎ𝑡
• 𝐸𝑖𝑗 = 𝑠𝑜𝑓𝑡𝑚𝑎𝑥 𝛼𝑖 + 𝛽𝑗 =
exp(𝛼𝑖+𝛽𝑗)
𝑘∈𝑁𝑖
exp(𝛼𝑖+𝛽𝑘)

10. 10
Methodology
Interest-extraction Graph Pooling Layer(Interest extraction via graph pooling)
• Assuming that a soft cluster assignment matrix 𝑆 ∈
ℝ𝑛𝑋𝑀exists, it can pool node information  cluster
information
• m : pre-defined model hyperparameter that reflects the
degree of pooling
• ℎ1
∗
, ℎ2
∗
, … , ℎ𝑚
∗
= 𝑆𝑇
ℎ1
′
, ℎ2
′
, … , ℎ𝑛
′
,
• 𝛾1
∗
, 𝛾2
∗
, … , 𝛾𝑚
∗
= 𝑠𝑇
𝛾1, 𝛾2, … , 𝛾𝑛
• 𝑆𝑖: = 𝑠𝑜𝑓𝑡𝑚𝑎𝑥 𝑊
𝑝 ∙ 𝐴𝑔𝑔𝑟𝑒𝑔𝑎𝑡𝑒 𝐴𝑖𝑗 ∗ ℎ𝑗
′
|𝑗 ∈ 𝑁𝑖

11. 11
Methodology
Interest-extraction Graph Pooling Layer(Assignment regularization)
• Same mapping regularization
• To make easier for two nodes with greater connection
strength to be mapped to the same cluster
• 𝐿𝑀 = 𝐴, 𝑆𝑆𝑇
𝐹
• Single affiliation regularization
• To clearly define the affiliation of each cluster
• 𝐿𝐴 =
1
𝑛 𝑖=1
𝑛
𝐻(𝑠𝑖:)
• Relative position regularization
• Design a position regularization to ensure the temporal
order between clusters during pooling
• 𝐿𝑃 = 𝑃𝑛𝑆, 𝑃𝑚 2

12. 12
Methodology
Interest-extraction Graph Pooling Layer(Graph readout)
• ℎ𝑔 = 𝑅𝑒𝑎𝑑𝑜𝑢𝑡 𝛾𝑖 ∗ ℎ′
𝑖, 𝑖 ∈ 𝒢

13. 13
Methodology
Prediction Layer (Interest evolution modeling)
• Under the joint influence of the external environment and
internal cognition, the user’s core interests are continually
evolving.
• From the relative position regularization, the pooled cluster
embedding matrix maintains the temporal order of the user’s
interest
• ℎ𝑠 = 𝐴𝑈𝐺𝑅𝑈 ℎ∗
1, ℎ∗
2, … , ℎ∗
𝑚

14. 14
Methodology
Prediction Layer (Precition)
• 𝑦 = 𝑃𝑟𝑒𝑑𝑖𝑐𝑡 ℎ𝑠 ℎ𝑔 ℎ𝑡||ℎ𝑔 ⊙ ℎ𝑡
• 𝐿 ==
1
𝒪 𝑜∈𝒪 𝑦𝑜𝑙𝑜𝑔𝑦𝑜 + 1 − 𝑦𝑜 log(1 − 𝑦𝑜) + 𝜆 Θ 2

15. 15
Experiment
RQ
• RQ1: How does the proposed method perform compared with state-of-the art sequential recommenders?
• RQ2: Can the proposed method be able to handle sequences with various length effectively and efficiently/
• RQ3: What is the effect of different components in the method?

16. 16
Experiment
Experimental Settings - Dataset
• Taobao
• Kuaishou
Experimental Settings – Evaluation Metrics
• AUC
• GAUC
• MRR
• NDCG@K

17. 17
Experiment
Experimental Settings – Baselines
• Non-sequential Models
• NCF
• DIN
• LightGCN
• Sequential Models
• Caser
• GRU4REC
• DIEN
• Sli-Rec

18. 18
Experiment
Performance comparisons

19. 19
Experiment
Performance breakdown by sequence lengths

20. 20
Experiment
Test performance of the baselines by iterations on two datasets

21. 21
Experiment
Total training time until convergence of baselines on two real-world datasets

22. 22
Experiment
Ablation study of the key designs

23. 23
Experiment
Performance comparison of the proposed method using different interest evolution layers

24. 24
Conclusions And Future Work
Conclusions
• They studies the task of sequential recommender systems
• They propose a graph-based solution that re-constructs loose item sequences into tight item-item interest
graphs.
• Extensive experiments on both public and proprietary industrial datasets demonstrate the effectiveness of
their proposal
Future Work
• They plan to conduct A/B tests on the online system to further evaluate their proposed solution’s
recommendation performance
• They plan to consider using multiple types of behaviors, such as clicks and favorites, to explore fine-
grained multiple interactions from noisy historical sequences