a research paper at SIGIR 2018

1. A Contextual Attention Recurrent Architecture
for Context-Aware Venue Recommendation
Jarana Manotumruksa, Craig Macdonald and Iadh Ounis
University of Glasgow
SIGIR 2018
1

2. Introduction
2
In Location-Based Social Networks (LBSNs), users can leave feedback
on venues they have visited
Shopping
Context-Aware Venue Recommendation (CAVR) Task: Given a user’s
sequence of checkins and preferred time, which venues should be
recommended to the user?
Attractions Restaurant
17:00
1 July 2018
20:00
1 July 2018
18:00
7 July 2018
Δt = 3 hours
Δg = 1 km
Δt = 6 days
Δg = 10 km
20:00
7 July 2018
Challenges
• How to capture users’ short-term (dynamic) preferences
• How to incorporate different types of context associated
with checkins
Transition
Context
Ordinary
Context
Δt = 2 hours
Δg = ?

3. Related Work
3
[1] Koren, et al. Matrix factorization techniques for recommender systems. Computer 8. 2009
[2] Zhang, Yuyu, et al. "Sequential Click Prediction for Sponsored Search with Recurrent Neural Networks." AAAI. 2014.
[3] Manotumruksa, Jarana, et al. "A Deep Recurrent Collaborative Filtering Framework for Venue Recommendation." CIKM. 2017.
Sequential-based MF approaches [2,3]
Matrix Factorisation (MF)[1]
? 1 1 1 1
1 1
1 1
1 1 1 1
users’ latent factors
venues’ latent factors
U1
U2
U3
U4
U1
U2
U3
U4
Assumption: similar users like to visit similar venues
User Checkin Matrix
Dot product
MF cannot model a user’s dynamic preferences
from their sequence of checkins
≈
U1 ?
Predicted checkin
≈
U1
Dot product
Recurrent
Model
[2,3] leverage recurrent models (e.g. LSTM and GRU) to capture the user’s dynamic
preference from sequence of user’s checkins
However, these approaches do not take the context associated with
the checkins into account

4. Related Work
4
TimeGRU is an extension of Gated Recurrent Unit (GRU) that
considers the time interval between two successive checkins [1]
[1] Zhu, Yu, et al. "What to do next: Modeling user behaviors by time-lstm." Proc. of IJCAI, 2017.
[2] Smirnova, Elena, and Flavian Vasile. "Contextual sequence modeling for recommendation with recurrent neural networks." Proc. of DLRS, 2017.
[3] Beutel, Alex, et al. "Latent Cross: Making Use of Context in Recurrent Recommender Systems." Proc. of WSDM, 2018.
ContextualGatedRecurrentUnit(CGRU)[2]andLatentCross[3]
[2,3] extent GRU to incorporate multiple types of context
TimeGRU cannot incorporate other types of transition contexts
(e.g. distance between two checkins)
Recurrent
Unit
Recurrent
Unit
Recurrent
Unit
Assumption: the shorter the time interval between checkins, the stronger the
correlation between them and vice versa.
Δt = 6 days Δt = 2 hours
Δt = 1 day
Δg = 200 meters
Recurrent
Unit
Recurrent
Unit
Recurrent
Unit
Hidden state Hidden state
Hidden state Hidden state
1 July 2018
Δt = 6 days
Δg = 2 km
7 July 2018
Δt = 1 days
Δg = 200 meters
7 July 2018
CGRU and LatentCross treat the ordinary and transition
context equally

5. Our Contributions
5
1. a Contextual Attention Recurrent Architecture (CARA), an
extension of GRU that incorporates different types of
context to model users’ dynamic preference for CAVR
2. a Contextual Attention Gate (CAG) that controls the
influences of ordinary context
3. a Time- and Spatial-based Gate (TSG) that controls the
influence of transition context

6. Our CARA Architecture
6
Input
Layer
U1
Embedding
Layer
Sequence of Previous checkins
Target user
U1
User’s static
latent factor
Venue’s latent
factor
Recurrent
Layer
Output
Layer
U1 ?
Predicted checkin
≈
U1
Dot product
Context’s latent
factor
Hidden state Hidden state
1 July 2018
Δt = 6 days
Δg = 1 km
7 July 2018 8 July 2018
Target venue
Δt = 1 days
Δg = 500 meters
Venue’s latent
factor
Context’s latent
factor
Venue’s latent
factor
Context’s latent
factor
1 July 2018 7 July 2018 8 July 2018
Recurrent Unit Recurrent Unit Recurrent Unit

7. Proposed CARA : GRUs
7
Hidden state of
previous unit
Venue’s latent
factor
Current Unit
u
1- Hidden state of
current unit
r
h
Previous Unit Next Unit
𝒖, 𝒓 = 𝝈(𝑾𝝓𝒗 + 𝑹𝒉 𝝉−𝟏 + 𝒃)
- Gated Recurrent Unit (GRU) controls the influences of the hidden state of previous unit
and the latent factor of the venue

8. Proposed CARA : GRUs
8
Hidden state of
previous unit
Venue’s latent
factor
Current Unit
u
1- Hidden state of
current unit
r
ĥ
Previous Unit Next Unit
ĥ = 𝒕𝒂𝒏𝒉(𝑾𝝓𝒗 + 𝑹(𝒓 ⊗ 𝒉 𝝉−𝟏) + 𝒃)
- Gated Recurrent Unit (GRU) controls the influences of the hidden state of previous unit
and the latent factor of the venue

9. Proposed CARA : GRUs
9
Hidden state of
previous unit
Venue’s latent
factor
Current Unit
u
1- Hidden state of
current unit
r
ĥ
Previous Unit Next Unit
h = 𝟏 − 𝒖 𝒉 𝝉−𝟏 + uĥ
- Gated Recurrent Unit (GRU) controls the influences of the hidden state of previous unit
and the latent factor of the venue

10. Proposed CARA : CAG
10
Hidden state of
previous unit
Venue’s latent
factor
Current Unit
u
1- Hidden state of
current unit
r
h
Previous Unit
Next Unit
Ordinary context’s
latent factor
Ordinary context’s
latent factor
a
- Contextual Attention Gate (CAG) controls the influences of the ordinary context (e.g. time-
of-day) and the hidden state of previous unit
𝒂 = 𝝈(𝑾𝒉 𝝉−𝟏 + 𝑾𝝓𝒕 + 𝒃)

11. Proposed CARA : CAG
11
Hidden state of
previous unit
Venue’s latent
factor
Current Unit
u
1- Hidden state of
current unit
r
ĥ
Previous Unit
Next Unit
Ordinary context’s
latent factor
Ordinary context’s
latent factor
a
- Next, we update the reset and update gates as well as the candidate hidden state as follow
𝒖, 𝒓 = 𝝈(𝑾𝝓𝒗 + 𝑹𝒉 𝝉−𝟏 + 𝑾 𝜶 ⊗ 𝝓𝒕 + 𝒃)
ĥ = 𝒕𝒂𝒏𝒉(𝑾𝝓𝒗 + 𝑹(𝒓 ⊗ 𝒉 𝝉−𝟏) + 𝑾 𝜶 ⊗ 𝝓𝒕 + 𝒃)
Pre-Fusion

12. Proposed CARA : CAG
12
Hidden state of
previous unit
Venue’s latent
factor
Current Unit
u
1- Hidden state of
current unit
r
h
Previous Unit Next Unit
Ordinary context’s
latent factor
Ordinary context’s
latent factor
a
1-
h = 𝟏 − 𝜶 ⊗ 𝝓𝒕 ⊗ (𝟏 − 𝒖 𝒉 𝝉−𝟏 + 𝒖ĥ)
Post-Fusion
- Contextual Attention Gate (CAG) controls the influences of the ordinary context (e.g. time-
of-day) and the hidden state of previous unit

13. Proposed CARA : CAG
13
Hidden state of
previous unit
Venue’s latent
factor
Current Unit
u
1- Hidden state of
current unit
r
h
Previous Unit Next Unit
Ordinary context’s
latent factor
Ordinary context’s
latent factor
a
1-
Post-Fusion
Pre-Fusion
Pre-Fusion
Post-Fusion
- the ordinary context has less impact to the current hidden state
- the ordinary context has more impact to the current hidden state

14. Proposed CARA : TSG
14
Hidden state of
previous unit
Venue’s latent
factor
Current Unit
z
1- Hidden state of
current unit
r
h
Previous Unit Next Unit
Ordinary context’s
latent factor
Ordinary context’s
latent factor
a
1-
Transition context
TS
𝐓𝐒 = 𝝈 𝑾𝝓𝒗 + 𝝈 𝑾𝚫𝐭 + 𝐛 ⊗ 𝝈 𝑾𝝓𝒗 + 𝝈 𝑾𝚫𝐠 + 𝐛
- Time- and Spatial Gate (TSG) controls the influences of previous hidden state based on the
transition context (i.e. time interval and distance between two checkins)
𝚫𝐭, 𝚫𝐠

15. Proposed CARA : TSG
15
Hidden state of
previous unit
Venue’s latent
factor
Current Unit
z
1- Hidden state of
current unit
r
ĥ
Previous Unit Next Unit
Ordinary context’s
latent factor
Ordinary context’s
latent factor
a
1-
Transition context
TS
ĥ = 𝒕𝒂𝒏𝒉(𝑾𝝓𝒗 + 𝑹(𝒓 ⊗ 𝑻𝑺 ⊗ 𝒉 𝝉−𝟏) + 𝑾 𝜶 ⊗ 𝝓𝒕 + 𝒃)
- Time- and Spatial Gate (TSG) controls the influences of hidden state based on the transition
context (i.e. time interval and geographical distance between two checkins
𝚫𝐭, 𝚫𝐠

16. Proposed CARA : Summary
16
Hidden state of
previous unit
Venue’s latent
factor
Current Unit
u
1- Hidden state of
current unit
r
ĥ
Previous Unit Next Unit
Ordinary context’s
latent factor
Ordinary context’s
latent factor
a
1-
Transition context
TS
User’s dynamic preferences ✔
Support multiple types of context ✔
Independently incorporate ordinary and transition context? ✔

17. Experimental Setup
17
Datasets: 3 large datasets (Brightkite, Foursquare and Yelp)
Evaluation Methodology: Leave-one-out
Experiment: Normal Users (those with >= 10 checkins)
Measures: Hit Ratio (HR) and NDCG
Significance testing: paired t-test with p < 0.01
Brightkite Foursquare Yelp
# of normal users 14,374 10,766 38,945
# of cold-start users 5,578 154 6903
# of venues 5,050 10,695 34,245
# of check-ins 681,024 1,336,278 981,379
% density of checkins matrix 0.93 1.16 0.07

18. Baselines and CARA
18
Model Context-aware Ordinary /
Transition
Special
gates
Pre & Post
Fusion
RNN ✘ ✘ ✘ ✘
TimeGRU Only time Only
transition
✔ ✘
CGRU ✔ Equally ✘ ✘
LatentCross ✔ Equally ✘ ✔
CARA ✔ Separately ✔ ✔
RQ 1: Is it important to model ordinary and transition contexts
separately?
RQ 2: Can our proposed Time- and Spatial-based Gates (TSG)
enhance the effectiveness of traditional recurrent units in capturing
users’ dynamic preferences?

19. Results
19
RQ 1: Is it important to model ordinary and transition context separately?
Brightkite Foursquare Yelp
Model HR NDCG HR NDCG HR NDCG
RNN 0.6657* 0.4407* 0.8302* 0.5762* 0.4164* 0.2146*
TimeGRU 0.7005* 0.4816* 0.8570* 0.6167* 0.4342* 0.2240*
CGRU 0.6969* 0.5659* 0.8592* 0.6985* 0.5194* 0.3005*
LatentCross 0.7063* 0.5727* 0.8616* 0.6964* 0.5210* 0.2991*
CARA 0.7385 0.6040 0.8851 0.7154 0.5587 0.3272

20. Effects of transition context
20
RQ 2: Can our proposed Time- and Spatial-based Gates (TSG) enhance the effectiveness of
traditional recurrent units in capturing the users’ dynamic preferences?

21. Conclusions
21
• We proposed Contextual Attention Recurrent Architecture
(CARA) for context-aware venue recommendation
• Contextual Attention Gate for ordinary context
• Time- and Spatial-based Gate for transition context
• Experimental results demonstrate the effectiveness of CARA in
comparison with several SOTA approaches
• The ordinary and transition context should be treated
separately
• Different types of transition context can enhance the
effectiveness of CAVR

22. 22
Thank You
Source code
https://github.com/feay1234/CARA