The document describes a ranking-based approach for contextual suggestions. It gathers candidate suggestions based on context like location, then ranks them using similarity to positive and negative examples in user profiles. Suggestions are filtered to match temporal context like business hours. Evaluation on TREC 2012 data found that using only category similarity worked best for weighted performance, while combining similarities worked best for binary relevance. Releasing a standard test set could help evaluate new methods.

1. An Exploration of Ranking-based
Strategy for Contextual Suggestion
TREC 2012 Contextual Suggestion Track
Peilin Yang and Hui Fang
University of Delaware
1
InfoLab@

2. The task is to provide suggestions based on
both user profiles and contexts
2
Contexts
e.g. New York City
e.g. Weekday/Morning/Summer
Profiles
Geographical Information
Temporal Information

3. Overview of Our methods
Gather
Candidate
Suggestions
Rank
Suggestions
Post-
Process
Results
3
Context
(geographical Info.)
User Profiles
Context
(temporal Info.)

4. Overview of Our methods
Gather
Candidate
Suggestions
Rank
Suggestions
Post-
Process
Results
4

5. Step 1. Gather candidate suggestions based on
geographical information in the contexts and
categories that occur in examples suggestions
5
What we used

6. Information about “Color Me Mine Tribeca”
Yelp FourSquare
Category Event Planning → Party;
Shopping → Arts & Crafts;
Event Planning → Venues
Arts & Entertainment → Art
Gallery
Web_site http://tribeca.colormemine.com/ http://tribeca.colormemine.com/
Business_hours Mon-Sat 11 am - 9 pm,
Sun 11 am - 8 pm
X
6

7. 7
Overview of Our methods
Gather
Candidate
Suggestions
Rank
Suggestions
Post-
Process
Results

8. E15E9E8 E14E13E12E11E10
Example Suggestions
Candidate Suggestions
like dislike dislikedislikedislikelike like like
C21 C22C20
Step 2: Rank suggestions Based on
User Profiles
8

9. E15E9E8 E14E13E12E11E10
Example Suggestions
C21
Candidate Suggestions
C22C20
Positive
Similarity
Negative
Similarity
Step 2: Rank suggestions Based on
User Profiles
9
like dislike dislikedislikedislikelike like like

10. E15E9E8 E14E13E12E11E10
Example Suggestions
C21
Candidate Suggestions
like dislike dislikedislikedislikelike like like
C22C20
Ranking
𝑆 𝑈, 𝐶 = 𝜑 ×
∑ 𝑆𝑆𝑆 𝑝, 𝐶𝑝∈𝑃 𝑈
𝑃 𝑈
− 1 − 𝜑 ×
∑ 𝑆𝑆𝑆 𝑛, 𝐶𝑛∈𝑁 𝑈
𝑁 𝑈
Step 2: Rank suggestions Based on
User Profiles
Similarity Function
Coefficients
10
Given a user profile, rank
candidate suggestions based
on their similarity scores with
respect to both positive and
negative examples in the
profile

11. Similarity Functions (1):
Description-based
• 𝑆𝑆𝑆 𝒟 𝑒, 𝐶 = 𝐹𝐹𝐹𝐹𝐹(𝐷𝐷𝐷 𝑒 , 𝑊𝑊𝑊(𝐶))
• F2EXP is an axiomatic retrieval function [Fang&Zhai,2005],
𝑆 𝑄, 𝐷 = � 𝑐 𝑡, 𝑄 ∙ (
𝑁
𝑑𝑑(𝑡)
) 𝑘
∙
𝑐(𝑡, 𝐷)
𝑐 𝑡, 𝐷 + 𝑠 +
𝑠 ∙ |𝐷|
𝑎𝑎𝑎𝑎𝑡∈𝑄∩𝐷
• DES(e) is the description of and example suggestion e,
• WEB(C) is all the information on the web sites of candidate C.
11
Playdium is the ultimate interactive, virtual and physical Family
Entertainment Center. The 40,000 sq. ft. indoor complex features more than
200 of today’s newest attractions, rides; simulators. Our outdoors feature
Go-Karts, Mini-Golf, Water Wars; Bungee Trampolines. Playdium Is The
Ultimate Place To Play!
Color Me Mine Tribeca
Example 49

12. Similarity Functions (2):
Category-based
• 𝑆𝑆𝑆 𝒞 𝑒, 𝐶 =
∑ ∑
|𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼(𝑐 𝑖,𝑐 𝑗)|
max{|𝑐 𝑖|,|𝑐 𝑗|}𝑐 𝑗∈𝒞(𝐶)𝑐 𝑖∈𝒞(𝑒)
|𝒞(𝑒)|×|𝒞(𝐶)|
• 𝒞 𝑠 is the set of categories of 𝑠,
• 𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼(𝑐𝑖, 𝑐𝑗) denotes the number of
common categories between 𝑐𝑖 𝑎𝑎𝑎 𝑐𝑗.
12
Event Planning → Party
Example 49
Color Me Mine Tribeca
Event Planning → Party
Similarity : 2/2Similarity : 0/2
Similarity : 1/2
Similarity : 0/2
Similarity : 0/2
Similarity : 0/2
Similarity :
(2/2+0/2+1/2+0/2+0/2+0/2)/6=1/4
Shopping → Arts & Crafts
Event Planning → Venues
Active Life → Amusement Parks

13. Similarity Functions (3):
Combined
• 𝑆𝑆𝑆𝑐𝑐𝑐𝑐𝑐𝑐𝑐 𝑒, 𝐶 =
𝛼 × 𝑆𝑆𝑆 𝒞 𝑒, 𝐶 + (1 − 𝛼) × 𝑆𝑆𝑆 𝒟(𝑒, 𝐶)
13

14. Overview of Our methods
14
Gather
Candidate
Suggestions
Rank
Suggestions
Post-
Process
Results

15. 15
Step 3: Post-Process Candidates
• How to filter out the suggestions that do not meet
the temporal requirement?
– Not every suggestion has listed business hours.
– To solve the data sparsity problem, we propose to
learn business hours for each category through
majority voting, and apply the learned business
hours for all the suggestions under the same
category.

16. Performance of Our Runs
Use Category only has better performance on WGT
Use Combination has better performance on GT
16
RunID Use
𝑆𝑆𝑆 𝒞
Use
𝑆𝑆𝑆 𝓓
P@5
WGT
P@5
W
P@5
GT
UDInfoCSTc Yes No 0.2481 0.35 0.4950
UDInfoCSTdc Yes Yes 0.2210 0.35 0.5442

17. 17
Results
Category Similarity (UDInfoCSTc)
Combination (UDInfoCSTdc)

18. 18
Results
Category Similarity (UDInfoCSTc)
Combination (UDInfoCSTdc)

19. 19
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
31 11 2 8 17 19 18 23 33 21 16 28 27 22 26 14 25 6 3
P@5WGT--Tc-Tdc
Profiles
Difference of P@5 WGT Performance of Two Runs (Subtract
combination from category)

20. 20
Like Dislike Relevant
Suggestions
Category
Relevant
Suggestions
Combination
31 Restaurants (6)
Tours(3)
Food(3)
Landmarks (2)
Museum (2)
…
Museum (2)
Shopping (2)
Bowling (1)
Planning (1)
…
Restaurants (5) Bowling (1)
11 Restaurants (3)
Food (3)
Bars (2)
Spa (1)
Amusement Parks
(1)
…
Shopping (5)
Museum (4)
Restaurants (3)
…
Food (1)
Amusement Parks (1)
…
Spa (2)
Food (1)
Amusement
Parks (1)
3 Food (4)
Museum (3)
Shopping (2)
Landmarks (1)
Theater (1)
…
Restaurants (6)
Theater (3)
Food (3)
Tours (3)
Museum (1)
Shopping (1)
…
None Theater (1)
Restaurants (1)
Museum (4)

21. 21
Summary
• We propose a ranking-based strategy for
the contextual suggestion task.
• It would be great if the track organizers
could release a standard evaluation set
so that new methods can be more
reliably evaluated.

22. Thank You!
Questions?
22