[SAC2014]Splitting Approaches for Context-Aware Recommendation: An Empirical Study

Splitting Approaches for
Context-Aware Recommendation:
An Empirical Study
Yong Zheng, Robin Burke, Bamshad Mobasher
Center for Web Intelligence
DePaul University, Chicago, IL USA
ACM SIGAPP the 29th Symposium On Applied Computing
Gyeongju, South Korea, March 26, 2014

Center for Web Intelligence DePaul University, Chicago, IL USA
Contents
Context-aware Recommender Systems
Context-aware Splitting Approaches
Empirical Study & Evaluation Results
Discussions, Conclusions & Future work
2

Contents
3

Recommender Systems
Recommender Systems (RS)
Two-dimension rating space: Users × Items  Ratings
M1 M2 M3
U1
U2
U3
U4
4
?

Context-aware Recommender Systems (CARS)
Multi-dimensional space: Users × Items × Contexts  Ratings
Center for Web Intelligence DePaul University, Chicago, IL USA5

Assumptions and Viewpoints in CARS:
 Users’preferences or decisions usually differ from contexts to contexts,
even towards the same item. E.g. buy a gift for someone.
 It’s better to infer user’s preferences by rating profiles within the same or
similar contexts. E.g. look at music others choose within same contexts
 Context is defined as “any information that can be used to characterize
the situation of an entity” by Dey, Anind K. (2001).However, the actual
contexts in CARS and the contextual effects are domain specific.
Movie domain: time, location, companion, mood, etc
Music domain: time, activity, mood, etc
Travel domain: season, weather, companion or trip type, etc

Example of CARS applications: Tour Plan Recommender
Yu, Chien-Chih and Chang, Hsiao-ping, "Towards Context-Aware Recommendation for Personalized
Mobile Travel Planning". International Conference on Context-Aware Systems and Applications, 2012

How to incorporate contexts into RS?
There are two methods to categorize those incorporations.
1).In terms of how contexts interacted with the RS algorithms

How to incorporate contexts into RS?
There are two methods to categorize those incorporations.
2).In terms of whether new CARS algorithms required to be developed
It can be simply categorized into:
a).Transformation Algorithms
A transformation is required, then all traditional RS algorithms can
be applied to. Do NOT need to develop new CARS algorithms.
such as Dimensions as Virtual Items (DaVI) and context-aware
splitting approaches (CASA).
b).Adaptation Algorithms
CARS algorithms are required, traditional algs can be modified.
Such as context-aware matrix factorization (CAMF).

Contents
10

Context-aware Splitting Approaches (CASA)
Context-aware Splitting Approaches (CASA)
In terms of the two categorizations (i.e. how to incorporate contexts into
recommender systems) above, CASA belongs to pre-filtering and
transformation algorithms.
There are three context-aware splitting approaches:
1). Item splitting by L. Baltrunas, F. Ricci, ACM RecSys, 2009
2). User splitting by A. Said et al, CARS@ACM RecSys, 2011
3). UI splitting by Y. Zheng et al, Decisions@ACM RecSys, 2013
User splitting and UI splitting are two approaches derived from
Item splitting, examined and evaluated by different authors.
According to feedbacks from researchers, CASA is one of most efficient
CARS algorithms, but there are no empirical study over them.

Item Splitting
The underlying idea is that the nature of an item, from the user's point of
view, may change in different contextual conditions, hence it may be
useful to consider it as two different items. (L. Baltrunas, F. Ricci,
RecSys'09) – In short, contexts are dependent with items.
Any dependent patterns involved in those ratings?
At Cinema At Home At Swimming Pool
12

Item Splitting -- Example
User Item Location Rating
U1 M1 Pool 5
U2 M1 Pool 5
U3 M1 Pool 5
U1 M1 Home 2
U4 M1 Home 3
U2 M1 Home 2
High Rating
Low Rating
Significant difference?
Let’s split it !!!
M11: being seen at Pool
M12: being seen at Home
M1
Same movie,
different IDs.
13

Item Splitting
 Step 1. Choose a contextual condition to split each item; The selection
process is done by measuring significance of rating differences (such
as the two-sample t test);
 Step 2. Contexts are fused to items and removed from original
multidimensional matrix. We get a 2D rating matrix, then traditional
algorithms like CF, MF can be applied to;
How to select an appropriate contextual conditions for splitting?
a). Binary contextual condition
b). Impurity criteria and significance test
See example in the next.

Item Splitting – Binary Contextual Condition
User Item Loc Rating
U1 M1 Pool 5
U2 M1 Pool 5
U3 M1 Pool 5
U1 M1 Home 2
U4 M1 Home 3
U2 M1 Cinema 2
User Item Rating
U1 M11 5
U2 M11 5
U3 M11 5
U1 M12 2
U4 M12 3
U2 M12 2
Transformation
If there is qualified split,
one item will be split to
two new ones.
A binary contextual condition
for splitting:
“Pool” vs. “Non-Pool”
Why use a binary condition? To alleviate or avoid cold-start problems!
15

Item Splitting – Impurity Criteria
There could be several binary context conditions, for example, “Pool” vs
“Non-Pool”, “Home” vs “Non-Home”, “Weekend” vs “Non-Weekend”.
Impurity criteria and significance test are used to make the selection.
There are 4 impurity criteria for splitting by L. Baltrunas, et al, RecSys'09;
tmean (t-test), tprop (z-test), tchi (chi-square test), tIG (Information gain)
Take tmean for example, tmean, is defined using the two-sample t test and
computes how significantly different are the means of the rating in the
two rating subsets, when the split c (c is a context condition, e.g.
location = Pool) is used. The bigger the t value of the test is, the
more likely the difference of the means in the two partitions is
significant (at 95% confidence value). Choose the largest one!

User Splitting and UI Splitting
Similarly, the splitting approach can be applied to user too!
• User Splitting: is a similar one. Instead of splitting items, it may be
useful to consider one user as two different users, if user demonstrates
significantly different preferences across contexts. (A. Said et al.,
CARS@RecSys 2011) In short, contexts are dependent with users.
• UI Splitting: simply a combination of item splitting and user splitting –
both approaches are applied to create a new rating matrix – new users
and new items are created in the rating matrix. (Y. Zheng, et al,
Decisions@ACM RecSys 2013). In short, it fuses dependent contexts
to users and items simultaneously at the same time.

An Example of Three CASA
After transformation:
Item Splitting: User + NewItem;
User Splitting: NewUser + Item;
UI Splitting: NewUser + NewItem;
UI Splitting fuses contexts to both
users and items, where it may
enlarge the contextual effects,
but it also increases sparsity. It is
hard to say whether UI splitting
will outperform the other two algs
or not. It varies from data to data.
18

Contents
19

Empirical Study and Evaluations
Experimental Goals
a> Comparison Among Three Splitting Approaches
Which one performs the best?
Which splitting criteria is the best appropriate one?
Any underlying patterns to indicate which one should be used?
b> Comparison Between CASA and other Contextual Algorithms
How about CASA competing with other CARS algorithms?

Data Sets
Contextual variables in the three survey data sets:
Food data: degree of hungriness in real and supposed situations;
Movie data: Location (home/cinema), Time (weekend/weekday), Companion (family, etc);
LDOS-CoMoDa: Location, Time, Companion, Weather, Emotions, Seasons, etc
We use a 5-fold cross validation for all data sets and examined algorithms.

Baseline Algorithms
We choose two other context-aware algorithms as the baselines:
1). Differential Context Modeling (DCM) by Y. Zheng, et al, 2012
There are two approaches falling into this category:
Differential Context Relaxation (DCR)
Differential Context Weighting (DCW)
Basic idea: Using rating profiles with same or similar contexts for rating predictions;
Take user-based collaborative filtering for example:
 Segment alg to various components;
 Apply context filter to each component;
 Filters could be different, and not necessary
to be the same.
 Filter could be realized by context relaxation
to find same contexts, or context weighting
to find similar contexts.
 Generally, DCW works better than DCR.

Baseline Algorithms
We choose two other context-aware algorithms as the baselines:
2). Context-aware Matrix Factorization (CAMF) by L. Baltrunas, et al, 2011
There are three approaches falling into this category: CAMF_C, CAMF_CI, CAMF_CU
CAMF_C: Assume contextual effect is associated with each contextual condition only.
CAMF_CI: Assume contextual effect is associated with item-context interactions.
CAMF_CU: Assume contextual effect is associated with user-context interactions.
CAMF is a kind of contextual modeling approach, where context-aware splitting
approaches are contextual pre-filtering approaches. Both of them take advantage of the
dependency between contexts and users or items.

Context-aware Splitting Approaches (CASA)
They are pre-filtering approaches. Any traditional recommendation algorithms can be applied
to, after the original multi-dimensional rating matrix was transformed to a 2D rating matrix.
We evaluate three CASA based on the configuration as follows:
1). Evaluated by different traditional RS algorithms
User-based Collaborative Filtering (UBCF), Item-based Collaborative Filtering (IBCF)
Traditional Matrix Factorization techniques (MF) without taking contexts into consideration
Implemented and evaluated by open-source Toolkit MyMediaLite v3.07
CF algorithms were tuned up by varying # of neighbors;
MF algorithms were examined by varying # of factors and training iterations;
2). Evaluated by different impurity criteria in splitting processes
tmean (t-test), tprop (z-test), tchi (chi-square test), tIG (Information gain)

Evaluation Metrics
We choose three metrics: RMSE, Precision, ROC.
RMSE is used to evaluate the accuracy of predicted ratings. Prediction error, is the most
popular and common used metric in CARS area, since context-aware data are usually sparse
and few users rated a same item for several times.
ROC = a visualization between recall and FPR by varying # of N in Top-N recommendations.
= x axis is FPT, y axis is Recall
In measuring Precision and ROC, we use a rating threshold to judge “relevance”.
For Movie data, the threshold is set as 7, and it is set as 3 for the other two data sets.

Evaluation Metrics
Traditional way to measure Precision and ROC:
1).We have training and testing set;
2).Train a model based on the training set and evaluate it on the testing set;
3).In the testing set, aggregate rating profiles by <user, a list of items he/she rated>;
4).For evaluation purpose, provide a list of ranked Top-N items to each user;
5).And examine the hit ratio between the Top-N list and the list of items rated in the testing;
However, in CARS, contexts should be taken into account; CPrecision and CROC curve
1).We have training and testing set;
2).Train a model based on the training set and evaluate it on the testing set;
3).In the testing set, aggregate rating profiles by <user, a list of items he/she rated, contexts>;
4).For evaluation purpose, provide a list of ranked Top-N items to each <user, contexts>;
5).And examine the hit ratio between the Top-N list and the list of items rated in the testing;
NOTICE: <user, a list of items he/she rated, contexts>; the list is pretty short and even just
one item, because users seldom rated items for several times within different contexts.
Thus the value of CPrecision and CROC will be much smaller than traditional ones.

Evaluation Challenge in CASA (Optional Part)
RMSE can be directly evaluated based on the transformed rating matrix in CASA
It is because the number of rating profiles in data is NOT changed.
CPrecision and CROC cannot be directly evaluated on the transformed rating matrix
1). # of users and # of items could be DIFFERENT
2). It is not comparable to other CARS algorithms
Solution: We only use transformed matrix to predict ratings, but evaluate IR metrics on
the original multi-dimensional rating matrix.
User Item Loc Rating
U1 M1 Pool 5
U2 M1 Pool 5
U3 M1 Pool 5
U1 M1 Home 2
U4 M1 Home 3
U2 M1 Cinema 2
User Item Rating
U1 M11 5
U2 M11 5
U3 M11 5
U1 M12 2
U4 M12 3
U2 M12 2
Transformation

Experimental Results
Experimental Results (in RMSE)
Goal-1: Comparisons among the three context-aware splitting approaches (in RMSE)
Q: Which one performs the best? The best impurity criteria?
A: UI Splitting using MF as the recommendation algorithm. MF works better than CFs.
The best choice varies from data to data. No consistent patterns.
Q: Any other patterns?
A: For Movie data, item splitting is better than user splitting; But user splitting is better
than item splitting for the other two ones, where they have emotional or feeling
contextual variables, we assume those contexts are more dependent with users.

Experimental Results (in RMSE)
Goal-2: Comparisons with other CARS algorithms (in terms of RMSE)
Q: Which one performs the best? The best impurity criteria?
A: UI Splitting using MF as the recommendation algorithm in terms of RMSE.
Q: Any other patterns?
A: If item splitting is better than user splitting, CAMF_CI is better than CAMF_CU;
If user splitting is better than item splitting, then CAMF_CU is better than CAMF_CI;
It is because both of them take advantage of context-dependency patterns!!

Experimental Results (CPrecision & CROC)
Goal-1: Comparisons among the three context-aware splitting approaches
In CPrecision, UI splitting > Item splitting > User Splitting;
In ROC Curve, UI splitting > User splitting > Item Splitting;
Goal-2: Comparisons with other CARS algorithms
In CPrecision, UI splitting > CAMF_CI > CAMF_CU > DCW > DCR;
In ROC Curve, UI splitting > CAMF_CU > CAMF_CI > DCW > DCR;
Patterns:
UI Splitting is the best in RMSE and IR metrics for LDOS-CoMoDa;
Consistent findings in context-dependency pattern in EACH METRIC;
In RMSE, context is more dependent with user;

In CPrecision, Item splitting > UI splitting > User splitting;
In ROC Curve, same patter as above;
In CPrecision, Item splitting > UI splitting > CAMF_CI > CAMF_CU > DCW;
In ROC Curve, Item splitting > UI splitting > CAMF_CI > DCW > CAMF_CU;
Patterns:
Item Splitting is the best in RMSE and IR metrics for Movie data;
Consistent findings in context-dependency pattern in EACH METRIC;

In CPrecision, UI splitting > User splitting > Item Splitting;
In ROC Curve, same pattern as above;
In CPrecision, UI splitting > CAMF_CU > CAMF_CI > DCR > DCW;
In ROC Curve, DCR > UI splitting > DCW > CAMF_CU > CAMF_CI
Patterns:
Overall, UI Splitting is the best in RMSE and IR metrics for Food Data;
Consistent findings in context-dependency pattern;

Contents
33

Conclusions
Conclusions & Future Work
Generally speaking, UI splitting is the best;
In Movie data, UI splitting is the best on RMSE, but item splitting is the best on IR metrics;
If context is not that dependent with users, merging effects by UI splitting may decrease the
joint effect on recommendations.
Any patterns or guidelines to choose which context-aware algorithms?
In terms of choices between item splitting & user splitting, and CAMF_CI & CAMF_CU, it
totally depends on which one contexts are more dependent to, user or item?
Whether UI splitting performs the best depends on three factors:
1). The dependency between contexts and users and items;
2). The sparsity after rating matrix transformation – cold-start problems in CASA;
3). The performance difference between user splitting and item splitting. If one of them
performs bad, it is not guaranteed that the joint effect UI splitting will perform better;
Future work:
1).how to judge contexts are more dependent with users or items?
Any numeric metrics to validate it? PS: Impurity values? no consistent patterns.
2). How to alleviate the cold-start problems in UI splitting.

Conclusions
References
Item Splitting
1) L. Baltrunas, and F. Ricci. "Context-based splitting of item ratings in collaborative filtering." ACM RecSys, 2009.
2) L. Baltrunas, and F. Ricci. "Experimental evaluation of context-dependent collaborative filtering using item splitting." User Modeling and
User-Adapted Interaction (2013): 1-28.
User Splitting
1) L. Batrunas and X. Amatriain."Towards Time-Dependent Recommendation Based on Implicit Feedback." CARS@RecSys, 2009
2) A. Said, E. Luca, S. Albayrak. "Inferring contextual user profiles—improving recommender performance.“ CARS@RecSys, 2011
UI Splitting
1) Y. Zheng, R. Burke, B. Mobasher. "The Role of Emotions in Context-aware Recommendation". Decisons@RecSys, 2013
2) Y. Zheng, R. Burke, B. Mobasher, “Splitting Approaches for Context-Aware Recommendation: An Empirical Study”, ACM SAC, 2014
Context-aware Matrix Factorization
1) L. Baltrunas, B. Ludwig, F. Ricci. "Matrix factorization techniques for context aware recommendation." ACM RecSys 2011.
Differential Context Modeling
1) Y. Zheng, R. Burke, B. Mobasher. "Differential Context Relaxation for Context-aware Travel Recommendation". EC-WEB, 2012
2) Y. Zheng, R. Burke, B. Mobasher. "Optimal Feature Selection for Context-Aware Recommendation using Differential Relaxation".
CARS@RecSys, 2012
3) Y. Zheng, R. Burke, B. Mobasher. "Recommendation with Differential Context Weighting". In UMAP, 2013

[SAC2014]Splitting Approaches for Context-Aware Recommendation: An Empirical Study

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (17)

Similar to [SAC2014]Splitting Approaches for Context-Aware Recommendation: An Empirical Study

Similar to [SAC2014]Splitting Approaches for Context-Aware Recommendation: An Empirical Study (20)

More from YONG ZHENG

More from YONG ZHENG (10)

Recently uploaded

Recently uploaded (20)

[SAC2014]Splitting Approaches for Context-Aware Recommendation: An Empirical Study