The document discusses research on recommender systems, highlighting the importance of user engagement and satisfaction beyond just ratings. It covers various recommendation methods, including collaborative filtering and content-based filtering, while introducing the concept of tag-based recommendations and implicit feedback. The author's contributions to the field include utilizing social tags for recommendations and exploring user-centric evaluation methods.

1. Research on Recommender
Systems: Beyond Ratings and Lists
Denis Parra, Ph.D. Information Sciences
Assistant Professor, CS Department
School of Engineering
Pontificia Universidad Católica de Chile
Tuesday, November 11th of 2014

2. Outline
• Personal Introduction
• Quick Overview of Recommender Systems
• My Work on Recommender Systems
– Tag-Based Recommendation
– Implicit-Feedback (time allowing …)
– Visual Interactive Interfaces
• Summary & Current & Future Work
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 2

3. Personal Introduction
• I’m from Valdivia!
• There are many reasons to love Valdivia
The City
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 3

4. Personal Introduction
• I’m from Valdivia!
• There are many reasons to love Valdivia
The Sports
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 4

5. Personal Introduction
• I’m from Valdivia!
• There are many reasons to love Valdivia
The Animals
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 5

6. Personal Introduction
• B.Eng. and professional title of Civil Engineer in
Informatics from Universidad Austral de Chile
(2004), Valdivia, Chile
• Ph.D. in Information Sciences at University of
Pittsburgh (2013), Pittsburgh, PA, USA
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 6

7. Personal Introduction
• B.Eng. and professional title of Civil Engineer in
Informatics from Universidad Austral de Chile
(2004), Valdivia, Chile
• Ph.D. in Information Sciences at University of
Pittsburgh (2013), Pittsburgh, PA, USA
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 7

8. Recommender Systems
INTRODUCTION
*
* Danboard (Danbo): Amazon’s cardboard robot, in these slides it represents a
recommender system
Nov 11th 2014 8

9. Recommender Systems (RecSys)
Systems that help (groups of) people to find relevant items in
a crowded item or information space (MacNee et al. 2006)
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 9

10. Why do we care about RecSys?
• RecSys have gained popularity due to several
domains & applications that require people to make
decisions among a large set of items.
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 10

11. A lil’ bit of History
• First recommender systems were built at the
beginning of 90’s (Tapestry, GroupLens, Ringo)
• Online contests, such as the Netflix prize, grew the
attention on recommender systems beyond
Computer Science
(2006-2009)
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 11

12. The Recommendation Problem
• The most popular way that the recommendation
problem has been presented is about rating
prediction:
Predict!
Item 1 Item 2 … Item m
User 1 1 5 4
User 2 5 1 ?
…
User n 2 5 ?
• How good is my prediction?
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 12

13. Recommendation Methods
• Without covering all possible methods, the two
most typical classifications on recommender
algorithms are
Classification 1 Classification 2
- Collaborative Filtering
- Content-based Filtering
- Hybrid
- Memory-based
- Model-based
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 13

14. Collaborative Filtering (User-based KNN)
• Step 1: Finding Similar Users (Pearson Corr.)
5
4
4
1
2
1
5
4
4
1
2
5
Active
user
User_1
User_2
User_3
active user
user_1
user_2
user_3
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 14

15. Collaborative Filtering (User-based KNN)
• Step 1: Finding Similar Users (Pearson Corr.)
5
4
4
1
2
1
5
4
4
1
2
5
Active
user
User_1
User_2
User_3
Σ
r r r r
( )( )
i ⊂
CR ui u ni n
r r r r
− −
u ,
n
( )2 ( )2
Σ Σ
− −
i ⊂ CR ui u i ⊂
CR ni n
=
u n u n
Sim u n
, ,
( , )
active user
user_1
0.4472136
user_2
0.49236596
user_3
-0.91520863
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 15

16. Collaborative Filtering (User-based KNN)
• Step 2: Ranking the items to recommend
5
4
4
Item 1
2
1
5
4
4
Active
user
2
User_1
3
User_2
4
2
Item 2
Item 3
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 16

17. Collaborative Filtering (User-based KNN)
• Step 2: Ranking the items to recommend
5
4
4
Item 1
2
1
5
4
4
Active
user
2 pred u i r
User_1
3
User_2
userSim u n r r
( , ) ( )
⊂ ⋅ −
n neighbors u ni n
u userSim u n
Σ
Σ
n ⊂
neighbors u
= +
( )
( )
( , )
( , )
4
2
Item 2
Item 3
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 17

18. Pros/Cons of CF
PROS:
• Very simple to implement
• Content-agnostic
• Compared to other techniques such as content-based,
is more accurate
CONS:
• Sparsity
• Cold-start
• New Item
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 18

19. Content-Based Filtering
• Can be traced back to techniques from IR, where
the User Profile represents a query.
Doc_1 = {w_1, w_2, …., w_3}
5
4
5
user_profile = {w_1, w_2, …., w_3} using TF-IDF, weighting
Doc_2 = {w_1, w_2, …., w_3}
Doc_3 = {w_1, w_2, …., w_3}
Doc_n = {w_1, w_2, …., w_3}
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 19

20. PROS/CONS of Content-Based Filtering
PROS:
• New items can be matched without previous
feedback
• It can exploit also techniques such as LSA or LDA
• It can use semantic data (ConceptNet, WordNet,
etc.)
CONS:
• Less accurate than collaborative filtering
• Tends to overspecialization
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 20

21. Hybridization
• Combine previous methods to overcome their
weaknesses (Burke, 2002)
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 21

22. C2. Model/Memory Classification
• Memory-based methods use the whole dataset in
training and prediction. User and Item-based CF are
examples.
• Model-based methods build a model during training
and only use this model during prediction. This
makes prediction performance way faster and
scalable
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 22

23. Model-based: Matrix Factorization
Latent vector of the item
Latent vector of the user
SVD ~
Singular Value Decomposition
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 23

24. PROS/CONS of MF and latent factors model
PROS:
• So far, state-of-the-art in terms of accuracy (these
methods won the Netflix Prize)
• Performance-wise, the best option nowadays: slow
at training time O((m+n)3) compared to correlation
O(m2n), but linear at prediction time O(m+n)
CONS:
• Recommendations are obscure: How to explain that
certain “latent factors” produced the
recommendation
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 24

25. Rethinking the Recommendation Problem
• Ratings are scarce: need for exploiting other sources
of user preference
• User-centric recommendation takes the problem
beyond ratings and ranked lists: evaluate user
engagement and satisfaction, not only RMSE
• Several other dimensions to consider in the
evaluation: novelty of the results, diversity,
coverage (user and catalog), serendipity
• Study de effect of interface characteristics: user-control,
explainability
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 25

26. My Take on RecSys Research
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 30

27. My Work on RecSys
• Traditional RecSys: accurate prediction and TopN
algorithms
• In my research I have contributed to RecSys by:
– Utilizing other sources of user preference (Social Tags)
– Exploiting implicit feedback for recommendation and for
mapping explicit feedback
– Studying user-centric evaluation: the effect of user
controllability on user satisfaction in interactive
interfaces
• And nowadays: Studying whether Virtual Worlds are a
good proxy for real world recommendation tasks
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 31

28. This is not only My work J
• Dr. Peter Brusilovsky
University of Pittsburgh, PA, USA
• Dr. Alexander Troussov
IBM Dublin and TCD, Ireland
• Dr. Xavier Amatriain
TID / Netflix, CA, USA
• Dr. Christoff Trattner
NTNU, Norway
• Dr. Katrien Verbert
KU Leuven, Belgium
• Dr. Leandro Balby-Marinho
UFCG, Brasil
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 32

29. TAG-BASED RECOMMENDATION

30. Tag-based Recommendation
• D. Parra, P. Brusilovsky. Improving Collaborative
Filtering in Social Tagging Systems for the
Recommendation of Scientific Articles. Web
Intelligence 2010, Toronto, Canada
• D. Parra, P. Brusilovsky. Collaborative Filtering
for Social Tagging Systems: an Experiment with
CiteULike. ACM Recsys 2009, New York, NY, USA
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 34

31. Motivation
• Ratings are scarce. Find another source of user
preference: Social Tagging Systems
User
Resource
Tags
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 35

32. A Folksonomy
• When a user u uses adds an item i using one or
more tags t1,…, tn, there is a tagging instance.
• The collection of tagging instances produces a
folksonomy
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 36

33. Applying CF over the Folksonomy
• In the first step: Calculate user similarity
Traditional CF Tag-based CF
Pearson Correlation
over ratings
BM25 over social tags
• In the second step: incorporate the amount of raters
to rank the items (NwCF)
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 37

34. Tag-based CF
=
Query
Doc_1
Doc_2
Doc_3
BM25
Active
User
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 38

35. Okapi BM25
BM25: We obtain the similarity between
users (neighbors) using their set of tags as
“documents” and performing an Okapi BM25
(probabilistic IR model) Retrieval Status
Value calculation.
Tag frequency in the neighbor (v) profile
1 ( 1)
RSV IDF k tf
sim(u, v) = ( 1)
Σ∈
+
td
k tf
3
d k ((1 − b ) + b × ( L / L ))
+
tf
k tf
= ⋅
t q 1
d ave td
tq
Tag frequency in the active user (u) profile
( , ) log (1 ( )) ( , ) 10 predʹ′ u i = + nbr i ⋅ pred u i
+
+
⋅
tq
3
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 39

36. Evaluation
• Crawl during 38 days during June-July 2009
Item # unique instances
# users 784
# items 26,599
# tags 26,009
# posts 71,413
# annotations 218,930
avg # items per user 91
avg # users per item 2.68
avg # tags per user 88.02
avg # users per tag 2.65
avg # tags per item 7.07
avg # items per tag 7.23
Item Phase 2
dataset
# users 5,849
# articles 574,907
# tags 139,993
#tagging
incidents
2,337,571
Filtering
process
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 40

37. Cross-validation
• Test-validation-train sets, 10-fold cross
validation
• Training to obtain parameter K: neighb. size
• One run the experiment: ~12 hours
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 41

38. Results & Statistical Significance
• BM25 is intended to bring more neighbors, at the
cost of more noise (neighbors not so similar)
• NwCF helps to decrease noise, so it was natural to
combine them and try just that option
CCF NwCF BM25+CCF BM25+NwCF
MAP@10 0.12875 0.1432* 0.1876** 0.1942***
K (neigh.size) 20 22 21 29
Ucov 81.12% 81.12% 99.23% 99.23%
Significance over the baseline: *p < 0.236, ** p < 0.033, *** p < 0.001
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 42

39. Take-aways
• We can exploit tags as a source for user similarity in
recommendation algorithms
• Tag-based (BM25) similarity can be considered as
an alternative to Pearson Correlation to calculate
user similarity in STS.
• Incorporating the number of raters helped to
decrease the noise produced by items with too few
ratings
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 43

40. Work with Xavier Amatriain
IMPLICIT FEEDBACK

41. Implicit-Feedback
• Slides are based on two articles:
– Parra-Santander, D., & Amatriain, X. (2011). Walk the
Talk: Analyzing the relation between implicit and
explicit feedback for preference elicitation. Proceedings
of UMAP 2011, Girona, Spain
– Parra, D., Karatzoglou, A., Amatriain, X., & Yavuz, I.
(2011). Implicit feedback recommendation via implicit-to-
explicit ordinal logistic regression mapping.
Proceedings of the CARS Workshop, Chicago, IL, USA,
2011.
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 45

42. Introduction (1/2)
• Most of recommender system approaches rely on
explicit information of the users, but…
• Explicit feedback: scarce (people are not especially
eager to rate or to provide personal info)
• Implicit feedback: Is less scarce, but (Hu et al., 2008)
There’s no negative feedback
… and if you watch a TV program just
once or twice?
Noisy
… but explicit feedback is also noisy
(Amatriain et al., 2009)
Preference & Confidence
… we aim to map the I.F. to preference
(our main goal)
Lack of evaluation metrics
… if we can map I.F. and E.F., we can
have a comparable evaluation
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 46

43. Introduction (2/2)
• Is it possible to map implicit behavior to explicit
preference (ratings)?
• Which variables better account for the amount of
times a user listens to online albums? [Baltrunas &
Amatriain CARS ‘09 workshop – RecSys 2009.]
• OUR APPROACH: Study with Last.fm users
– Part I: Ask users to rate 100 albums (how to sample)
– Part II: Build a model to map collected implicit feedback
and context to explicit feedback
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 47

44. Walk the Talk (2011)
Albums they listened to during last:
7days, 3months, 6months, year,
overall For each album in the list we obtained:
# user plays (in each period), # of
global listeners and # of global plays
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 48

45. Walk the Talk - 2
• Requirements: 18 y.o., scrobblings > 5000
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 49

46. Quantization of Data for Sampling
• What items should they rate? Item (album) sampling:
– Implicit Feedback (IF): playcount for a user on a given album.
Changed to scale [1-3], 3 means being more listened to.
– Global Popularity (GP): global playcount for all users on a given
album [1-3]. Changed to scale [1-3], 3 means being more listened to.
– Recentness (R) : time elapsed since user played a given album.
Changed to scale [1-3], 3 means being listened to more recently.
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 50

47. 4 Regression Analysis
M1: implicit feedback
M2:
implicit
feedback &
recentness
M4:
Interaction of
implicit
feedback &
recentness
M3: implicit
feedback,
recentness,
global
popularity
• Including Recentness increases R2 in more than 10% [ 1 -> 2]
• Including GP increases R2, not much compared to RE + IF [ 1 -> 3]
• Not Including GP, but including interaction between IF and RE
improves the variance of the DV explained by the regression model.
[ 2 -> 4 ]
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 51

48. 4.1 Regression Analysis
Model RMSE1 RMSE2
User average 1.5308 1.1051
M1: Implicit feedback 1.4206 1.0402
M2: Implicit feedback + recentness 1.4136 1.034
M3: Implicit feedback + recentness + global popularity 1.4130 1.0338
M4: Interaction of Implicit feedback * recentness 1.4127 1.0332
• We tested conclusions of regression analysis by
predicting the score, checking RMSE in 10-fold
cross validation.
• Results of regression analysis are supported.
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 52

49. Conclusions of Part I
• Using a linear model, Implicit feedback and
recentness can help to predict explicit feedback (in
the form of ratings)
• Global popularity doesn’t show a significant
improvement in the prediction task
• Our model can help to relate implicit and explicit
feedback, helping to evaluate and compare explicit
and implicit recommender systems.
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 53

50. Part II: Extension of Walk the Talk
• Implicit Feedback Recommendation via Implicit-to-
Explicit OLR Mapping (Recsys 2011, CARS
Workshop)
– Consider ratings as ordinal variables
– Use mixed-models to account for non-independence of
observations
– Compare with state-of-the-art implicit feedback
algorithm
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 54

51. Recalling the 1st study (5/5)
• Prediction of rating by multiple Linear Regression
evaluated with RMSE.
• Results showed that Implicit feedback (play count
of the album by a specific user) and recentness (how
recently an album was listened to) were important
factors, global popularity had a weaker effect.
• Results also showed that listening style (if user
preferred to listen to single tracks, CDs, or either)
was also an important factor, and not the other
ones.
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 55

52. ... but
• Linear Regression didn’t account for the nested
nature of ratings
User 1
1 3 5 3 0 4 5 2 2 1 5 4 3 2
User n
3 2 1 0 4 5 2 5 4 3 2 1 3 5
. . .
• And ratings were treated as continuous, when they
are actually ordinal.
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 56

53. So, Ordinal Logistic Regression!
• Actually Mixed-Efects Ordinal Multinomial Logistic
Regression
• Mixed-effects: Nested nature of ratings
• We obtain a distribution over ratings (ordinal
multinomial) per each pair USER, ITEM -> we
predict the rating using the expected value.
• … And we can compare the inferred ratings with
a method that directly uses implicit information
(playcounts) to recommend ( by Hu, Koren et al.
2007)
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 57

54. Ordinal Regression for Mapping
• Model
• Predicted value
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 58

55. Datasets
• D1: users, albums, if, re, gp, ratings,
demographics/consumption
• D2: users, albums, if, re, gp, NO RATINGS.
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 59

56. Results
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 60

57. Conclusions & Current Work
Problem/ Challenge
1. Ground truth: How many Playcounts to relevancy?
> Sensibility Analysis needed
2. Quantization of playcounts (implicit feedback),
recentness, and overall number of listeners of an album
(global popularity) [1-3] scale v/s raw playcounts >
modifiy and compare
3. Additional/Alternative metrics for evaluation [MAP
and nDCG used in the paper]
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 61

58. Part of this work with Katrien Verbert
VISUALIZATION + USER
CONTROLLABILITY

59. Visualization & User Controllability
• Motivation: Can user controllability and
explainability improve user engagement and
satisfaction with a recommender system?
• Specific research question: How intersections of
contexts of relevance (of recommendation
algorithms) might be better represented for user
experience with the recommender?
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 63

60. The Concept of Controllability
MovieLens: example of traditional recommender list
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 64

61. Visualization & User Controllability
• Motivation: Can user controllability and
explainability improve user engagement and
satisfaction with a recommender system?
• Specific research question: How intersections of
contexts of relevance (of recommendation
algorithms) might be better represented for user
experience with the recommender?
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 65

62. Research Platform
• The studies were conducted using Conference
Navigator, a Conference Support System
• Our goal was recommending conference talks
http://halley.exp.sis.pitt.edu/cn3/
Program
Proceedings
Author List
Recommendations
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 66

63. Hybrid RecSys: Visualizing Intersections
• Clustermap vs. Venn Diagram
Clustermap Venn diagram
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 67

64. TalkExplorer – IUI 2013
• Adaptation of Aduna Visualization to CN
• Main research question: Does fusion (intersection) of
contexts of relevance improve user experience?
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 68

65. TalkExplorer - I
Entities
Tags, Recommender Agents,
Users
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 69

66. TalkExplorer - II
• Canvas Area: Intersections of Different Entities
Recommender
Recommender
Cluster
with
intersect
ion of
entities Cluster (of
talks)
associated
to only one
entity
User
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 70

67. TalkExplorer - III
Items
Talks explored by the
user
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 71

68. Our Assumptions
• Items which are relevant in more that
one aspect could be more valuable to the
users
• Displaying multiple aspects of relevance
visually is important for the users in the
process of item’s exploration
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 72

69. TalkExplorer Studies I & II
• Study I
– Controlled Experiment: Users were asked to discover
relevant talks by exploring the three types of entities: tags,
recommender agents and users.
– Conducted at Hypertext and UMAP 2012 (21 users)
– Subjects familiar with Visualizations and Recsys
• Study II
– Field Study: Users were left free to explore the interface.
– Conducted at LAK 2012 and ECTEL 2013 (18 users)
– Subjects familiar with visualizations, but not much with
RecSys
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 73

70. Evaluation: Intersections & Effectiveness
• What do we call an “Intersection”?
• We used #explorations on intersections and their
effectiveness, defined as:
Effectiveness = |푏표표푘푚푎푟푘푒푑 푖푡푒푚푠|/|
푖푛푡푒푟푒푠푒푐푡푖표푛푠 푒푥푝푙표푟푒푑|
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 74

71. Results of Studies I & II
• Effectiveness increases
with intersections of
more entities
• Effectiveness wasn’t
affected in the field
study (study 2)
• … but exploration
distribution was
affected
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 75

72. SETFUSION: VENN DIAGRAM FOR
USER-CONTROLLABLE INTERFACE
76

73. SetFusion – IUI 2014
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 77

74. SetFusion I
Traditional
Ranked List
Papers sorted by
Relevance.
It combines 3
recommendation
approaches.
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 78

75. SetFusion - II
Sliders
Allow the user to control the importance of
each data source or recommendation method
Interactive Venn Diagram
Allows the user to inspect and to filter papers
recommended. Actions available:
- Filter item list by clicking on an area
- Highlight a paper by mouse-over on a circle
- Scroll to paper by clicking on a circle
- Indicate bookmarked papers
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 79

76. SetFusion – UMAP 2012
• Field Study: let users freely explore the interface
- ~50% (50 users) tried the
SetFusion recommender
- 28% (14 users) bookmarked at
least one paper
- Users explored in average 14.9
talks and bookmarked 7.36
talks in average.
A AB ABC AC B BC C
15 7 9 26 18 4 17
16% 7% 9% 27% 19% 4% 18%
Distribution of bookmarks per method or combination of methods
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 80

77. TalkExplorer vs. SetFusion
• Comparing distributions of explorations
In studies 1 and 2 over
talkExplorer we observed an
important change in the
distribution of explorations.
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 81

78. TalkExplorer vs. SetFusion
• Comparing distributions of explorations
Comparing the field studies:
- In TalkExplorer, 84% of
the explorations over
intersections were
performed over clusters of
1 item
- In SetFusion, was only
52%, compared to 48%
(18% + 30%) of multiple
intersections, diff. not
statistically significant
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 82

79. Summary & Conclusions
• We presented two implementations of visual
interactive interfaces that tackle exploration on a
recommendation setting
• We showed that intersections of several contexts of
relevance help to discover relevant items
• The visual paradigm used can have a strong effect on
user behavior: we need to keep working on visual
representation that promote exploration without
increasing the cognitive load over the users
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 83

80. Limitations & Future Work
• Apply our approach to other domains (fusion of
data sources or recommendation algorithms)
• For SetFusion, find alternatives to scale the
approach to more than 3 sets, potential alternatives:
– Clustering and
– Radial sets
• Consider other factors that interact with the user
satisfaction:
– Controllability by itself vs. minimum level of accuracy
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 84

81. More Details on SetFusion?
• Effect of other variables: gender, age, experience
with in the domain, or familiarity with the system
• Check our upcoming paper in the IJHCS “User-controllable
Personalization: A Case Study with
SetFusion”: Controlled Laboratory study with
SetFusion versus traditional ranked list
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 85

82. CONCLUSIONS (& CURRENT) &
FUTURE WORK

83. Challenges in Recommender Systems
• Recommendation to groups
• Cross-Domain recommendation
• User-centric evaluation
• Interactive interfaces and visualization
• Improve Evaluation for comparison (P. Campos of U.
of Bio-Bio on doing fair evaluations considering time)
• ML: Active learning, multi-armed bandits (exploration,
exploitation)
• Prevent the “Filter Bubble”
• Make algorithms resistant to attacks
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 87

84. Are Virtual Worlds Good Proxies for Real
World ?
• Why? We have a Second Life dataset with 3
connected dimensions of information
Social Network
Marketplace
Virtual World
• 2 undergoing projects: Entrepreneurship and LBSN
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 88

85. Entrepreneurship
• Can we predict whether a user will create a store
and how successful will she/he be? Literature on
this area is extremely scarce.
Social Network Marketplace
James Gaskin
SEM, Causal models
BYU, USA
Stephen Zhang
Entrepreneurship
PUC Chile
Christoph Trattner
Social Networks
NTNU, Norway
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 89

86. Location-Based Social Networks (LBSN)
• How similar are the patterns of mobility in real
world and virtual world ?
Social Network
Virtual World
Christoph Trattner
Social Networks
NTNU, Norway
Leandro Balby-Marinho
LBSN and RecSys
UFCG, Brasil
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 90

87. Other RecSys Activities
• I am part of the Program Committee of the 2015
RecSys challenge. Don’t miss it!
» Is the user going to buy items in this session? Yes|No
» If yes, what are the items that are going to be bought?
• Part of team creating the upcoming RecSys Forum
(like SIGIR Forum). Coming Soon! (Alan Said,
Cataldo Musto, Alejandro Bellogin, etc.)
Nov 11th 2014 D.Parra ~ JCC 2014 ~ Invited Talk 92

88. dparra@ing.puc.cl
THANKS!