Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
[UMAP 2016] User-Oriented Context Suggestion
Next
Download to read offline and view in fullscreen.

1

Share

Download to read offline

Can Trailers Help to Alleviate Popularity Bias in Choice-Based Preference Elicitation?

Download to read offline

Slides for my talk at IntRS workshop @ RecSys 2016, Boston.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Can Trailers Help to Alleviate Popularity Bias in Choice-Based Preference Elicitation?

  1. 1. Can Trailers Help to Alleviate Popularity Bias in Choice-Based Preference Elicitation? Mark P. Graus Martijn C. Willemsen Human-Technology Interaction Group Eindhoven University of Technology
  2. 2. Summary • We wanted to see if we could make people chose less popular items in a choice-based preference elicitation recommender system by showing them trailers. • We tested this in a between subjects user study. • We found that after watching trailers people chose less popular items, while user experience was not negatively affected. 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 2
  3. 3. Motivation 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 3
  4. 4. Latent Feature Diversification • Can we reduce choice overload through diversification based on the latent features of a matrix factorization model? Willemsen, M. C., Graus, M. P., & Knijnenburg, B. P. (2016). Understanding the role of latent feature diversification on choice difficulty and satisfaction. User Modeling and User-Adapted Interaction, 1– 43. http://doi.org/10.1007/s11257-016-9178-6 Latent Feature 1 LatentFeature2 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 4
  5. 5. Latent Feature Diversification Findings 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 5 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 low mid high standardizedscore diversification Perceived diversity -1 -0.8 -0.6 -0.4 -0.2 0 0.2 low mid high standardizedscore diversification Expected choice difficulty
  6. 6. LatentFeature2 Latent Feature 1 4) Iteration 2 Choice-Based Preference Elicitation • Can we improve the user experience during cold start by having people choose between items instead of rating items? Graus, M. P., & Willemsen, M. C. (2015). Improving the User Experience during Cold Start through Choice- Based Preference Elicitation. In Proceedings of the 9th ACM Conference on Recommender Systems - RecSys ’15 (pp. 273–276). New York, New York, USA: ACM Press. http://doi.org/10.1145/2792838.2799681 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 6
  7. 7. How does this work? Step 1 Latent Feature 1 LatentFeature2 Iteration 1a: Diversified choice set is calculated from a matrix factorization model (red items) Iteration 1b: User vector (blue arrow) is moved towards chosen item (green item), items with lowest predicted rating are discarded (greyed out items) 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 7
  8. 8. How does this work? Step 2 Iteration 2: New diversified choice set (blue items) End of Iteration 2: with updated vector and more items discarded based on second choice (green item) 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 8
  9. 9. Choice-Based Preference Elicitation Findings • People are more satisfied with choice-based than rating-based interfaces • This comes mainly because of increased popularity (items with many ratings) But we do not want to recommend popular items! 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 9 Satisfaction with Chosen Item Popularity Difficulty Intra List Similarity -2.407(.381) p<.001 -.240 (.145) p<.1 -.479 (.111) p<.001 -.257 (.045) p<.001 14.00 (4.51) p<.01 Choice- Based List + + - - +
  10. 10. Why do people end up with popular items? • Our hypothesis • Users don’t know all movies, hard to judge based on metadata alone • People choose movies they know • People know movies that are popular • Choosing popular movies results in popular recommendations • Our Solution • Provide trailers as additional information for making choices 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 10
  11. 11. Rationale • In the music domain • Implicit Feedback • Movie domain • Implicit feedback is sparse • I (can) listen to 100s of tracks in a week, but I can’t watch 100s of movies a week (and sustain my job). • We can approximate experiencing movies by providing trailers 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 11
  12. 12. Study 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 12
  13. 13. Choice-Based Interface with or without Trailers • N = 71 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 13
  14. 14. Expected Effects Trailers Perceived Diversity Informativeness Perceived Novelty Choice Satisfaction System Satisfaction Popularity of Chosen Items - + + +? -?- - + 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 14
  15. 15. Set Up • Random assignment • Choice Based Preference Elicitation – 9 choices of 10 items [with/without trailers] • Recommendation List – Top-10 Items [with/without trailers] • Survey to measure User Experience • Informativeness • Perceived Diversity • Perceived Novelty • System Satisfaction • Choice Satisfaction 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 15
  16. 16. Results 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 16
  17. 17. Do trailers affect the popularity of chosen items? • Checked through repeated measures (10 choices) • Popularity is expressed as the rank ordering by number of ratings in MovieLens dataset • Trailers do not decrease popularity of choices • The popularity rank of the item chosen in each choice set • The average popularity rank of all items in each choice set 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 17
  18. 18. However: Relative Popularity of Choice • average popularity rank of choice set – popularity rank of chosen item 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 18
  19. 19. If we look at people that actually watched trailers • People that watch trailers are more likely to pick less popular movies from the lists 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 19
  20. 20. User Experience Choice Satisfaction Perceived Diversity System Satisfaction Informativeness .570 (.295) p < 0.1 -.604 (.091) p < 0.01 .244 (.162) n.s. .785 (.115) p < 0.01 -.266 (.122) p < 0.05 Trailers .611 (.256) p < 0.05 -.570 (.259) p < 0.05 - + - - + 9/16/2016 + Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 20
  21. 21. Conclusions 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 21
  22. 22. What we found • Providing people with trailers does make them choose less popular items. • No indication that the overall satisfaction is affected negatively or positively • As opposed to initial study where popularity resulted in increased satisfaction 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 22
  23. 23. Limitations • When were trailers watched? • In the preference elicitation task? • In the decision task? Future Work • How do trailers affect a more standard rating interface? 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 23
  24. 24. Thank You • Questions/remarks? Mark Graus – PhD Student Human-Technology Interaction Group Eindhoven University of Technology m.p.graus@tue.nl https://twitter.com/newmarrk https://linkedin.com/in/markgraus http://www.marrk.nl 9/16/2016 Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @ RecSys '16 24
  • abellogin

    Sep. 17, 2016

Slides for my talk at IntRS workshop @ RecSys 2016, Boston.

Views

Total views

558

On Slideshare

0

From embeds

0

Number of embeds

5

Actions

Downloads

3

Shares

0

Comments

0

Likes

1

×