Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
cacooアイコンの話
cacooアイコンの話
Loading in …3
×
1 of 45

Toward Better Interactions in Recommender Systems: Cycling and Serpentining Approaches for Top-N Item Lists

1

Share

Download to read offline

An experience design perspective on recommenders: There is a tradeoff between serving come-and-go users vs. encouraging deeper interaction/engagement!

Better understanding of the trade-off between efficiency vs. engagement can help design a better recommender user experience!

Cycling and serpentining top-N recommendation lists have benefits (higher engagement) but also costs (negative perception)!

More work combining algorithms and user experience is needed!

Related Books

Free with a 30 day trial from Scribd

See all

Toward Better Interactions in Recommender Systems: Cycling and Serpentining Approaches for Top-N Item Lists

  1. 1. Toward Better Interactions in Recommender Systems: Cycling and Serpentining Approaches for Top-N Item Lists Qian Zhao, F. Maxwell Harper, Joseph A. Konstan (GroupLens) Gediminas Adomavicius (Dept. of Information and Decision Sciences, University of Minnesota) Martijn Willemsen (Human-Technology Interaction group, Eindhoven University of Technology) 1
  2. 2. Recommender Systems » Recommender systems typically display the top-N recommended items in order. 2
  3. 3. Example: MovieLens.org Top Picks 3
  4. 4. What’s wrong with this recommender? 4
  5. 5. 1st Visit 5
  6. 6. 2nd Visit 6
  7. 7. 3rd ... Visit 7
  8. 8. 8
  9. 9. What’s wrong with this recommender? 9
  10. 10. 1st Page 10
  11. 11. 5th Page 11
  12. 12. 10th … Page 12
  13. 13. 13 Excitement Show Time/Further Exploration
  14. 14. Re-thinking Top-N Recommendation Lists » Static Top-N in order à the best design for user interaction and temporal experience » Two missing factors • Fresh vs. stale • Further exploration à worse quality/experience 14
  15. 15. Prior Work: Dynamic/Interactive Recommenders » Classics • Temporal dynamics (Koren et al. 2010) • CARS (Adomavicius et al. 2011) • Incremental matrix factorization (Luo et al. 2012) » Interactive (reinforcement) machine learning • Markov decision processes (Shani et al. 2002) • Contextual bandits (Lu et al. 2010) 15
  16. 16. This Work: Cycling and Serpentining » Cycling demotes items that have been viewed several (3+) times, exposing fresher recommendations. » Serpentining spreads top recommended items across several pages, offering high-quality items on each page as a user continues to explore. 16
  17. 17. Cycling Movie Score M1 5.0 M2 4.9 M3 4.6 M4 4.0 17 Movie Score #display M1 5.0 3 M2 4.9 3
  18. 18. Cycling Movie Score M1 5.0 M2 4.9 M3 4.6 M4 4.0 18 Movie Score #display M3 4.6 0 M4 4.0 0 M1 5.0 3 M2 4.9 3
  19. 19. Serpentining Movie Score M1 5.0 M2 4.9 M3 4.8 M4 4.7 M5 4.6 M6 4.5 M7 4.4 M8 4.3 19 Movie Score M1 5.0 M3 4.8 M5 4.6 M7 4.4 p. 1 p. 2 p. 1 p. 2
  20. 20. Serpentining Movie Score M1 5.0 M2 4.9 M3 4.8 M4 4.7 M5 4.6 M6 4.5 M7 4.4 M8 4.3 20 Movie Score M1 5.0 M3 4.8 M5 4.6 M7 4.4 M2 4.9 M4 4.7 M6 4.5 M8 4.3 p. 1 p. 2 p. 1 p. 2
  21. 21. When to Cycle » Within-session: each time when users go back to home page • More perceived change but more confusion? • Reflective of current recommenders in terms of the change, e.g. Youtube. » Between-session: when users sign in next time • Less disorienting? • But, can users perceive the change? 21
  22. 22. The novel contribution of this work » Cycling and serpentining approaches are not studied before. • e.g. Youtube is not cycling (Davidson et al. 2010). » Better understand the effects of dynamic top-N lists on user experience. 22
  23. 23. A between-subjects field experiment on MovieLens.org » Measurements • Objective activity level • Subjective perception (through surveys) 23
  24. 24. Objective Activity Level » opt out rate (bad experience) » number of page views (negative efficiency or positive engagement) » number of interested actions. i.e. clicks, wishlist (positive engagement) » interested rate: number of interested actions per page view (positive efficiency) 24
  25. 25. Subjective Perception » Specific aspects • accuracy, familiarity, diversity, novelty • change, freshness, confusion, boredom » Overall experience • usefulness and satisfaction » Surveys embedded in user browsing activities, i.e. sampling user experience 25
  26. 26. Survey Prompt 26
  27. 27. Survey Display 27
  28. 28. 28
  29. 29. Data » March 22 – May 14, 2016 (1.6 months) » 5158 invited users (having more than 15 ratings and two sessions) » 987 users joined the experiment » Analyzing activities of the first half month (802 users) and all survey responses (~900) 29
  30. 30. Activity Statistics in the Experiment Activity #Activities #Users front page 43,371 987 top picks page 43,231 (mean page depth: 9.14) 821 rating 163,242 943 click 107,955 924 wishlisting 23,032 473 30
  31. 31. Results No Cycling Within-session Cycling Between-session Cycling No Serpentining control condition ? ? Serpentining ? ? ? 31
  32. 32. Results No Cycling Within-session Cycling Between-session Cycling No Serpentining control condition opt out rate: + #page views: + #interested: + interested rate: + accuracy: - familiarity: - usefulness: - change: + freshness: + Serpentining 32 Only significant results are shown. Italic: objective metrics; Non-italic: subjective metrics
  33. 33. Results 33 No Cycling Within-session Cycling Between-session Cycling No Serpentining control condition opt out rate: + #page views: + #interested: + interested rate: + accuracy: - familiarity: - usefulness: - change: + freshness: + #page views: + #interested: + accuracy: - confusion: + change: + Serpentining Only significant results are shown. Italic: objective metrics; Non-italic: subjective metrics
  34. 34. 34 No Cycling Within-session Cycling Between-session Cycling No Serpentining control condition opt out rate: + #page views: + #interested: + interested rate: + accuracy: - familiarity: - usefulness: - change: + freshness: + #page views: + #interested: + accuracy: - confusion: + change: + Serpentining #page views: + #interested: + accuracy: - familiarity: - usefulness: - Only significant results are shown. Italic: objective metrics; Non-italic: subjective metrics
  35. 35. 35 No Cycling Within-session Cycling Between-session Cycling No Serpentining control condition opt out rate: + #page views: + #interested: + interested rate: + accuracy: - familiarity: - usefulness: - change: + freshness: + #page views: + #interested: + accuracy: - confusion: + change: + Serpentining #page views: + #interested: + accuracy: - familiarity: - usefulness: - too complicated manipulation no interesting sig. results see the paper for details Only significant results are shown. Italic: objective metrics; Non-italic: subjective metrics
  36. 36. 36 No Cycling Within-session Cycling Between-session Cycling No Serpentining control condition opt out rate: + #page views: + #interested: + interested rate: + accuracy: - familiarity: - usefulness: - change: + freshness: + #page views: + #interested: + accuracy: - confusion: + change: + Serpentining #page views: + #interested: + accuracy: - familiarity: - usefulness: - see the paper Only significant results are shown. Italic: objective metrics; Non-italic: subjective metrics Some users hate it! L
  37. 37. 37 No Cycling Within-session Cycling Between-session Cycling No Serpentining control condition opt out rate: + #page views: + #interested: + interested rate: + accuracy: - familiarity: - usefulness: - change: + freshness: + #page views: + #interested: + accuracy: - confusion: + change: + Serpentining #page views: + #interested: + accuracy: - familiarity: - usefulness: - see the paper Only significant results are shown. Italic: objective metrics; Non-italic: subjective metrics Staying users engage more!
  38. 38. 38 No Cycling Within-session Cycling Between-session Cycling No Serpentining control condition opt out rate: + #page views: + #interested: + interested rate: + accuracy: - familiarity: - usefulness: - change: + freshness: + #page views: + #interested: + accuracy: - confusion: + change: + Serpentining #page views: + #interested: + accuracy: - familiarity: - usefulness: - see the paper Only significant results are shown. Italic: objective metrics; Non-italic: subjective metrics Staying users engage more. More browsing to get recommendations! L
  39. 39. 39 No Cycling Within-session Cycling Between-session Cycling No Serpentining control condition opt out rate: + #page views: + #interested: + interested rate: + accuracy: - familiarity: - usefulness: - change: + freshness: + #page views: + #interested: + accuracy: - confusion: + change: + Serpentining #page views: + #interested: + accuracy: - familiarity: - usefulness: - see the paper Only significant results are shown. Italic: objective metrics; Non-italic: subjective metrics Staying users engage more. Delighted to explore more interesting recommendations! J
  40. 40. Summary » Within-session cycling • Higher churning risk • Negative subjective perception, but positive freshness • Higher level of user activities and interested rate » Between-session cycling or serpentining • Higher level of user activities • Negative effects on subjective perception 40
  41. 41. Efficiency or engagement? 41
  42. 42. Efficiency or engagement? 42 An experience design perspective on recommenders: There is a tradeoff between serving come-and-go users vs. encouraging deeper interaction/engagement !
  43. 43. Messages » Better understanding of the trade-off between efficiency vs. engagement can help design a better recommender user experience! » Cycling and serpentining top-N recommendation lists have benefits (higher engagement) but also costs (negative perception)! » More work combining algorithms and user experience is needed! 43
  44. 44. Thanks! Questions? » Title: Toward Better Interactions in Recommender Systems: Cycling and Serpentining Approaches for Top-N Item Lists » Authors: Qian Zhao, Gediminas Adomavicius, F. Maxwell Harper, Martijn Willemsen, Joseph A. Konstan » Contact • zhaox331@umn.edu • http://www-users.cs.umn.edu/~qian/ 44
  45. 45. 45

×