Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Agents vs Users: Visual Recommendation of Research Talks with Multiple Dimension of Relevance

137 views

Published on

Published in ACM TiiS: Verbert, K., Parra, D., & Brusilovsky, P. (2016). Agents Vs. Users: Visual Recommendation of Research Talks with Multiple Dimension of Relevance. ACM Transactions on Interactive Intelligent Systems (TiiS), 6(2), 11.
Presented at IUI 2017

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Agents vs Users: Visual Recommendation of Research Talks with Multiple Dimension of Relevance

  1. 1. Agents vs Users: Visual Recommendation of Research Talks with Multiple Dimension of Relevance Katrien Verbert – KU Leuven – @katrien_v Denis Parra – PUC Chile - @denisparra Peter Brusilovsky – University of Pittsburgh - @peterpaws Presented at IUI 2017
  2. 2. Motivation • multiple relevance prospects in personalized social tagging systems o community relevance prospects o social relevance prospect o content relevance prospect • existing personalized social systems o do not allow to explore and combine multiple relevance prospects o only one prospect can be explored at a given time 2
  3. 3. Also recommendations  personalized relevance prospect 3
  4. 4. Shortcomings recommender systems • cold-start issues • difficult to explain the rationale behind recommendations • user control has positive effect o several ways to control elements o which are most effective for the user experience? 4
  5. 5. 5 He, C., Parra, D., & Verbert, K. (2016). Interactive recommender systems: A survey of the state of the art and future research challenges and opportunities. Expert Systems with Applications, 56, 9-27.
  6. 6. John O'Donovan, Barry Smyth, Brynjar Gretarsson, Svetlin Bostandjiev, and Tobias Höllerer. 2008. PeerChooser: visual interactive recommendation. CHI '08 Related work: PeerChooser 6
  7. 7. Related work: Smallworlds Gretarsson, B., O'Donovan, J., Bostandjiev, S., Hall, C. and Höllerer, T. SmallWorlds: Visualizing Social Recommendations. Comput. Graph. Forum, 29, 3 (2010), 833-842. 7
  8. 8. Related work: TasteWeights Bostandjiev,S.,O'Donovan,J.andHöllerer,T.TasteWeights:avisualinteractivehybrid recommendersystem.InProceedingsofthesixthACMconferenceonRecommender systems(RecSys'12).ACM,NewYork,NY,USA(2012),35-42. 8
  9. 9. Contributions • new approach to support exploration, transparency and controllability o recommender systems are shown as agents o in parallel to real users and tags o users can interrelate entities to find items • evaluation study that assesses o effectiveness o probability of item selection 9
  10. 10. Conference Navigator The studies were conducted using Conference Navigator 3 http://halley.exp.sis.pitt.edu/cn3/10
  11. 11. TalkExplorer - I 11 Entities Tags, Recommender Agents, Users
  12. 12. TalkExplorer - II Recommender Recommender Cluster with intersect ion of entities Cluster (of talks) associated to only one entity Canvas Area: Intersections of Different Entities User
  13. 13. TalkExplorer - III 13 Items Talks explored by the user
  14. 14. Our Assumptions • Items which are relevant in more that one aspect could be more valuable to the users • Displaying multiple aspects of relevance visually is important for the users in the process of item’s exploration 14
  15. 15. User study 1 • Setup o supervised user study o 21 participants at UMAP 2012 and ACM Hypertext 2012 conferences • Procedure o Tasks • interact with users and their bookmarks • interact with agents • interact with tags o Post-questionnaire 15
  16. 16. Evaluation • Data collection o recordings of voice and screen using camtasia studio o system logs • Measurements o effectiveness: # bookmarked items / #explorations o yield: : # bookmarked items / sum of items in selection set 16
  17. 17. Effectiveness results Summary explored actions and their effectiveness Effectiveness = # bookmarked items / #explorations 17
  18. 18. Summary results   Sign. effect. Sign. yield multiple versus one entity 0.003 <0.001 user versus (user + entity) 0.593 <0.001 agent versus (agent + entity) 0.341 <0.001
  19. 19. Post-questionnaire 19
  20. 20. Post-questionnaire 20
  21. 21. User study 2 • Setup o Unsupervised user study o Conducted at LAK 2012 and ECTEL 2013 (18 users) o Subjects familiar with visualizations, but not much with RecSys • Procedure o Users were left free to explore the interface. o Interactions were logged o Post-questionnaire 21
  22. 22. Summary results 22   Sign. effect. multiple versus one entity <0.001 user versus (user + entity) 0.3682 agent versus (agent + entity) 0.4426
  23. 23. Summary results   Sign. effectiveness ranked list versus agents 0.7008 ranked list versus (agent + entity) <0.001 ranked list versus multiple entities 0.0012 23
  24. 24. Results of Studies 1 & 2 • Effectiveness increases with intersections of more entities • Effectiveness wasn’t affected in the field study (study 2) • … but exploration distribution was affected Average effectiveness Total number of explorations 24
  25. 25. Drawback: visualizing intersections Clustermap Venn diagram Venn diagram: more natural way to visualize intersections 25 Verbert, K., Parra, D., Brusilovksy, P. (2014). The effect of different set-based visualizations on user exploration of recommendations. In : IntRS@RecSys, 2014 (pp. 37-44).
  26. 26. IntersectionExplorer (IEx) 26 Verbert, K., Seipp, K., He, C., Parra, D., Wongchokprasitti, C., Brusilovsky, P. (2016). Scalable exploration of relevance prospects to support decision making. In : IIntRS@RecSys. 2016 (pp. 28-35). CEUR-WS.
  27. 27. IntersectionExplorer (IEx) 27
  28. 28. Feedback very welcome! Please participate in a short user study: http://halley.exp.sis.pitt.edu/cn3/iestudy3.php?conferenceID= 148 28
  29. 29. Thank you! Katrien.verbert@cs.kuleuven.be peterb@pitt.edu denisparra@gmail.com 29

×