Mixed-initiative recommender systems
•
2 likes•310 views
This document summarizes Katrien Verbert's talk on mixed-initiative recommender systems at the 12th RecSysNL meetup. It discusses how recommender systems can increase user trust and acceptance by explaining recommendations and enabling user interaction with the recommendation process. Examples of Verbert's research include systems like TasteWeights and IntersectionExplorer that provide transparency, user control, and support for exploration in recommender interfaces. Verbert's work also examines how personal characteristics affect user experience with different types and levels of recommender system controllability.
Recommended
More Related Content
Similar to Mixed-initiative recommender systems
Similar to Mixed-initiative recommender systems (20)
More from Katrien Verbert
More from Katrien Verbert (13)
Recently uploaded
Recently uploaded (20)
Mixed-initiative recommender systems
- 1. Mixed-initiative Recommender Systems 12th RecSysNL meetup @ Elsevier Amsterdam -18 Dec 2018 Katrien Verbert Augment/HCI - KU Leuven @katrien_v
- 2. Human-Computer Interaction group recommender systems – visualization – intelligent user interfaces Learning analytics Media consumption Research Information Systems Wellness & health Augment prof. Katrien Verbert ARIA prof. Adalberto Simeone Computer Graphics prof. Phil Dutré Language Intelligence & Information Retrieval prof. Sien Moens
- 3. Augment/HCI team Robin De Croon Postdoc researcher Katrien Verbert Associate Professor Francisco Gutiérrez PhD researcher Tom Broos PhD researcher Martijn Millecamp PhD researcher Sven Charleer Postdoc researcher Nyi Nyi Htun Postdoc researcher Houda Lamqaddam PhD researcher Yucheng Jin PhD researcher Oscar Alvarado PhD researcher http://augment.cs.kuleuven.be/ Diego Rojo Carcia PhD researcher
- 4. Interactive recommender systems Core objectives: • Explaining recommendations to increase user trust and acceptance • Enable users to interact with the recommendation process
- 7. Interactive recommender systems ¤ Transparency: explaining the rational of recommendations ¤ User control: closing the gap between browse and search ¤ Diversity – novelty ¤ Cold start ¤ Context-aware interfaces 7 He, C., Parra, D. and Verbert, K., 2016. Interactive recommender systems: A survey of the state of the art and future research challenges and opportunities. Expert Systems with Applications, 56, pp.9-27.
- 8. Flexible interaction with RecSys Research visit ¤ Host: Carnegie Mellon University & University of Pittsburg ¤ Collaboration: John Stamper, Peter Brusilovsky, Denis Parra ¤ Period: April 2012 – June 2012 Second post-doctoral fellowship FWO ¤ host university: KU Leuven, Belgium ¤ supervisor: Erik Duval ¤ period: Oct 2012 – Sept 2015 8
- 9. Overview research topics 9 2012 | 2013 | 2014 | 2015 | 2016 | 2017 | 2018 Learning Analytics - Media Consumption – Research Information Systems - Healthcare
- 11. Also recommendations è personalized relevance prospect 11
- 12. Contributions ¤ new approach to support exploration, transparency and controllability ¤ recommender systems are shown as agents ¤ in parallel to real users and tags ¤ users can interrelate entities to find items ¤ evaluation study that assesses ¤ effectiveness ¤ probability of item selection 12 Verbert, K., Parra, D., Brusilovsky, P., & Duval, E. (2013). Visualizing recommendations to support exploration, transparency and controllability. In Proceedings of the IUI 2013 international conference on Intelligent user interfaces (pp. 351-362). ACM.
- 13. TalkExplorer 13
- 14. Results of studies 1 & 2 ¤ Effectiveness: # bookmarked items / #explorations ¤ Effectiveness increases with intersections of more entities ¤ Effectiveness wasn’t affected in the field study (study 2) ¤ … but exploration distribution was affected 14 Average effectiveness Total number of explorations Verbert, K., Parra, D., & Brusilovsky, P. (2016). Agents vs. users: Visual recommendation of research talks with multiple dimension of relevance. ACM Transactions on Interactive Intelligent Systems (TIIS), 6(2), 11.
- 16. Three user studies ¤ Study 1: ¤ Within-subjects study with 20 users ¤ baseline: exploration of recommendations in CN3 ¤ Second condition: exploration of recommendations in IEx ¤ Data from two conferences EC-TEL 2014, EC-TEL 2015 ¤ Study 2: ¤ Field study at Digital Humanities conference ¤ + 1000 participants, less technically oriented ¤ Study 3: ¤ Field study at IUI conference ¤ Smaller scale, technical audience 16
- 17. Study 1 vs Study 2 vs Study 3 ¤ Overall combinations of users and agents (“augmented agents”) were used in all three studies ¤ Precision scores significantly higher for augmented agents in study 1 and study 3 ¤ Participants of study 2 (Digital Humanities) ¤ more interested in content perspective ¤ Rated several dimensions lower (use intention, fun, information sufficiency, control) 17 Cardoso, B., Sedrakyan, G., Gutiérrez, F., Parra, D., Brusilovsky, P., & Verbert, K. (2018). IntersectionExplorer, a multi-perspective approach for exploring recommendations. International Journal of Human-Computer Studies.
- 18. Overview research topics 18 2012 | 2013 | 2014 | 2015 | 2016 | 2017 | 2018 Learning analytics - Media Consumption – Research Information Systems - Healthcare
- 19. Explanations 19 http://bellows.experiments.cs.kuleuven.be:3002/ Millecamp, M., Htun, N.N., Conati, C. and Verbert, K. (2019)To Explain or not to Explain: the Effects of Personal Characteristics when Explaining Music Recommendations. Proceedings of IUI 2019 (to appear)
- 20. Personal characteristics Need for cognition •Measurement of the tendency for an individual to engage in, and enjoy, effortful cognitive activities •Measured by test of Cacioppo et al. [1984] Visualisation literacy •Measurement of the ability to interpret and make meaning from information presented in the form of images and graphs •Measured by test of Boy et al. [2014] Locus of control (LOC) •Measurement of the extent to which people believe they have power over events in their lives •Measured by test of Rotter et al. [1966] Visual working memory •Measurement of the ability to recall visual patterns [Tintarev and Mastoff, 2016] •Measured by Corsi block-tapping test Musical experience •Measurement of the ability to engage with music in a flexible, effective and nuanced way [Müllensiefen et al., 2014] •Measured using the Goldsmiths Musical Sophistication Index (Gold-MSI) Tech savviness •Measured by confidence in trying out new technology 20
- 21. User study ¤ Within-subjects design: 105 participants recruited with Amazon Mechanical Turk ¤ Baseline version (without explanations) compared with explanation interface ¤ Pre-study questionnaire for all personal characteristics ¤ Task: Based on a chosen scenario for creating a play-list, explore songs and rate all songs in the final playlist ¤ Post-study questionnaire: ¤ Recommender effectiveness ¤ Trust ¤ Good understanding ¤ Use intentions ¤ Novelty ¤ Satisfaction ¤ Confidence
- 22. Results 22 The interaction effect between NFC (divided into 4 quartiles Q1-Q4) and interfaces in terms of confidence
- 23. Design implications ¤ Explanations should be personalised for different groups of end-users. ¤ Users should be able to choose whether or not they want to see explanations. ¤ Explanation components should be flexible enough to present varying levels of details depending on a user’s preference. 23
- 24. User control Users tend to be more satisfied when they have control over how recommender systems produce suggestions for them (Konstan and Riedl, 2012) Control recommendations Douban FM Control user profile Spotify Control algorithm parameters TasteWeights
- 25. Controllability Cognitive load Additional controls may increase cognitive load (Andjelkovic et al. 2016; Jin et al. 2017)
- 26. Different levels of user control 26 Level Recommender components Controls low Recommendations (REC) Rating, removing, and sorting medium User profile (PRO) Select which user profile data will be considered by the recommender high Algorithm parameters (PAR) Modify the weight of different parameters Jin, Y., Tintarev, N., & Verbert, K. (2018, September). Effects of personal characteristics on music recommender systems with different levels of controllability. In Proceedings of the 12th ACM Conference on Recommender Systems (pp. 13-21). ACM.
- 27. User profile (PRO) Algorithm parameters (PAR) Recommendations (REC) 8 control settings No control REC PAR PRO REC*PRO REC*PAR PRO*PAR REC*PRO*PAR
- 28. Evaluation method ¤ Between-subjects – 240 participants recruited with AMT ¤ Independent variable: settings of user control ¤ 2x2x2 factorial design ¤ Dependent variables: ¤ Acceptance (ratings) ¤ Cognitive load (NASA-TLX), Musical Sophistication, Visual Memory ¤ Framework Knijnenburg et al. [2012]
- 29. Results ¤ Main effects: from REC to PRO to PAR → higher cognitive load ¤ Two-way interaction: does not necessarily result in higher cognitive load. Adding an additional control component to PAR increases the acceptance. PRO*PAR has less cognitive load than PRO and PAR ¤ High Musical Sofistication leads to higher quality, and thereby result in higher acceptance 29
- 30. Overview research topics 30 2012 | 2013 | 2014 | 2015 | 2016 | 2017 | 2018 Learning Analytics - Media Consumption – Research Information Systems - Healthcare
- 31. Recommender systems for food 31
- 32. Augmented reality 32 Gutiérrez, Francisco, Htun, Nyi Nyi, Charleer, Sven, De Croon, Robin, Verbert, Katrien (2019) Designing Augmented Reality Applications for Personal Health Decision-Making. Proceedings of HICSS-52. (to appear)
- 33. Job explorer 33
- 35. RECOMMENDER ALGORITHMS MACHINE LEARNING INTERACTIVE DASHBOARDS SMART ALERTS RICH CARE PLANS OPEN IoT ARCHITECTURE
- 36. Peter Brusliovsky Nava Tintarev Cristina ConatiDenis Parra Collaborations Bart Knijnenburg Jurgen Ziegler
- 37. Open PhD and Postdoc positions ¤ PhD/Postdoc position on Human-Computer Interaction: the focus of this research position is on user interfaces for recommender systems ¤ PhD position on Visual Analytics for Healthcare 37 http://augment.cs.kuleuven.be
- 39. References ¤ Boy, J., Rensink, R. A., Bertini, E., & Fekete, J. D. (2014). A principled way of assessing visualization literacy. IEEE transactions on visualization and computer graphics, 20(12), 1963-1972. ¤ Cacioppo, J.T., Petty, R.E. and Feng Kao, C., 1984. The efficient assessment of need for cognition. Journal of personality assessment, 48(3), pp.306-307. ¤ B. P. Knijnenburg, M. C. Willemsen, Z. Gantner, H. Soncu, and C. Newell. Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction, 22(4-5):441–504, 2012. ¤ Konstan, J.A. and Riedl, J., 2012. Recommender systems: from algorithms to user experience. User modeling and user-adapted interaction, 22(1-2), pp.101-123. ¤ Müllensiefen, D., Gingras, B., Musil, J., & Stewart, L. (2014). The musicality of non-musicians: an index for assessing musical sophistication in the general population. PloS one, 9(2), e89642. ¤ Rotter, J. B. (1966). Generalized expectancies for internal versus external control of reinforcement. Psychological monographs: General and applied, 80(1), 1. ¤ Tintarev, N., & Masthoff, J. (2016). Effects of Individual Differences in Working Memory on Plan Presentational Choices. Frontiers in psychology, 7, 1793.