SlideShare a Scribd company logo
1 of 24
Download to read offline
See What you Want to See:
Visual User-Driven Approach
for Recommendation
Denis Parra, PUC Chile
Peter Brusilovsky, University of Pittsburgh
Christoph Trattner, Graz University of Technology
IUI 2014, Haifa, Israel
Outline
•  Short intro to some Challenges in Recommender
Systems
•  Our Approach to User Controllability (demo)
•  User Study & Results
•  Summary & Future Work
02/27/2014 D.Parra et al.~ IUI 2014 2
INTRODUCTION
Recommender Systems: Introduction & Challenges addressed in this research
3
* Danboard (Danbo): Amazon’s cardboard robot, in these slides represents a recommender system
*
Recommender Systems (RecSys)
Systems that help people to find relevant items in a
crowded item or information space (McNee et al. 2006)
02/27/2014 D.Parra et al.~ IUI 2014 4
Challenges of RecSys Addressed Here
Traditionally, RecSys has focused on producing
accurate recommendation algorithms. In this research,
these challenges are addressed:
1.  Human Factors in RecSys: Study controllability by
introducing a novel visualization that presents
fusion of different recommenders
2.  Evaluation: Use of Objective,
Subjective & Behavioral metrics
02/27/2014 D.Parra et al.~ IUI 2014 5
Research Goals & User Studies
Research Goal
•  To understand the effect of controllability on the
user engagement and on the overall user
experience of a RecSys
(on this paper) Through
•  Two studies conducted using Conference Navigator:
02/27/2014 D.Parra et al.~ IUI 2014 6
Program Proceedings Author List Recommendations
http://halley.exp.sis.pitt.edu/cn3/
WHYIUISHOULDCARE:HCI+RECSYS
COMMUNITY
Previous research related to this work / Motivating results from TalkExplorer study
7/22/2013 D.Parra ~ PhD. Dissertation Defense 7
TasteWeights (Bostandjev et ala 2012)
7/22/2013 D.Parra ~ PhD. Dissertation Defense 8
Preliminary Work: TalkExplorer
•  Adaptation of Aduna Visualization in CN
•  Main research question: Do fusion (intersection) of
contexts of relevance improve user experience?
7/22/2013 D.Parra ~ PhD. Dissertation Defense 9
Center user
CN user
RecommenderRecommender
Cluster
with
intersect
ion of
entities
Cluster (of
talks)
associated
to only one
entity
SETFUSION:USER-CONTROLLABLE
HYBRIDINTERFACE
10
Our Proposed Interface: SetFusion
02/27/2014 D.Parra et al.~ IUI 2014 11
Our Proposed Interface - II
02/27/2014 D.Parra et al.~ IUI 2014 12
Traditional
Ranked List
Papers sorted by
Relevance.
It combines 3
recommendation
approaches.
Our Proposed Interface - III
02/27/2014 D.Parra et al.~ IUI 2014 13
Sliders
Allow the user to control the importance of
each data source or recommendation method
Interactive Venn Diagram
Allows the user to inspect and to filter papers
recommended. Actions available:
-  Filter item list by clicking on an area
-  Highlight a paper by mouse-over on a circle
-  Scroll to paper by clicking on a circle
-  Indicate bookmarked papers
Mixed Hybridization: Item Score
7/22/2013 D.Parra ~ PhD. Dissertation Defense 14
M: The set of all methods available to fuse
rankreci,mj : rank–position in the list of a recommended item
reci : recommended item i
mj, : recommendation method j
Wmj : weight given by the user to the method mj using the controllable
interface
|Mreci| represents the number of methods by which item reci was
recommended
Slider weight
RESEARCH:DETAILS&RESULTS
Description and Analysis of the results of the 3 user studies
Studies: CSCW 2013 & UMAP 2013
02/27/2014 D.Parra et al.~ IUI 2014 16
CSCW 2013
Conditions Static
List
Interactive
SetFusion
# Attendants ~400
# RecSys
Users
15 22
Study type Between Subjects
UMAP 2013
Interactive
SetFusion
~ 100
50
1 group
Preliminary User study: Here we learned that
the Interactive interface had a positive effect
on user behavior and perception of the recsys
Second study:
Only interactive
interface
CHANGES:
1.  Preference Elicitation:
In CSCW we avoided
cold start. In UMAP
we had no constraints
2.  Use of the ratings to
update the
recommended items
3.  Tuning of Content-
based recommender
Comparing CSCW and UMAP
02/27/2014 D.Parra et al.~ IUI 2014 17
(Only Interactive Interfaces) CSCW 2013 UMAP 2013
# Users exposed to recommendation 84 95
# Users who used the recommender 22 ( ~ 26 %) 50 ( ~52.6 %)
# Users bookmarked papers 6 ( ~ 27.2 %) 14 (~28 %)
# Talks bookmarked / user avg. 28 / 4.67 103 / 7.36
Average User rating 3.73 / 10 ( ~45.4 %) 3.62 / 8 (~16%)
Usage at Recommender Page
# Talks explored (user avg.) 16.84 14.9
# People returning 7 (~31.8%) 14 (28%)
Average time spent in page (seconds) 261.72 353.8
Comparing CSCW and UMAP
02/27/2014 D.Parra et al.~ IUI 2014 18
(Only Interactive Interfaces) CSCW 2013 UMAP 2013
# Users exposed to recommendation 84 95
# Users who used the recommender 22 50
# Users bookmarked papers 6 ( ~ 27.2 %) 14 (28 %)
# Talks bookmarked / user avg. 28 / 4.67 103 / 7.36
Average User rating 3.73 / 10 ( ~45.4 %) 3.62 / 8 (~16%)
Usage at Recommender Page
# Talks explored (user avg.) 16.84 14.9
# People returning 7 (~31.8%) 14 (28%)
Average time spent in page (seconds) 261.72 353.8
Comparing CSCW and UMAP
02/27/2014 D.Parra et al.~ IUI 2014 19
(Only Interactive Interfaces) CSCW 2013 UMAP 2013
# Users exposed to recommendation 84 95
# Users who used the recommender 22 50
# Users bookmarked papers 6 ( ~ 27.2 %) 14 (~28 %)
# Talks bookmarked / user avg. 28 / 4.67 103 / 7.36
Average User rating 3.73 / 10 ( ~45.4 %) 3.62 / 8 (~16%)
Usage at Recommender Page
# Talks explored (user avg.) 16.84 14.9
# People returning 7 (~31.8%) 14 (28%)
Average time spent in page (seconds) 261.72 353.8
From the Final Survey
CSCW 2013
(11 users)
UMAP 2013
(8 users)
I don’t think that Conference
Navigator needs a
Recommender System
M = 2.36, S.E. =
0.2
M = 1.5 , S.E. = 0.21
(p < 0.05)
I would recommend this
system to my colleagues
M = 3.36, S.E. =
0.28
M = 4.25, S.E. = 0.33
(p < 0.05)
02/27/2014 D.Parra et al.~ IUI 2014 20
- Users perceived SetFusion significantly as a more
useful tool in UMAP than in CSCW
CONCLUSIONS&FUTUREWORK
Summary of Results
•  From Study 1 we showed that User Controllability
had an effect on the user experience with RecSys.
•  Comparing SetFusion in Study 1 and Study 2:
– A natural elicitation setting (UMAP) allowed users to
be more engaged on using the system for the task of the
interface: bookmark papers recommended.
– Users also perceived the system as more useful in
UMAP 2013.
– Ratings are a form of giving user control, a big lesson
from Study 1: if you ask user for feedback, use it!
02/27/2014 D.Parra et al.~ IUI 2014 22
Limitations & Future Work
•  Apply our approach to other domains (fusion of
data sources or recommendation algorithms)
•  Find alternatives to scale the approach to more than
3 sets, potential alternatives:
– Clustering and
– Radial sets
•  Consider other factors that might interact with the
user experience:
– Controllability by itself vs. minimum level of accuracy
02/27/2014 D.Parra et al.~ IUI 2014 23
THANKS!
QUESTIONS?DPARRA@ING.PUC.CL

More Related Content

What's hot

Answering Twitter Questions: a Model for Recommending Answerers through Socia...
Answering Twitter Questions: a Model for Recommending Answerers through Socia...Answering Twitter Questions: a Model for Recommending Answerers through Socia...
Answering Twitter Questions: a Model for Recommending Answerers through Socia...UPMC - Sorbonne Universities
 
Flipped Classroom and Scorm - 2014 Brightspace Wisconsin Ignite
Flipped Classroom and Scorm - 2014 Brightspace Wisconsin Ignite Flipped Classroom and Scorm - 2014 Brightspace Wisconsin Ignite
Flipped Classroom and Scorm - 2014 Brightspace Wisconsin Ignite D2L Barry
 
Blending synchronous asynchronous
Blending synchronous asynchronousBlending synchronous asynchronous
Blending synchronous asynchronousLisa Yamagata-Lynch
 
Recommender Systems in TEL
Recommender Systems in TELRecommender Systems in TEL
Recommender Systems in TELtelss09
 
Comparative and non comparative studies
Comparative and non comparative studiesComparative and non comparative studies
Comparative and non comparative studiesu069072
 
Collaborative Information Retrieval: Frameworks, Theoretical Models and Emerg...
Collaborative Information Retrieval: Frameworks, Theoretical Models and Emerg...Collaborative Information Retrieval: Frameworks, Theoretical Models and Emerg...
Collaborative Information Retrieval: Frameworks, Theoretical Models and Emerg...UPMC - Sorbonne Universities
 
Investigating Crowdsourcing as an Evaluation Method for (TEL) Recommender Sy...
Investigating Crowdsourcing as an Evaluation Method for (TEL) Recommender Sy...Investigating Crowdsourcing as an Evaluation Method for (TEL) Recommender Sy...
Investigating Crowdsourcing as an Evaluation Method for (TEL) Recommender Sy...Christoph Rensing
 
Universities and their web-based library services : a study of their relati...
Universities and their  web-based library services :  a study of their relati...Universities and their  web-based library services :  a study of their relati...
Universities and their web-based library services : a study of their relati...Sangeeta Dhamdhere
 
A Data-driven Method for the Detection of Close Submitters in Online Learning...
A Data-driven Method for the Detection of Close Submitters in Online Learning...A Data-driven Method for the Detection of Close Submitters in Online Learning...
A Data-driven Method for the Detection of Close Submitters in Online Learning...MIT
 
The User Side of Personalization: How Personalization Affects the Users
The User Side of Personalization: How Personalization Affects the UsersThe User Side of Personalization: How Personalization Affects the Users
The User Side of Personalization: How Personalization Affects the UsersPeter Brusilovsky
 
Collaborative Information Retrieval: Concepts, Models and Evaluation
Collaborative Information Retrieval: Concepts, Models and EvaluationCollaborative Information Retrieval: Concepts, Models and Evaluation
Collaborative Information Retrieval: Concepts, Models and EvaluationUPMC - Sorbonne Universities
 
Learning analytics exemplar template
Learning analytics exemplar templateLearning analytics exemplar template
Learning analytics exemplar templateSimon Buckingham Shum
 
Survey on Study Partners Recommendation for Online Courses
Survey on Study Partners Recommendation for Online CoursesSurvey on Study Partners Recommendation for Online Courses
Survey on Study Partners Recommendation for Online CoursesIRJET Journal
 
Opening the Black Box of User Profiles in Content-based Recommender Systems
Opening the Black Box of User Profiles in Content-based Recommender SystemsOpening the Black Box of User Profiles in Content-based Recommender Systems
Opening the Black Box of User Profiles in Content-based Recommender SystemsDavid Graus
 

What's hot (14)

Answering Twitter Questions: a Model for Recommending Answerers through Socia...
Answering Twitter Questions: a Model for Recommending Answerers through Socia...Answering Twitter Questions: a Model for Recommending Answerers through Socia...
Answering Twitter Questions: a Model for Recommending Answerers through Socia...
 
Flipped Classroom and Scorm - 2014 Brightspace Wisconsin Ignite
Flipped Classroom and Scorm - 2014 Brightspace Wisconsin Ignite Flipped Classroom and Scorm - 2014 Brightspace Wisconsin Ignite
Flipped Classroom and Scorm - 2014 Brightspace Wisconsin Ignite
 
Blending synchronous asynchronous
Blending synchronous asynchronousBlending synchronous asynchronous
Blending synchronous asynchronous
 
Recommender Systems in TEL
Recommender Systems in TELRecommender Systems in TEL
Recommender Systems in TEL
 
Comparative and non comparative studies
Comparative and non comparative studiesComparative and non comparative studies
Comparative and non comparative studies
 
Collaborative Information Retrieval: Frameworks, Theoretical Models and Emerg...
Collaborative Information Retrieval: Frameworks, Theoretical Models and Emerg...Collaborative Information Retrieval: Frameworks, Theoretical Models and Emerg...
Collaborative Information Retrieval: Frameworks, Theoretical Models and Emerg...
 
Investigating Crowdsourcing as an Evaluation Method for (TEL) Recommender Sy...
Investigating Crowdsourcing as an Evaluation Method for (TEL) Recommender Sy...Investigating Crowdsourcing as an Evaluation Method for (TEL) Recommender Sy...
Investigating Crowdsourcing as an Evaluation Method for (TEL) Recommender Sy...
 
Universities and their web-based library services : a study of their relati...
Universities and their  web-based library services :  a study of their relati...Universities and their  web-based library services :  a study of their relati...
Universities and their web-based library services : a study of their relati...
 
A Data-driven Method for the Detection of Close Submitters in Online Learning...
A Data-driven Method for the Detection of Close Submitters in Online Learning...A Data-driven Method for the Detection of Close Submitters in Online Learning...
A Data-driven Method for the Detection of Close Submitters in Online Learning...
 
The User Side of Personalization: How Personalization Affects the Users
The User Side of Personalization: How Personalization Affects the UsersThe User Side of Personalization: How Personalization Affects the Users
The User Side of Personalization: How Personalization Affects the Users
 
Collaborative Information Retrieval: Concepts, Models and Evaluation
Collaborative Information Retrieval: Concepts, Models and EvaluationCollaborative Information Retrieval: Concepts, Models and Evaluation
Collaborative Information Retrieval: Concepts, Models and Evaluation
 
Learning analytics exemplar template
Learning analytics exemplar templateLearning analytics exemplar template
Learning analytics exemplar template
 
Survey on Study Partners Recommendation for Online Courses
Survey on Study Partners Recommendation for Online CoursesSurvey on Study Partners Recommendation for Online Courses
Survey on Study Partners Recommendation for Online Courses
 
Opening the Black Box of User Profiles in Content-based Recommender Systems
Opening the Black Box of User Profiles in Content-based Recommender SystemsOpening the Black Box of User Profiles in Content-based Recommender Systems
Opening the Black Box of User Profiles in Content-based Recommender Systems
 

Viewers also liked

A Hybrid Peer Recommender System for a Online Community Teachers
A Hybrid Peer Recommender System for a Online Community TeachersA Hybrid Peer Recommender System for a Online Community Teachers
A Hybrid Peer Recommender System for a Online Community TeachersDenis Parra Santander
 
Evaluation of Collaborative Filtering Algorithms for Recommending Articles on...
Evaluation of Collaborative Filtering Algorithms for Recommending Articles on...Evaluation of Collaborative Filtering Algorithms for Recommending Articles on...
Evaluation of Collaborative Filtering Algorithms for Recommending Articles on...Denis Parra Santander
 
Currents steps to be a researcher and faculty
Currents steps to be a researcher and facultyCurrents steps to be a researcher and faculty
Currents steps to be a researcher and facultyDenis Parra Santander
 
Identifying Relevant Messages in a Twitter-based Citizen Channel for Natural ...
Identifying Relevant Messages in a Twitter-based Citizen Channel for Natural ...Identifying Relevant Messages in a Twitter-based Citizen Channel for Natural ...
Identifying Relevant Messages in a Twitter-based Citizen Channel for Natural ...Denis Parra Santander
 
Walk the Talk: Analyzing the relation between implicit and explicit feedback ...
Walk the Talk: Analyzing the relation between implicit and explicit feedback ...Walk the Talk: Analyzing the relation between implicit and explicit feedback ...
Walk the Talk: Analyzing the relation between implicit and explicit feedback ...Denis Parra Santander
 
Implicit Feedback Recommendation via Implicit-to-Explicit Ordinal Logistic Re...
Implicit Feedback Recommendation via Implicit-to-Explicit Ordinal Logistic Re...Implicit Feedback Recommendation via Implicit-to-Explicit Ordinal Logistic Re...
Implicit Feedback Recommendation via Implicit-to-Explicit Ordinal Logistic Re...Denis Parra Santander
 
Network Visualization guest lecture at #DataVizQMSS at @Columbia / #SNA at PU...
Network Visualization guest lecture at #DataVizQMSS at @Columbia / #SNA at PU...Network Visualization guest lecture at #DataVizQMSS at @Columbia / #SNA at PU...
Network Visualization guest lecture at #DataVizQMSS at @Columbia / #SNA at PU...Denis Parra Santander
 

Viewers also liked (9)

LDA on social bookmarking systems
LDA on social bookmarking systemsLDA on social bookmarking systems
LDA on social bookmarking systems
 
Twitter in Academic Conferences
Twitter in Academic ConferencesTwitter in Academic Conferences
Twitter in Academic Conferences
 
A Hybrid Peer Recommender System for a Online Community Teachers
A Hybrid Peer Recommender System for a Online Community TeachersA Hybrid Peer Recommender System for a Online Community Teachers
A Hybrid Peer Recommender System for a Online Community Teachers
 
Evaluation of Collaborative Filtering Algorithms for Recommending Articles on...
Evaluation of Collaborative Filtering Algorithms for Recommending Articles on...Evaluation of Collaborative Filtering Algorithms for Recommending Articles on...
Evaluation of Collaborative Filtering Algorithms for Recommending Articles on...
 
Currents steps to be a researcher and faculty
Currents steps to be a researcher and facultyCurrents steps to be a researcher and faculty
Currents steps to be a researcher and faculty
 
Identifying Relevant Messages in a Twitter-based Citizen Channel for Natural ...
Identifying Relevant Messages in a Twitter-based Citizen Channel for Natural ...Identifying Relevant Messages in a Twitter-based Citizen Channel for Natural ...
Identifying Relevant Messages in a Twitter-based Citizen Channel for Natural ...
 
Walk the Talk: Analyzing the relation between implicit and explicit feedback ...
Walk the Talk: Analyzing the relation between implicit and explicit feedback ...Walk the Talk: Analyzing the relation between implicit and explicit feedback ...
Walk the Talk: Analyzing the relation between implicit and explicit feedback ...
 
Implicit Feedback Recommendation via Implicit-to-Explicit Ordinal Logistic Re...
Implicit Feedback Recommendation via Implicit-to-Explicit Ordinal Logistic Re...Implicit Feedback Recommendation via Implicit-to-Explicit Ordinal Logistic Re...
Implicit Feedback Recommendation via Implicit-to-Explicit Ordinal Logistic Re...
 
Network Visualization guest lecture at #DataVizQMSS at @Columbia / #SNA at PU...
Network Visualization guest lecture at #DataVizQMSS at @Columbia / #SNA at PU...Network Visualization guest lecture at #DataVizQMSS at @Columbia / #SNA at PU...
Network Visualization guest lecture at #DataVizQMSS at @Columbia / #SNA at PU...
 

Similar to SetFusion Visual Hybrid Recommender - IUI 2014

Scalable Exploration of Relevance Prospects to Support Decision Making
Scalable Exploration of Relevance Prospects to Support Decision MakingScalable Exploration of Relevance Prospects to Support Decision Making
Scalable Exploration of Relevance Prospects to Support Decision MakingKatrien Verbert
 
Interactive Recommender Systems
Interactive Recommender SystemsInteractive Recommender Systems
Interactive Recommender SystemsKatrien Verbert
 
Mixed-initiative recommender systems: towards a next generation of recommende...
Mixed-initiative recommender systems: towards a next generation of recommende...Mixed-initiative recommender systems: towards a next generation of recommende...
Mixed-initiative recommender systems: towards a next generation of recommende...Katrien Verbert
 
ICT Intervention for Empowerment of Maternal Healthcare in Assam
ICT Intervention for Empowerment of Maternal Healthcare in AssamICT Intervention for Empowerment of Maternal Healthcare in Assam
ICT Intervention for Empowerment of Maternal Healthcare in AssamMannu Amrit
 
Language Models for Collaborative Filtering Neighbourhoods [ECIR '16 Slides]
Language Models for Collaborative Filtering Neighbourhoods [ECIR '16 Slides]Language Models for Collaborative Filtering Neighbourhoods [ECIR '16 Slides]
Language Models for Collaborative Filtering Neighbourhoods [ECIR '16 Slides]Daniel Valcarce
 
Preliminary PhD Defence - Student-facing Dashboards
Preliminary PhD Defence - Student-facing DashboardsPreliminary PhD Defence - Student-facing Dashboards
Preliminary PhD Defence - Student-facing DashboardsSven Charleer
 
Classification of Researcher's Collaboration Patterns Towards Research Perfor...
Classification of Researcher's Collaboration Patterns Towards Research Perfor...Classification of Researcher's Collaboration Patterns Towards Research Perfor...
Classification of Researcher's Collaboration Patterns Towards Research Perfor...Nur Hazimah Khalid
 
Masters Thesis - Smart Cafeteria
Masters Thesis - Smart CafeteriaMasters Thesis - Smart Cafeteria
Masters Thesis - Smart CafeteriaRichard Philip
 
Explaining recommendations: design implications and lessons learned
Explaining recommendations: design implications and lessons learnedExplaining recommendations: design implications and lessons learned
Explaining recommendations: design implications and lessons learnedKatrien Verbert
 
Data analytics to support awareness and recommendation
Data analytics to support awareness and recommendationData analytics to support awareness and recommendation
Data analytics to support awareness and recommendationKatrien Verbert
 
201 - Using Qualitative Metasummary to Synthesize Empirical Findings in Liter...
201 - Using Qualitative Metasummary to Synthesize Empirical Findings in Liter...201 - Using Qualitative Metasummary to Synthesize Empirical Findings in Liter...
201 - Using Qualitative Metasummary to Synthesize Empirical Findings in Liter...ESEM 2014
 
Towards the next generation of interactive and adaptive explanation methods
Towards the next generation of interactive and adaptive explanation methodsTowards the next generation of interactive and adaptive explanation methods
Towards the next generation of interactive and adaptive explanation methodsKatrien Verbert
 
Assessing Perceived Usability of the Data Curation Profiles Toolkit Using th...
Assessing Perceived Usability of the Data Curation Profiles Toolkit  Using th...Assessing Perceived Usability of the Data Curation Profiles Toolkit  Using th...
Assessing Perceived Usability of the Data Curation Profiles Toolkit Using th...Tao Zhang
 
using-qualitative-metasummary-to-synthesize-empirical-findings-in-literature-...
using-qualitative-metasummary-to-synthesize-empirical-findings-in-literature-...using-qualitative-metasummary-to-synthesize-empirical-findings-in-literature-...
using-qualitative-metasummary-to-synthesize-empirical-findings-in-literature-...Danilo Monteiro
 
Welcome to International Journal of Engineering Research and Development (IJERD)
Welcome to International Journal of Engineering Research and Development (IJERD)Welcome to International Journal of Engineering Research and Development (IJERD)
Welcome to International Journal of Engineering Research and Development (IJERD)IJERD Editor
 
Mixed-initiative recommender systems
Mixed-initiative recommender systemsMixed-initiative recommender systems
Mixed-initiative recommender systemsKatrien Verbert
 

Similar to SetFusion Visual Hybrid Recommender - IUI 2014 (20)

Scalable Exploration of Relevance Prospects to Support Decision Making
Scalable Exploration of Relevance Prospects to Support Decision MakingScalable Exploration of Relevance Prospects to Support Decision Making
Scalable Exploration of Relevance Prospects to Support Decision Making
 
Crowdsourcing Software Evaluation
Crowdsourcing Software EvaluationCrowdsourcing Software Evaluation
Crowdsourcing Software Evaluation
 
Interactive Recommender Systems
Interactive Recommender SystemsInteractive Recommender Systems
Interactive Recommender Systems
 
Mixed-initiative recommender systems: towards a next generation of recommende...
Mixed-initiative recommender systems: towards a next generation of recommende...Mixed-initiative recommender systems: towards a next generation of recommende...
Mixed-initiative recommender systems: towards a next generation of recommende...
 
ICT Intervention for Empowerment of Maternal Healthcare in Assam
ICT Intervention for Empowerment of Maternal Healthcare in AssamICT Intervention for Empowerment of Maternal Healthcare in Assam
ICT Intervention for Empowerment of Maternal Healthcare in Assam
 
Software evaluation via users’ feedback at runtime
Software evaluation via users’ feedback at runtimeSoftware evaluation via users’ feedback at runtime
Software evaluation via users’ feedback at runtime
 
lms final ppt.pptx
lms final ppt.pptxlms final ppt.pptx
lms final ppt.pptx
 
Language Models for Collaborative Filtering Neighbourhoods [ECIR '16 Slides]
Language Models for Collaborative Filtering Neighbourhoods [ECIR '16 Slides]Language Models for Collaborative Filtering Neighbourhoods [ECIR '16 Slides]
Language Models for Collaborative Filtering Neighbourhoods [ECIR '16 Slides]
 
Preliminary PhD Defence - Student-facing Dashboards
Preliminary PhD Defence - Student-facing DashboardsPreliminary PhD Defence - Student-facing Dashboards
Preliminary PhD Defence - Student-facing Dashboards
 
Classification of Researcher's Collaboration Patterns Towards Research Perfor...
Classification of Researcher's Collaboration Patterns Towards Research Perfor...Classification of Researcher's Collaboration Patterns Towards Research Perfor...
Classification of Researcher's Collaboration Patterns Towards Research Perfor...
 
Masters Thesis - Smart Cafeteria
Masters Thesis - Smart CafeteriaMasters Thesis - Smart Cafeteria
Masters Thesis - Smart Cafeteria
 
Explaining recommendations: design implications and lessons learned
Explaining recommendations: design implications and lessons learnedExplaining recommendations: design implications and lessons learned
Explaining recommendations: design implications and lessons learned
 
Data analytics to support awareness and recommendation
Data analytics to support awareness and recommendationData analytics to support awareness and recommendation
Data analytics to support awareness and recommendation
 
201 - Using Qualitative Metasummary to Synthesize Empirical Findings in Liter...
201 - Using Qualitative Metasummary to Synthesize Empirical Findings in Liter...201 - Using Qualitative Metasummary to Synthesize Empirical Findings in Liter...
201 - Using Qualitative Metasummary to Synthesize Empirical Findings in Liter...
 
Towards the next generation of interactive and adaptive explanation methods
Towards the next generation of interactive and adaptive explanation methodsTowards the next generation of interactive and adaptive explanation methods
Towards the next generation of interactive and adaptive explanation methods
 
Assessing Perceived Usability of the Data Curation Profiles Toolkit Using th...
Assessing Perceived Usability of the Data Curation Profiles Toolkit  Using th...Assessing Perceived Usability of the Data Curation Profiles Toolkit  Using th...
Assessing Perceived Usability of the Data Curation Profiles Toolkit Using th...
 
Us
UsUs
Us
 
using-qualitative-metasummary-to-synthesize-empirical-findings-in-literature-...
using-qualitative-metasummary-to-synthesize-empirical-findings-in-literature-...using-qualitative-metasummary-to-synthesize-empirical-findings-in-literature-...
using-qualitative-metasummary-to-synthesize-empirical-findings-in-literature-...
 
Welcome to International Journal of Engineering Research and Development (IJERD)
Welcome to International Journal of Engineering Research and Development (IJERD)Welcome to International Journal of Engineering Research and Development (IJERD)
Welcome to International Journal of Engineering Research and Development (IJERD)
 
Mixed-initiative recommender systems
Mixed-initiative recommender systemsMixed-initiative recommender systems
Mixed-initiative recommender systems
 

Recently uploaded

slides CapTechTalks Webinar May 2024 Alexander Perry.pptx
slides CapTechTalks Webinar May 2024 Alexander Perry.pptxslides CapTechTalks Webinar May 2024 Alexander Perry.pptx
slides CapTechTalks Webinar May 2024 Alexander Perry.pptxCapitolTechU
 
REPRODUCTIVE TOXICITY STUDIE OF MALE AND FEMALEpptx
REPRODUCTIVE TOXICITY  STUDIE OF MALE AND FEMALEpptxREPRODUCTIVE TOXICITY  STUDIE OF MALE AND FEMALEpptx
REPRODUCTIVE TOXICITY STUDIE OF MALE AND FEMALEpptxmanishaJyala2
 
Dementia (Alzheimer & vasular dementia).
Dementia (Alzheimer & vasular dementia).Dementia (Alzheimer & vasular dementia).
Dementia (Alzheimer & vasular dementia).Mohamed Rizk Khodair
 
ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...
ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...
ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...Nguyen Thanh Tu Collection
 
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT VẬT LÝ 2024 - TỪ CÁC TRƯỜNG, TRƯ...
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT VẬT LÝ 2024 - TỪ CÁC TRƯỜNG, TRƯ...TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT VẬT LÝ 2024 - TỪ CÁC TRƯỜNG, TRƯ...
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT VẬT LÝ 2024 - TỪ CÁC TRƯỜNG, TRƯ...Nguyen Thanh Tu Collection
 
Post Exam Fun(da) Intra UEM General Quiz 2024 - Prelims q&a.pdf
Post Exam Fun(da) Intra UEM General Quiz 2024 - Prelims q&a.pdfPost Exam Fun(da) Intra UEM General Quiz 2024 - Prelims q&a.pdf
Post Exam Fun(da) Intra UEM General Quiz 2024 - Prelims q&a.pdfPragya - UEM Kolkata Quiz Club
 
Essential Safety precautions during monsoon season
Essential Safety precautions during monsoon seasonEssential Safety precautions during monsoon season
Essential Safety precautions during monsoon seasonMayur Khatri
 
Basic Civil Engineering notes on Transportation Engineering, Modes of Transpo...
Basic Civil Engineering notes on Transportation Engineering, Modes of Transpo...Basic Civil Engineering notes on Transportation Engineering, Modes of Transpo...
Basic Civil Engineering notes on Transportation Engineering, Modes of Transpo...Denish Jangid
 
The basics of sentences session 4pptx.pptx
The basics of sentences session 4pptx.pptxThe basics of sentences session 4pptx.pptx
The basics of sentences session 4pptx.pptxheathfieldcps1
 
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽中 央社
 
MichaelStarkes_UncutGemsProjectSummary.pdf
MichaelStarkes_UncutGemsProjectSummary.pdfMichaelStarkes_UncutGemsProjectSummary.pdf
MichaelStarkes_UncutGemsProjectSummary.pdfmstarkes24
 
An overview of the various scriptures in Hinduism
An overview of the various scriptures in HinduismAn overview of the various scriptures in Hinduism
An overview of the various scriptures in HinduismDabee Kamal
 
Features of Video Calls in the Discuss Module in Odoo 17
Features of Video Calls in the Discuss Module in Odoo 17Features of Video Calls in the Discuss Module in Odoo 17
Features of Video Calls in the Discuss Module in Odoo 17Celine George
 
Envelope of Discrepancy in Orthodontics: Enhancing Precision in Treatment
 Envelope of Discrepancy in Orthodontics: Enhancing Precision in Treatment Envelope of Discrepancy in Orthodontics: Enhancing Precision in Treatment
Envelope of Discrepancy in Orthodontics: Enhancing Precision in Treatmentsaipooja36
 
Navigating the Misinformation Minefield: The Role of Higher Education in the ...
Navigating the Misinformation Minefield: The Role of Higher Education in the ...Navigating the Misinformation Minefield: The Role of Higher Education in the ...
Navigating the Misinformation Minefield: The Role of Higher Education in the ...Mark Carrigan
 
2024_Student Session 2_ Set Plan Preparation.pptx
2024_Student Session 2_ Set Plan Preparation.pptx2024_Student Session 2_ Set Plan Preparation.pptx
2024_Student Session 2_ Set Plan Preparation.pptxmansk2
 
Open Educational Resources Primer PowerPoint
Open Educational Resources Primer PowerPointOpen Educational Resources Primer PowerPoint
Open Educational Resources Primer PowerPointELaRue0
 
Danh sách HSG Bộ môn cấp trường - Cấp THPT.pdf
Danh sách HSG Bộ môn cấp trường - Cấp THPT.pdfDanh sách HSG Bộ môn cấp trường - Cấp THPT.pdf
Danh sách HSG Bộ môn cấp trường - Cấp THPT.pdfQucHHunhnh
 
factors influencing drug absorption-final-2.pptx
factors influencing drug absorption-final-2.pptxfactors influencing drug absorption-final-2.pptx
factors influencing drug absorption-final-2.pptxSanjay Shekar
 

Recently uploaded (20)

slides CapTechTalks Webinar May 2024 Alexander Perry.pptx
slides CapTechTalks Webinar May 2024 Alexander Perry.pptxslides CapTechTalks Webinar May 2024 Alexander Perry.pptx
slides CapTechTalks Webinar May 2024 Alexander Perry.pptx
 
REPRODUCTIVE TOXICITY STUDIE OF MALE AND FEMALEpptx
REPRODUCTIVE TOXICITY  STUDIE OF MALE AND FEMALEpptxREPRODUCTIVE TOXICITY  STUDIE OF MALE AND FEMALEpptx
REPRODUCTIVE TOXICITY STUDIE OF MALE AND FEMALEpptx
 
Post Exam Fun(da) Intra UEM General Quiz - Finals.pdf
Post Exam Fun(da) Intra UEM General Quiz - Finals.pdfPost Exam Fun(da) Intra UEM General Quiz - Finals.pdf
Post Exam Fun(da) Intra UEM General Quiz - Finals.pdf
 
Dementia (Alzheimer & vasular dementia).
Dementia (Alzheimer & vasular dementia).Dementia (Alzheimer & vasular dementia).
Dementia (Alzheimer & vasular dementia).
 
ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...
ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...
ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...
 
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT VẬT LÝ 2024 - TỪ CÁC TRƯỜNG, TRƯ...
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT VẬT LÝ 2024 - TỪ CÁC TRƯỜNG, TRƯ...TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT VẬT LÝ 2024 - TỪ CÁC TRƯỜNG, TRƯ...
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT VẬT LÝ 2024 - TỪ CÁC TRƯỜNG, TRƯ...
 
Post Exam Fun(da) Intra UEM General Quiz 2024 - Prelims q&a.pdf
Post Exam Fun(da) Intra UEM General Quiz 2024 - Prelims q&a.pdfPost Exam Fun(da) Intra UEM General Quiz 2024 - Prelims q&a.pdf
Post Exam Fun(da) Intra UEM General Quiz 2024 - Prelims q&a.pdf
 
Essential Safety precautions during monsoon season
Essential Safety precautions during monsoon seasonEssential Safety precautions during monsoon season
Essential Safety precautions during monsoon season
 
Basic Civil Engineering notes on Transportation Engineering, Modes of Transpo...
Basic Civil Engineering notes on Transportation Engineering, Modes of Transpo...Basic Civil Engineering notes on Transportation Engineering, Modes of Transpo...
Basic Civil Engineering notes on Transportation Engineering, Modes of Transpo...
 
The basics of sentences session 4pptx.pptx
The basics of sentences session 4pptx.pptxThe basics of sentences session 4pptx.pptx
The basics of sentences session 4pptx.pptx
 
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
 
MichaelStarkes_UncutGemsProjectSummary.pdf
MichaelStarkes_UncutGemsProjectSummary.pdfMichaelStarkes_UncutGemsProjectSummary.pdf
MichaelStarkes_UncutGemsProjectSummary.pdf
 
An overview of the various scriptures in Hinduism
An overview of the various scriptures in HinduismAn overview of the various scriptures in Hinduism
An overview of the various scriptures in Hinduism
 
Features of Video Calls in the Discuss Module in Odoo 17
Features of Video Calls in the Discuss Module in Odoo 17Features of Video Calls in the Discuss Module in Odoo 17
Features of Video Calls in the Discuss Module in Odoo 17
 
Envelope of Discrepancy in Orthodontics: Enhancing Precision in Treatment
 Envelope of Discrepancy in Orthodontics: Enhancing Precision in Treatment Envelope of Discrepancy in Orthodontics: Enhancing Precision in Treatment
Envelope of Discrepancy in Orthodontics: Enhancing Precision in Treatment
 
Navigating the Misinformation Minefield: The Role of Higher Education in the ...
Navigating the Misinformation Minefield: The Role of Higher Education in the ...Navigating the Misinformation Minefield: The Role of Higher Education in the ...
Navigating the Misinformation Minefield: The Role of Higher Education in the ...
 
2024_Student Session 2_ Set Plan Preparation.pptx
2024_Student Session 2_ Set Plan Preparation.pptx2024_Student Session 2_ Set Plan Preparation.pptx
2024_Student Session 2_ Set Plan Preparation.pptx
 
Open Educational Resources Primer PowerPoint
Open Educational Resources Primer PowerPointOpen Educational Resources Primer PowerPoint
Open Educational Resources Primer PowerPoint
 
Danh sách HSG Bộ môn cấp trường - Cấp THPT.pdf
Danh sách HSG Bộ môn cấp trường - Cấp THPT.pdfDanh sách HSG Bộ môn cấp trường - Cấp THPT.pdf
Danh sách HSG Bộ môn cấp trường - Cấp THPT.pdf
 
factors influencing drug absorption-final-2.pptx
factors influencing drug absorption-final-2.pptxfactors influencing drug absorption-final-2.pptx
factors influencing drug absorption-final-2.pptx
 

SetFusion Visual Hybrid Recommender - IUI 2014

  • 1. See What you Want to See: Visual User-Driven Approach for Recommendation Denis Parra, PUC Chile Peter Brusilovsky, University of Pittsburgh Christoph Trattner, Graz University of Technology IUI 2014, Haifa, Israel
  • 2. Outline •  Short intro to some Challenges in Recommender Systems •  Our Approach to User Controllability (demo) •  User Study & Results •  Summary & Future Work 02/27/2014 D.Parra et al.~ IUI 2014 2
  • 3. INTRODUCTION Recommender Systems: Introduction & Challenges addressed in this research 3 * Danboard (Danbo): Amazon’s cardboard robot, in these slides represents a recommender system *
  • 4. Recommender Systems (RecSys) Systems that help people to find relevant items in a crowded item or information space (McNee et al. 2006) 02/27/2014 D.Parra et al.~ IUI 2014 4
  • 5. Challenges of RecSys Addressed Here Traditionally, RecSys has focused on producing accurate recommendation algorithms. In this research, these challenges are addressed: 1.  Human Factors in RecSys: Study controllability by introducing a novel visualization that presents fusion of different recommenders 2.  Evaluation: Use of Objective, Subjective & Behavioral metrics 02/27/2014 D.Parra et al.~ IUI 2014 5
  • 6. Research Goals & User Studies Research Goal •  To understand the effect of controllability on the user engagement and on the overall user experience of a RecSys (on this paper) Through •  Two studies conducted using Conference Navigator: 02/27/2014 D.Parra et al.~ IUI 2014 6 Program Proceedings Author List Recommendations http://halley.exp.sis.pitt.edu/cn3/
  • 7. WHYIUISHOULDCARE:HCI+RECSYS COMMUNITY Previous research related to this work / Motivating results from TalkExplorer study 7/22/2013 D.Parra ~ PhD. Dissertation Defense 7
  • 8. TasteWeights (Bostandjev et ala 2012) 7/22/2013 D.Parra ~ PhD. Dissertation Defense 8
  • 9. Preliminary Work: TalkExplorer •  Adaptation of Aduna Visualization in CN •  Main research question: Do fusion (intersection) of contexts of relevance improve user experience? 7/22/2013 D.Parra ~ PhD. Dissertation Defense 9 Center user CN user RecommenderRecommender Cluster with intersect ion of entities Cluster (of talks) associated to only one entity
  • 11. Our Proposed Interface: SetFusion 02/27/2014 D.Parra et al.~ IUI 2014 11
  • 12. Our Proposed Interface - II 02/27/2014 D.Parra et al.~ IUI 2014 12 Traditional Ranked List Papers sorted by Relevance. It combines 3 recommendation approaches.
  • 13. Our Proposed Interface - III 02/27/2014 D.Parra et al.~ IUI 2014 13 Sliders Allow the user to control the importance of each data source or recommendation method Interactive Venn Diagram Allows the user to inspect and to filter papers recommended. Actions available: -  Filter item list by clicking on an area -  Highlight a paper by mouse-over on a circle -  Scroll to paper by clicking on a circle -  Indicate bookmarked papers
  • 14. Mixed Hybridization: Item Score 7/22/2013 D.Parra ~ PhD. Dissertation Defense 14 M: The set of all methods available to fuse rankreci,mj : rank–position in the list of a recommended item reci : recommended item i mj, : recommendation method j Wmj : weight given by the user to the method mj using the controllable interface |Mreci| represents the number of methods by which item reci was recommended Slider weight
  • 15. RESEARCH:DETAILS&RESULTS Description and Analysis of the results of the 3 user studies
  • 16. Studies: CSCW 2013 & UMAP 2013 02/27/2014 D.Parra et al.~ IUI 2014 16 CSCW 2013 Conditions Static List Interactive SetFusion # Attendants ~400 # RecSys Users 15 22 Study type Between Subjects UMAP 2013 Interactive SetFusion ~ 100 50 1 group Preliminary User study: Here we learned that the Interactive interface had a positive effect on user behavior and perception of the recsys Second study: Only interactive interface CHANGES: 1.  Preference Elicitation: In CSCW we avoided cold start. In UMAP we had no constraints 2.  Use of the ratings to update the recommended items 3.  Tuning of Content- based recommender
  • 17. Comparing CSCW and UMAP 02/27/2014 D.Parra et al.~ IUI 2014 17 (Only Interactive Interfaces) CSCW 2013 UMAP 2013 # Users exposed to recommendation 84 95 # Users who used the recommender 22 ( ~ 26 %) 50 ( ~52.6 %) # Users bookmarked papers 6 ( ~ 27.2 %) 14 (~28 %) # Talks bookmarked / user avg. 28 / 4.67 103 / 7.36 Average User rating 3.73 / 10 ( ~45.4 %) 3.62 / 8 (~16%) Usage at Recommender Page # Talks explored (user avg.) 16.84 14.9 # People returning 7 (~31.8%) 14 (28%) Average time spent in page (seconds) 261.72 353.8
  • 18. Comparing CSCW and UMAP 02/27/2014 D.Parra et al.~ IUI 2014 18 (Only Interactive Interfaces) CSCW 2013 UMAP 2013 # Users exposed to recommendation 84 95 # Users who used the recommender 22 50 # Users bookmarked papers 6 ( ~ 27.2 %) 14 (28 %) # Talks bookmarked / user avg. 28 / 4.67 103 / 7.36 Average User rating 3.73 / 10 ( ~45.4 %) 3.62 / 8 (~16%) Usage at Recommender Page # Talks explored (user avg.) 16.84 14.9 # People returning 7 (~31.8%) 14 (28%) Average time spent in page (seconds) 261.72 353.8
  • 19. Comparing CSCW and UMAP 02/27/2014 D.Parra et al.~ IUI 2014 19 (Only Interactive Interfaces) CSCW 2013 UMAP 2013 # Users exposed to recommendation 84 95 # Users who used the recommender 22 50 # Users bookmarked papers 6 ( ~ 27.2 %) 14 (~28 %) # Talks bookmarked / user avg. 28 / 4.67 103 / 7.36 Average User rating 3.73 / 10 ( ~45.4 %) 3.62 / 8 (~16%) Usage at Recommender Page # Talks explored (user avg.) 16.84 14.9 # People returning 7 (~31.8%) 14 (28%) Average time spent in page (seconds) 261.72 353.8
  • 20. From the Final Survey CSCW 2013 (11 users) UMAP 2013 (8 users) I don’t think that Conference Navigator needs a Recommender System M = 2.36, S.E. = 0.2 M = 1.5 , S.E. = 0.21 (p < 0.05) I would recommend this system to my colleagues M = 3.36, S.E. = 0.28 M = 4.25, S.E. = 0.33 (p < 0.05) 02/27/2014 D.Parra et al.~ IUI 2014 20 - Users perceived SetFusion significantly as a more useful tool in UMAP than in CSCW
  • 22. Summary of Results •  From Study 1 we showed that User Controllability had an effect on the user experience with RecSys. •  Comparing SetFusion in Study 1 and Study 2: – A natural elicitation setting (UMAP) allowed users to be more engaged on using the system for the task of the interface: bookmark papers recommended. – Users also perceived the system as more useful in UMAP 2013. – Ratings are a form of giving user control, a big lesson from Study 1: if you ask user for feedback, use it! 02/27/2014 D.Parra et al.~ IUI 2014 22
  • 23. Limitations & Future Work •  Apply our approach to other domains (fusion of data sources or recommendation algorithms) •  Find alternatives to scale the approach to more than 3 sets, potential alternatives: – Clustering and – Radial sets •  Consider other factors that might interact with the user experience: – Controllability by itself vs. minimum level of accuracy 02/27/2014 D.Parra et al.~ IUI 2014 23