TasteWeights                        A Visual Interactive Hybrid Recommender System                     Svetlin Bostandjiev...
MotivationProblems• Many traditional  recommenders are “black  boxes” and lack explanation  and control [Herlocker]• “Why ...
MotivationChallenges:  Need for more dynamic, more  adaptable algorithms that can cope  with diverse data from APIs.  An...
TasteWeights: BackgroundInitial Work on Graph-based    Representations of Collaborative    Filtering Algorithms:  PeerChoo...
Interactive, Trust-based Recommender for Facebook DataSupports user interaction toupdate information atrecommendation time...
Combining Social and Semantic                       Recommendations                                                   Face...
TasteWeights Design
Demohttp://www.youtube.com/watch?v=9_JgynePm9w&hd=1
ApproachParallel hybrid recommender system               Entity Resolution          Recommendation         Hybridization  ...
RecommendationSourcesInput Data Resolution  Mapping between Wikipedia articles  Facebook pages, and Twitter #tagsSimilarit...
Generating Recs.Individual SourceHybrid Strategies  Weighted  Mixed  Cross-source
EvaluationGoals  Evaluate combining social and semantic recommendations  Evaluate explanation and transparency in a hybrid...
Evaluation: AccuracyExperiment  One-way repeated measures ANOVA  Compared 9 recommendation methods (below) in terms of rec...
Results: Accuracy                                             Results from a Tukey post-hoc analysis of the               ...
Results: Diversity                                                             Diversity                        140Number ...
TasteWeights on LinkedIn DataCase Study: Portability of TW interface  -Developed a Social-Semantic Recommendation algorith...
ConclusionsUI and interaction design are important considerations for RSs  -Increased explanation, provenance  -Expose oth...
After the break… Inspectability and  Control in Social Recommenders In this work we touched on the ideas of inspectability...
Thanks for listening!
TasteWeights: Visual Interactive Hybrid Recommendations
TasteWeights: Visual Interactive Hybrid Recommendations
TasteWeights: Visual Interactive Hybrid Recommendations
TasteWeights: Visual Interactive Hybrid Recommendations
Upcoming SlideShare
Loading in …5
×

TasteWeights: Visual Interactive Hybrid Recommendations

272 views
237 views

Published on

Slides from ACM Recommender Systems conference, September 2012. Presented by John O'Donovan

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
272
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Grand challenge? Seamlessly ingest data from newly arriving sources and produce real-time recommendations.
  • Show demo ifpossible1.50 jenny node interaction.
  • General ideaExplain 3 columnsLeftprofileMiddleWeb APIs wiki – semantic FB – social Twitter – socio-semantic expert-basedmeta data usually not available in modern rec syspandora has just started sharing why they rec certain bands to youwe argue that there is value in displaying meta data and letting the user fine-tune itRight
  • Demo the interactivity and UI featuresConnectionsReal time feedback Item weights Domain weights more popular bands go to top
  • Parallel hybrid rec sysalgos run in parallel and are later combined to produce the final recsInput data resolutionto the space of different data domains: W, F, Tmap input music bands -> W articles, F pages, T hash tagsdone mainly through search calloutsSimilarity modelstake in W articles, F pages, T hash tags -> produce related W articles / F friends / T experts Recalgosindividual recalgo per data domaincombine individual algos via a hybrid recalgo-> related music bands
  • Input data resolution mainly tru search engine calloutsSimilarity models
  • First, we compute recs from each individual sourceThen, combine recsWeightedlinear combination: weight of rec from individual source * weight of the sourceMixedcompile ranked list of recs, pick top recs from each list, then 2nd best recs, and so onCross-sourcesame as weighted, but give more weights to recs that appear in multiple sources
  • GoalsCompare hybrid methods to individual source methods 3. Compare interactive methods to hybrid methods
  • Utilitymetric for how useful a list of items ismore weight to items on top of the listexponential decay of utility as we go down the listFor each method, generate top 15 recs.Ask users to rate theseMethod 1: (top 5 recs ratings) 5, 4, 4, 5, 3 -> utilty 10Method 2: 1, 2, 3, 1, 2 -> utility 3Prepare some response for a diversity question.
  • First three individual sourcesYellow hybridsPurple interaction methodsimprovedrec accuracy: Cross-source hybrid > single source (contribution 1)Full interaction > best hybrid (Cross-source) (contributions 3)
  • Explicit qualitative evaluationExplanationHighest rating (4.4) “the system helped me understand how I got the recommendations”InteractionPerceived: highest: “interaction helped me get better recommendations”
  • Explicit qualitative evaluationExplanationHighest rating (4.4) “the system helped me understand how I got the recommendations”InteractionPerceived: highest: “interaction helped me get better recommendations”
  • This should be a bar graph.Explicit qualitative evaluationExplanationHighest rating (4.4) “the system helped me understand how I got the recommendations”InteractionPerceived: highest: “interaction helped me get better recommendations”
  • Utilitymetric for how useful a list of items ismore weight to items on top of the listexponential decay of utility as we go down the listFor each method, generate top 15 recs.Ask users to rate theseMethod 1: (top 5 recs ratings) 5, 4, 4, 5, 3 -> utilty 10Method 2: 1, 2, 3, 1, 2 -> utility 3Prepare some response for a diversity question.
  • Utilitymetric for how useful a list of items ismore weight to items on top of the listexponential decay of utility as we go down the listFor each method, generate top 15 recs.Ask users to rate theseMethod 1: (top 5 recs ratings) 5, 4, 4, 5, 3 -> utilty 10Method 2: 1, 2, 3, 1, 2 -> utility 3Prepare some response for a diversity question.
  • Utilitymetric for how useful a list of items ismore weight to items on top of the listexponential decay of utility as we go down the listFor each method, generate top 15 recs.Ask users to rate theseMethod 1: (top 5 recs ratings) 5, 4, 4, 5, 3 -> utilty 10Method 2: 1, 2, 3, 1, 2 -> utility 3Prepare some response for a diversity question.
  • TasteWeights: Visual Interactive Hybrid Recommendations

    1. 1. TasteWeights A Visual Interactive Hybrid Recommender System Svetlin Bostandjiev, John O’Donovan, Tobias Höllerer Department of Computer Science University of California, Santa Barbara {alex, jod, holl}@cs.ucsb.eduACM Recommender Systems. Burlington Hotel, Dublin. September 10th 2012
    2. 2. MotivationProblems• Many traditional recommenders are “black boxes” and lack explanation and control [Herlocker]• “Why am I being recommended this movie? I don’t like horror films.”• Even in modern recommenders, data can be static, outdated or simply irrelevant from the beginning.• Data about users and items is spread far and wide.
    3. 3. MotivationChallenges:  Need for more dynamic, more adaptable algorithms that can cope with diverse data from APIs.  And, we need an interface that can keep up…Solutions? User interfaces help to explain provenance of a recommendation. This can improve users’ understanding of the underlying system and contribute to better user experience and greater satisfaction Interaction allows users to: tweak otherwise hidden systems settings; provide updated preference data, recommendation feedback etc. etc.
    4. 4. TasteWeights: BackgroundInitial Work on Graph-based Representations of Collaborative Filtering Algorithms: PeerChooser : Based on static MovieLens data SmallWorlds: Web-based, dynamic data from Facebook API.Issues discovered during evaluations: PeerChooser: Interaction with nodes that represent movie genres …too coarse. SmallWorlds: A “complete” representation, but far too complicated view.Learning from evaluations: Abstraction, Detail-on-demand, Interactive Visual Cues, Cleaner game-like graphics, and more flexible API connectivity. Focus on “social” recommendation.
    5. 5. Interactive, Trust-based Recommender for Facebook DataSupports user interaction toupdate information atrecommendation time>Solves stale data problem.Makes the ACF algorithmtransparent and understandable.>increases satisfaction,acceptance etc.Enables fast visual exploration ofthe data>what-if scenarios>increases learning
    6. 6. Combining Social and Semantic Recommendations Facebook / Twitter (Social Recs)DBPedia/Freebas e(Semantic Recs)
    7. 7. TasteWeights Design
    8. 8. Demohttp://www.youtube.com/watch?v=9_JgynePm9w&hd=1
    9. 9. ApproachParallel hybrid recommender system Entity Resolution Recommendation Hybridization Wikipedia Input Data Sources Rec Resolution Similarity (W articles) Algorithm (W articles) Model (recs) Input Input Data Facebook Rec Hybrid Output Similarity Sources Rec Data Resolution (FB pages) Model (band pages) Algorithm (recs) Algorithm Data (FB likes) (recs) Input Data Twitter Rec Similarity Resolution Algorithm (TW #tags) Model Sources (recs) (WF Experts)
    10. 10. RecommendationSourcesInput Data Resolution Mapping between Wikipedia articles Facebook pages, and Twitter #tagsSimilarity Models Wikipedia (Data source: DBpedia) Facebook (Data source: Facebook Graph API) Twitter (Data source: wefollow.com)
    11. 11. Generating Recs.Individual SourceHybrid Strategies Weighted Mixed Cross-source
    12. 12. EvaluationGoals Evaluate combining social and semantic recommendations Evaluate explanation and transparency in a hybrid recommender Evaluate interaction in a hybrid recommenderSetup Supervised user study. 32 participants from the human subject pool at UCSBProcedure Pre-questionnaire Tasks Interact with Profile Interact with Sources Interact with Full interface Rate recommendations Post-questionnaire (Explanation & Interaction)
    13. 13. Evaluation: AccuracyExperiment One-way repeated measures ANOVA Compared 9 recommendation methods (below) in terms of rec. accuracyMethod (independent variable) Single-source: Wikipedia, Facebook, Twitter Hybrid: Weighted, Mixed, Cross-source Interaction: Profile, Sources, FullAccuracy (dependent variable) Measured in terms of “Utility”
    14. 14. Results: Accuracy Results from a Tukey post-hoc analysis of the recommendation methods: multiple comparisons of means with 95% family-wise confidence levelPlot of means of recommendation methodsover utility with 95% confidence intervals
    15. 15. Results: Diversity Diversity 140Number of Recommended Items Wikipeda 120 Facebook (CF) 100 Hybrid 80 Hybrid + Interaction 60 40 20 0 1000 10000 100000 1000000 10000000 100000000 Bins for Number of Facebook Likes
    16. 16. TasteWeights on LinkedIn DataCase Study: Portability of TW interface -Developed a Social-Semantic Recommendation algorithm for data from LinkedIn API -Personalized for one “active” logged-in user. -Visualized the algorithm in TasteWeights interfaceAlgorithm: -Map profile items to noun-phrases -Resolve to Wikipedia articles -e.g.: ph.D => PHD, UCSB => UC Santa Barbara -Compute similarity based on overlap in resolved entities.Features -Segmented / Organized user profile -Interactive profile weighting -Interactive weighting of social connections -Dynamic re-ranking of recommendations (visual feedback) -Provenance views to show effects of each interaction.
    17. 17. ConclusionsUI and interaction design are important considerations for RSs -Increased explanation, provenance -Expose otherwise hidden controls (e.g: control of hybrid recommender) -Helps ease the stale data problem -Support user input at various granularity (recommended item, recommendation partner, profile items etc) -Increase ambient learning. -Promote interest in the recommender system (game-like feel)Contributions: -Demonstrated a novel interactive RS -Hybrid of recommendations from Wikipedia, Facebook and Twitter -Evaluated via a 32 person supervised user study at UCSB. -Demonstrated portability of the system on LinkedIn’s API.Results -Interaction increases user satisfaction in all conditions. (more interaction = higher accuracy) -Cross-source hybrid strategy outperformed individual source strategies.
    18. 18. After the break… Inspectability and Control in Social Recommenders In this work we touched on the ideas of inspectability and control in the context of our hybrid recommender system. In the next talk, Bart Knijnenburg (UC Irvine) will present results from a larger study that focuses on a general analysis of inspectability and control in social recommenders. This study used some components from our TasteWeights system.
    19. 19. Thanks for listening!

    ×