Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lecture 5: Personalization on the Social Web (2013)


Published on

Lecture 5: Personalization on the Social Web (2013)

  1. 1. Social Web Lecture V: Personalization on the Social Web (some slides were adopted from Fabian Abel) Lora Aroyo The Network Institute VU University AmsterdamMonday, March 4, 13
  2. 2. Personalization & Social Web • To design ‘social’ functionality we need to understand to data the application can provide to filter the relevant information (what users perceive as relevant) • Therefore we need to understand • how good personalization (recommenders) are • how good the user models are they are based on • In this lecture, we consider theory & techniques for how to design and evaluate recommenders and user models (for use in SW applications)Monday, March 4, 13
  3. 3. User Modeling How to infer & represent user information that supports a given application or context? Kevin KellyMonday, March 4, 13
  4. 4. User Modeling Challenge • Application has to obtain, understand & exploit information about the user • Information (need & context) about user • Inferring information about user & representing it so that it can be consumed by the application • Data relevant for inferring information about userMonday, March 4, 13
  5. 5. User & Usage Data is everywhere • People leave traces on the Web and on their computers: • Usage data, e.g., query logs, click-through-data • Social data, e.g., tags, (micro-)blog posts, comments, bookmarks, friend connections • Documents, e.g., pictures, videos • Personal data, e.g., affiliations, locations • Products, applications, services - bought, used, installed • Not only a user’s behavior, but also interactions of other users • “people can make statements about me”, “people who are similar to me can reveal information about me” --> “social learning” collaborative recommender systemsMonday, March 4, 13
  6. 6. UM: Basic Concepts • User Profile: a data structure that represents a characterization of a user at a particular moment of time represents what, from a given (system) perspective, there is to know about a user. The data in the profile can be explicitly given by the user or derived by the system • User Model: contains the definitions & rules for the interpretation of observations about the user and about the translation of that interpretation into the characteristics in a user profile user model is the recipe for obtaining and interpreting user profiles • User Modeling: the process of representing the userMonday, March 4, 13
  7. 7. User Modeling Approaches • Overlay User Modeling: describe user characteristics, e.g. “knowledge of a user”, “interests of a user” with respect to “ideal” characteristics • Customizing: user explicitly provides & adjusts elements of the user profile • User model elicitation: ask & observe the user; learn & improve user profile successively “interactive user modeling” • Stereotyping: stereotypical characteristics to describe a user • User Relevance Modeling: learn/infer probabilities that a given item or concept is relevant for a user Related scientific conference: Related journal: http:/, March 4, 13
  8. 8. Which approach suits best the conditions of applications?, March 4, 13
  9. 9. Overlay User Models • among the oldest user models • used for modeling student knowledge • the user is typically characterized in terms of domain concepts & hypotheses of the user’s knowledge about these concepts in relation to an (ideal) expert’s knowledge • concept-value pairsMonday, March 4, 13
  10. 10. User Model Elicitation • Ask the user explicitly learn • NLP, intelligent dialogues • Bayesian networks, Hidden Markov models • Observe the user learn • Logs, machine learning • Clustering, classification, data mining • Interactive user modeling: mixture of direct inputs of a user, observations and inferencesMonday, March 4, 13
  11. 11. http://hunch.comMonday, March 4, 13
  12. 12. Monday, March 4, 13
  13. 13. Stereotyping • set of characteristics (e.g. attribute-value pairs) that describe a group of users. • user is not assigned to a single stereotype - user profile can feature characteristics of several different stereotypesMonday, March 4, 13
  14. 14. Why are stereotypes useful?, March 4, 13
  15. 15. Monday, March 4, 13
  16. 16. Can we infer a Twitter- based user profile? Personalized News Recommender Profile I want my ? personalized news recommendations! User Modeling (4 building blocks) Semantic Enrichment, Linkage and Alignment Example from Abel et al. (2011)Monday, March 4, 13
  17. 17. User Modeling Building Blocks 1. Temporal Constraints Profile? (a) time period 1. Which tweets of concept weight the user should be analyzed? ? (b) temporal patterns start weekends end Morning: Afternoon: time Night: June 27 July 4 July 11Monday, March 4, 13
  18. 18. User Modeling Building Blocks 1. Temporal Constraints Francesca T Sport Schiavone 2. Profile Profile? Type concept weight ? Francesca Schiavone won # hashtag-based French Open #fo2010 entity-based T topic-based French Open # fo2010 2. What type of concepts should represent “interests”? time June 27 July 4 July 11Monday, March 4, 13
  19. 19. User Modeling Building Blocks 1. Temporal Constraints Francesca (a) tweet-based Schiavone 2. Profile Profile? Type concept weight Francesca Francesca Schiavone won! Schiavone 3. Semantic French Open Enrichment Tennis Francesca wins French Open French Thirty in womens Open (b) further enrichment tennis is primordially old, an age when agility and desire Tennis recedes as the … 3. Further enrich the semantics of tweets?Monday, March 4, 13
  20. 20. User Modeling Building Blocks 1. Temporal Constraints Profile? 2. Profile concept weight Type 4. How to weight the Francesca 4 ? Schiavone 3. Semantic concepts? French Open 3 Enrichment Tennis 6 Concept frequency (TF) 4. Weighting TFxIDF weight(French Open) Scheme weight(Francesca Time-sensitive Schiavone) weight(Tennis) time June 27 July 4 July 11Monday, March 4, 13
  21. 21. Observations • Profile characteristics: • Semantic enrichment solves sparsity problems • Profiles change over time: fresh profiles reflect better current user demands • Temporal patterns: weekend profiles differ significantly from weekday profiles • Impact on news recommendations: • The more fine-grained the concepts the better the recommendation performance: entity-based > topic-based > hashtag-based • Semantic enrichment improves recommendation quality • Time-sensitivity (adapting to trends) improves performanceMonday, March 4, 13
  22. 22. User Modeling it is not about putting everything in a user profile it is about making the right choicesMonday, March 4, 13
  23. 23. User Adaptation Knowing the user - this knowledge - can be applied to adapt a system or interface to the user to improve the system functionality and user experienceMonday, March 4, 13
  24. 24. Fig. 1 Functional model of tasks and sub-tasks specifically suited for SASs Fig. 1 Functional model of tasks and sub-tasks specifically suited for SASs (Ilaria Torre (2009)Monday, March 4, 13
  25. 25. User-Adaptive Systems user profile user modeling profile analysis observations, data and adaptation information decisions about user A. Jameson. Adaptive interfaces and agents. The HCI handbook: fundamentals, evolving technologies and emerging applications, pp. 305–330, 2003.Monday, March 4, 13
  26. 26. adapts to your music taste user profile interests in genres, artists, tags user modeling compare profile (infer current with possible next musical taste) songs to play history of next song to songs, like, be played ban, pause, skipMonday, March 4, 13
  27. 27. Issues in User-Adaptive Systems • Overfitting, “bubble effects”, loss of serendipity problem: • systems may adapt too strongly to the interests/behavior • e.g., an adaptive radio station may always play the same or very similar songs • We search for the right balance between novelty and relevance for the user • “Lost in Hyperspace” problem: • when adapting the navigation – i.e. the links on which users can click to find/access information • e.g., re-ordering/hiding of menu items may lead to confusionMonday, March 4, 13
  28. 28. personalization - are we in control? personalization - FoF weaken recommendations? personalization - self fulfilling prophecy?Monday, March 4, 13
  29. 29. What is good user modeling & personalization?, March 4, 13
  30. 30. Success perspectives • From the consumer perspective of an adaptive system: Adaptive system maximizes hard to measure/obtain satisfaction of the user • From the provider perspective of an adaptive system: influence of UM & Adaptive system maximizes personalization may be the profit hard to measure/obtainMonday, March 4, 13
  31. 31. Evaluation Strategies • User studies: ask/observe (selected) people whether you did a good job • Log analysis: Analyze (click) data and infer whether you did a good job, • Evaluation of user modeling: • measure quality of profiles directly, e.g. measure overlap with existing (true) profiles, or let people judge the quality of the generated user profiles • measure quality of application that exploits the user profile, e.g., apply user modeling strategies in a recommender systemMonday, March 4, 13
  32. 32. Evaluating User Modeling in RecSys training data test data (ground truth) item C item G item A item H item E item B measure item D item F time quality training data Recommendations: Strategy X X Y Z ? Strategy Y Recommender item R item H item F item H item G item H Strategy Z ? ? ? User Modeling strategies to compare item M item N item MMonday, March 4, 13
  33. 33. Possible Metrics • The usual IR metrics: • Precision: fraction of retrieved items that are relevant • Recall: fraction of relevant items that have been retrieved • F-Measure: (harmonic) mean of precision and recall • Metrics for evaluating recommendation (rankings): • Mean Reciprocal Rank (MRR) of first relevant item • Success@k: probability that a relevant item occurs within the top k • If a true ranking is given: rank correlations performance strategy X • Precision@k, Recall@k & F-Measure@k baseline • Metrics for evaluating prediction of user preferences: • MAE = Mean Absolute Error • True/False Positives/Negatives runs Is strategy X better than the baseline?Monday, March 4, 13
  34. 34. Example Evaluation • [Rae et al.] shows a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks) • Task: test how well the different strategies (different tag contexts) can be used for tag prediction/recommendation • Steps: 1. Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data 2. Use the input data and calculate for the different strategies the predictions 3. Measure the performance using standard (IR) metrics: Precision of the top 5 recommended tags (P@5), Mean Reciprocal Rank (MRR), Mean Average Precision (MAP) 4. Test the results for statistical significance using T-test, relative to the baseline (e.g. existing approach, competitive approach) [Rae et al. Improving Tag Recommendations Using Social Networks, RIAO’10]]Monday, March 4, 13
  35. 35. Example Evaluation • [Guy et al.] shows another example of a similar evaluation approach • The different strategies differ in the way people and tags are used in the strategies: with these tag-based systems, there are complex relationships between users, tags and items, and strategies aim to find the relevant aspects of these relationships for modeling and recommendation • Their baseline is the strategy of the ‘most popular’ tags: this is a strategy often used, to compare the globally most popular tags to the tags predicted by a particular personalization strategy, thus investigating whether the personalization is worth the effort and is able to outperform the easily available baseline. [Guy et al. Social Media Recommendation based on People and Tags, SIGIR’10]]Monday, March 4, 13
  36. 36. Recommendation Systems Predict items that are relevant/useful/interesting (and to what extent) for given user (in a given context) it’s often a ranking taskMonday, March 4, 13
  37. 37. Monday, March 4, 13
  38. 38. Monday, March 4, 13
  39. 39. Monday, March 4, 13
  40. 40., March 4, 13
  41. 41. Collaborative Filtering • Memory-based: User-Item matrix: ratings/preferences of users => compute similarity between users & recommend items of similar users • Model-based: Item-Item matrix: similarity (e.g. based on user ratings) between items => recommend items that are similar to the ones the user likes • Model-based: Clustering: cluster users according to their preferences => recommend items of users that belong to the same cluster • Model-based: Bayesian networks: P(u likes item B | u likes item A) = how likely is it that a user, who likes item A, will like item B learn probabilities from user ratings/preferences • Others: rule-based, other data mining techniques • u1 likes likes likes ! u1 likes u2 Pulp Fiction?Monday, March 4, 13
  42. 42. Memory vs. Model-based • complete input data is • abstraction (model) of input required data • pre-computation not possible • pre-computation (partially) possible (model has to be re- • does not scale well (“tricks” built from time to time) are needed) • scales better • high quality of recommendations • abstraction may reduce recommendation qualityMonday, March 4, 13
  43. 43. Collaborative Filtering: Pros & Cons • No domain knowledge required • Cold-start problem / New User (recommendations are just problem based on the social interactions) • Sparsity problem / New Item • Quality may improve over time problem (e.g. the more users are interacting with items) • SPAM (e.g. SPAM users might promote items by rating those • Implicit user feedback is sufficient items and other non-similar items positively) • Word-of-mouth approach may lead to more diversity in the recommendationsMonday, March 4, 13
  44. 44. Rating Issues • Reliability, consistency: Would a person give the same grade when re-grading? • Validity, accuracy: Does the grade reflect the deeper quality? Do opinions shift over time, e.g. after grading many? • Anonymity, trust: Does anonymity or explicit identification impact the reviewing? Are people led by other motives?Monday, March 4, 13
  45. 45. Social Networks & Interest Similarity • collaborative filtering: ‘neighborhoods’ of people with similar interest & recommending items based on likings in neighborhood • limitations: next to ‘cold start’ and ‘sparsity’ the lack of control (over one’s neighborhood) is also a problem, i.e. cannot add ‘trusted’ people, nor exclude ‘strange’ ones • therefore, interest in ‘social recommenders’, where presence of social connections defines the similarity in interests (e.g. social tagging CiteULike): • does a social connection indicate user interest similarity? • how much users interest similarity depends on the strength of their connection? • is it feasible to use a social network as a personalized recommendation? [Lin & Brusilovsky, Social Networks and Interest Similarity: The Case of CiteULike, HT’10]Monday, March 4, 13
  46. 46. Conclusions • pairs unilaterally connected have more common information items, metadata, and tags than non-connected pairs. • the similarity was largest for direct connections and decreased with the increase of distance between users in the social networks • users involved in a reciprocal relationship exhibited significantly larger similarity than users in a unidirectional relationship • traditional item-level similarity may be less reliable way to find similar users in social bookmarking systems. • items collections of peers connected by self-defined social connections could be a useful source for cross-recommendation.Monday, March 4, 13
  47. 47. personalization semantics = good or bad? social personalization - are people aware?Monday, March 4, 13
  48. 48. Content-based Recommendations • Input: characteristics of items & interests of a user into characteristics of items => Recommend items that feature characteristics which meet the user’s interests • Techniques: • Data mining methods: Cluster items based on their characteristics => Infer users’ interests into clusters • IR methods: Represent items & users as term vectors => Compute similarity between user profile vector and items • Utility-based methods: Utility function that gets an item as input; the parameters of the utility function are customized via preferences of a userMonday, March 4, 13
  49. 49. Government stops Tower Bridge is a combined bascule and suspension renovation of tower bridge in London, England, over the River Thames. bridge Oct 13th 2011 Category: politics, england Related Twiper news: @bob: Why do they stop to… [more] @mary: London stops reno… [more] Tower Bridge today Under construction db:Politics 0.2 Content db:Sports db:Education 0 0 Features db:London 0.2 = a db:Tower_Bridge 0.4 db:Government 0.1 Weighting strategy: db:UK 0.1 -  occurrence frequency -  normalize vectors (1-norm ! sum of vector equals 1)Monday, March 4, 13
  50. 50. User’s Twitter history RT: Government stops User renovation of tower bridge Oct 13th 2011 Model I am in London at the moment Oct 13th 2011 db:Politics 0 I am doing sports db:Sports 0.1 Oct 12th 2011 db:Education 0 db:London 0.5 = u db:Tower_Bridge 0.2 db:Government 0.2 Weighting strategy: db:UK 0 -  occurrence frequency (e.g. smoothened by occurrence time ! recent concepts are more important -  normalize vectors (1-norm ! sum of vector equals 1)Monday, March 4, 13
  51. 51. candidate items user a b c u db:Politics 0.2 0 0 0 db:Sports 0 0 0.5 0.1 db:Education 0 0 0.2 0 db:London 0.2 0.8 0 0.5 db:Tower_Bridge 0.4 0.2 0 0.2 db:Government 0.1 0 0 0.2 db:UK 0.1 0 0.3 0 cosine a b c similarities u 0.67 0.92 0.14Recommendations Ranking of recommended items: 1.  b 2.  a 3.  cMonday, March 4, 13
  52. 52. Content-based Filtering: Pros & Cons • New item problem may not occur • Cold-start problem / New User (depends slightly on how the problem item representation is created) • Changing User Preferences • Quality may improve over time (e.g. if more profile information is available about a • Lack of Diversity / Overfitting user) • SPAM (e.g. if item descriptions can be added/modified by • Implicit user feedback is sufficient users, then SPAM users might add non-valid descriptions)Monday, March 4, 13
  53. 53. RecSys Problems • Cold-start problem / New User problem: no or little data is available to infer the preferences of new users • Changing User Preferences: user interests may change over time • Sparsity problem / New Item problem: item descriptions are sparse, e.g. not many user rated or tagged an item • Lack of Diversity / Overfitting: for many applications good recommendations should be relevant and new to the user (exceptions: predicting re-visiting behavior, etc.). When adapting too strongly to the preferences of a user, the user might see again and again same/similar recommendations • Use the right context: users do lots of things, which might not be relevant for their user model, e.g. try out things, do stuff for other people • Research challenge: find right balance between serendipity & personalization • Research challenge: find right way to use the influence of the recommendations on the user’s behaviorMonday, March 4, 13
  54. 54. personalization, privacy & laws - are there here or not?Monday, March 4, 13
  55. 55. Monday, March 4, 13
  56. 56. Hands-on Teaser • Your Facebook Friends’ popularity in a spread sheet • Locations of your Facebook Friends • Tag Cloud of your wall posts image source:, March 4, 13