Your SlideShare is downloading. ×
0
Social Web
2014
Lecture V: Personalization on the Social Web
(some slides adopted from Fabian Abel)
Lora Aroyo
The Network...
theory & techniques for 	

how to design & evaluate 	

recommenders & user models 	

to use in Social Web applications

So...
Fig. 1 Functional model of tasks and sub-tasks specifically suited for SASs

Fig. 1 Functional model of tasks and sub-tasks...
User Modeling

How to infer & represent 	

user information that supports a given
application or context?
Kevin Kelly

Soc...
User Modeling Challenge
• Application has to obtain,

understand & exploit information
about the user	


• Information (ne...
User & Usage Data
is Everywhere
• People leave traces on the Web and on their computers:
• Usage data, e.g., query logs, c...
UM: Basic Concepts
• User Profile = data structure = a characterization of a user at a
particular moment
represents what, f...
User Modeling Approaches
• Overlay User Modeling: describe user characteristics, e.g.

“knowledge of a user”, “interests o...
Which approach suits best
the conditions of
applications?

Social Web 2014, Lora Aroyo! http://farm7.staticflickr.com/6240/...
Overlay User Models
• among the oldest user models	

• used for modeling student
knowledge	

• the user is typically chara...
User Model Elicitation
• Ask the user explicitly
learn	

• NLP, intelligent dialogues	

• Bayesian networks, Hidden Markov...
http://hunch.com
Social Web 2014, Lora Aroyo!
User
Stereotypes
•

set of characteristics (e.g.
attribute-value pairs) that
describe a group of users.	


•

user is not ...
based on slides from Fabien Abel
Can we infer a
Twitter-based User Profile?
Personalized News
Recommender
Profile

?

User Modeling
(4 building blocks)

Se...
User Modeling Building
Blocks

1. Temporal
Constraints

1. Which tweets of
the user should be
analyzed?

start

Profile?

...
User Modeling Building
Blocks
Francesca
Schiavone

T Sport
concept weight

# hashtag-based
entity-based

T topic-based

#
...
User Modeling Building
Blocks
Francesca
Schiavone

1. Temporal
Constraints

(a) tweet-based
Profile?

Francesca Schiavone ...
User Modeling Building
Blocks

1. Temporal
Constraints

Profile?

concept

4. How to weight the
concepts?
Concept frequenc...
Observations
• Profile characteristics:
• Semantic enrichment solves sparsity problems	

• Profiles change over time: recent...
User Modeling
it is not about putting everything in a user profile 	

it is about making the right choices

Social Web 2014...
User Adaptation
Knowing the user to adapt a system or interface	

to improve the system functionality and user experience
...
User-Adaptive Systems
user
profile
user modeling

observations,
data and
information
about user

profile analysis

adaptat...
Last.fm adapts to
your music taste
user profile
interests in
genres,
artists, tags

user modeling
(infer current
musical t...
Issues in User-Adaptive
Systems
• Overfitting, “bubble effects”, loss of serendipity problem: 	

• systems may adapt too st...
What is good user modelling
& personalisation?

Social Web 2014, Lora Aroyo!

http://www.flickr.com/photos/bellarosebyliz/4...
Success Perspectives

• From the consumer perspective of an
adaptive system: 	


! Adaptive system maximizes
satisfaction ...
Evaluation Strategies
• User studies: ask/observe (selected) people whether you did a
good job	


• Log analysis: Analyze ...
Evaluating User Modeling
in RecSys
training data

test data (ground truth)

item C
item A
item B

training
data

item G
it...
Possible Metrics
• The usual IR metrics:	

• Precision: fraction of retrieved items that are relevant	

• Recall: fraction...
Example Evaluation
• [Rae et al.] a typical example of how to investigate and evaluate a proposal for
improving (tag) reco...
Example Evaluation
• [Guy et al.] another example of a similar evaluation approach	

• The different strategies differ in ...
recommendation 	

dimensions
Recommendation Systems
Predict relevant/useful/interesting items	

for a given user (in a given context)	

it’s often a ra...
Social Web 2014, Lora Aroyo!
March 28, 2013

Social Web 2014, Lora Aroyo!
Social Web 2014, Lora Aroyo!
Social Web 2014, Lora Aroyo!
commercial 	

personalisation
http://www.wired.com/magazine/2011/11/mf_artsy/all/1

Social Web 2014, Lora Aroyo!
filter bubble
Collaborative Filtering
• Memory-based: User-Item matrix: ratings/preferences of users => compute
similarity between users...
Memory vs. Model-based
• complete input data is
required	

• pre-computation not
possible	

• does not scale well 	

• hig...
Social Networks &
Interest Similarity
• collaborative filtering: ‘neighborhoods’ of people with similar interest
& recommen...
Conclusions
• unilaterally connected pairs have more common items/metadata/tags than non-connected pairs	

• highest simil...
Content-based
Recommendations
• Input: characteristics of items & interests of a user into

characteristics of items => Re...
Government stops
renovation of tower
bridge Oct 13th 2011

Tower Bridge
is a combined bascule and suspension
bridge in Lon...
User’s Twitter
history
RT: Government stops
renovation of tower
bridge Oct 13th 2011
I am in London at the
moment Oct 13th...
db:Politics
db:Sports
db:Education
db:London
db:Tower_Bridge
db:Government
db:UK

candidate
a
b
0.2 0
0
0
0
0
0.2 0.8
0.4 ...
RecSys Issues
• Cold-start problem (new user problem): no/little data available to infer preferences of new users
• Changi...
one machine	

vs. 	

humans
Hands-on Teaser
• Your Facebook Friends’ popularity in a spread sheet	

• Locations of your Facebook Friends	

• Tag Cloud...
Upcoming SlideShare
Loading in...5
×

Lecture 5: Personalization on the Social Web (2014)

644

Published on

This is the fifth lecture in the Social Web course (2014) at the VU University Amsterdam. Visit the website for more information: http://thesocialweb2014.wordpress.com/

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
644
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
20
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Lecture 5: Personalization on the Social Web (2014)"

  1. 1. Social Web 2014 Lecture V: Personalization on the Social Web (some slides adopted from Fabian Abel) Lora Aroyo The Network Institute VU University Amsterdam
  2. 2. theory & techniques for how to design & evaluate recommenders & user models to use in Social Web applications Social Web 2014, Lora Aroyo!
  3. 3. Fig. 1 Functional model of tasks and sub-tasks specifically suited for SASs Fig. 1 Functional model of tasks and sub-tasks specifically suited for SASs (Ilaria Torre, 2009) Social Web 2014, Lora Aroyo!
  4. 4. User Modeling How to infer & represent user information that supports a given application or context? Kevin Kelly Social Web 2014, Lora Aroyo!
  5. 5. User Modeling Challenge • Application has to obtain, understand & exploit information about the user • Information (need & context) about user • Inferring information about user & representing it so that it can be consumed by the application • Data relevant for inferring information about user Social Web 2014, Lora Aroyo!
  6. 6. User & Usage Data is Everywhere • People leave traces on the Web and on their computers: • Usage data, e.g., query logs, click-through-data • Social data, e.g., tags, (micro-)blog posts, comments, bookmarks, friend connections • Documents, e.g., pictures, videos • Personal data, e.g., affiliations, locations • Products, applications, services - bought, used, installed • Not only a user’s behavior, but also interactions of other users • “people can make statements about me” • “people who are similar to me can reveal information about me” • “social learning” collaborative recommender systems Social Web 2014, Lora Aroyo!
  7. 7. UM: Basic Concepts • User Profile = data structure = a characterization of a user at a particular moment represents what, from a given (system) perspective, there is to know about a user. The data in the profile can be explicitly given by user or derived by system • User Model = definitions & rules for the interpretation of observations about the user & about the translation of that interpretation into the characteristics in a user profile user model is the recipe for obtaining & interpreting user profiles • User Modeling = the process of representing the user Social Web 2014, Lora Aroyo!
  8. 8. User Modeling Approaches • Overlay User Modeling: describe user characteristics, e.g. “knowledge of a user”, “interests of a user” with respect to “ideal” characteristics • Customizing: user explicitly provides & adjusts elements of the user profile • User model elicitation: ask & observe the user; learn & improve user profile successively modeling” “interactive user • Stereotyping: stereotypical characteristics to describe a user • User Relevance Modeling: learn/infer probabilities that a given item or concept is relevant for a user Related scientific conference: http://umap2011.org/ Related journal: http:/umuai.org/ Social Web 2014, Lora Aroyo!
  9. 9. Which approach suits best the conditions of applications? Social Web 2014, Lora Aroyo! http://farm7.staticflickr.com/6240/6346803873_e756dd9bae_b.jpg
  10. 10. Overlay User Models • among the oldest user models • used for modeling student knowledge • the user is typically characterized in terms of domain concepts & hypotheses of the user’s knowledge about these concepts in relation to an (ideal) expert’s knowledge • concept-value pairs Social Web 2014, Lora Aroyo!
  11. 11. User Model Elicitation • Ask the user explicitly learn • NLP, intelligent dialogues • Bayesian networks, Hidden Markov models • Observe the user learn • Logs, machine learning • Clustering, classification, data mining
 • Interactive user modeling: mixture of direct inputs of a user, observations and inferences Social Web 2014, Lora Aroyo!
  12. 12. http://hunch.com Social Web 2014, Lora Aroyo!
  13. 13. User Stereotypes • set of characteristics (e.g. attribute-value pairs) that describe a group of users. • user is not assigned to a single stereotype - user profile can feature characteristics of several different stereotypes Social Web 2014, Lora Aroyo! http://farm1.staticflickr.com/155/413650229_31ef379b0b_b.jpg
  14. 14. based on slides from Fabien Abel
  15. 15. Can we infer a Twitter-based User Profile? Personalized News Recommender Profile ? User Modeling (4 building blocks) Semantic Enrichment, Linkage and Alignment based on slides from Fabien Abel I want my personalized news recommendations!
  16. 16. User Modeling Building Blocks 1. Temporal Constraints 1. Which tweets of the user should be analyzed? start Profile? concept weight ? weekends Morning: Afternoon: Night: (a) time period (b) temporal patterns end time June 27 July 4 based on slides from Fabien Abel July 11
  17. 17. User Modeling Building Blocks Francesca Schiavone T Sport concept weight # hashtag-based entity-based T topic-based # 2. Profile Type Profile? Francesca Schiavone won French Open #fo2010 French Open 1. Temporal Constraints ? fo2010 2. What type of concepts should represent “interests”? time June 27 July 4 based on slides from Fabien Abel July 11
  18. 18. User Modeling Building Blocks Francesca Schiavone 1. Temporal Constraints (a) tweet-based Profile? Francesca Schiavone won! http://bit.ly/2f4t7a concept weight Francesca Schiavone French Open Tennis Francesca wins French Open Thirty in women's tennis is primordially old, an age when agility and desire recedes as the … French Open 2. Profile Type 3. Semantic Enrichment (b) further enrichment Tennis 3. Further enrich the semantics of tweets? based on slides from Fabien Abel
  19. 19. User Modeling Building Blocks 1. Temporal Constraints Profile? concept 4. How to weight the concepts? Concept frequency (TF) TFxIDF Time-sensitive weight Francesca Schiavone 4 French Open Tennis ? weight(French Open) weight(Francesca Schiavone) 3 6 2. Profile Type 3. Semantic Enrichment 4. Weighting Scheme weight(Tennis) time June 27 July 4 based on slides from Fabien Abel July 11
  20. 20. Observations • Profile characteristics: • Semantic enrichment solves sparsity problems • Profiles change over time: recent profiles reflect better current user demands • Temporal patterns: weekend profiles differ significantly from weekday profiles • Impact on recommendations: • The more fine-grained the concepts the better the recommendation performance: entity-based > topic-based > hashtag-based • Semantic enrichment improves recommendation quality • Time-sensitivity (adapting to trends) improves performance Social Web 2014, Lora Aroyo!
  21. 21. User Modeling it is not about putting everything in a user profile it is about making the right choices Social Web 2014, Lora Aroyo!
  22. 22. User Adaptation Knowing the user to adapt a system or interface to improve the system functionality and user experience Social Web 2014, Lora Aroyo!
  23. 23. User-Adaptive Systems user profile user modeling observations, data and information about user profile analysis adaptation decisions A. Jameson. Adaptive interfaces and agents. The HCI handbook: fundamentals, evolving technologies and emerging applications, pp. 305–330, 2003.
  24. 24. Last.fm adapts to your music taste user profile interests in genres, artists, tags user modeling (infer current musical taste) compare profile with possible next songs to play history of songs, like, ban, pause, skip next song to be played based on slides from Fabien Abel
  25. 25. Issues in User-Adaptive Systems • Overfitting, “bubble effects”, loss of serendipity problem: • systems may adapt too strongly to the interests/behavior • e.g., an adaptive radio station may always play the same or very similar songs • We search for the right balance between novelty and relevance for the user • “Lost in Hyperspace” problem: • when adapting the navigation – i.e. the links on which users can click to find/access information • e.g., re-ordering/hiding of menu items may lead to confusion Social Web 2014, Lora Aroyo!
  26. 26. What is good user modelling & personalisation? Social Web 2014, Lora Aroyo! http://www.flickr.com/photos/bellarosebyliz/4729613108
  27. 27. Success Perspectives • From the consumer perspective of an adaptive system: ! Adaptive system maximizes satisfaction of the user hard to measure/obtain ! • From the provider perspective of an adaptive system: Adaptive system maximizes the profit Social Web 2014, Lora Aroyo! influence of UM & personalization may be hard to measure/obtain
  28. 28. Evaluation Strategies • User studies: ask/observe (selected) people whether you did a good job • Log analysis: Analyze (click) data and infer whether you did a good job, • Evaluation of user modeling: • measure quality of profiles directly, e.g. measure overlap with • existing (true) profiles, or let people judge the quality of the generated user profiles measure quality of application that exploits the user profile, e.g., apply user modeling strategies in a recommender system Social Web 2014, Lora Aroyo!
  29. 29. Evaluating User Modeling in RecSys training data test data (ground truth) item C item A item B training data item G item E item D measure quality time item F Recommendations: Z X Y Strategy X Strategy Y item H Recommender ? User Modeling strategies to compare Social Web 2014, Lora Aroyo! item H item F item H Strategy Z item R item G item H ? ? ? item M item N item M
  30. 30. Possible Metrics • The usual IR metrics: • Precision: fraction of retrieved items that are relevant • Recall: fraction of relevant items that have been retrieved • F-Measure: (harmonic) mean of precision and recall • Metrics for evaluating recommendation (rankings): • Mean Reciprocal Rank (MRR) of first relevant item • Success@k: probability that relevant item occurs within the top k • If a true ranking is given: rank correlations • Precision@k, Recall@k & F-Measure@k • Metrics for evaluating prediction of user preferences: • MAE = Mean Absolute Error • True/False Positives/Negatives performance Social Web 2014, Lora Aroyo! strategy X baseline runs Is strategy X better than the baseline?
  31. 31. Example Evaluation • [Rae et al.] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks) • Task: test how well the different strategies (different tag contexts) can be used for tag prediction/recommendation • Steps: 1. Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data 2. Use the input data and calculate for the different strategies the predictions 3. Measure the performance using standard (IR) metrics: Precision of the top 5 recommended tags (P@5), Mean Reciprocal Rank (MRR), Mean Average Precision (MAP) 4. Test the results for statistical significance using T-test, relative to the baseline (e.g. existing approach, competitive approach) [Rae et al. Improving Tag Recommendations Using Social Networks, RIAO’10]] Social Web 2014, Lora Aroyo!
  32. 32. Example Evaluation • [Guy et al.] another example of a similar evaluation approach • The different strategies differ in the way people & tags are used: with tag-based systems, there are complex relationships between users, tags and items, and strategies aim to find the relevant aspects of these relationships for modeling and recommendation • The baseline is the ‘most popular’ tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline. [Guy et al. Social Media Recommendation based on People and Tags, SIGIR’10]] Social Web 2014, Lora Aroyo!
  33. 33. recommendation dimensions
  34. 34. Recommendation Systems Predict relevant/useful/interesting items for a given user (in a given context) it’s often a ranking task Social Web 2014, Lora Aroyo!
  35. 35. Social Web 2014, Lora Aroyo!
  36. 36. March 28, 2013 Social Web 2014, Lora Aroyo!
  37. 37. Social Web 2014, Lora Aroyo!
  38. 38. Social Web 2014, Lora Aroyo!
  39. 39. commercial personalisation
  40. 40. http://www.wired.com/magazine/2011/11/mf_artsy/all/1 Social Web 2014, Lora Aroyo!
  41. 41. filter bubble
  42. 42. Collaborative Filtering • Memory-based: User-Item matrix: ratings/preferences of users => compute similarity between users & recommend items of similar users • Model-based: Item-Item matrix: similarity (e.g. based on user ratings) between items => recommend items that are similar to the ones the user likes • Model-based: Clustering: cluster users according to their preferences => recommend items of users that belong to the same cluster • Model-based: Bayesian networks: P(u likes item B | u likes item A) = how likely is it that a user, who likes item A, will like item B learn probabilities from user ratings/preferences • Others: rule-based, other data mining techniques u1 likes likes u2 likes Social Web 2014, Lora Aroyo! ! u1 likes Pulp Fiction?
  43. 43. Memory vs. Model-based • complete input data is required • pre-computation not possible • does not scale well • high quality of recommendations ! • abstraction (model) of input data • pre-computation (partially) possible (model has to be re-built from time to time) • scales better • abstraction may reduce recommendation quality Social Web 2014, Lora Aroyo!
  44. 44. Social Networks & Interest Similarity • collaborative filtering: ‘neighborhoods’ of people with similar interest & recommending items based on likings in neighborhood • limitations: next to ‘cold start’ and ‘sparsity’ the lack of control (over one’s neighborhood) is also a problem, i.e. cannot add ‘trusted’ people, nor exclude ‘strange’ ones • therefore, interest in ‘social recommenders’, where presence of social connections defines the similarity in interests (e.g. social tagging CiteULike): • does a social connection indicate user interest similarity? • how much users interest similarity depends on the strength of their connection? • is it feasible to use a social network as a personalized recommendation? [Lin & Brusilovsky, Social Social Web 2014, Lora Aroyo! Similarity: The Case of CiteULike, HT’10] Networks and Interest
  45. 45. Conclusions • unilaterally connected pairs have more common items/metadata/tags than non-connected pairs • highest similarity for direct connections - decreasing with the increase of distance between users in SN • reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship • traditional item-level similarity may be less reliable to find similar users in social bookmarking systems • peers connected by self-defined social connections could be a useful source for cross-recommendation Social Web 2014, Lora Aroyo!
  46. 46. Content-based Recommendations • Input: characteristics of items & interests of a user into characteristics of items => Recommend items that feature characteristics which meet the user’s interests • Techniques: • Data mining methods: Cluster items based on their • • characteristics => Infer users’ interests into clusters IR methods: Represent items & users as term vectors => Compute similarity between user profile vector and items Utility-based methods: Utility function that gets an item as input; the parameters of the utility function are customized via preferences of a user Social Web 2014, Lora Aroyo!
  47. 47. Government stops renovation of tower bridge Oct 13th 2011 Tower Bridge is a combined bascule and suspension bridge in London, England, over the River Thames. Category: politics, england Related Twiper news: @bob: Why do they stop to… [more] @mary: London stops reno… [more] Tower Bridge today Under construction Content Features db:Politics db:Sports db:Education db:London db:Tower_Bridge db:Government db:UK Weighting strategy: -  occurrence frequency -  normalize vectors (1-norm ! sum of vector equals 1) based on slides from Fabien Abel 0.2 0 0 0.2 = a 0.4 0.1 0.1
  48. 48. User’s Twitter history RT: Government stops renovation of tower bridge Oct 13th 2011 I am in London at the moment Oct 13th 2011 I am doing sports Oct 12th 2011 User Model db:Politics db:Sports db:Education db:London db:Tower_Bridge db:Government db:UK Weighting strategy: -  occurrence frequency (e.g. smoothened by occurrence time ! recent concepts are more important -  normalize vectors (1-norm ! sum of vector equals 1) based on slides from Fabien Abel 0 0.1 0 0.5 = u 0.2 0.2 0
  49. 49. db:Politics db:Sports db:Education db:London db:Tower_Bridge db:Government db:UK candidate a b 0.2 0 0 0 0 0 0.2 0.8 0.4 0.2 0.1 0 0.1 0 items user c u 0 0 0.5 0.1 0.2 0 0 0.5 0 0.2 0 0.2 0.3 0 cosine similarities Recommendations based on slides from Fabien Abel u a b c 0.67 0.92 0.14 Ranking of recommended items: 1.  b 2.  a 3.  c
  50. 50. RecSys Issues • Cold-start problem (new user problem): no/little data available to infer preferences of new users • Changing User Preferences: user interests may change over time • Sparsity problem (new item problem): item descriptions are sparse, e.g. not many user rated or tagged an item • Lack of Diversity (overfitting): when adapting too strongly to the preferences of users they might see same/similar recommendations • Use the right context: users do things, which might not be relevant for their user model, e.g. try out things, do stuff for other people • Research challenge: right balance between serendipity & personalization • Research challenge: right way to use the influence of recommendations on user’s behavior Social Web 2014, Lora Aroyo!
  51. 51. one machine vs. humans
  52. 52. Hands-on Teaser • Your Facebook Friends’ popularity in a spread sheet • Locations of your Facebook Friends • Tag Cloud of your wall posts image Social Web 2014, Lora Aroyo! source: http://www.flickr.com/photos/bionicteaching/1375254387/
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×