Lecture 4: Social Web Personalization (2012)

Social Web
Lecture IV: Personalization on the Social Web
(some slides were adopted from Fabian Abel)

Lora Aroyo
The Network Institute
VU University Amsterdam

Monday, February 27, 12

Personalization &
Social Web
• Applications on the Social Web use web data [last
week] & are ‘social’
• To design ‘social’ functionality we need to understand how
out of the data the application can provide relevant
information (what users perceive as relevant)
• Therefore we need to understand
• how good personalization (recommenders) are
• how good the user models are they are based on
• In this lecture, we consider theory & techniques for
how to design and evaluate recommenders and user
models (for use in SW applications)

total transparency - is it desired?


User Modeling
How to infer & represent user
information that supports a given
application or context?

http://farm5.staticﬂickr.com/4038/4553496383_5b6a5f1485_o.jpg

User Modeling
Challenge
• Application has to obtain,
understand & exploit information
about the user
• Information (need & context) about
user
• Inferring information about user &
representing it so that it can be
consumed by the application
• Data relevant for inferring
information about user


User & Usage Data is
everywhere
• People leave traces on the Web and on their computers:
• Usage data, e.g., query logs, click-through-data
• Social data, e.g., tags, (micro-)blog posts, comments,
bookmarks, friend connections
• Documents, e.g., pictures, videos
• Personal data, e.g., afﬁliations, locations
• Products, applications, services - bought, used, installed
• Not only a user’s behavior, but also interactions of other users
• “people can make statements about me”, “people who are
similar to me can reveal information about me” --> “social
learning” collaborative recommender systems


UM: Basic Concepts

• User Profile: a data structure that represents a characterization of
a user at a particular moment of time represents what, from
a given (system) perspective, there is to know about a user. The data
in the profile can be explicitly given by the user or derived by the
system
• User Model: contains the definitions & rules for the interpretation
of observations about the user and about the translation of that
interpretation into the characteristics in a user profile user
model is the recipe for obtaining and interpreting user profiles
• User Modeling: the process of representing the user


User Modeling
Approaches
• Overlay User Modeling: describe user characteristics, e.g.
“knowledge of a user”, “interests of a user” with respect to
“ideal” characteristics
• Customizing: user explicitly provides & adjusts elements of the
user profile
• User model elicitation: ask & observe the user; learn & improve
user profile successively “interactive user modeling”
• Stereotyping: stereotypical characteristics to describe a user
• User Relevance Modeling: learn/infer probabilities that a given
item or concept is relevant for a user

Related scientific conference: http://umap2011.org/ Related journal: http:/umuai.org/

Which
approach suits
best the
conditions of
applications?

http://farm7.staticﬂickr.com/6240/6346803873_e756dd9bae_b.jpg

Overlay User Models

• among the oldest user models
• used for modeling student
knowledge
• the user is typically characterized
in terms of domain concepts &
hypotheses of the user’s knowledge
about these concepts in relation
to an (ideal) expert’s knowledge
• concept-value pairs


User Model Elicitation
• Ask the user explicitly learn
• NLP, intelligent dialogues
• Bayesian networks, Hidden Markov models
• Observe the user learn
• Logs, machine learning
• Clustering, classiﬁcation, data mining

• Interactive user modeling: mixture of direct inputs of a user,
observations and inferences


http://hunch.com

Stereotyping

• set of characteristics (e.g. attribute-value
pairs) that describe a group of users.
• user is not assigned to a single stereotype -
user proﬁle can feature characteristics of
several different stereotypes


Why are
stereotypes
useful?

http://farm1.staticﬂickr.com/155/413650229_31ef379b0b_b.jpg

Can we use
Social Web
data for
user
modeling?


Can we infer a Twitter-
based user proﬁle? *
Personalized News
Recommender

Profile I want my

?
personalized news
recommendations!

User Modeling
(4 building blocks)

Semantic Enrichment,
Linkage and Alignment

* Example from Abel et al. (2011)

User Modeling Building
Blocks
1. Temporal
Constraints

Profile?
(a) time period
1. Which tweets of concept weight

the user should be
analyzed? ? (b) temporal patterns

start weekends end

Morning:
Afternoon:
time
Night:
June 27 July 4 July 11


Blocks
1. Temporal
Constraints
Francesca T Sport
Schiavone 2. Profile
Profile? Type
concept weight

?
Francesca Schiavone won # hashtag-based
French Open #fo2010 entity-based

T topic-based
French
Open # fo2010
2. What type of concepts
should represent “interests”?

time

Blocks
1. Temporal
Constraints
Francesca (a) tweet-based
Schiavone 2. Profile
Profile? Type
concept weight
Francesca
Francesca Schiavone won! Schiavone
3. Semantic
http://bit.ly/2f4t7a French Open
Enrichment
Tennis

Francesca wins French Open
French
Thirty in women's Open (b) further enrichment
tennis is primordially
old, an age when
agility and desire Tennis
recedes as the …

3. Further enrich the semantics of tweets?

Blocks
1. Temporal
Constraints

Profile? 2. Profile
concept weight Type
4. How to weight the Francesca
4

?
Schiavone
3. Semantic
concepts? French Open 3
Enrichment
Tennis 6
Concept frequency (TF)
4. Weighting
TFxIDF weight(French Open) Scheme
weight(Francesca
Time-sensitive Schiavone)
weight(Tennis)

time

Observations
• Profile characteristics:
• Semantic enrichment solves sparsity problems
• Profiles change over time: fresh profiles reflect better current
user demands
• Temporal patterns: weekend profiles differ significantly from
weekday profiles
• Impact on news recommendations:
• The more fine-grained the concepts the better the
recommendation performance: entity-based > topic-based >
hashtag-based
• Semantic enrichment improves recommendation quality
• Time-sensitivity (adapting to trends) improves performance


User Modeling
it is not about putting everything in a user proﬁle
it is about making the right choices


User Adaptation
Knowing the user - this knowledge - can be applied to adapt
a system or interface to the user
to improve the system functionality and user experience


User-Adaptive Systems
user
profile

user modeling profile analysis

observations,
data and adaptation
information decisions
about user

A. Jameson. Adaptive interfaces and agents. The HCI handbook: fundamentals,
evolving technologies and emerging applications, pp. 305–330, 2003.

Last.fm: adapts to your
music taste
user profile
interests in
genres,
artists, tags

user modeling compare profile
(infer current with possible next
musical taste) songs to play

history of next song to
songs, like, be played
ban, pause,
skip

Google:
Adaptive Search

http://www.google.com/goodtoknow/

Issues in User-Adaptive
Systems
• Overﬁtting, “bubble effects”, loss of serendipity problem:
• systems may adapt too strongly to the interests/behavior
• e.g., an adaptive radio station may always play the same or
very similar songs
• We search for the right balance between novelty and
relevance for the user
• “Lost in Hyperspace” problem:
• when adapting the navigation – i.e. the links on which
users can click to ﬁnd/access information
• e.g., re-ordering/hiding of menu items may lead to
confusion

What is good
user modeling &
personalization?

http://www.ﬂickr.com/photos/bellarosebyliz/4729613108

Success perspectives

• From the consumer perspective of an
adaptive system:
Adaptive system maximizes hard to measure/obtain
satisfaction of the user

• From the provider perspective of an
adaptive system:
influence of UM &
Adaptive system maximizes personalization may be
the profit hard to measure/obtain


Evaluation Strategies
• User studies: Clean-room study: ask/observe (selected) people
whether you did a good job
• Log analysis: Analyze (click) data and infer whether you did a
good job, e.g., cross-validation by “Leave-one-out”
• Evaluation of user modeling:
• measure quality of profiles directly, e.g. measure overlap with
existing (true) profiles, or let people judge the quality of
the generated user profiles
• measure quality of application that exploits the user profile,
e.g., apply user modeling strategies in a recommender
system


Possible Metrics
• The usual IR metrics:
• Precision: fraction of retrieved items that are relevant
• Recall: fraction of relevant items that have been
retrieved
• F-Measure: (harmonic) mean of precision and recall
• Metrics for evaluating recommendation (rankings):
• Mean Reciprocal Rank (MRR) of ﬁrst relevant item
• Success@k: probability that a relevant item occurs performance
within the top k strategy X
baseline

• If a true ranking is given: rank correlations
• Precision@k, Recall@k & F-Measure@k
• Metrics for evaluating prediction of user preferences:
• MAE = Mean Absolute Error runs
Is strategy X better than the baseline?
• True/False Positives/Negatives


Example Evaluation
• [Rae et al.] shows a typical example of how to investigate and evaluate a
proposal for improving (tag) recommendations (using social networks)
• Task: test how well the different strategies (here different tag contexts) can be
used for tag prediction/recommendation
• Steps:
1. Gather a dataset of tag data part of which can be used as input and aim to
test the recommendation on the remaining tag data
2. Use the input data and calculate for the different strategies the predictions
3. Measure the performance using standard (IR) metrics: Precision of the
top 5 recommended tags (P@5), Mean Reciprocal Rank (MRR), Mean
Average Precision (MAP)
4. Test the results for statistical signiﬁcance using Student’s T-test, relative
to the baseline (e.g. existing approach, competitive approach)

[Rae et al. Improving Tag Recommendations Using Social Networks, RIAO’10]]


Example Evaluation
• [Guy et al.] shows another example of a similar evaluation
approach
• Here, the different strategies differ in the way people and tags
are used in the strategies: with these tag-based systems, there are
complex relationships between users, tags and items, and
strategies aim to ﬁnd the relevant aspects of these relationships
for modeling and recommendation
• Here, their baseline is the strategy of the ‘most popular’
tags: this is a strategy often used, to compare the globally most
popular tags to the tags predicted by a particular personalization
strategy, thus investigating whether the personalization is worth
the effort and is able to outperform the easily available baseline.

[Guy et al. Social Media Recommendation based on People and Tags, SIGIR’10]]


user interactions (level & type) instead of general social
context - better for recommendations?

does hybrid always work worse?

Recommendation
Systems
Predict items that are relevant/useful/interesting
(and to what extent)
for given user (in a given context)

it’s often a ranking task


http://www.wired.com/magazine/2011/11/mf_artsy/all/1

Collaborative Filtering
• Memory-based: User-Item matrix: ratings/preferences of users => compute
similarity between users & recommend items of similar users
• Model-based: Item-Item matrix: similarity (e.g. based on user ratings) between
items => recommend items that are similar to the ones the user likes
• Model-based: Clustering: cluster users according to their preferences =>
recommend items of users that belong to the same cluster
• Model-based: Bayesian networks: P(u likes item B | u likes item A) = how
likely is it that a user, who likes item A, will like item B learn probabilities
from user ratings/preferences
• Others: rule-based, other data mining techniques
•
u1 likes likes likes ! u1 likes
u2
Pulp Fiction?


Memory vs. Model-based
• complete input data is • abstraction (model) of input
required data

• pre-computation not possible • pre-computation (partially)
possible (model has to be re-
• does not scale well (“tricks” built from time to time)
are needed)
• scales better
• high quality of
recommendations • abstraction may reduce
recommendation quality


Social Networks &
Interest Similarity
• collaborative ﬁltering: ‘neighborhoods’ of people with similar
interest & recommending items based on likings in neighborhood
• limitations: next to ‘cold start’ and ‘sparsity’ the lack of control
(over one’s neighborhood) is also a problem, i.e. cannot add ‘trusted’
people, nor exclude ‘strange’ ones
• therefore, interest in ‘social recommenders’, where presence of social
connections deﬁnes the similarity in interests (e.g. social tagging CiteULike):
• does a social connection indicate user interest similarity?
• how much users interest similarity depends on the strength of their
connection?
• is it feasible to use a social network as a personalized
recommendation?

[Lin & Brusilovsky, Social Networks and Interest Similarity: The Case of CiteULike, HT’10]

Conclusions
• pairs unilaterally connected have more common information
items, metadata, and tags than non-connected pairs.
• the similarity was largest for direct connections and
decreased with the increase of distance between users in the
social networks
• users involved in a reciprocal relationship exhibited
significantly larger similarity than users in a unidirectional
relationship

• traditional item-level similarity may be less reliable way to find similar
users in social bookmarking systems.
• items collections of peers connected by self-defined social
connections could be a useful source for cross-recommendation.


Social recommendations on conﬂicting groups

are all interests of our friends relevant? Is it
application generic?

Content-based
Recommendations
• Input: characteristics of items & interests of a user into
characteristics of items => Recommend items that feature
characteristics which meet the user’s interests
• Techniques:
• Data mining methods: Cluster items based on their
characteristics => Infer users’ interests into clusters
• IR methods: Represent items & users as term vectors =>
Compute similarity between user proﬁle vector and items
• Utility-based methods: Utility function that gets an item as
input; the parameters of the utility function are customized
via preferences of a user


Government stops Tower Bridge
is a combined bascule and suspension
renovation of tower bridge in London, England, over the
River Thames.
bridge Oct 13th 2011
Category: politics, england
Related Twiper news:
@bob: Why do they stop to… [more]
@mary: London stops reno… [more]

Tower Bridge today Under construction

db:Politics 0.2
Content db:Sports
db:Education
0
0
Features db:London 0.2 = a
db:Tower_Bridge 0.4
db:Government 0.1
Weighting strategy: db:UK 0.1
-  occurrence frequency
-  normalize vectors (1-norm ! sum of vector equals 1)

User’s Twitter
history
RT: Government stops User
renovation of tower
bridge Oct 13th 2011
Model
I am in London at the
moment Oct 13th 2011
db:Politics 0
I am doing sports db:Sports 0.1
Oct 12th 2011 db:Education 0
db:London 0.5 = u
db:Tower_Bridge 0.2
db:Government 0.2
Weighting strategy:
db:UK 0
-  occurrence frequency (e.g. smoothened by occurrence time ! recent
concepts are more important
-  normalize vectors (1-norm ! sum of vector equals 1)

candidate items user
a b c u
db:Politics 0.2 0 0 0
db:Sports 0 0 0.5 0.1
db:Education 0 0 0.2 0
db:London 0.2 0.8 0 0.5
db:Tower_Bridge 0.4 0.2 0 0.2
db:Government 0.1 0 0 0.2
db:UK 0.1 0 0.3 0
cosine a b c
similarities
u 0.67 0.92 0.14
Recommendations
Ranking of recommended items:
1.  b
2.  a
3.  c

RecSys Problems
• Cold-start problem / New User problem: no or little data is available to infer
the preferences of new users
• Changing User Preferences: user interests may change over time
• Sparsity problem / New Item problem: item descriptions are sparse, e.g. not
many user rated or tagged an item
• Lack of Diversity / Overfitting: for many applications good recommendations
should be relevant and new to the user (exceptions: predicting re-visiting
behavior, etc.). When adapting too strongly to the preferences of a user, the
user might see again and again same/similar recommendations
• Use the right context: users do lots of things, which might not be relevant for
their user model, e.g. try out things, do stuff for other people
• Research challenge: find right balance between serendipity & personalization
• Research challenge: find right way to use the influence of the
recommendations on the user’s behavior


What is true personalization?


Hands-on Teaser

• Build your own recommender
system 101
• Recommend pages on
del.icio.us
• Recommend pages to your
Facebook friends

image source: http://www.ﬂickr.com/photos/bionicteaching/1375254387/


Lecture 4: Social Web Personalization (2012)

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (8)

Similar to Lecture 4: Social Web Personalization (2012)

Similar to Lecture 4: Social Web Personalization (2012) (20)

More from Lora Aroyo

More from Lora Aroyo (20)

Recently uploaded

Recently uploaded (20)

Lecture 4: Social Web Personalization (2012)