UMAP 2011: Analyzing User Modeling on Twitter for Personalized News Recommendations

Analyzing User Modeling on Twitter for Personalized News Recommendations
UMAP, Girona, July 13, 2011
Fabian Abel, QiGao, Geert-Jan Houben, Ke Tao
Web Information Systems, TU Delft

The Social Web
Help me to tackle the information overload!
Who is this? What are his
personal demands? How
can we make him happy?
Recommend me news articles that now interest me!
Help me to find interesting (social) media!
Give me personalized support when I do my online training!
Personalize my Web experience!
Do not bother me with advertisements that are not interesting for me!

What we do: Science and Engineering for the Personal Web
domains: news social mediacultural heritage public datae-learning
Personalized
Recommendations
Personalized Search
Adaptive Systems
Analysis and
User Modeling
Semantic Enrichment, Linkage and Alignment
user/usage data
Social Web

User Modeling Challenge
Personalized News Recommender
I want my personalized news recommendations!
Profile
Analysis and
User Modeling
?
(How) can we infer a Twitter-based user profile that supports the news recommender?
Semantic Enrichment, Linkage and Alignment

1. Temporal Constraints
time period
temporal patterns
hashtag-based
entity-based
topic-based
2. Profile Type
tweet-based
further enrichment
3. Semantic Enrichment
concept frequency
4. Weighting Scheme
User Modeling Framework
Building Blocks for generating valuable user profiles

User Modeling Building Blocks
(a) time period
1. Which tweets of the user should be analyzed?
?
(b) temporal patterns
Profile?
concept weight
end
start
weekends
Morning:
Afternoon:
Night:
time
June 27
July 4
July 11

Francesca
Schiavone
T
Sport
2. Profile Type
Francesca Schiavone won French Open #fo2010
Francesca Schiavone French Open
?
#fo2010
Profile?
concept weight
#
hashtag-based
entity-based
French
Open
T
topic-based
#
fo2010
2. What type of concepts should represent “interests”?
time
June 27
July 4
July 11

(a) tweet-based
Francesca
Schiavone
2. Profile Type
Francesca wins French Open
Thirty in women's
tennis is primordially
old, an age when
agility and desire
recedes as the …
Francesca Schiavone
Francesca Schiavone won! http://bit.ly/2f4t7a
Profile?
concept weight
French Open
Tennis
French
Open
(b) further enrichment
Tennis
3. Further enrich the semantics of tweets?

2. Profile Type
?
Francesca Schiavone
4
4. How to weight the concepts?
Profile?
concept weight
3
French Open
6
Tennis
Concept frequency
4. Weighting Scheme
weight(FrancescaSchiavone)
weight(French Open)
weight(Tennis)
time
June 27
July 4
July 11

time period
temporal patterns
hashtag-based
entity-based
topic-based
2. Profile Type
tweet-based
further enrichment
concept frequency
4. Weighting Scheme

time period
temporal patterns
hashtag-based
entity-based
topic-based
2. Profile Type
tweet-based
further enrichment
concept frequency
4. Weighting Scheme
Analysis
How do the user modeling building blocks impact the (temporal) characteristics of Twitter-based user profiles?

Dataset
more than:
20,000
Twitter users
2
months
10,000,000
WikiLeaks founder, Julian Assange, under arrest in London
tweets
75,000
news articles
time
Dec 15
Jan 15
Nov 15

Size of user profiles
Profile Type
~5% of the users do not make use of hashtags
hashtag-based profiles are empty
entity-based
Entity-based user modeling succeeds for 100% of the users
topic-based
hashtag-based

Semantic Enrichment
More distinct topics per profile
further enrichment
(e.g. exploiting links)
further enrichment
(e.g. exploiting links)
More distinct entities per profile
Exploiting external resources allows for significantly richer user profiles (quantitatively)
Tweet-based
Tweet-based
entity-based user profiles
topic-based user profiles
Impact of Semantic Enrichment

User Profiles change over time
Temporal Constraints
Hashtag-based profiles change stronger than entity-based and topic-based profiles
d1-distance:
difference between current profile and past profile
Example:
#
old
new
?
music
The older the profile the more it differs from the current profile
tennis
football
T

Temporal patterns of user profiles
2
1. Weekend profiles differ significantly from weekday profiles
2. the difference is stronger than between day and night profiles
weekday vs. weekend profiles
d1(pweekday, pweekend)
day vs. night profiles
d1(pday, pnight)
topic-based user profiles

Observations
Semantic enrichment allows for richer user profiles
Profiles change over time: fresh profiles seem to better reflect current user demands
Temporal patterns: weekend profiles differ significantly form weekday profiles

time period
temporal patterns
hashtag-based
entity-based
topic-based
2. Profile Type
tweet-based
further enrichment
concept frequency
4. Weighting Scheme
Evaluation
How do the user modeling building blocks impact the quality of Twitter-based profiles for personalized news recommendations?
And can we benefit from the findings of the analysis to improve recommendations?

Twitter-based Profiles for Personalization
Task: Recommending news articles (= tweets with URLs pointing to news articles)
Recommender algorithm: cosine similarity between user profile and tweets
Ground truth: re-tweets of users
Candidate items: news article tweets posted during evaluation period
5.5 relevant tweets per user
5529 candidate news articles
Recommendations = ?
P(u)= ?
time
1 week

Profile Type
Overview: Performance of User Modeling strategies
Topic-based strategy improves S@10 significantly
#
Entity-based strategy improves the recommendation quality significantly (MRR & S@10)
T

Impact of Semantic Enrichment
Semantic Enrichment
T
Tweet-based
Further enrichment
Further semantic enrichment (exploiting links) improves the quality of the Twitter-based profiles!

Impact of temporal characteristics
startcomplete
startfresh
end
Adapting to temporal context helps?
Selection of temporal constraints depends on type of user profile.
Topic-based profiles:
adapting to temporal
context is beneficial
Entity-based profiles:
long-term profiles
perform better
Recommendations = ?
yes
T
time
no
complete: 2 months
fresh: 2 weeks
end
start
yes
weekends
T
Recommendations = ?
no
time

Conclusions and Future Work
What we did: Twitter-based User Modeling for Recommending News Articles
Analysis:
Semantic enrichment results in richer user profiles (quantitative)
User interest profiles change over time (hashtag-based stronger than others)
Weekend/weekday pattern more significant than day/night pattern
Evaluation:
Best user modeling strategy: Entity-based > topic-based > hashtag-based
Semantic enrichment improves recommendation quality
Adapting to temporal context helps for topic-based strategy
Future work: for what type of personalization tasks can we exploit what type of Twitter profiles?

Thank you!
Fabian Abel, QiGao, Geert-Jan Houben, Ke Tao
Twitter: @persweb
http://persweb.org/
http://u-sem.org/

Research Questions
What type of user interest profiles can we infer from Twitter activities?
Can we exploit Twitter-based profiles for personalizing users’ Social Web experience?
Personalized news
recommendations
in time:
interest
twitter
Good Morning! #tooearly
?
?
I like this http://bit.ly/5d4r2t
Why do people now blame Julian Assange?
time
time
Ajax deserves it! #sport

Analyzing Twitter-based Profiles for Personalized News Recommendations (in time)
News Recommendations in time:
Interests:
Tennis
Football
Francesca Schiavone is great!
Thirty in women's
tennis is primordially
old, an age when
agility and desire
recedes as the
next wave of younger/faster/stronger players encroaches. It's uncommon for any athlete to have a breakthrough season at 30, but it's exceedingly…
Ajax gives
De Jong
a break
Ajax manager Frank de
Boer announced that…
Personalized news recommendations
interest
interest
I like this http://bit.ly/4Gfd2
Analysis and
User Modeling
time
time
topic:Tennis
Semantic Enrichment, Linkage, Alignment
dbpedia:Schiavone
Nice, thank you!
oc:Sports
event:FrenchOpen
tweets

User Modeling Challenge
Wednesday, July 13th 2011, 9:10am
Personalized news recommender
Profile?
I want my personalized news recommendations!
?
(How) can we infer a Twitter-based user profile that supports the news recommender?

Bob tweets…
Why do people now blame Julian Assange?
Ajax deserves it! #sport
Good Morning! #tooearly
I like this http://bit.ly/5d4r2t
time
Fr, 6am
Fr, 3pm
Fr, 8pm
Sa, 5pm
People publish more than 60 million tweets per day!

UMAP 2011: Analyzing User Modeling on Twitter for Personalized News Recommendations

by Web Information Systems, TU Delft

Accessibility

Categories

Upload Details

Usage Rights

Statistics