This document summarizes a research paper that explores using Twitter data to improve the ranking of fresh, time-sensitive search results. The researchers extracted various content, aggregate, and Twitter-specific features to train machine learning models to rank regular web pages and Twitter URLs. In experiments, models trained on Twitter features in addition to standard features significantly outperformed models using only standard features in ranking fresh Twitter URLs higher for breaking news queries. Key Twitter features included textual similarity between tweets and URLs, social network properties of users sharing URLs, and metrics of Twitter accounts associated with shortened URLs. The findings suggest Twitter data can help search engines better satisfy users' immediate needs for up-to-date information on recent events.

1. Time is of the Essence : Improving
Recency Ranking Using Twitter Data
(WWW2k10)
Anlei Dong, Ruiqiang Zhang, Pranam Kolari , Jing Bai, Fernando
Diaz, Yi Chang, Zhaohui Zheng
Presenter : ChinHui Chen
(陳晉暉)

2. Author
• Anlei Dong
• Area: Yahoo! Search Sciences
• Location: Yahoo! Labs Silicon Valley

3. Agenda
• Introduction
• Motivation
• Method
• Experiment
• Discussion

4. Introduction
• Recency Sensitive Queries
ex: earthquake -> relevant , timely
• Problem :
1. 0 recall prob
2. user’s need for relevant content is
immediate
• Use Micro-Blogging site

5. Motivation
• General web search algorithm :
– match signals (Language/VSM Model)
– query-independent signals (PageRank)
• But when we issue recency sensitive queries
– Fresh docs may have very few in-links.
– Fresh docs may have very few clicks.

6. Motivation (con’t)
• So how ?

8. Methods
• Learning to rank
• What ?
– 想像是個黑盒子
– 給一堆Query – Doc Pair的feature
– 就會Train一個Model
– 之後遇到未知Query抽Feature即可對Doc排序

9. Feature
Score
Prediction

11. Methods (con’t)
• So the goal is :
Query : 章魚哥
Ranking List
Regular URL
Fresh URL

12. Methods (con’t)
• So the goal is :
Query : 章魚哥
Regular URL Fresh URL
標準答案 測試
Regular URL Fresh URL
Query : 新的Query
抽feature 抽feature
3
5
1
4
4
4
2
5
抽feature 抽feature
?
?
?
?
?
?
?
?
Train Predict

13. Methods(con’t)
• Therefore, the main steps :
1. Extract Features.
2. Apply Learning to rank.

14. Methods – FeatureSet
• Content Features
– Functions of the content of the doc
(ex. query term match, …)
• Aggregate Features
– A doc’s long term popularity, usage
(in-link stat, clicks, PageRank, …)
• Twitter Features

15. Methods – FeatureSet (con’t)
• Content Features
• Aggregate Features
• Twitter Features
– Textual Features
– Social Network Features
– Other Features

16. Methods – FeatureSet (con’t)
• Twitter - Textual Features (q vs url)
Goal : 將URL用text表示, 可算Cosine
Mm post
w URLs
Dm post
v words
Represents a URL by combination
of twitter contents

17. Methods – FeatureSet (con’t)
• Twitter - Textual Features
Goal : 沒有match的term應該懲罰
PS:

18. Methods – FeatureSet (con’t)
• Twitter - Textual Features
Goal : phrase

19. Methods – FeatureSet (con’t)
• 整理 - Textual Features (q vs url)

20. Methods – FeatureSet (con’t)
• Twitter - Social Network Features
A user i posted URL j

21. Methods – FeatureSet (con’t)
• Twitter - Other Features
– The next page

22. Avg stat of users who issued
the tiny url.
First user who issued
the tiny url.
Issued the tiny url with
highest Twitter score

23. Methods – FeatureSet (con’t)
Regular URL Twitter URL
Content Features
(ex. term info )
O O
Aggregate Features
(ex. PageRank)
O poor
Twitter Features No Have

24. Methods – Ranking
• We have feature set now.
• How does Learning To Rank work ??
1. Build Relevance Model (Training)
2. Predict ranking (Prediction)

25. Methods – Ranking(con’t)
• 1. Build Relevance Model :
– Train一個query與url是否相關的Model.
– Straightforward :
• 1. sample query-url pairs (regular + twitter) , and label them.
• 2. train a ranking function
(RankSVM, RankBoost, Gbrank, RankNet,…)
• But … regular url >>>>> twitter url
(twitter feature只有twitter url有, 會被忽略)

26. Methods – Ranking(con’t)
• 1. Build Relevance Model :
– Modified :
M represents ranking function
D represents data set
F represents feature set
TRAIN-MLR (D,F) : train D using F
PREDICT(D,M) : scores dataset, D using M

27. Methods – Ranking(con’t)
• 2. Prediction Ranking
Straightforward :
1. apply step1. model to regular/twitter urls.
2. rank url by sorting scores.
Regular Twitter

28. Methods – Ranking(con’t)

29. Experiments
• Dataset :
– Queries :
• only consider time-sensitive queries in one hour.
– Regular URL :
• in the search engine index during one hour.
– Twitter URL :
• 9-hour period before the query time.

30. Experiments(con’t)
• Label :
– query-url pairs: perfect, excellent, good, fair, bad.
– documents :
提升 fresh
降低 out of date

31. Experiments(con’t)

32. Experiments(con’t)
Evaluation :

33. Experiments(con’t)
• Q :

34. Experiments(con’t)
Mregular = content + aggre
Mcontent = content only
Mtwitter = twitter only
Regular Twitter
Content Feature
(ex. term info )
O O
Aggregate
(ex. PageRank)
O poor
Twitter Features No Have
• Result:

35. Experiments(con’t)
• Feature Importance
Authority & activity
of users

36. Q&A