Retrieval and Feedback Models for Blog Feed Search

Retrieval and Feedback Models for Blog Feed Search SIGIR 2008 Singapore Jonathan Elsas, Jaime Arguello, Jamie Callan & Jaime Carbonell LTI/SCS/CMU

Outline
The task
Overview of Blogs & Blog Search
Challenges in Blog Search
Our approach
Retrieval Models
Query Expansion Models
Conclusion

Background

What is a Blog?

What is a Feed? <xml> <feed> <entry> <author>Peter …</> <title>Good, Evil…</> <content>I’ve said…</> </entry> <entry> <author>Peter …</> <title>Agreeing…</> <content>Some peo…</> </entry> …

Blog-Feed Correspondence Blog Feed Post Entry HTML XML

Why are Blogs important?
Technorati currently tracking > 112.8 Million Blogs > 175,000 new Blogs per day > 1.6 Million posts per day
[http://www.technorati.com/about/]

The Task

Feed Search at TREC
Ranking Blogs/Feeds (collections of posts) in response to a user’s query, [X]
“ A relevant feed should have a principle and recurring interest in X ”
— TREC 2007 Blog Track
(a.k.a. Blog Distillation)

Feed Search at TREC
[Gardening]
[Apple iPod]
[Violence in Sudan]
[Gun Control]
[Food]
[Wine]
Represent Ongoing Information Needs Frequently Very General

Challenges in Feed Search

A feed is a collection of documents
entries time feed

A feed is a collection of documents
How does relevance at the entry level correspond to relevance at the feed level?
Challenges in Feed Search entries time feed

2. Even a topical feed is topically diverse
time Space Exploration topic NASA China’s plans for the moon shuttle launch My dog Mars rover Boeing

2. Even a topical feed is topically diverse
Can we favor entries close to the central topic of the feed?
Space Exploration time topic

3. Feeds are noisy
Spam blogs, Spam & off topic comments
time

4. General & Ongoing Information Needs
[Mac] [Music] [Food] [Wine] … post regularly about new products , features , or application software of Apple Mac computers. … describing songs , biographies of musicians, musical styles and their influences of music on people are discussed. … such as tastings , reviews , food matching or pairing , and oenophile news and events . … describing experiences eating cuisines, culinary delights , recipes , nutrition plans .

Our Approach

Feeds:
Topically Diverse
Noisy
Collections
Information Needs: General & Ongoing Challenges Our Approach Retrieval Models Feedback Models

Retrieval Models
Challenge: ranking topically diverse collections
Representation: feed vs. entry
Model topical relationship between entries

Large Document (Feed) Model [Q] <?xml… … </…> `<?xml… … </…> <?xml… … </…> <?xml… <feed> <entry> <entry> <entry> <entry> <entry> … </…> <?xml… … </…> <?xml… … </…> <?xml… … </…> <?xml… <feed> <entry> <entry> <entry> <entry> <entry> … </…> Feed Document Collection Ranked Feeds Rank by Indri’s standard retrieval model [Metzler and Croft, 2004; 2005]

Large Document (Feed) Model
Advantages:
A straightforward application of existing retrieval techniques
Potential Pitfalls:
Large entries dominate a feed’s language model
Ignores relationship among entries
Feed Entry E E Entry Entry E

Small Document (Entry) Model Ranked Entries [Q] <entry> <entry> <entry> <entry> <?xml… <entry> Entry Document Collection <entry> <entry> <entry> <entry> <?xml… <entry> <entry> <entry> <entry> <entry> <?xml… <entry> <entry> <entry> <entry> <entry> <?xml… <entry> <entry> <entry> <entry> <entry> <?xml… <entry> <entry> <entry> <entry> <entry> <?xml… <entry> <entry> <entry> <entry> <entry> <?xml… <entry> Ranked Feeds document = entry Apply some rank aggregation function Rank By

Small Document (Entry) Model
Query Likelihood
Entry Centrality
Feed Prior: favors longer feeds
ReDDE Federated Search Algortihm [Si & Callan, 2003]

Entry Centrality
Uniform :
Geometric Mean :
time topic

Small Document (Entry) Model
Advantages:
Controls for differing entry length
Models topical relationship among entries
Disadvantages:
Centrality computation is slow(er)
Not only improves speed, Also performance Q

Retrieval Model Results

Retrieval Model Results
45 Queries from the TREC 2007 Blog Distillation Task
BLOG06 test collection, XML feeds only
5-Fold Cross Validation for all retrieval model smoothing parameters

Retrieval Model Results Mean Average Precision Large Document (Feed) Model Small Document (Entry) Models

Retrieval Model Results Mean Average Precision Uniform Log(Feed Length) Uniform Log Prior Map 0.188

Retrieval Model Results Mean Average Precision Uniform Log(Feed Length) Uniform n/a

Feedback Models
Challenge: Noisy collection with general & ongoing information needs
Use a cleaner external collection for query expansion (Wikipedia)
With an expansion technique designed to identify multiple query facets

Query Expansion (PRF) [Q] BLOG06 Collection Related Terms from top K documents [Q + Terms] [Lavrenko & Croft, 2001]

Query Expansion Example
Ideal
digital photography
depth of field
photographic film
photojournalism
cinematography
[Photography] PRF photography nude erotic art girl free teen fashion women

Feedback Model Results Mean Average Precision None PRF

Query Expansion (Wikipedia PRF) [Q] BLOG06 Collection [Q + Terms] [Lavrenko & Croft, 2001] Wikipedia [Diaz & Metzler, 2006] Related Terms from top K documents

Ideal
digital photography
depth of field
photographic film
photojournalism
cinematography
[Photography] PRF photography nude erotic art girl free teen fashion women Wikipedia PRF photography director special film art camera music cinematographer photographic

Feedback Model Results Mean Average Precision None PRF Wiki. PRF

Query Expansion (Wikipedia Link) [Q] BLOG06 Collection [Q + Terms] Wikipedia Related Terms from link structure

Wikipedia Link-Based Query Expansion

Wikipedia Link-Based Expansion Wikipedia … Q

Wikipedia Link-Based Expansion … Relevance Set, Top R = 100 Working Set, Top W = 1000 Q Wikipedia

Wikipedia Link-Based Expansion … Wikipedia Q Relevance Set, Top R = 100 Working Set, Top W = 1000

Wikipedia Link-Based Expansion Relevance Set, Top R = 100 Working Set, Top W = 1000 … Wikipedia Extract anchor text from Working Set that link to the Relevance Set . Q

Wikipedia Link-Based Expansion Relevance Set, Top R = 500 Working Set, Top W = 1000 … Wikipedia Extract anchor text from Working Set that link to the Relevance Set . Q Combines relevance and popularity Relevance: An anchor phrase that links to a high ranked article gets a high score Popularity: An anchor phrase that links many times to a mid-ranked articles also gets high score

Wikipedia Link-Based
photography
photographer
digital photography
photographic
depth of field
feature photography
film
photographic film
photojournalism
[Photography] PRF photography nude erotic art girl free teen fashion women Ideal digital photography depth of field photographic film photojournalism cinematography

Feedback Model Results Mean Average Precision None PRF Wiki. PRF Wiki. Link

Conclusion
Feed Search Challenges:
Feeds are topically diverse, noisy collections
Ranked against ongoing & general information needs
Novel Retrieval Models:
Ranking collections, sensitive to topical relationship among entries
Novel Feedback Models:
Discover multiple query facets & robust to collection noise

Thank You! Student Travel Grant funding from: ACM SIGIR, Amit Singhal, Microsoft Research

Entry Centrality GM Derivation where Entry Generation Likelihood: |E|

Query Expansion Examples
Wikipedia Expansion
Music
Folk music
Electronic music
Folk
Music video
World music
Ambient
Electronic
Country music
[Music] PRF Music Country Download Free MP3 Mp3andmore Lyric Listen Song

Wikipedia Expansion
scotland
scottish parliament
scottish
scottish national party
wars of scottish independence
scottish independence
william wallace
glasgow
scottish socialist party
[Scottish Independence] PRF scotland independence party convention politics snp national people scot

Wikipedia Expansion
machine learning
learning
artificial intelligence
turing machine
machine gun
neural network
support vector machine
supervised learning
artificial neural network
[Machine Learning] PRF learn machine credit card karaoke journal sex model sew

Query Generality Characteristics
Query Length:
BLOG: 1.9 words
TB04: 3.2 words
TB05: 3.0 words
ODP Depth
BLOG: 4.7 levels
TB04: 5.2 levels
TB05: 5.3 levels

Relevance Set Cohesiveness … Relevance Set, Top R = 100 Wikipedia Cohesiveness = | L in | | L in U L out |

Relevant Set Cohesiveness

Is it the Queries?
Feed Search Queries
≠
TB Adhoc Queries
But, none of these measures predict whether wikipedia expansions helps…

http://windowoffice.tumblr.com	33
http://www.searchenginecaffe.com	16
http://www.slideshare.net	3

Retrieval and Feedback Models for Blog Feed Search

by jelsas on Jul 22, 2008

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 52

Statistics

Retrieval and Feedback Models for Blog Feed Search — Presentation Transcript