Set Retrieval 2.0 Daniel Tunkelang Chief Scientist, Endeca

let’s quickly review some history…

1947: Hans Peter Luhn

1968: Gerald Salton

1972: Karen Spärck Jones

1980s: lots of progress

1990s – 2000s: WWW

today

so, do we all feel lucky?

recession? what recession?

ask the users…

selection bias?

the rebuttal "Tell us what to do, and we will do it."

besides, search is 90% solved

let’s go back to the 80s for a moment

requirements

transparency

control

guidance

set retrieval precision = fraction of retrieved documents that are relevant recall = fraction of relevant documents that are retrieved retrieved documents relevant documents

the classic trade-off recall precision

set retrieval: 2 out of 3

set retrieval 2.0 = set retrieval + guidance Did you mean : guidance Related Searches Guidance Counselor Salary Guidance Counselor Job Description Definition of Guidance Guidance Counseling History of Guidance Counseling Child Guidance Career Guidance What Is the Meaning of Guidance Free Marriage Counseling Problems in Marriage Career Exploration Role of School Counselor

so where does guidance come from?

it’s people!

hcir cheats the trade-off recall precision

but how do we get there?

faceted search guides refinement

showing the right facets: microwaves

showing the right facets: ceiling fans

results-driven clarification before refinement Search : storage

taxonomies are so 1990s

facets populated using entity extraction apple production

bootstrap on folksonomies

or learn from users