Set Retrieval 2.0 Daniel Tunkelang Chief Scientist, Endeca

howdy! 1988 – 1992 1993 – 1998 1999 -

1988 – 1992

1993 – 1998

1999 -

overview what’s right with search today? what’s wrong with search today? how do we fix it?

what’s right with search today?

what’s wrong with search today?

how do we fix it?

let’s quickly review some history…

1947: Hans Peter Luhn

1968: Gerald Salton

1972: Karen Spärck Jones

1980s: lots of progress

1990s – 2000s: WWW

today

so, do we all feel lucky?

recession? what recession?

ask the users…

…though they do have complaints 78% wish search engines could read their minds what frustrates users most? 25%: deluge of results 24%: too many paid listings 19%: inability to understand their keywords 19%: disorganized / random results The State of Search Autobytel & Kelton Research, Oct ’07

78% wish search engines could read their minds

what frustrates users most?

25%: deluge of results

24%: too many paid listings

19%: inability to understand their keywords

19%: disorganized / random results

The State of Search

Autobytel & Kelton Research, Oct ’07

web search vs. enterprise search “ Search on the internet is solved. I always find what I need. But why not in the enterprise? Seems like a solution waiting to happen.” - a Fortune 500 CTO

“ Search on the internet is solved. I always find what I need. But why not in the enterprise? Seems like a solution waiting to happen.”

- a Fortune 500 CTO

enterprise users really have complaints Why is Joe the Knowledge Worker so upset? 49%: finding the information needed to do their job is difficult and time consuming 50%: findability within organization worse than on their own consumer-facing site Market IQ Report on Findability AIIM, June ’08

Why is Joe the Knowledge Worker so upset?

49%: finding the information needed to do their job is difficult and time consuming

50%: findability within organization worse than on their own consumer-facing site

Market IQ Report on Findability

AIIM, June ’08

selection bias?

the library and information science critique models relevance is subjective evaluation neglects interactivity tools no support for exploration

models

relevance is subjective

evaluation

neglects interactivity

tools

no support for exploration

the rebuttal "Tell us what to do, and we will do it."

besides, search is 90% solved

we need to call a truce real, effective systems that support interaction cost-effective to evaluate

real, effective systems

that support interaction

cost-effective to evaluate

let’s go back to the 80s for a moment

then vs. now known-item search was an open problem now it’s a commodity library and information science ideas of the 80s ahead of their time now we can find known items let’s tackle more ambitious information needs

known-item search was an open problem

now it’s a commodity

library and information science ideas of the 80s

ahead of their time

now we can find known items

let’s tackle more ambitious information needs

requirements

transparency

control

guidance

set retrieval precision = fraction of retrieved documents that are relevant recall = fraction of relevant documents that are retrieved retrieved documents relevant documents

the classic trade-off recall precision

set retrieval: 2 out of 3

set retrieval 2.0 = set retrieval + guidance Did you mean : guidance Related Searches Guidance Counselor Salary Guidance Counselor Job Description Definition of Guidance Guidance Counseling History of Guidance Counseling Child Guidance Career Guidance What Is the Meaning of Guidance Free Marriage Counseling Problems in Marriage Career Exploration Role of School Counselor

guidance vs. mind reading system can’t read your mind spouse / best friend can’t read your mind sometimes you can’t read your own mind

system can’t read your mind

spouse / best friend can’t read your mind

sometimes you can’t read your own mind

so where does guidance come from?

it’s people!

human-computer information retrieval don’t just guess the user’s intent optimize communication de-emphasize the top ten documents response is a set of documents think beyond single queries support refinement and exploration

don’t just guess the user’s intent

optimize communication

de-emphasize the top ten documents

response is a set of documents

think beyond single queries

support refinement and exploration

hcir cheats the trade-off recall precision

but how do we get there?

set retrieval 2.0 set retrieval that responds to queries with overview of the user's current context organized set of options for exploration contextual summaries of document sets optimize system’s communication with user query refinement options optimize user’s communication with system

set retrieval that responds to queries with

overview of the user's current context

organized set of options for exploration

contextual summaries of document sets

optimize system’s communication with user

query refinement options

optimize user’s communication with system

faceted search guides refinement

showing the right facets: microwaves

showing the right facets: ceiling fans

query-driven clarification before refinement Matching Categories include: Appliances > Small Appliances > Irons & Steamers Appliances > Small Appliances > Microwaves & Steamers Bath > Sauna & Spas > Steamers Kitchen > Bakeware & Cookware > Cookware > Open Stock Pots > Double Boilers & Steamers Kitchen > Small Appliances > Steamers

Matching Categories include:

Appliances > Small Appliances > Irons & Steamers

Appliances > Small Appliances > Microwaves & Steamers

Bath > Sauna & Spas > Steamers

Kitchen > Bakeware & Cookware > Cookware >

Open Stock Pots > Double Boilers & Steamers

Kitchen > Small Appliances > Steamers

results-driven clarification before refinement Search : storage

taxonomies are so 1990s

dynamic topic facet Subject Electronic data processing (1002) Distributed processing (937) Parallel processing (619) Computer networks (562) Fault-tolerant-computing (365) Show more…

Subject

Electronic data processing (1002)

Distributed processing (937)

Parallel processing (619)

Computer networks (562)

Fault-tolerant-computing (365)

Show more…

facets populated using entity extraction apple production

bootstrap on folksonomies

or learn from users

hcir using set retrieval 2.0 emphasize set summaries over ranked lists establish a dialog between the user and the data enable exploration and discovery

emphasize set summaries over ranked lists

establish a dialog between the user and the data

enable exploration and discovery

think outside the (search) box best-first search works for many use cases but not for some of the most valuable ones set retrieval 2.0 = set retrieval + guidance human-computer information retrieval

best-first search works for many use cases

but not for some of the most valuable ones

set retrieval 2.0 = set retrieval + guidance

human-computer information retrieval

thank you communication 1.0 email: [email_address] communication 2.0 blog: http://thenoisychannel.com twitter: http://twitter.com/dtunkelang

communication 1.0

email: [email_address]

communication 2.0

blog: http://thenoisychannel.com

twitter: http://twitter.com/dtunkelang