Reconsidering Relevance We've become complacent about relevance. The overwhelming success of web search engines has lulled even information retrieval (IR) researchers to expect only incremental improvements in relevance in the near future. And beyond web search, there are still broad search problems where relevance still feels hopelessly like the pre-Google web. But even some of the most basic IR questions about relevance are unresolved. We take for granted the very idea that a computer can determine which documents are relevant to a person's needs. And we still rely on two-word queries (on average) to communicate a user's information need. But this approach is a contrivance; in reality, we need to think of information-seeking as a problem of optimizing the communication between people and machines. We can do better. In fact, there are a variety of ongoing efforts to do so, often under the banners of "interactive information retrieval", "exploratory search", and "human computer information retrieval". In this talk, I'll discuss these initiatives and how they are helping to move "relevance" beyond today's outdated assumptions. About the Speaker Daniel Tunkelang is co-founder and Chief Scientist at Endeca, a leading provider of enterprise information access solutions. He leads Endeca's efforts to develop features and capabilities that emphasize user interaction. Daniel has spearheaded the annual Workshops on Human Computer Information Retrieval (HCIR) and is organizing the Industry Track for SIGIR '09. Daniel also publishes The Noisy Channel, a widely read and cited blog that focuses on how people interact with information. Daniel holds undergraduate degrees in mathematics and computer science from the Massachusetts Institute of Technology, with a minor in psychology. He completed a PhD at Carnegie Mellon University for his work on information visualization. His work previous to Endeca includes stints at the IBM T. J. Watson Research Center and AT&T Labs,

1. Reconsidering Relevance Daniel Tunkelang Chief Scientist, Endeca

4. but first let’s set the stage

5. iconic businesses of the 20 th and 21 st centuries I’m Feeling Lucky

6. process and scale orchestration

7. but there’s a dark side

8. users are satisfied

14. we need more definitions

18. a relevance-centric approach information Need query select from results rank using IR model USER: SYSTEM: tf-idf PageRank

23. queries are misinterpreted Results 1-10 out of about 344,000,000 for ir

24. ranked lists are inefficient

26. can we do better?

29. human computer information retrieval

31. google him!

32. google scholar him?

33. rexa him?

34. getting better

35. hcir-inspired interface

36. tags provide summarization and guidance

37. my information need evolves as i learn

38. hcir – implementing the vision

39. scatter/gather: a search for “star”

40. faceted search

42. showing the right facets: microwaves

43. showing the right facets: ceiling fans

45. results-driven clarification before refinement Search : storage

46. crowd-sourcing to tag documents

47. hcir cheats the precision / recall trade-off recall precision

51. one more thing …

52. “ Google's mission is to organize the world's information and make it universally accessible and useful.”

53. organizer or referee?