The document discusses various aspects of the TREC anniversary, touching on themes like 'deep personalization' in search and the concept of 'slow search.' It highlights significant elements such as the importance of user context, access to personal data, and challenges related to search log data collection. Additionally, it emphasizes the need for balance between reproducibility and representativeness in TREC tasks.

1. Looking Forward
Prof.dr.ir. Arjen P. de Vries
arjen@acm.org
Gaithersburg MD, November 15th, 2016

2. Q: “TREC Anniversary”

3. Top Result:
 50 years of Star Trek
(Article on the Verge about Facebook Like buttons)

4. Science Fiction
 Defining a TREC task or a track is like time-travel in Back
to the Future
Note to the audience: that is just 74 characters
You could even add the hashtag #TREC #TRECCelebrations
and my Twitter handle @arjenpdevries

6. Better Search – “Deep Personalization”
 “Even more broadly than trying to get people the right
content based on their context, we as a community need to
be thinking about how to support people through the entire
search experience.”
Jaime Teevan on “Slow Search”
 Search as a dialogue
My first journal paper:
De Vries, Van der Veer and Blanken: Let’s talk about it: dialogues with multimedia databases (1998)

7. Moving Forward
 Elements of the “Slow Search movement” at TREC today:
- Sessions
- Tasks
- Dynamic domains
- Total recall
- Complex Answer Retrieval (new!)

8. Missing from TREC!
 Access to rich personal data including email, browsing
history, documents read and contents of the user’s home
directory…

10. Trade log data!
IR-809: (2011) Feild, H., Allan, J. and Glatt, J.,
"CrowdLogging: Distributed, private, and
anonymous search logging," Proceedings of the
International Conference on Research and
Development in Information Retrieval (SIGIR'11),
pp. 375-384. [View bibtex]
We describe an approach for distributed search log collection, storage, and mining,
with the dual goals of preserving privacy and making the mined information broadly
available. [..] The approach works with any search behavior artifact that can be
extracted from a search log, including queries, query reformulations, and query-
click pairs.

11. Open challenges
 How to select the part of your log data you are willing to
trade?
 How to estimate the value of this log data?
 And a social challenge, not so much scientific:
How to get people to participate?

12. Branding

13. Branding (NL)

14. The TREC Brand
 A community that creates reusable test collections

15. Extra Slides

16. Reproducibility vs. Representativeness
 Increasing representativeness of a TREC task should not
come at the cost of sacrificing reproducibility
(104 characters )
Samar, T., Bellogín, A. & de Vries, A.P. Inf Retrieval J (2016) 19: 230.
doi: 10.1007/s10791-015-9276-9

17. Baltimore

18. Baltimore
 Title query of TREC topic 478 for the information need “Who is
the mayor of Baltimore”
 “The honest conclusion of this year’s evaluation should be that we
underestimated the problem of handling Web data. Surprising is
the performance of the title-only queries doing better than queries
including description or even narrative. It seems that the web-track
topics are really different from the previous TREC topics in the ad-
hoc task, for which we never weighted title terms different from
description or narrative.”
(Quote from the CWI TREC-9 paper)